JSON Stream Merger

A Node.js package for merging large JSON files using streams, designed to handle files that are too large to fit in memory.

Features

• Stream-based processing for memory-efficient handling of large JSON files
• Supports merging multiple JSON files into a single JSON array
• No memory limitations as it processes files in chunks
• Maintains JSON formatting in the output file
• Command-line interface for easy use
• Support for merging all JSON files from a directory

Installation

npm install json-stream-merger

Usage

As a Node.js Module

const { mergeJsonFiles } = require('json-stream-merger');

// Example usage
async function example() {
  try {
    await mergeJsonFiles(
      ['file1.json', 'file2.json', 'file3.json'],
      'merged-output.json'
    );
    console.log('Files merged successfully!');
  } catch (error) {
    console.error('Error merging files:', error);
  }
}

Command Line Interface

After installation, you can use the package from the command line:

# Merge specific files
json-stream-merger output.json input1.json input2.json input3.json

# Merge all JSON files from a directory
json-stream-merger output.json --directory ./json-files

# Combine directory and specific files
json-stream-merger output.json --directory ./json-files additional1.json additional2.json

# Suppress console output
json-stream-merger -s output.json --directory ./json-files

# Show help
json-stream-merger --help

Features

Directory Support

The --directory option allows you to merge all JSON files from a specified directory. This is useful when you have multiple JSON files that need to be combined. The output file will be written to the current working directory, and if it's in the same directory as the input files, it will be automatically excluded from the merge process.

API

mergeJsonFiles(inputFiles, outputFile)

Merges multiple JSON files into a single JSON array.

Parameters

• inputFiles (string[]): Array of input file paths to merge
• outputFile (string): Path where the merged JSON will be written

Returns

• Promise: Resolves when all files are merged

Throws

• Error: If input files array is empty or invalid

License

MIT