📄 readline-pager

⚡ Memory-efficient paginated file reader for Node.js with async and sync iteration, prefetching, backward reading, and optional worker support. readline-pager reads large text files page-by-page without loading the entire file into memory.

• ✅ Zero dependencies
• ✅ Async iterator (for await...of) + manual next() API
• ✅ Sync iterator (for...of) + manual nextSync() API
• ✅ Forward & backward reading (EOF → BOF)
• ✅ Optional worker thread mode (forward only)
• ✅ Up to ~3× faster than Node.js readline
• ✅ ~97% test coverage & fully typed (TypeScript)

Important:
Performance depends heavily on the chunkSize option. Tune it for your specific I/O hardware. A value of 64 KB is usually a good starting point. Increasing it may gradually improve throughput until reaching the optimal point for your hardware.

📦 Installation

npm install readline-pager

🚀 Quick start

import { createPager } from "readline-pager";
// const { createPager } = require("readline-pager");

const pager = createPager("./bigfile.txt");

// Async iteration
for await (const page of pager) {
  console.log(page[0]); // first line of the current page
}

// Sync iteration
for (const page of pager) {
}

// Manual next
while (true) {
  const page = await pager.next();
  if (!page) break;
}

// Manual nextSync (also with condition variation)
let page;
while ((page = pager.nextSync()) !== null) {
  console.log(page[0]);
}

⚙️ Options

createPager(filepath, {
  chunkSize?: number,     // default: 64 * 1024 (64 KiB)
  pageSize?: number,      // default: 1_000
  delimiter?: string,     // default: "\n"
  prefetch?: number,      // default: 8
  backward?: boolean,     // default: false
  useWorker?: boolean,    // default: false
});

• chunkSize — number of bytes read per I/O operation.
• pageSize — number of lines per page.
• delimiter — line separator.
• prefetch — maximum number of pages buffered internally.
• backward — read the file from end → start (not supported with useWorker).
• useWorker — offload reading to a worker thread (forward reading only).

📚 API

pager.next(): Promise<string[] | null>

Returns the next page asynchronously.

Returns null when the end of the file is reached.

Empty lines are preserved.

pager.nextSync(): string[] | null

Synchronous version of pager.next().

Returns the next page immediately or null when the end of the file is reached.

pager.close(): Promise<void>

Stops reading and releases resources asynchronously. Safe to call at any time.

Note: Unlike Node.js readline, which may skip empty files or leading empty lines, readline-pager always returns all lines.

• A completely empty file (0 bytes) produces [""] on the first read.
• A file containing multiple empty lines returns each line as an empty string (for example ["", ""] for two empty lines).

📊 Benchmark

Run the included benchmark:

# default run
npm run benchmark

# or customize with args
node test/_benchmark.ts --lines=20000 --page-size=500 --backward

Test setup: generated text files with uuid, run on a fast NVMe machine with default options; values are averages from multiple runs. Results are machine-dependent.

The Average Throughput (MB/s) is computed for two strategies: reading files line by line and page by page.

In addition to Node, the two other popular JavaScript runtimes were also tested with readline-pager.

Line by line

Page by page

Runtime Environment: Node.js v25.6.1 & Bun v1.3.9 & Deno 2.6.10

🛠 Development & Contributing

• Minimum supported Node.js: v18.12 (lts/hydrogen).
• Development/test environment: Node v25.6 & TypeScript v5.9.

Run tests:

npm ci
npm test

Contributions are welcome — feel free to open an issue or PR.

📜 License

MIT — © Morteza Jamshidi