zstdify

Pure JavaScript/TypeScript zstd compression/decompression library. No native dependencies, works in Node.js and browsers.

Features

• Pure JS/TS zstd in Node and browsers: No native dependencies, portable by default.
• Decoder support across real-world zstd frames:
  • Raw, RLE, and compressed blocks (including Huffman/FSE-based paths).
  • Single and concatenated frames, plus skippable frame support.
  • Content checksum validation.
  • Dictionary-aware decompression, including dictionary ID checks.
• Encoder support with adaptive strategy:
  • Raw blocks, RLE blocks, and compressed blocks.
  • Compression-level driven behavior with automatic raw fallback when compressed output is not smaller.
  • Optional frame content checksums.
  • Optional dictionary-aware frame headers (dictID) for dictionary workflows.
• Dictionary generation:
  • Pure TypeScript dictionary training from sample payloads.
  • Zstd-inspired training options (fastcover/cover/legacy style knobs).
• Interop-focused: zstdify output is decoded by the official zstd CLI, and zstd CLI output is decoded by zstdify.
• Extensively tested:
  • Round-trip and property-based tests.
  • Conformance fixtures from known-good archives generated by the official zstd tool.
  • Differential tests against the official zstd CLI (both directions).
  • Corruption, boundary, and compression-regression coverage.

Usage

import { compress, decompress } from 'zstdify';

const data = new TextEncoder().encode('hello world');
const compressed = compress(data);
const restored = decompress(compressed);
// restored equals data

API

• compress(input: Uint8Array, options?: { level?: number; checksum?: boolean; dictionary?: Uint8Array | { bytes: Uint8Array; id?: number }; noDictId?: boolean }): Uint8Array
• decompress(input: Uint8Array, options?: { maxSize?: number; dictionary?: Uint8Array | { bytes: Uint8Array; id?: number } }): Uint8Array
• generateDictionary(samples: Uint8Array[], options?: { maxDictSize?: number; dictId?: number; algorithm?: "fastcover" | "cover" | "legacy"; k?: number; d?: number; steps?: number; split?: number; f?: number; accel?: number; selectivity?: number; shrink?: boolean | number }): Uint8Array

Dictionary generation outputs a raw-content dictionary. If you want a specific dictID written into compressed frames, pass it to compress() via dictionary: { bytes, id }.

Dictionary workflow example

import { compress, decompress, generateDictionary } from 'zstdify';

const encoder = new TextEncoder();
const samples = [
  encoder.encode('alpha beta gamma delta'),
  encoder.encode('header vertex texture normal index'),
  encoder.encode('offset match literal sequence table'),
];
const dictionary = generateDictionary(samples, { maxDictSize: 2048, algorithm: 'fastcover' });

const payload = encoder.encode('header vertex texture offset match literal');
const compressed = compress(payload, { dictionary: { bytes: dictionary, id: 42 } });
const restored = decompress(compressed, { dictionary: { bytes: dictionary, id: 42 } });

CLI Tool

The zstdify-cli package is a command-line tool for compressing and decompressing files with zstd. Install from npm:

pnpm add -g zstdify-cli

zstdify compress input.txt output.zst
zstdify extract output.zst restored.txt

See packages/cli/README.md for full CLI documentation.

Development

pnpm install
pnpm build
pnpm test
pnpm check

How we validate

All of the following run as part of the test suite (pnpm test / pnpm vitest):

• Round-trip: decompress(compress(x)) === x for a variety of payloads and levels, plus property-based tests with fast-check.
• Conformance fixtures: Pre-generated .zst files from the official zstd CLI (legacy fixtures and a committed decodecorpus-style corpus with manifest); we decompress and compare. See packages/zstdify-tests/fixtures/README.md.
• Differential (zstd ↔ zstdify): We test zstd compress → zstdify decompress and zstdify compress → zstd decompress across payloads and levels.
• Corruption: Truncation, checksum mismatch, invalid header bits, and related error paths.
• Compression regression: Compressed sizes for fixed payloads are checked against golden values (ratio stability).
• Decompress robustness: Each corpus fixture is decompressed in its own test (one test per file), so the suite tracks decompress behavior per input. See upstream zstd TESTING.md for comparison.

Publishing

Publish the npm packages (library first, then CLI so it gets the correct zstdify version):

pnpm make-release:zstdify
pnpm make-release:cli

Project structure

• packages/zstdify - Core library
• packages/zstdify-tests - Integration tests
• packages/cli - CLI tool (zstdify-cli on npm)
• packages/cli-tests - Tests of the CLI tool

Acknowledgements

This project is made possible by the original zstd project by Meta and its contributors. The monorepo, project, and CLI structure were bootstrapped from hdrify, which made this project much easier to build.

License

MIT

Author

Ben Houston, Sponsored by Land of Assets