json-url

The goal of this module is to generate a URL-friendly representation of some arbtirary JSON data in as small a space as possible. This is in contrast to e.g. "short URLs" where the representation requires some dictionary or database to be known beforehand.

Essentially, whatever is generated by this is a fixed universal representation in a particular format that has decent compression.

The motivation is to avoid having to maintain a separate database that you may need to replicate locally or otherwise make acessible remotely in order to decompress the data.

Usage

All functions are callback-style async.

Compress

    require('json-url').compress(someJson, function cb(err, result) {
        console.log(result); // will be url-friendly string
    });

Decompress

    require('json-url').decompress(someCompressedString, function cb(err, result) {
        console.log(result); // will be a JS object (NOT JSON STRING)
    });

Stats

    require('json-url').stats(someJson, function cb(err, result) {
        console.log(result); // will be a JS object giving some stats on the compression performance
    });

Approach

I explored several options, the most popular one being MessagePack. However, I noticed that it did not give the best possible compression as compared to LZMA and LZW.

At first I tried to apply the binary compression directly on a stringified JSON, then I realised that packing it first resulted in better compression. For short strings, LZW largely outperformed LZMA, but for the most part you'd probably be looking to compress large JSON data rather than small amounts (otherwise a simple stringify + base64 is sufficient), so I went with LZMA - this may be configurable in later (major) versions, although it will likely break backward compatibility.

Finally, I went with urlsafe-base64 to encode it in a URL-friendly format.

TODO

Add tests