it-length-prefixed

Streaming length prefixed buffers with async iterables

Install
Usage
API
Contribute
License
Contribution

Install

$ npm i it-length-prefixed

Usage

import { pipe } from 'it-pipe'
import * as lp from 'it-length-prefixed'

const encoded = []

// encode
await pipe(
  [uint8ArrayFromString('hello world')],
  lp.encode(),
  async source => {
    for await (const chunk of source) {
      encoded.push(chunk.slice()) // (.slice converts Uint8ArrayList to Uint8Array)
    }
  }
)

console.log(encoded)
// => [Buffer <0b 68 65 6c 6c 6f 20 77 6f 72 6c 64>]

const decoded = []

// decode
await pipe(
  encoded, // e.g. from above
  lp.decode(),
  async source => {
    for await (const chunk of source) {
      decoded.push(chunk.slice()) // (.slice converts Uint8ArrayList to Uint8Array)
    }
  }
)

console.log(decoded)
// => [Buffer <68 65 6c 6c 6f 20 77 6f 72 6c 64>]

API

import {
  encode, decode
} from 'it-length-prefixed'

import {
  encode
} from 'it-length-prefixed/encode'

import {
  decode,
  MAX_LENGTH_LENGTH,
  MAX_DATA_LENGTH
} from 'it-length-prefixed/decode'

`encode([opts])`

opts: Object, optional
- lengthEncoder: Function: A function that encodes the length that will prefix each message. By default this is a varint encoder. It is passed a value to encode, an (optional) target buffer to write to and an (optional) offset to start writing from. The function should encode the value into the target (or alloc a new Buffer if not specified), set the lengthEncoder.bytes value (the number of bytes written) and return the target.

Returns a transform that yields Uint8ArrayList objects. All messages will be prefixed with a length, determined by the lengthEncoder function.

`encode.single(chunk, [opts])`

chunk: Buffer|Uint8ArrayList chunk to encode
opts: Object, optional
- lengthEncoder: Function: See description above. Note that this encoder will not be passed a target or offset and so will need to allocate a buffer to write to.

Returns a Uint8ArrayList containing the encoded chunk.

`decode([opts])`

opts: Object, optional
- maxLengthLength: If provided, will not decode messages whose length section exceeds the size specified, if omitted will use the default of 147 bytes.
- maxDataLength: If provided, will not decode messages whose data section exceeds the size specified, if omitted will use the default of 4MB.
- onLength(len: Number): Called for every length prefix that is decoded from the stream
- onData(data: Uint8ArrayList): Called for every chunk of data that is decoded from the stream
- lengthDecoder: Function: A function that decodes the length that prefixes each message. By default this is a varint decoder. It is passed some data to decode which is a Uint8ArrayList. The function should decode the length, set the lengthDecoder.bytes value (the number of bytes read) and return the length. If the length cannot be decoded, the function should throw a RangeError.

Returns a transform that yields Uint8ArrayList objects.

`decode.fromReader(reader, [opts])`

Behaves like decode except it only reads the exact number of bytes needed for each message in reader.

reader: Reader: An it-reader
opts: Object, optional
- maxLengthLength: If provided, will not decode messages whose length section exceeds the size specified, if omitted will use the default of 147 bytes.
- maxDataLength: If provided, will not decode messages whose data section exceeds the size specified, if omitted will use the default of 4MB.
- onData(data: Uint8ArrayList): Called for every chunk of data that is decoded from the stream
- lengthEncoder: Function: See description above.

Returns a transform that yields Uint8ArrayList objects.

Contribute

PRs and issues gladly accepted! Check out the issues.

License

Licensed under either of

Apache 2.0, (LICENSE-APACHE / http://www.apache.org/licenses/LICENSE-2.0)
MIT (LICENSE-MIT / http://opensource.org/licenses/MIT)

Contribution

Unless you explicitly state otherwise, any contribution intentionally submitted for inclusion in the work by you, as defined in the Apache-2.0 license, shall be dual licensed as above, without any additional terms or conditions.

JSPM