Package Exports

token-types

This package does not declare an exports field, so the exports above have been automatically detected and optimized by JSPM instead. If any package subpath is missing, it is recommended to post an issue to the original package (token-types) to support the "exports" field. If that is not possible, create a JSPM override to customize the exports field for this package.

Readme

token-types

A primitive token library used to read and write from a node Buffer. Although it is possible to use this module directly, it is primary designed to be used with strtok3 tokenizer.

Installation

npm install --save token-types

Usually in combination with strtok3:

npm install --save strtok3

Example

const strtok3 = require('strtok3');
const token = require('token-types');
    
(async () => {

  const tokenizer = await strtok3.fromFile("somefile.bin");
  try {
    const myNumber = await tokenizer.readToken(token.Float32_BE);
    console.log(`My number: ${myNumber}`);
  } finally {
    tokenizer.close(); // Close the file
  } 
})();

Tokens

Numeric tokens

node-strtok supports a wide variety of numerical tokens out of the box:

Token	Number	Bits	Endianness
`UINT8`	Unsigned integer	8	n/a
`UINT16_BE`	Unsigned integer	16	big endian
`UINT16_LE`	Unsigned integer	16	little endian
`UINT24_BE`	Unsigned integer	24	big endian
`UINT24_LE`	Unsigned integer	24	little endian
`UINT32_BE`	Unsigned integer	32	big endian
`UINT32_LE`	Unsigned integer	32	little endian
`UINT64_BE`	Unsigned integer	64	big endian
`UINT64_LE`*	Unsigned integer	64	little endian
`INT8`	Signed integer	8	n/a
`INT16_BE`	Signed integer	16	big endian
`INT16_LE`	Signed integer	16	little endian
`INT24_BE`	Signed integer	24	big endian
`INT24_LE`	Signed integer	24	little endian
`INT32_BE`	Signed integer	32	big endian
`INT32_LE`	Signed integer	32	little endian
`INT64_BE`	Signed integer	64	big endian
`INT64_LE`*	Signed integer	64	little endian
`Float16_BE`	IEEE 754 float	16	big endian
`Float16_LE`	IEEE 754 float	16	little endian
`Float32_BE`	IEEE 754 float	32	big endian
`Float32_LE`	IEEE 754 float	32	little endian
`Float64_BE`	IEEE 754 float	64	big endian
`Float64_LE`	IEEE 754 float	64	little endian
`Float80_BE`*	IEEE 754 float	80	big endian
`Float80_LE`*	IEEE 754 float	80	little endian

Other tokens

String types:

Windows-1252
ISO-8859-1

*) The tokens exceed the JavaScript IEEE 754 64-bit Floating Point precision, decoding and encoding is best effort based.

Custom token

Complex tokens can be added, which makes very suitable for reading binary files or network messages:

 ExtendedHeader = {
    len: 10,

    get: (buf, off) => {
      return {
        // Extended header size
        size: Token.UINT32_BE.get(buf, off),
        // Extended Flags
        extendedFlags: Token.UINT16_BE.get(buf, off + 4),
        // Size of padding
        sizeOfPadding: Token.UINT32_BE.get(buf, off + 6),
        // CRC data present
        crcDataPresent: common.strtokBITSET.get(buf, off + 4, 31)
      };
    }
  };