JSPM

nai-js-tokenizer

1.1.0
  • ESM via JSPM
  • ES Module Entrypoint
  • Export Map
  • Keywords
  • License
  • Repository URL
  • TypeScript Types
  • README
  • Created
  • Published
  • Downloads 69
  • Score
    100M100P100Q69694F
  • License MIT

BPE tokenizer used for the NovelAI frontend.

Package Exports

  • nai-js-tokenizer
  • nai-js-tokenizer/lib/index.js

This package does not declare an exports field, so the exports above have been automatically detected and optimized by JSPM instead. If any package subpath is missing, it is recommended to post an issue to the original package (nai-js-tokenizer) to support the "exports" field. If that is not possible, create a JSPM override to customize the exports field for this package.

Readme

The BPE tokenizer used for NovelAI frontend.