JSPM

Skyflow SDK for Node.js

Basis Theory utility for decrypting Apple Pay Tokens

A NLP library for Russian language

Basis Theory utility for decrypting Google Pay Tokens

Small, fast, event-driven, fault-tolerant html tokenizer. Works in node or browsers.

text segmentation into sentences

Core chunking library for Chonkie - lightweight and efficient text chunking

Easily create payment tokens using Flex API

Isomorphic utilities for GPT-3 tokenization and prompt building.

n8n community node for Brickken API V2 - tokenization and STO operations on blockchain

A payment system module for SIBS payments integration with card tokenization support

ASON (Aliased Serialization Object Notation) - Token-optimized JSON compression for LLMs. Reduces tokens by 20-60% while maintaining perfect round-trip fidelity.

Official TypeScript SDK for Oumla - Blockchain integration made simple

n8n节点插件：基于jieba-wasm的中文分词功能

Compare and convert JSON to TOON format with token efficiency analysis – includes CLI, API, and Web UI server.

Node.js bindings for BlingFire tokenization library

<div align="center">

Tokenizing strings of text. Extracting arrays of words and optionally number, emojis, tags, usernames and email addresses from strings. For Node.js and the browser. When you need more than just [a-z] regular expressions.

Easily create keys for use in a Flex API integration and verifying Flex token contents

HRON (Hierarchical Reference Object Notation) is a structured text format focused on being readable, compact, and fast to parse.

API client for token processing by connecting to assca

Wix Restaurants credit-cards tokenizer

Official AstroPay client for easy frontend integration to Tokenizer

🌌 FR3K NEXUS v2.0 - Advanced Quantum Trading Platform with AI Agents, Neural Interface, Cosmic Timing, and Mystical Analysis

Main NLPTools package - Complete suite of NLP algorithms, text distance, similarity, splitting, and tokenization utilities

Babel Street Analytics API Node.js client SDK

Waffo Payment SDK for secure card tokenization

RWA Token SDK for the development of permissioned tokens on SVM blockchains.

Very fast Porter Stemmer implementation written in Javascript.

English word and sentence tokenizer, for natural language processing.

TypeScript/Node.js 版 wordfreq，提供跨语言的词频查询能力。

A high-performance wrapper around `Intl.Segmenter` for efficient text segmentation. This class resolves memory handling issues seen with large strings and can enhance performance by 50-500x. Only ~70 loc (with comments) and no dependencies.

text segmentation into sentences

OpenVINO™ Tokenizers adds text processing operations to openvino-node package

A full-featured command-line ZIP application with NeoZipKit integration

DeFarm SDK - Git for traceability with multi-role permissions and global item discovery for agriculture supply chain

Textmate token-based language service for Visual Studio Code.

Official TypeScript/JavaScript SDK for DAPP AI Cost Platform

Tokenization utilities - HuggingFace tokenizers wrapper for NLPTools

A parser and analyzer for advanced search queries, supporting tokenization, abstract syntax tree generation, and condition extraction.

DeonPay Elements SDK - Embeddable payment UI components with multi-acquirer routing

Cielo eCommerce nodes for n8n (tokenization, token payments, credential validation)

Bedrock VC Issuer Coordinator Storage

Taiwanese Hokkien Transliterator and Tokeniser

es6 version of aaditmshah/lexer

Salient is a natural language processing and sentiment analysis library

A library for natural language processing tasks including tokenization, entity recognition, anonymization, semantic chunking, and much more.

A powerful, lightweight TypeScript library for dynamic mathematical expression evaluation with support for variables, formulas, Unicode strings, and compiled expressions for high performance.

## How To Use

Bedrock Tokenization Engine

High-performance library for tokenizing text.

Here is a README generated from the code snippet:

Tools for manipulating sentences

RWA Token SDK for the development of permissioned tokens on SVM blockchains.

Official Node.js SDK for Portous Payment Platform - Accept payments, manage tokens, and handle transactions

A simple iterative lexer written in TypeScript

React Native payment processing SDK for Tuna with native Apple Pay and Google Pay support

A JSON parser implemented in TypeScript

Adebiet

tokenizing text by rules

Fat Zebra payment processing library for Next.js and standalone environments

Use LLMs to run map-reduce summarization tasks on large documents until a target token size is met.

Library that allows you to make payments and tokenize debit and credit cards with Niubiz.

A Bedrock module to provide auto-rotating tokenizers

A tokenizer and lemmatizer for canonical terms in text

Salient is a natural language processing and sentiment analysis library

POS tokenization of words int meaningful components usable in POS-Bayes & Elastic Search Indexes

A powerful and flexible text search library for JavaScript that enables you to build a simple text search engine.

Jawi

NLP (natural language processing) for server and the browser in TypeScript. All lightweight and super-fast.

An interface to the pruf network written in javascript

Break down text into array of words.

a lexer in javascript

Professional SDK for tokenized assets and security tokens on blockchain

#### Description I needed the SentenceSplitter from llamaindex but had to import the entire llamaindex package which is 1GB. I pulled it out and had GPT make a standalone version. It's not exactly the same but close.

Tokenization Service's Smart Contracts

TokenEX Node.js library for API

Tools for working with human language data.

Official SDK for MAS Hub - Enterprise blockchain platform built on MasChain

Functions for filtering SQL-like json with conjunctive normal form filters

Bedrock Tokenized Cache

A command-line tool for managing VGS aliases for tokenization

Well typed, string transmutation via splicing, driven by definitions of regular expressions, priority match and recursive clauses

package to deploy the smart contracts that associate the tokens to the various clients proportional to their contribution

A WebAssembly binding for the charabia multilingual text tokenizer used by Meilisearch.

Bedrock Tokenization Engine

Multi-criteria Cantonese segmentation with dashes, intermediates, pipes, and spaces.

Basic English tokenizer

A reverse engineered Node.js client for the Blackbox.ai API, supporting chat completions with streaming and aggregated responses.

MCP server implementation for XRP Ledger

TypeScript SDK for AddPay Cloud API - CNP payment processing

Define a function to execute after completing token-defined tasks.

Modular backend service for tokenization and smart contracts in real estate applications

A TypeScript library for building powerful, composable parsers using the parser combinator pattern. Parsco provides small, reusable parsing tools that can be combined to create complex grammars and interpreters, all without separate tokenization or extern

A Bedrock module to provide auto-rotating tokenizers

CLI tool to perform NLP on selected files

A simple and efficient tokenizer for natural language processing tasks.

This template provides a minimal setup to get React working in Vite with HMR and some ESLint rules.

a wrapper around the LunaSec CLI enabling it to be used as an NPM package

MCP library for AI-assisted Real World Asset tokenization on XRPL

TypeScript SDK for Prime Power Plant Tokenization Platform

A TypeScript/JavaScript Toolkit for Persian NLP

Node.js implementation of OpenAI's Harmony response format for structured conversational AI interactions

a wrapper around the LunaSec CLI enabling it to be used as an NPM package

A standalone, type-safe LLM SDK with multi-provider support, retry logic, context management, and intelligent tokenization.

CLI for GOON (Greatly Optimized Object Notation) - Convert JSON to ultra-compact GOON format

n8n node for Skyflow data privacy operations including detokenization

Greatly Optimized Object Notation - Hyper-compact LLM-friendly data format