tokenx
Fast and lightweight token estimation for any LLM without requiring a full tokenizer
Found 25 results for tiktoken
Fast and lightweight token estimation for any LLM without requiring a full tokenizer
Calculate the token consumption and amount of openai gpt message
Universal LLM context manager supporting 14+ languages with method-level filtering, token optimization, and GitIngest format support
A service for calculating, managing, truncating openai prompt tokens
Anthropic/AWS Bedrock Typescript SDK.
#### Description I needed the SentenceSplitter from llamaindex but had to import the entire llamaindex package which is 1GB. I pulled it out and had GPT make a standalone version. It's not exactly the same but close.
🚀 CLI tool to analyze projects, concatenate source files, count tokens, and export HTML with syntax highlighting and copy functionality
Weigh your code in tokens - calculate AI API costs for your codebase
UltraToken Utility - CLI tool for token cost analysis
A simple CLI tool to count tokens in files or clipboard content using tiktoken
CLI tool to process PDFs and create local vector databases using ChromaDB
GPT token estimation and context size utilities without a full tokenizer
Use web-only models (e.g., GPT-5 Pro) with your local codebase: scan, pack to token limits, and copy a paste-ready bundle.
A javascript library designed to estimate AI models tokens and its API costs.
High-level binding to tiktoken-rs for Node.js and browsers
tiktoken for nest.js
Model Context Protocol(MCP)服务器专为LLMs与Obsidian vaults进行交互而设计。通过标准化接口提供安全、token感知的工具,实现无缝知识库管理。
High-level binding to tiktoken-rs for Node.js and browsers
High-level binding to tiktoken-rs for Node.js and browsers
Fast tokenizer.
High-level binding to tiktoken-rs for Node.js and browsers
offline-capable ESM module for cl100k_base tokenization in the browser
High-level binding to tiktoken-rs for Node.js and browsers
Model Context Protocol (MCP) server designed for LLMs to interact with Obsidian vaults. Provides secure, token-aware tools for seamless knowledge base management through a standardized interface.
Pure JavaScript version of OpenAI tiktoken