Package Exports
- llmist
- llmist/testing
Readme
llmist
Universal TypeScript LLM client with streaming-first tool execution and simple, extensible agent framework
โ ๏ธ EARLY WORK IN PROGRESS - This library is under active development. APIs may change without notice. Use in production at your own risk.
llmist is an asynchonous, streaming-first, provider-agnostic LLM client that makes it easy to build AI agents with any modelโno structured outputs or native tool calling required. Switch between OpenAI, Anthropic, and Gemini without changing your code, plug into any part of the Agent workflow, have tools (Gadgets) triggered while still streaming.
๐ฏ Why llmist?
- ๐ Universal - Works with any LLM provider (OpenAI, Anthropic, Gemini, custom)
- ๐ No Structured Outputs - Flexible YAML/JSON grammar works with any text model
- โก Streaming-First - Built for real-time responses and efficient error handling
- ๐ช Powerful Hooks - Monitor, customize, and control every step of execution
- ๐จ Beautiful API - Fluent builder pattern with model shortcuts and presets
- ๐งช Testing-Friendly - Built-in mocking system for zero-cost testing
๐ Quick Start
Installation
npm install llmist
# or
bun add llmist๐ฅ๏ธ Command Line Interface
# Quick completion
bunx llmist complete "Explain TypeScript generics" --model haiku
# Agent with tools
bunx llmist agent "Calculate 15 * 23" --gadget ./calculator.ts --model sonnet
# Pipe input
cat document.txt | llmist complete "Summarize" --model gpt-5-nanoBuilt-in gadgets are included by default in agent mode:
AskUser- Prompts for user input when clarification is neededTellUser- Displays important messages (info/success/warning/error) and can end conversations
# Disable built-in gadgets
bunx llmist agent "Task" --no-builtins -g ./my-tools.ts๐ CLI Reference | CLI Gadgets Guide
Your First Agent
import { LLMist, Gadget, z } from 'llmist';
// Define a tool (called "gadget" in llmist)
class Calculator extends Gadget({
description: 'Performs arithmetic operations',
schema: z.object({
operation: z.enum(['add', 'multiply', 'subtract', 'divide']),
a: z.number(),
b: z.number(),
}),
}) {
execute(params: this['params']): string {
const { operation, a, b } = params; // Automatically typed!
switch (operation) {
case 'add': return `${a + b}`;
case 'multiply': return `${a * b}`;
case 'subtract': return `${a - b}`;
case 'divide': return `${a / b}`;
default: throw new Error('Unknown operation');
}
}
}
// Create and run agent with fluent API
const answer = await LLMist.createAgent()
.withModel('gpt-5-nano') // Model shortcuts: sonnet, haiku, etc.
.withSystem('You are a helpful math assistant')
.withGadgets(Calculator)
.askAndCollect('What is 15 times 23?');
console.log(answer); // "15 times 23 equals 345"That's it! N
๐ Getting Started Guide - Learn more in 5 minutes
โจ Key Features
๐ Multi-Provider Support
// Use model shortcuts
.withModel('gpt-5-nano') // OpenAI gpt-5-nano
.withModel('sonnet') // Claude Sonnet 4.5
.withModel('haiku') // Claude Haiku 4.5
.withModel('flash') // Gemini 2.0 Flash
// Or full names
.withModel('openai:gpt-5-nano')
.withModel('anthropic:claude-sonnet-4-5')
.withModel('gemini:gemini-2.0-flash')Automatic provider discovery - Just set API keys as env vars (OPENAI_API_KEY, ANTHROPIC_API_KEY, GEMINI_API_KEY)
๐ Providers Guide | Model Catalog
๐ ๏ธ Flexible Gadgets (Tools)
Two ways to create tools:
// 1. Class-based with type safety
class Weather extends Gadget({
description: 'Get weather for a city',
schema: z.object({ city: z.string() }),
}) {
async execute(params: this['params']) {
// params is auto-typed!
const data = await fetch(`https://api.weather.com/${params.city}`);
return `Weather: ${data.temp}ยฐC`;
}
}
// 2. Functional for simplicity
const calculator = createGadget({
description: 'Arithmetic operations',
schema: z.object({ operation: z.enum(['add']), a: z.number(), b: z.number() }),
execute: ({ operation, a, b }) => `${a + b}`,
});๐ Gadgets Guide | Examples
๐ช Lifecycle Hooks
Monitor, transform, and control agent execution with ready-to-use presets or custom hooks:
Quick start with presets:
import { LLMist, HookPresets } from 'llmist';
// Full monitoring suite (recommended for development)
await LLMist.createAgent()
.withHooks(HookPresets.monitoring())
.ask('Your prompt');
// Output: Logs + timing + token tracking + error logging
// Combine specific presets for focused monitoring
await LLMist.createAgent()
.withHooks(HookPresets.merge(
HookPresets.timing(),
HookPresets.tokenTracking()
))
.ask('Your prompt');Available presets:
logging()/logging({ verbose: true })- Event logging with optional detailstiming()- Execution time measurementstokenTracking()- Cumulative token usage and cost trackingerrorLogging()- Detailed error informationsilent()- No output (for testing)monitoring()- All-in-one preset combining logging, timing, tokens, and errorsmerge()- Combine multiple presets or add custom hooks
Production vs Development patterns:
// Environment-based configuration
const isDev = process.env.NODE_ENV === 'development';
const hooks = isDev
? HookPresets.monitoring({ verbose: true }) // Full visibility in dev
: HookPresets.merge(
HookPresets.errorLogging(), // Only errors in prod
HookPresets.tokenTracking() // Track costs
);
await LLMist.createAgent()
.withHooks(hooks)
.ask('Your prompt');Custom hooks for advanced control:
// Observers: read-only monitoring
.withHooks({
observers: {
onLLMCallComplete: async (ctx) => {
console.log(`Used ${ctx.usage?.totalTokens} tokens`);
await sendMetricsToDataDog(ctx);
},
},
})
// Interceptors: transform data in flight
.withHooks({
interceptors: {
interceptTextChunk: (chunk) => chunk.toUpperCase(),
},
})
// Controllers: control execution flow
.withHooks({
controllers: {
beforeLLMCall: async (ctx) => {
if (shouldCache(ctx)) {
return { action: 'skip', syntheticResponse: cachedResponse };
}
return { action: 'proceed' };
},
},
})๐ Hooks Guide | Examples
๐ฌ Human-in-the-Loop
class AskUser extends Gadget({
description: 'Ask the user a question',
schema: z.object({ question: z.string() }),
}) {
execute(params: this['params']) {
throw new HumanInputException(params.question);
}
}
await LLMist.createAgent()
.withGadgets(AskUser)
.onHumanInput(async (question) => {
return await promptUser(question);
})
.ask('Help me plan my vacation');๐ Human-in-the-Loop Guide | Examples
โก Streaming & Event Handling
// Collect all text
const answer = await LLMist.createAgent()
.withModel('haiku')
.askAndCollect('Tell me a joke');
// Handle specific events
await LLMist.createAgent()
.withModel('sonnet')
.withGadgets(Calculator)
.askWith('Calculate 2 + 2', {
onText: (text) => console.log('LLM:', text),
onGadgetCall: (call) => console.log('Calling:', call.gadgetName),
onGadgetResult: (result) => console.log('Result:', result.result),
});
// Manual control
const agent = LLMist.createAgent().withModel('gpt-5-nano').ask('Question');
for await (const event of agent.run()) {
if (event.type === 'text') console.log(event.content);
}๐ Streaming Guide | Examples
๐งช Mock Testing
import { LLMist, mockLLM, createMockClient } from 'llmist';
mockLLM()
.forModel('gpt-5')
.whenMessageContains('calculate')
.returns('The answer is 42')
.register();
const mockClient = createMockClient();
const answer = await mockClient.createAgent()
.withModel('gpt-5')
.askAndCollect('Calculate 2 + 2');
console.log(answer); // "The answer is 42" - no API call made!๐ Testing Guide | Examples
๐ Model Catalog & Cost Estimation
const client = new LLMist();
// Get model specs
const gpt5 = client.modelRegistry.getModelSpec('gpt-5');
console.log(gpt5.contextWindow); // 272000
console.log(gpt5.pricing.input); // 1.25 per 1M tokens
// Estimate costs
const cost = client.modelRegistry.estimateCost('gpt-5', 10_000, 2_000);
console.log(`$${cost.totalCost.toFixed(4)}`);
// Find cheapest model
const cheapest = client.modelRegistry.getCheapestModel(10_000, 2_000);๐ Model Catalog Guide | Custom Models
๐ข Native Token Counting
const messages = [
{ role: 'system', content: 'You are helpful' },
{ role: 'user', content: 'Explain quantum computing' }
];
const tokens = await client.countTokens('openai:gpt-5', messages);
const cost = client.modelRegistry.estimateCost('gpt-5', tokens, 1000);Uses provider-specific methods (tiktoken for OpenAI, native APIs for Anthropic/Gemini).
๐ Documentation
Getting Started
- Getting Started - Your first agent in 5 minutes
- Configuration - All available options
- Quick Methods - Simple APIs for basic tasks
Core Concepts
- Gadgets (Tools) - Creating custom functions
- Hooks - Lifecycle monitoring and control
- Streaming - Real-time response handling
- Human-in-the-Loop - Interactive workflows
Advanced
- Providers - Multi-provider configuration
- Model Catalog - Querying models and costs
- Custom Models - Register fine-tuned models
- Error Handling - Recovery strategies
- Testing - Mocking and test strategies
Reference
- CLI Reference - Command-line interface
- Architecture - Technical deep-dive
- Debugging - Capture raw prompts/responses
- Troubleshooting - Common issues
๐ Examples
Comprehensive examples are available in the examples/ directory:
| Example | Description |
|---|---|
| 01-basic-usage.ts | Simple agent with calculator gadget |
| 02-custom-gadgets.ts | Async gadgets, validation, loop termination |
| 03-hooks.ts | Lifecycle hooks for monitoring |
| 04-human-in-loop.ts | Interactive conversations |
| 05-streaming.ts | Real-time streaming |
| 06-model-catalog.ts | Model queries and cost estimation |
| 07-logging.ts | Logging and debugging |
| 13-syntactic-sugar.ts | Fluent API showcase |
Run any example:
bun install && bun run build
bunx tsx examples/01-basic-usage.tsSee examples/README.md for full list and details.
๐๏ธ Architecture
llmist follows SOLID principles with a composable architecture.
Key components:
- LLMist - Provider-agnostic streaming client
- Agent - Full agent loop with automatic orchestration
- StreamProcessor - Process LLM streams with custom event loops
- GadgetExecutor - Execute tools with timeout and error handling
- GadgetRegistry - Registry for available tools
๐ Architecture Guide for detailed design documentation
๐งช Development
bun install
# Run tests
bun test # All tests
bun run test:unit # Unit tests only
bun run test:e2e # E2E tests only
# Build and lint
bun run build
bun run lint
bun run format๐ค Contributing
Contributions welcome! Please ensure:
- โ
All tests pass:
bun test - โ
Code is formatted:
bun run format - โ
Linting passes:
bun run lint - โ Types are properly defined
- โ Examples/docs updated for API changes
Commit Message Convention
This project follows Conventional Commits specification. All commit messages must be formatted as:
<type>(<scope>): <subject>Types:
feat:- New feature (triggers minor version bump)fix:- Bug fix (triggers patch version bump)docs:- Documentation only changesstyle:- Code style changes (formatting, missing semi-colons, etc)refactor:- Code refactoring without feature changesperf:- Performance improvementstest:- Adding or updating testsbuild:- Build system or dependency changesci:- CI configuration changeschore:- Other changes that don't modify src or test files
Breaking Changes: Add BREAKING CHANGE: in the footer to trigger major version bump.
Examples:
feat(agent): add support for streaming tool calls
fix(cli): prevent crash on invalid gadget path
docs: update API documentation for v2Note: Git hooks will validate your commit messages locally.
Release Process
Releases are fully automated using semantic-release:
- Merge PR to
mainbranch - CI workflow runs automatically
- If CI passes, release workflow:
- Analyzes commits since last release
- Determines version bump based on commit types
- Updates
package.jsonandCHANGELOG.md - Creates git tag and GitHub release
- Publishes to npm
- Syncs changes back to
devbranch
No manual version bumps needed!
๐ License
MIT - see LICENSE for details.
๐ Links
- ๐ฆ npm Package
- ๐ GitHub Repository
- ๐ Full Documentation
- ๐ Issue Tracker
Made with ๐คช by the llmist team