Token Warden

Track LLM costs per feature, team, and user — with budget alerts and anomaly detection.

Why

LLM API costs are opaque and hard to attribute. When multiple features share the same API keys, you can't tell which feature is responsible for cost spikes. Token Warden wraps your LLM clients to automatically capture token usage and attribute costs per feature.

Quick Start

npm install token-warden

import { warden } from "token-warden";
import OpenAI from "openai";

// Initialize Token Warden
warden.init({
  apiKey: "tw_your_api_key",
  endpoint: "https://your-token-warden-endpoint.com/v1/events",
});

// Wrap your OpenAI client — tracking happens automatically
const openai = warden.wrap(new OpenAI(), {
  feature: "chat-support",
  team: "cx-team",
});

// Use the client as usual
const response = await openai.chat.completions.create({
  model: "gpt-4o",
  messages: [{ role: "user", content: "Hello!" }],
});

// In serverless environments, flush before the function exits
await warden.flush();

Manual Tracking

For custom providers or direct HTTP calls that bypass SDK clients, use warden.track() to record usage manually.

warden.track({
  provider: "together",
  model: "llama-3.1-70b",
  feature: "document-summary",
  inputTokens: 1200,
  outputTokens: 350,
  durationMs: 820,
});

Supported Providers

API Reference

warden.init(config)

Initialize Token Warden. Call once at application startup.

warden.wrap(client, opts)

Wrap an LLM SDK client to automatically capture usage.

warden.track(event)

Manually record a usage event.

warden.flush()

Flush all pending events to the ingestion endpoint. Returns a promise that resolves when all events have been sent. Call this before process exit in serverless environments.

warden.shutdown()

Flush pending events and stop the background flush interval. Call this during graceful application shutdown.

Dashboard

Token Warden includes a self-hosted dashboard for visualizing costs, setting budgets, and detecting anomalies.

Features

• Per-feature cost breakdown with daily spend charts and sparkline trends
• Feature drill-down with model usage, cost over time, and recent request history
• Budget alerts — set daily, weekly, or monthly limits per feature with configurable thresholds
• Anomaly detection — automatically flags cost spikes exceeding 3x the 7-day hourly average
• Webhook notifications — Slack-compatible alerts when budgets are breached
• Dual database support — SQLite for local dev, PostgreSQL for production

Setup

cd apps/dashboard
npm install
npm run db:setup        # Create tables + seed model pricing
npm run db:seed-demo    # Optional: generate 30 days of demo data
npm run dev             # Start on http://localhost:3100

Connect the SDK

Point your Token Warden SDK at the dashboard's ingestion endpoint:

import { warden } from "token-warden";

warden.init({
  apiKey: process.env.WARDEN_API_KEY,      // Set this on the dashboard too
  endpoint: "http://localhost:3100/api/events",
});

Environment Variables

Routes

Contributing

See CONTRIBUTING.md for development setup and guidelines.

License

MIT — see LICENSE for details.