Summoned AI Gateway

The open-source AI gateway built for India — and the world.

Route to 28+ LLM providers through one OpenAI-compatible API.

Quickstart · Providers · SDK · Features · Console · Contributing

A lightweight, open-source AI gateway that sits between your app and every LLM provider. Bring your own provider API keys — Summoned adds intelligent routing, automatic failover, response caching, cost governance, guardrails, and a full self-hosted console on top.

No code changes in your app. Drop-in replacement for the OpenAI API.

Zero infra to start. Just set ADMIN_API_KEY + one provider key and run.

• 28 providers, one API — OpenAI, Anthropic, Google, AWS Bedrock, Azure, Groq, Mistral, DeepSeek, Together, Fireworks, Cohere, Cerebras, Perplexity, xAI, OpenRouter, HuggingFace, DeepInfra, Hyperbolic, SambaNova, Novita, Moonshot, Z.AI, Nvidia NIM, Ollama, vLLM, Voyage + India-first Sarvam and Yotta.
• Zero infra required — runs completely stateless. Add Redis for caching + rate limits. Add Postgres for audit history.
• India-native — Sarvam AI, Yotta Labs, AWS Bedrock ap-south-1 (DPDP-compliant defaults), INR cost tracking.
• Full console, self-hosted, free — dashboard, live logs, playground, cost analytics. No cloud subscription needed.
• Guardrails in the free tier — PII blocking, content filters, regex rules. Competitors lock this behind enterprise.
• Virtual keys — store provider credentials encrypted (AES-256-GCM). Callers reference a vk_... id; raw keys never leave the server.
• Daily token budgets — hard caps per API key. Critical for agents. Free.
• Official TypeScript SDK — @summoned/ai on npm.

What can you do?

• Route to 28 providers through one endpoint — Supported Providers
• Zero downtime when a provider goes down — Automatic Failover & Circuit Breakers
• Route to the cheapest or fastest model automatically — Intelligent Routing
• Stop runaway agent loops before they drain your budget — Daily Token Budgets
• Cache repeated queries — Response Caching
• Block PII, profanity, injection attempts — Guardrails
• See cost (USD + INR), latency, and token usage in real time — Observability
• Encrypt and store provider keys on the gateway — Virtual Keys
• Works with OpenAI SDK, LangChain, LlamaIndex, CrewAI, Vercel AI SDK — Framework Support
• Add any OpenAI-compatible provider in 5 lines — Custom Providers

[!TIP] Starring this repo helps more developers discover the gateway 🙏

⭐ Star us on GitHub — it takes 2 seconds and means a lot.

Console

The gateway ships with a built-in web console at /console — no separate app, no extra services.

Access control: the console and its API (/console/api/*) are protected by ADMIN_API_KEY. On first visit you'll be prompted for the key; it's stored in localStorage and sent on every request. Clicking "Sign out" clears it.

Quickstart — pick one, gateway up in 15 seconds

🚀 npx (zero install)

ADMIN_API_KEY=$(openssl rand -hex 32) \
OPENAI_API_KEY=sk-... \
npx @summoned/gateway

Gateway → http://localhost:4000 · Console → http://localhost:4000/console

No clone, no Docker, no Bun — just Node 18+.

🐳 Docker (for production)

docker run -p 4000:4000 \
  -e ADMIN_API_KEY=$(openssl rand -hex 32) \
  -e OPENAI_API_KEY=sk-... \
  ghcr.io/summoned-tech/summoned-ai-gateway:latest

🛠 Full stack with Postgres + Redis (persistent logs + managed keys)

git clone https://github.com/summoned-tech/summoned-ai-gateway.git
cd summoned-ai-gateway

cp .env.example .env
# Edit .env — set ADMIN_API_KEY + POSTGRES_URL + REDIS_URL + provider keys

docker compose up -d

Or for local dev with hot reload:

make setup   # deps + Postgres + Redis + migrations + console build
make dev     # gateway with hot reload via Bun

What works without Postgres / Redis:

2. Make your first request

Option A — pass your provider key directly (no gateway key needed):

curl http://localhost:4200/v1/chat/completions \
  -H "x-provider-key: sk-YOUR_OPENAI_KEY" \
  -H "Content-Type: application/json" \
  -d '{"model": "openai/gpt-4o-mini", "messages": [{"role":"user","content":"Hello!"}]}'

Option B — use a gateway-managed key (recommended for teams):

# Create a key
curl -X POST http://localhost:4200/v1/keys \
  -H "x-admin-key: YOUR_ADMIN_KEY" \
  -H "Content-Type: application/json" \
  -d '{"name": "my-app", "tenantId": "team-a"}'

# Use it
curl http://localhost:4200/v1/chat/completions \
  -H "Authorization: Bearer sk-smnd-YOUR_KEY" \
  -H "Content-Type: application/json" \
  -d '{"model": "openai/gpt-4o-mini", "messages": [{"role":"user","content":"Hello!"}]}'

3. Add gateway features

Control retries, fallbacks, caching, routing, and guardrails per request via the x-summoned-config header (or the SDK's config field):

import json, base64
from openai import OpenAI

client = OpenAI(
    base_url="http://localhost:4200/v1",
    api_key="sk-smnd-...",
)

config = {
    "retry":    { "attempts": 3, "backoff": "exponential" },
    "fallback": ["anthropic/claude-haiku-4", "groq/llama-3.3-70b-versatile"],
    "cache":    True,
    "routing":  "cost",   # cheapest provider first
    "guardrails": {
        "input": [{ "type": "pii", "deny": True }]
    }
}

response = client.chat.completions.create(
    model="openai/gpt-4o",
    messages=[{"role": "user", "content": "Summarize this contract"}],
    extra_headers={
        "x-summoned-config": base64.b64encode(json.dumps(config).encode()).decode()
    }
)

Works with any OpenAI-compatible library — LangChain, LlamaIndex, CrewAI, Autogen, Vercel AI SDK and more.

Use from your code

The official TypeScript SDK ships on npm. It's a thin typed wrapper around the OpenAI-compatible surface, plus first-class support for the config object (retry / fallback / cache / guardrails / virtual keys).

npm install @summoned/ai

import { Summoned } from "@summoned/ai"

const client = new Summoned({
  apiKey: "sk-smnd-...",
  baseURL: "http://localhost:4200",
})

const res = await client.chat.completions.create({
  model: "openai/gpt-4o",
  messages: [{ role: "user", content: "Hello" }],
  config: {
    cache: true,
    fallback: ["anthropic/claude-sonnet-4-20250514", "groq/llama-3.3-70b-versatile"],
    routing: "cost",
  },
})

console.log(res.choices[0].message.content)
console.log(res.summoned) // { provider, cost, latency_ms, ... }

Streaming:

for await (const chunk of await client.chat.completions.create({
  model: "openai/gpt-4o",
  messages: [{ role: "user", content: "Write a haiku" }],
  stream: true,
})) {
  process.stdout.write(chunk.choices[0]?.delta?.content ?? "")
}

Prefer the OpenAI SDK? Point it at the gateway:

import OpenAI from "openai"
const openai = new OpenAI({
  baseURL: "http://localhost:4200/v1",
  apiKey: "sk-smnd-...",
})

Package: @summoned/ai · ESM-only · Node 18+ · ~13 KB tarball.

Supported Providers

28 providers out of the box. Every provider listed here is enabled by setting its env var — unset ones are simply skipped at startup, so you only pay for the surface you use.

Pure proxy — no static model catalog. Any model the upstream provider accepts works immediately, zero config changes when new models launch.

Any OpenAI-compatible provider — use CUSTOM_PROVIDERS to add any private endpoint in JSON config. No code changes needed.

Works with any framework

// @summoned/ai (official)
import { Summoned } from "@summoned/ai"
const client = new Summoned({ apiKey: "sk-smnd-...", baseURL: "http://localhost:4200" })

// OpenAI SDK
import OpenAI from "openai"
const openai = new OpenAI({ baseURL: "http://localhost:4200/v1", apiKey: "sk-smnd-..." })

// Vercel AI SDK
import { createOpenAI } from "@ai-sdk/openai"
const openai = createOpenAI({ baseURL: "http://localhost:4200/v1", apiKey: "sk-smnd-..." })

# LangChain
from langchain_openai import ChatOpenAI
llm = ChatOpenAI(base_url="http://localhost:4200/v1", api_key="sk-smnd-...")

# LlamaIndex
from llama_index.llms.openai import OpenAI
llm = OpenAI(base_url="http://localhost:4200/v1", api_key="sk-smnd-...")

# CrewAI, Autogen — just set OPENAI_BASE_URL=http://localhost:4200/v1

Core Features

Reliability

Intelligent Routing

Cost Governance

Security

Performance

Observability

API Reference

Configuration

See .env.example for the full reference.

Core

Provider credentials

Set only the ones you plan to use — unset providers are skipped at startup.

Adding a New Provider

Takes ~10 lines of code and ~5 minutes if the provider speaks the OpenAI API format:

// src/providers/your-provider.ts
import { createOpenAICompatProvider } from "./openai-compat"

export function createYourProvider(apiKey: string) {
  return createOpenAICompatProvider({
    id: "yourprovider",
    name: "Your Provider",
    apiKey,
    baseURL: "https://api.yourprovider.com/v1",
  })
}

Then add the env var in src/lib/env.ts, register it in src/index.ts, and optionally add pricing in src/lib/models/your-provider.ts. See CONTRIBUTING.md for a full walkthrough.

Development

make setup          # Full setup: deps + Postgres + Redis + migrations + console
make dev            # Gateway with hot reload
make dev-console    # Console Vite dev server
make check-types    # TypeScript type check
make migrate        # Run DB migrations
make create-key     # Quick-create an API key for testing
make help           # All commands

Run the test suite:

bun test tests      # ~45 tests across pricing, guardrails, fallback, config, circuit breaker

SDKs

Contributing

The easiest way to contribute is to add a new LLM provider — it's ~10 lines of code and ~5 minutes. See CONTRIBUTING.md.

Bug report? Open an issue → Feature request? Start a discussion →

License

MIT — free to use, fork, modify, and self-host.

Built by Summoned Tech
_{Made with ♥ for developers who care about production AI infra}