Patter

Connect AI agents to phone numbers with 10 lines of code

Quickstart • Features • Installation • Documentation • Self-Hosting

Patter is an open-source platform that gives your AI agent a voice and a phone number. Point it at any function that returns a string, and Patter handles the rest: telephony, speech-to-text, text-to-speech, and real-time audio streaming.

Quickstart

import asyncio
from patter import Patter, IncomingMessage

async def on_message(msg: IncomingMessage) -> str:
    # Your agent logic here — return what the AI should say
    return f"You said: {msg.text}"

async def main():
    phone = Patter(api_key="pt_xxx")
    await phone.connect(on_message=on_message)  # starts listening for inbound calls

asyncio.run(main())

import { Patter } from "getpatter";

const phone = new Patter({ apiKey: "pt_xxx" });

await phone.connect({
  onMessage: async (msg) => {
    // Your agent logic here — return what the AI should say
    return `You said: ${msg.text}`;
  },
});

Local Mode (No Cloud Required)

Run Patter entirely in your process — no Patter account, no cloud backend.

import asyncio
from patter import Patter

async def main():
    phone = Patter(
        mode="local",
        twilio_sid="AC...", twilio_token="...",
        openai_key="sk-...",
        phone_number="+1...",
        webhook_url="xxx.ngrok-free.dev",
    )

    agent = phone.agent(
        system_prompt="You are a friendly customer service agent for Acme Corp.",
        voice="alloy",
        first_message="Hello! Thanks for calling. How can I help?",
    )

    print("Listening for calls...")
    await phone.serve(agent=agent, port=8000)

asyncio.run(main())

import { Patter } from "getpatter";

const phone = new Patter({
  mode: "local",
  twilioSid: "AC...", twilioToken: "...",
  openaiKey: "sk-...",
  phoneNumber: "+1...",
  webhookUrl: "xxx.ngrok-free.dev",
});

const agent = phone.agent({
  systemPrompt: "You are a friendly customer service agent for Acme Corp.",
  voice: "alloy",
  firstMessage: "Hello! Thanks for calling. How can I help?",
});

console.log("Listening for calls...");
await phone.serve({ agent, port: 8000 });

Local vs Cloud

Features

Voice

• Three voice modes: OpenAI Realtime, ElevenLabs ConvAI, Pipeline (any STT + TTS)
• Any STT: Deepgram, OpenAI Whisper
• Any TTS: ElevenLabs, OpenAI TTS
• Natural barge-in with mark-based audio tracking
• DTMF keypad input forwarded to agent as [DTMF: 1]

Agent

• Bring your own agent (any LLM in pipeline mode)
• System prompt with dynamic {variable} substitution
• Conversation history tracked per call (data.history in all callbacks)
• Tool calling via webhooks with automatic 3x retry
• Built-in tools: transfer_call, end_call (auto-injected)

Telephony

• Twilio and Telnyx carriers
• Inbound and outbound calls
• Call transfer to humans (transfer_call system tool)
• Call recording (recording: true in serve())
• Answering machine detection (machineDetection: true for outbound)
• Voicemail drop (voicemailMessage: "..." plays on voicemail detection)
• Custom parameters passthrough via TwiML

Developer Experience

• pip install patter / npm install getpatter
• 10 lines of code to connect an agent to a phone
• Local mode (embedded, no backend) + Cloud mode
• Python + TypeScript SDKs with full parity
• MCP server for Claude Desktop
• Open-source (MIT)

Complete Example

const phone = new Patter({
  mode: 'local',
  twilioSid: process.env.TWILIO_SID,
  twilioToken: process.env.TWILIO_TOKEN,
  openaiKey: process.env.OPENAI_KEY,
  phoneNumber: '+16592214527',
  webhookUrl: 'your-domain.ngrok-free.dev',
});

const agent = phone.agent({
  systemPrompt: `You are a customer service agent for Acme Corp.
The customer is {customer_name} with order #{order_id}.
Check inventory before answering stock questions.
Transfer to a human if the customer is upset.`,
  voice: 'alloy',
  language: 'en',
  firstMessage: 'Hi {customer_name}! How can I help with order #{order_id}?',
  variables: {
    customer_name: 'John',
    order_id: '12345',
  },
  tools: [{
    name: 'check_inventory',
    description: 'Check product stock',
    parameters: { type: 'object', properties: { product: { type: 'string' } } },
    webhookUrl: 'https://api.acme.com/inventory',
  }],
  // Built-in: transfer_call, end_call (auto-injected)
});

await phone.serve({
  agent,
  port: 8000,
  recording: true,
  onCallStart: async (data) => console.log(`Call from ${data.caller}`),
  onCallEnd: async (data) => console.log(`Transcript: ${data.transcript?.length} turns`),
  onTranscript: async (data) => console.log(`${data.role}: ${data.text}`),
});

// Outbound with machine detection
await phone.call({
  to: '+1234567890',
  machineDetection: true,
  voicemailMessage: 'Hi, please call us back at 555-0123.',
});

How It Works

Your Code (on_message handler)
        │
        ▼
   Patter SDK  ──WebSocket──►  Patter Backend  ──────────────────────────────┐
                                      │                                       │
                              ┌───────┴────────┐                              │
                              ▼                ▼                              ▼
                          STT Engine       TTS Engine          Telephony Provider
                       (Deepgram /      (ElevenLabs /          (Twilio / Telnyx)
                        Whisper /        OpenAI TTS)                  │
                       OpenAI RT)              │                       │
                              │               └───────────────────────►│
                              └────────────────────────────────────────►│
                                                                        ▼
                                                                  Phone Call

The audio path: Phone → Telephony → WebSocket → Backend → STT → your handler → TTS → Backend → WebSocket → Phone

Installation

# Python
pip install patter

# TypeScript / Node.js
npm install getpatter

Documentation

Inbound Calls (AI answers the phone)

import asyncio
from patter import Patter, IncomingMessage

async def agent(msg: IncomingMessage) -> str:
    if "hours" in msg.text.lower():
        return "We're open Monday through Friday, 9 to 5."
    return "How can I help you today?"

async def main():
    phone = Patter(api_key="pt_xxx")
    await phone.connect(
        on_message=agent,
        on_call_start=lambda data: print(f"Call from {data['caller']}"),
        on_call_end=lambda data: print("Call ended"),
    )
    await asyncio.Event().wait()  # keep the process alive

asyncio.run(main())

import { Patter } from "getpatter";

const phone = new Patter({ apiKey: "pt_xxx" });

await phone.connect({
  onMessage: async (msg) => {
    if (msg.text.toLowerCase().includes("hours")) {
      return "We're open Monday through Friday, 9 to 5.";
    }
    return "How can I help you today?";
  },
  onCallStart: (data) => console.log(`Call from ${data.caller}`),
  onCallEnd: () => console.log("Call ended"),
});

Outbound Calls (AI calls someone)

import asyncio
from patter import Patter, IncomingMessage

async def agent(msg: IncomingMessage) -> str:
    return "Thanks for picking up. This is a reminder about your appointment tomorrow."

async def main():
    phone = Patter(api_key="pt_xxx")
    await phone.connect(on_message=agent)
    await phone.call(
        to="+14155551234",
        first_message="Hi, this is an automated reminder from Acme Corp.",
    )

asyncio.run(main())

import { Patter } from "getpatter";

const phone = new Patter({ apiKey: "pt_xxx" });

await phone.connect({
  onMessage: async () =>
    "Thanks for picking up. This is a reminder about your appointment tomorrow.",
});

await phone.call({
  to: "+14155551234",
  firstMessage: "Hi, this is an automated reminder from Acme Corp.",
});

Outbound with Machine Detection + Voicemail Drop

await phone.call(
    to="+14155551234",
    first_message="Hi, this is an automated reminder from Acme Corp.",
    machine_detection=True,
    voicemail_message="Hi, we tried to reach you. Please call us back at 555-0123.",
)

await phone.call({
  to: "+14155551234",
  firstMessage: "Hi, this is an automated reminder from Acme Corp.",
  machineDetection: true,
  voicemailMessage: "Hi, we tried to reach you. Please call us back at 555-0123.",
});

Dynamic Variables in Prompts

Inject call-specific data into system prompts and first messages using {variable} placeholders.

agent = phone.agent(
    system_prompt="You are helping {customer_name}, account #{account_id}.",
    first_message="Hi {customer_name}! How can I help you today?",
    variables={
        "customer_name": "Jane",
        "account_id": "A-789",
    },
)

const agent = phone.agent({
  systemPrompt: "You are helping {customer_name}, account #{account_id}.",
  firstMessage: "Hi {customer_name}! How can I help you today?",
  variables: {
    customer_name: "Jane",
    account_id: "A-789",
  },
});

Tool Calling via Webhooks

Agents can call external APIs mid-conversation. Patter POSTs to your webhook URL and retries up to 3 times on failure.

agent = phone.agent(
    system_prompt="You are a booking assistant. Check availability before confirming.",
    tools=[{
        "name": "check_availability",
        "description": "Check appointment availability for a given date",
        "parameters": {
            "type": "object",
            "properties": {
                "date": {"type": "string", "description": "ISO date, e.g. 2025-06-15"},
            },
            "required": ["date"],
        },
        "webhook_url": "https://api.example.com/availability",
    }],
)

const agent = phone.agent({
  systemPrompt: "You are a booking assistant. Check availability before confirming.",
  tools: [{
    name: "check_availability",
    description: "Check appointment availability for a given date",
    parameters: {
      type: "object",
      properties: {
        date: { type: "string", description: "ISO date, e.g. 2025-06-15" },
      },
      required: ["date"],
    },
    webhookUrl: "https://api.example.com/availability",
  }],
});

Built-in Tools: Transfer & End Call

transfer_call and end_call are automatically injected into every agent — no configuration needed.

• The agent calls transfer_call when it decides to route to a human (e.g. "Let me transfer you now.")
• The agent calls end_call when the conversation is complete (e.g. after a confirmed booking.)

Call Recording

await phone.serve(agent=agent, port=8000, recording=True)

await phone.serve({ agent, port: 8000, recording: true });

Conversation History

Every callback receives data.history — the full conversation so far as a list of {role, text} turns.

await phone.serve(
    agent=agent,
    port=8000,
    on_transcript=lambda data: print(f"[{data['role']}] {data['text']}"),
    on_call_end=lambda data: print(f"Full history: {data['history']}"),
)

await phone.serve({
  agent,
  port: 8000,
  onTranscript: (data) => console.log(`[${data.role}] ${data.text}`),
  onCallEnd: (data) => console.log(`Full history:`, data.history),
});

Custom Voice (choose your providers)

phone = Patter(api_key="pt_xxx", backend_url="ws://localhost:8000")

await phone.connect(
    on_message=agent,
    provider="twilio",
    provider_key="ACxxxxxxxx",
    provider_secret="your_auth_token",
    number="+14155550000",
    stt=Patter.deepgram(api_key="dg_xxx", language="en"),
    tts=Patter.elevenlabs(api_key="el_xxx", voice="rachel"),
)

const phone = new Patter({ apiKey: "pt_xxx", backendUrl: "ws://localhost:8000" });

await phone.connect({
  onMessage: agent,
  provider: "twilio",
  providerKey: "ACxxxxxxxx",
  providerSecret: "your_auth_token",
  number: "+14155550000",
  stt: Patter.deepgram({ apiKey: "dg_xxx", language: "en" }),
  tts: Patter.elevenlabs({ apiKey: "el_xxx", voice: "rachel" }),
});

Voice Modes

The voice mode is configured on the backend. Your on_message handler works identically regardless of mode.

MCP Server (Claude Desktop)

Patter ships an MCP server so you can control calls directly from Claude Desktop.

{
  "mcpServers": {
    "patter": {
      "command": "patter-mcp",
      "env": { "PATTER_API_KEY": "pt_xxx" }
    }
  }
}

Self-Hosting

Run the full stack yourself — no Patter Cloud account needed.

# 1. Clone the repo
git clone https://github.com/your-org/patter
cd patter

# 2. Copy env and fill in your keys
cp .env.example .env

# 3. Start the backend
cd backend
pip install -e ".[dev]"
alembic upgrade head
uvicorn app.main:app --reload

Then point your SDK at your local backend:

phone = Patter(
    api_key="pt_xxx",
    backend_url="ws://localhost:8000",
    rest_url="http://localhost:8000",
)

Required environment variables:

See backend/.env.example for the full list.

API Reference

Patter (Python & TypeScript)

serve() options:

agent() options:

call() options:

Static provider helpers (for self-hosted mode):

IncomingMessage

Contributing

Pull requests are welcome.

# Run tests
cd backend && pytest tests/ -v
cd sdk && pytest tests/ -v

# Install dev dependencies
cd backend && pip install -e ".[dev]"
cd sdk && pip install -e ".[dev]"

Please open an issue before submitting large changes so we can discuss the approach first.

License

MIT — see LICENSE.