Package Exports
- ag-ui-cloudflare
Readme
ag-ui-cloudflare
Native AG-UI protocol implementation for Cloudflare Workers AI Power your CopilotKit apps with edge AI - 68% faster, 93% cheaper than OpenAI
Why Use This?
- โก 68% Faster - ~110ms response time from 200+ edge locations
- ๐ฐ 93% Cheaper - $11 per million tokens vs $150+ for OpenAI
- ๐ Global Edge - Runs at Cloudflare's 200+ locations worldwide
- ๐ Drop-in Replacement - Works with all CopilotKit components
- ๐ ๏ธ 10+ Models - Llama 3.1, Mistral, and more with auto-selection
- โจ Full AG-UI Protocol - All 16 event types for rich interactions
Features
- ๐ Full AG-UI Protocol Support - All ~16 standard event types implemented
- โก Edge-Native Performance - Sub-110ms response times from 200+ locations
- ๐ฐ 93% Cost Reduction - ~$11 per million tokens vs $150+ for OpenAI
- ๐ Real-Time Streaming - Progressive text and tool call streaming
- ๐ ๏ธ Function Calling - Support for tools with Llama 3.3 70B
- ๐ฏ Multiple Models - 10+ Cloudflare models with auto-selection
- ๐ Gateway Support - Built-in AI Gateway for caching and analytics
Installation
npm install ag-ui-cloudflare
# Peer dependencies (if not already installed)
npm install @ag-ui/core @ag-ui/protoQuick Start
Basic Usage
import { CloudflareAGUIAdapter } from 'ag-ui-cloudflare';
const adapter = new CloudflareAGUIAdapter({
accountId: 'your-account-id',
apiToken: 'your-api-token',
model: '@cf/meta/llama-3.1-8b-instruct'
});
// Execute with AG-UI events
const messages = [
{ role: 'user', content: 'Hello, how are you?' }
];
for await (const event of adapter.execute(messages)) {
console.log(event.type, event.data);
}With CopilotKit
import { CopilotRuntime } from '@copilotkit/runtime';
import { CloudflareAGUIAdapter } from 'ag-ui-cloudflare';
const adapter = new CloudflareAGUIAdapter({
accountId: process.env.CLOUDFLARE_ACCOUNT_ID!,
apiToken: process.env.CLOUDFLARE_API_TOKEN!,
});
const runtime = new CopilotRuntime({
adapter,
});
export async function POST(req: Request) {
return runtime.handle(req);
}Using Providers
import { CloudflareProviders } from 'ag-ui-cloudflare';
// Auto-select best model based on capabilities
const adapter = CloudflareProviders.auto({
accountId: 'your-account-id',
apiToken: 'your-api-token',
tools: [...], // Will use Llama 3.3 70B if tools provided
});
// Or choose specific model providers
const fastAdapter = CloudflareProviders.llama3_8b(config);
const powerfulAdapter = CloudflareProviders.llama3_70b(config);
const functionAdapter = CloudflareProviders.llama3_3_70b(config); // Supports toolsWith AI Gateway
import { CloudflareProviders } from 'ag-ui-cloudflare';
const adapter = CloudflareProviders.createWithGateway(
accountId,
apiToken,
gatewayId,
'@cf/meta/llama-3.1-70b-instruct'
);
// Benefits: Response caching, rate limiting, analyticsTool Calling
const tools = [{
type: 'function',
function: {
name: 'get_weather',
description: 'Get current weather',
parameters: {
type: 'object',
properties: {
location: { type: 'string' }
}
}
}
}];
const adapter = new CloudflareAGUIAdapter({
accountId: 'your-account-id',
apiToken: 'your-api-token',
model: '@cf/meta/llama-3.3-70b-instruct', // Only model with function calling
tools
});
for await (const event of adapter.execute(messages)) {
if (event.type === 'TOOL_CALL_START') {
console.log('Calling tool:', event.data.toolName);
}
}Progressive Generation
const stages = [
{ name: 'Research', instruction: 'Research the topic' },
{ name: 'Outline', instruction: 'Create an outline' },
{ name: 'Write', instruction: 'Write the full article' }
];
for await (const event of adapter.progressiveGeneration('Write about AI', stages)) {
if (event.type === 'PROGRESS') {
console.log(`Progress: ${event.data.progress}% - ${event.data.message}`);
}
}Available Models
| Model | Speed | Cost | Context | Function Calling |
|---|---|---|---|---|
@cf/meta/llama-3.1-8b-instruct |
โกโกโก | $ | 128K | โ |
@cf/meta/llama-3.1-70b-instruct |
โกโก | $$ | 128K | โ |
@cf/meta/llama-3.3-70b-instruct |
โกโก | $$ | 128K | โ |
@cf/mistral/mistral-7b-instruct-v0.2 |
โกโกโก | $ | 32K | โ |
@cf/google/gemma-7b-it |
โกโกโก | $ | 8K | โ |
@cf/qwen/qwen1.5-14b-chat-awq |
โกโก | $$ | 32K | โ |
AG-UI Events
The adapter emits standard AG-UI protocol events (16 total):
RUN_STARTED- Execution beginsRUN_FINISHED- Execution completesRUN_ERROR- Error during executionTEXT_MESSAGE_START- Text generation startsTEXT_MESSAGE_CONTENT- Streaming text chunksTEXT_MESSAGE_END- Text generation endsTOOL_CALL_START- Tool invocation beginsTOOL_CALL_ARGS- Tool arguments streamingTOOL_CALL_END- Tool invocation completesTOOL_CALL_RESULT- Tool execution resultsSTEP_STARTED- Processing step beginsSTEP_FINISHED- Processing step endsSTATE_SYNC- State synchronizationPROGRESS- Progress updates for multi-stage operationsMETADATA- Usage statistics and model infoCUSTOM- Application-specific events
API Reference
CloudflareAGUIAdapter
class CloudflareAGUIAdapter {
constructor(options: CloudflareAGUIAdapterOptions)
execute(messages: CloudflareMessage[], context?: Record<string, any>): AsyncGenerator<AGUIEvent>
executeWithTools(messages: CloudflareMessage[], tools: Tool[], context?: Record<string, any>): AsyncGenerator<AGUIEvent>
progressiveGeneration(prompt: string, stages: Stage[]): AsyncGenerator<AGUIEvent>
setModel(model: CloudflareModel): void
getCapabilities(): ModelCapabilities
listAvailableModels(): Promise<string[]>
}Configuration Options
interface CloudflareAGUIAdapterOptions {
accountId: string; // Cloudflare account ID
apiToken: string; // API token with Workers AI permissions
model?: CloudflareModel; // Model to use (defaults to llama-3.1-8b)
baseURL?: string; // Custom API endpoint
gatewayId?: string; // AI Gateway ID for caching/analytics
systemPrompt?: string; // System prompt for all requests
tools?: Tool[]; // Available tools for function calling
streamingEnabled?: boolean; // Enable streaming (default: true)
}Testing
npm test # Run tests
npm test:watch # Run tests in watch mode
npm test:coverage # Generate coverage reportBuilding
npm run build # Build for production
npm run dev # Build in watch mode
npm run typecheck # Type checkingContributing
Contributions are welcome! Please read our contributing guidelines and submit PRs to our GitHub repository.
License
MIT
Credits
Built with โค๏ธ by the AG-UI community for seamless edge AI integration.