@node-llm/core

The production-grade LLM engine for Node.js. Provider-agnostic by design.

@node-llm/core provides a single, unified API for interacting with over 540+ models across all major providers. It is built for developers who need stable infrastructure, standard streaming, and automated tool execution without vendor lock-in.

🚀 Key Features

• Unified API: One interface for OpenAI, Anthropic, Gemini, DeepSeek, OpenRouter, and Ollama.
• Automated Tool Loops: Recursive tool execution handled automatically—no manual loops required.
• Streaming + Tools: Seamlessly execute tools and continue the stream with the final response.
• Structured Output: Native Zod support for rigorous schema validation (.withSchema()).
• Multimodal engine: Built-in handling for Vision, Audio (Whisper), and Video (Gemini).
• Security-First: Integrated circuit breakers for timeouts, max tokens, and infinite tool loops.

📋 Supported Providers

Provider	Supported Features
OpenAI	Chat, Streaming, Tools, Vision, Audio, Images, Transcription, Reasoning
Anthropic	Chat, Streaming, Tools, Vision, PDF, Structured Output, Extended Thinking (Claude 3.7)
Gemini	Chat, Streaming, Tools, Vision, Audio, Video, Embeddings
DeepSeek	Chat (V3), Extended Thinking (R1), Streaming, Tools
Bedrock	Chat, Streaming, Tools, Image Gen (Titan/SD), Embeddings, Prompt Caching
OpenRouter	540+ models, Chat, Streaming, Tools, Vision, Embeddings, Reasoning
Ollama	Local Inference, Chat, Streaming, Tools, Vision, Embeddings

⚡ Quick Start

Installation

npm install @node-llm/core

Basic Chat & Streaming

NodeLLM automatically reads your API keys from environment variables (e.g., OPENAI_API_KEY).

import { createLLM } from "@node-llm/core";

const llm = createLLM({ provider: "openai" });

// 1. Standard Request
const res = await llm.chat("gpt-4o").ask("What is the speed of light?");
console.log(res.content);

// 2. Real-time Streaming
for await (const chunk of llm.chat().stream("Tell me a long story")) {
  process.stdout.write(chunk.content);
}

Structured Output (Zod)

Stop parsing markdown. Get typed objects directly.

import { z } from "@node-llm/core";

const PlayerSchema = z.object({
  name: z.string(),
  powerLevel: z.number(),
  abilities: z.array(z.string())
});

const chat = llm.chat("gpt-4o-mini").withSchema(PlayerSchema);
const response = await chat.ask("Generate a random RPG character");

console.log(response.parsed.name); // Fully typed!

🛡️ Security Circuit Breakers

NodeLLM protects your production environment with four built-in safety pillars:

const llm = createLLM({
  requestTimeout: 15000, // 15s DoS Protection
  maxTokens: 4096, // Cost Protection
  maxRetries: 3, // Retry Storm Protection
  maxToolCalls: 5 // Infinite Loop Protection
});

🔌 Middleware System

NodeLLM 1.9.0 introduces a powerful lifecycle hook system for audit, security, and observability.

import { createLLM, PIIMaskMiddleware, UsageLoggerMiddleware } from "@node-llm/core";

const llm = createLLM({
  provider: "openai",
  middlewares: [
    new PIIMaskMiddleware(), // Redact emails/phone numbers automatically
    new UsageLoggerMiddleware() // Log structured token usage & costs
  ]
});

// All chats created from this instance inherit these middlewares
const chat = llm.chat("gpt-4o");

Decisive Tool Safety

Middlewares can control the engine's recovery strategy during tool failures.

const safetyMiddleware = {
  name: "Audit",
  onToolCallError: async (ctx, tool, error) => {
    if (tool.function.name === "delete_user") return "STOP"; // Kill the loop
    return "RETRY"; // Attempt recovery
  }
};

💾 Ecosystem

Looking for persistence? use @node-llm/orm.

• Automatically saves chat history to PostgreSQL/MySQL/SQLite via Prisma.
• Tracks tool execution results and API metrics (latency, cost, tokens).

📚 Full Documentation

Visit node-llm.eshaiju.com for:

• Deep Dive into Tool Calling
• Multi-modal Vision & Audio Guide
• Custom Provider Plugin System

License

MIT © [NodeLLM Contributors]