NodeLLM

An opinionated architectural layer for integrating Large Language Models in Node.js.

Provider-agnostic by design.

Most LLM SDKs tightly couple your application to vendors, APIs, and churn. NodeLLM provides a unified, production-oriented API for interacting with over 540+ models across multiple providers (OpenAI, Gemini, Anthropic, DeepSeek, OpenRouter, Ollama, etc.) without the SDK fatigue.

🛑 What NodeLLM is NOT

NodeLLM represents a clear architectural boundary between your system and LLM vendors.

NodeLLM is NOT:

• A wrapper around a single provider SDK (like openai or @google/generative-ai)
• A prompt-engineering framework
• An agent playground or experimental toy

🏗️ Why NodeLLM?

Most AI integrations today are provider-specific, SDK-driven, and leaky at abstraction boundaries. This creates long-term architectural risk. LLMs should be treated as infrastructure, and NodeLLM exists to help you integrate them without vendor lock-in.

NodeLLM exists to solve architectural problems, not just provide API access. It is the core architectural layer for LLMs in the Node.js ecosystem.

Strategic Goals

• Provider Isolation: Decouple your services from vendor SDKs.
• Production-Ready: Native support for streaming, retries, and unified error handling.
• Predictable API: Consistent behavior for Tools, Vision, and Structured Outputs across all models.

⚡ The Architectural Path

import { NodeLLM } from "@node-llm/core";

// 1. Configure once
NodeLLM.configure({ provider: "openai" });

// 2. Chat (High-level request/response)
const chat = NodeLLM.chat("gpt-4o");
const response = await chat.ask("Explain event-driven architecture");
console.log(response.content);

// 3. Streaming (Standard AsyncIterator)
for await (const chunk of chat.stream("Explain event-driven architecture")) {
  process.stdout.write(chunk.content);
}

🔧 Strategic Configuration

NodeLLMprovides a flexible configuration system designed for enterprise usage:

// Recommended for multi-provider pipelines
NodeLLM.configure((config) => {
  config.openaiApiKey = process.env.OPENAI_API_KEY;
  config.anthropicApiKey = process.env.ANTHROPIC_API_KEY;
  config.ollamaApiBase = process.env.OLLAMA_API_BASE;
});

// Switch providers at the framework level
NodeLLM.configure({ provider: "anthropic" });

// Support for Custom Endpoints (e.g., Azure or LocalAI)
NodeLLM.configure({
  openaiApiKey: process.env.AZURE_KEY,
  openaiApiBase: "https://your-resource.openai.azure.com/openai/deployments/...",
});

Full Configuration Guide →

🔮 Capabilities

💬 Unified Chat

Stop rewriting code for every provider. NodeLLM normalizes inputs and outputs into a single, predictable mental model.

const chat = NodeLLM.chat(); // Defaults to GPT-4o
await chat.ask("Hello world");

👁️ Smart Vision & Files

Pass images, PDFs, or audio files directly. We handle the heavy lifting: fetching remote URLs, base64 encoding, and MIME type mapping.

await chat.ask("Analyze this interface", { 
  files: ["./screenshot.png", "https://example.com/spec.pdf"] 
});

🛠️ Auto-Executing Tools

Define tools once;NodeLLM manages the recursive execution loop for you, keeping your controller logic clean. Works seamlessly with both regular chat and streaming!

// Class-based DSL
class WeatherTool extends Tool {
  name = "get_weather";
  description = "Get current weather";
  schema = z.object({ location: z.string() });
  async execute({ location }) { return `Sunny in ${location}`; }
}

// Register tools
chat.withTools([WeatherTool]);

// Now the model can use it automatically
await chat.ask("What's the weather in Tokyo?");

Full Tool Calling Guide →

🔍 Comprehensive Debug Logging

Enable detailed logging for all API requests and responses across every feature and provider:

// Set environment variable
process.env.NODELLM_DEBUG = "true";

// Now see detailed logs for every API call:
// [NodeLLM] [OpenAI] Request: POST https://api.openai.com/v1/chat/completions
// { "model": "gpt-4o", "messages": [...] }
// [NodeLLM] [OpenAI] Response: 200 OK
// { "id": "chatcmpl-123", ... }

Covers: Chat, Streaming, Images, Embeddings, Transcription, Moderation - across all providers!

✨ Structured Output

Get type-safe, validated JSON back using Zod schemas.

import { z } from "@node-llm/core";
const Product = z.object({ name: z.string(), price: z.number() });

const res = await chat.withSchema(Product).ask("Generate a gadget");
console.log(res.parsed.name); // Full type-safety

🎨 Image Generation

await NodeLLM.paint("A cyberpunk city in rain");

🎤 Audio Transcription

await NodeLLM.transcribe("meeting-recording.wav");

⚡ Scoped Parallelism

Run multiple providers in parallel safely without global configuration side effects using isolated contexts. You can also override credentials (API keys) for specific instances.

const [gpt, claude] = await Promise.all([
  // Each call branch off into its own isolated context
  NodeLLM.withProvider("openai").chat("gpt-4o").ask(prompt),
  NodeLLM.withProvider("anthropic", { anthropicApiKey: "..." }).chat("claude-3.5-sonnet").ask(prompt),
]);

🧠 Deep Reasoning

Direct access to the thought process of models like DeepSeek R1 or OpenAI o1/o3 using the .reasoning field.

const res = await NodeLLM.chat("deepseek-reasoner").ask("Solve this logical puzzle");
console.log(res.reasoning); // Chain-of-thought

🚀 Why use this over official SDKs?

📋 Supported Providers

Provider	Supported Features
OpenAI	Chat, Streaming + Tools, Vision, Audio, Images, Transcription, Reasoning
Gemini	Chat, Streaming + Tools, Vision, Audio, Video, Embeddings
Anthropic	Chat, Streaming + Tools, Vision, PDF, Structured Output
DeepSeek	Chat (V3), Reasoning (R1), Streaming + Tools
OpenRouter	Aggregator, Chat, Streaming, Tools, Vision, Embeddings, Reasoning
Ollama	Local Inference, Chat, Streaming, Tools, Vision, Embeddings

📚 Documentation & Installation

npm install @node-llm/core

View Full Documentation ↗

🫶 Credits

Heavily inspired by the elegant design of RubyLLM.

📄 License

MIT © [NodeLLMcontributors]