llm-interface

Introduction

llm-interface is a wrapper designed to interact with multiple Large Language Model (LLM) APIs. llm-interface simplifies integrating various LLM providers, including OpenAI, AI21 Studio, Anthropic, Cloudflare AI, Cohere, DeepInfra, Fireworks AI, Friendli AI, Google Gemini, Goose AI, Groq, Hugging Face, Mistral AI, Monster API, Octo AI, Perplexity, Reka AI, watsonx.ai, and LLaMA.cpp (ollama compatible), into your applications. It is available as an NPM package.

This goal of llm-interface is to provide a single, simple, unified interface for sending messages and receiving responses from different LLM services. This will make it easier for developers to work with multiple LLMs without worrying about the specific intricacies of each API.

Features

• Unified Interface: LLMInterfaceSendMessage is a single, consistent interface to interact with 19 different LLM APIs.
• Dynamic Module Loading: Automatically loads and manages LLM interfaces only when they are invoked, minimizing resource usage.
• Error Handling: Robust error handling mechanisms to ensure reliable API interactions.
• Extensible: Easily extendable to support additional LLM providers as needed.
• Response Caching: Efficiently caches LLM responses to reduce costs and enhance performance.
• Graceful Retries: Automatically retry failed prompts with increasing delays to ensure successful responses.
• JSON Output: Simple to use native JSON output for OpenAI, Fireworks AI, and Gemini responses.
• JSON Repair: Detect and repair invalid JSON responses.

Updates

v2.0.7

• New LLM Providers: Added support for DeepInfra, FriendliAI, Monster API, Octo AI, Together AI, and NVIDIA.
• Improved Test Coverage: New DeepInfra, FriendliAI, Monster API, NVIDIA, Octo AI, Together AI, and watsonx.ai test cases.
• Refactor: Improved support for OpenAI compatible APIs using new BaseInterface class.

v2.0.6

• New LLM Provider: Added support for watsonx.ai.

v2.0.3

• New LLM Providers Functions: LLMInterface.getAllModelNames() and LLMInterface.getModelConfigValue(provider, configValueKey).

Dependencies

The project relies on several npm packages and APIs. Here are the primary dependencies:

• axios: For making HTTP requests (used for various HTTP AI APIs).
• @anthropic-ai/sdk: SDK for interacting with the Anthropic API.
• @google/generative-ai: SDK for interacting with the Google Gemini API.
• groq-sdk: SDK for interacting with the Groq API.
• openai: SDK for interacting with the OpenAI API.
• dotenv: For managing environment variables. Used by test cases.
• flat-cache: For optionally caching API responses to improve performance and reduce redundant requests.
• jsonrepair: Used to repair invalid JSON responses.
• jest: For running test cases.

Installation

To install the llm-interface package, you can use npm:

npm install llm-interface

Usage

Example

First import LLMInterfaceSendMessage. You can do this using either the CommonJS require syntax:

const { LLMInterfaceSendMessage } = require('llm-interface');

or the ES6 import syntax:

import { LLMInterfaceSendMessage } from 'llm-interface';

then send your prompt to the LLM provider of your choice:

try {
  const response = LLMInterfaceSendMessage('openai', process.env.OPENAI_API_KEY, 'Explain the importance of low latency LLMs.');
} catch (error) {
  console.error(error);
}

or if you'd like to chat, use the message object. You can also pass through options such as max_tokens.

const message = {
  model: 'gpt-3.5-turbo',
  messages: [
    { role: 'system', content: 'You are a helpful assistant.' },
    { role: 'user', content: 'Explain the importance of low latency LLMs.' },
  ],
};

try {
  const response = LLMInterfaceSendMessage('openai', process.env.OPENAI_API_KEY, message, { max_tokens: 150 });
} catch (error) {
  console.error(error);
}

If you need API Keys, use this starting point. Additional usage examples and an API reference are available. You may also wish to review the test cases for further examples.

Running Tests

The project includes tests for each LLM handler. To run the tests, use the following command:

npm test

Test Results (v2.0.0)

Test Suites: 52 passed, 52 total
Tests:       2 skipped, 215 passed, 217 total
Snapshots:   0 total
Time:        76.236 s

Note: Currently skipping NVIDIA test cases due to API key limits.

Contribute

Contributions to this project are welcome. Please fork the repository and submit a pull request with your changes or improvements.

License

This project is licensed under the MIT License - see the LICENSE file for details.

Blogs

• Comparing 13 LLM Providers API Performance with Node.js: Latency and Response Times Across Models

Share