JSPM

ollama-middleware

1.2.1
  • ESM via JSPM
  • ES Module Entrypoint
  • Export Map
  • Keywords
  • License
  • Repository URL
  • TypeScript Types
  • README
  • Created
  • Published
  • Downloads 5
  • Score
    100M100P100Q69122F
  • License MIT

Complete middleware infrastructure for Ollama-based backends

Package Exports

  • ollama-middleware
  • ollama-middleware/dist/index.js

This package does not declare an exports field, so the exports above have been automatically detected and optimized by JSPM instead. If any package subpath is missing, it is recommended to post an issue to the original package (ollama-middleware) to support the "exports" field. If that is not possible, create a JSPM override to customize the exports field for this package.

Readme

๐Ÿš€ Ollama Middleware

A comprehensive TypeScript middleware library for building robust Ollama-based AI backends with advanced features like JSON cleaning, logging, error handling, and more.

npm version npm downloads TypeScript Node.js MIT License GitHub

๐Ÿ“‹ Table of Contents

โœจ Features

  • ๐Ÿ—๏ธ Clean Architecture: Base classes and interfaces for scalable AI applications
  • ๐Ÿค– Ollama Integration: Complete service layer with retry logic and authentication
  • ๐Ÿงน JSON Cleaning: Recipe-based JSON repair system with automatic strategy selection
  • ๐ŸŽจ FlatFormatter System: Advanced data formatting for LLM consumption
  • ๐Ÿ“Š Comprehensive Logging: Multi-level logging with metadata support
  • โš™๏ธ Configuration Management: Flexible model and application configuration
  • ๐Ÿ›ก๏ธ Error Handling: Robust error handling and recovery mechanisms
  • ๐Ÿ”ง TypeScript First: Full type safety throughout the entire stack
  • ๐Ÿ“ฆ Modular Design: Use only what you need
  • ๐Ÿงช Testing Ready: Includes example implementations and test utilities

๐Ÿš€ Quick Start

Installation

Install from npm:

npm install ollama-middleware

Or install directly from GitHub:

npm install github:loonylabs-dev/ollama-middleware

Or using a specific version/tag:

npm install github:loonylabs-dev/ollama-middleware#v1.1.0

Basic Usage

import { BaseAIUseCase, BaseAIRequest, BaseAIResult } from 'ollama-middleware';

// Define your request/response interfaces
interface MyRequest extends BaseAIRequest<string> {
  message: string;
}

interface MyResult extends BaseAIResult {
  response: string;
}

// Create your use case
class MyChatUseCase extends BaseAIUseCase<string, MyRequest, MyResult> {
  protected readonly systemMessage = "You are a helpful assistant.";
  
  // Required: return user message template function
  protected getUserTemplate(): (formattedPrompt: string) => string {
    return (message) => message;
  }
  
  protected formatUserMessage(prompt: any): string {
    return typeof prompt === 'string' ? prompt : prompt.message;
  }
  
  protected createResult(content: string, usedPrompt: string, thinking?: string): MyResult {
    return {
      generatedContent: content,
      model: this.modelConfig.name,
      usedPrompt: usedPrompt,
      thinking: thinking,
      response: content
    };
  }
}
๐ŸŽญ Advanced Example with FlatFormatter
import { 
  FlatFormatter, 
  personPreset
} from 'ollama-middleware';

class ProfileGeneratorUseCase extends BaseAIUseCase {
  protected readonly systemMessage = `You are a professional profile creator.
  
IMPORTANT: Respond with ONLY valid JSON following this schema:
{
  "name": "Person name",
  "title": "Professional title", 
  "summary": "Brief professional overview",
  "skills": "Key skills and expertise",
  "achievements": "Notable accomplishments"
}`;

  // Use FlatFormatter and presets for rich context building
  protected formatUserMessage(prompt: any): string {
    const { person, preferences, guidelines } = prompt;
    
    const contextSections = [
      // Use preset for structured data
      personPreset.formatForLLM(person, "## PERSON INFO:"),
      
      // Use FlatFormatter for custom structures
      `## PREFERENCES:\n${FlatFormatter.flatten(preferences, {
        format: 'bulleted',
        keyValueSeparator: ': '
      })}`,
      
      // Format guidelines with FlatFormatter
      `## GUIDELINES:\n${FlatFormatter.flatten(
        guidelines.map(g => ({ 
          guideline: g,
          priority: "MUST FOLLOW" 
        })),
        {
          format: 'numbered',
          entryTitleKey: 'guideline',
          ignoredKeys: ['guideline']
        }
      )}`
    ];
    
    return contextSections.join('\n\n');
  }
  
  protected createResult(content: string, usedPrompt: string, thinking?: string): MyResult {
    return {
      generatedContent: content,
      model: this.modelConfig.name,
      usedPrompt,
      thinking,
      profile: JSON.parse(content)
    };
  }
}

// Use it
const profileGen = new ProfileGeneratorUseCase();
const result = await profileGen.execute({ 
  prompt: { 
    person: { name: "Alice", occupation: "Engineer" },
    preferences: { tone: "professional", length: "concise" },
    guidelines: ["Highlight technical skills", "Include leadership"]
  },
  authToken: "optional-token"
});

๐Ÿ“‹ Prerequisites

๐Ÿ“ฆ Required Dependencies
  • Node.js 18+
  • TypeScript 4.9+
  • Ollama server running (local or remote)

โš™๏ธ Configuration

๐Ÿ”ง Environment Setup

Create a .env file in your project root:

# Server Configuration
PORT=3000
NODE_ENV=development

# Logging
LOG_LEVEL=info

# Ollama Model Configuration (REQUIRED)
MODEL1_NAME=phi3:mini              # Required: Your model name
MODEL1_URL=http://localhost:11434  # Optional: Defaults to localhost
MODEL1_TOKEN=optional-auth-token   # Optional: For authenticated servers

๐Ÿ—๏ธ Architecture

The middleware follows Clean Architecture principles:

src/
โ”œโ”€โ”€ middleware/
โ”‚   โ”œโ”€โ”€ controllers/base/     # Base HTTP controllers
โ”‚   โ”œโ”€โ”€ usecases/base/        # Base AI use cases
โ”‚   โ”œโ”€โ”€ services/             # External service integrations
โ”‚   โ”‚   โ”œโ”€โ”€ ollama/          # Ollama API service
โ”‚   โ”‚   โ”œโ”€โ”€ json-cleaner/    # JSON repair and validation
โ”‚   โ”‚   โ””โ”€โ”€ response-processor/ # AI response processing
โ”‚   โ””โ”€โ”€ shared/              # Common utilities and types
โ”‚       โ”œโ”€โ”€ config/          # Configuration management
โ”‚       โ”œโ”€โ”€ types/           # TypeScript interfaces
โ”‚       โ””โ”€โ”€ utils/           # Utility functions
โ””โ”€โ”€ examples/               # Example implementations
    โ””โ”€โ”€ simple-chat/        # Basic chat example

๐Ÿ“– Documentation

๐Ÿงช Testing

The middleware includes comprehensive test suites covering unit tests, integration tests, robustness tests, and end-to-end workflows.

Quick Start

# Build the middleware first
npm run build

# Run all automated tests
npm run test:all

# Run unit tests only
npm run test:unit

๐Ÿ“– For complete testing documentation, see tests/README.md

The test documentation includes:

  • ๐Ÿ“‹ Quick reference table for all tests
  • ๐Ÿš€ Detailed test descriptions and prerequisites
  • โš ๏ธ Troubleshooting guide
  • ๐Ÿ”ฌ Development workflow best practices

๐Ÿฆ Tweet Generator Example

๐Ÿ’ฌ Demonstrating Token Limiting with Social Media Content

The Tweet Generator example showcases parameter configuration for controlling output length:

import { TweetGeneratorUseCase } from 'ollama-middleware';

const tweetGenerator = new TweetGeneratorUseCase();

const result = await tweetGenerator.execute({
  prompt: 'The importance of clean code in software development'
});

console.log(result.tweet);          // Generated tweet
console.log(result.characterCount); // Character count
console.log(result.withinLimit);    // true if โ‰ค 280 chars

Key Features:

  • ๐ŸŽฏ Token Limiting: Uses num_predict: 70 to limit output to ~280 characters
  • ๐Ÿ“Š Character Validation: Automatically checks if output is within Twitter's limit
  • ๐ŸŽจ Marketing Preset: Optimized parameters for engaging, concise content
  • โœ… Testable: Integration test verifies parameter effectiveness

Parameter Configuration:

protected getParameterOverrides(): ModelParameterOverrides {
  return {
    num_predict: 70,        // Limit to ~280 characters
    temperatureOverride: 0.7,
    repeatPenalty: 1.3,
    frequencyPenalty: 0.3,
    presencePenalty: 0.2,
    topP: 0.9,
    topK: 50,
    repeatLastN: 32
  };
}

This example demonstrates:

  • How to configure parameters for specific output requirements
  • Token limiting as a practical use case
  • Validation and testing of parameter effectiveness
  • Real-world application (social media content generation)

See src/examples/tweet-generator/ for full implementation.

๐ŸŽฏ Example Application

๐Ÿš€ Quick Example Setup

Run the included examples:

# Clone the repository
git clone https://github.com/loonylabs-dev/ollama-middleware.git
cd ollama-middleware

# Install dependencies
npm install

# Copy environment template
cp .env.example .env

# Start Ollama (if running locally)
ollama serve

# Run the example
npm run dev

Test the API:

curl -X POST http://localhost:3000/api/chat \
  -H "Content-Type: application/json" \
  -d '{"message": "Hello, how are you?"}'

๐Ÿ”ง Advanced Features

๐Ÿงน Recipe-Based JSON Cleaning System

Advanced JSON repair with automatic strategy selection and modular operations:

import { JsonCleanerService, JsonCleanerFactory } from 'ollama-middleware';

// Simple usage (async - uses new recipe system with fallback)
const result = await JsonCleanerService.processResponseAsync(malformedJson);
console.log(result.cleanedJson);

// Legacy sync method (still works)
const cleaned = JsonCleanerService.processResponse(malformedJson);

// Advanced: Quick clean with automatic recipe selection
const result = await JsonCleanerFactory.quickClean(malformedJson);
console.log('Success:', result.success);
console.log('Confidence:', result.confidence);
console.log('Changes:', result.totalChanges);

Features:

  • ๐ŸŽฏ Automatic strategy selection (Conservative/Aggressive/Adaptive)
  • ๐Ÿ”ง Modular detectors & fixers for specific problems
  • โœจ Extracts JSON from Markdown/Think-Tags
  • ๐Ÿ”„ Checkpoint/Rollback support for safe repairs
  • ๐Ÿ“Š Detailed metrics (confidence, quality, performance)
  • ๐Ÿ›ก๏ธ Fallback to legacy system for compatibility

Available Templates:

import { RecipeTemplates } from 'ollama-middleware';

const conservativeRecipe = RecipeTemplates.conservative();
const aggressiveRecipe = RecipeTemplates.aggressive();
const adaptiveRecipe = RecipeTemplates.adaptive();

See Recipe System Documentation for details.

๐Ÿ“ Request Formatting (FlatFormatter & RequestFormatterService)

For simple data: Use FlatFormatter

const flat = FlatFormatter.flatten({ name: 'Alice', age: 30 });

For complex nested prompts: Use RequestFormatterService

import { RequestFormatterService } from 'ollama-middleware';

const prompt = {
  context: { genre: 'sci-fi', tone: 'dark' },
  instruction: 'Write an opening'
};

const formatted = RequestFormatterService.formatUserMessage(
  prompt, (s) => s, 'MyUseCase'
);
// Outputs: ## CONTEXT:\ngenre: sci-fi\ntone: dark\n\n## INSTRUCTION:\nWrite an opening

See Request Formatting Guide for details.

๐Ÿ“Š Performance Monitoring & Metrics

Automatic performance tracking with UseCaseMetricsLoggerService:

// Automatically logged for all use cases:
// - Execution time
// - Token usage (input/output)
// - Generation speed (tokens/sec)
// - Parameters used

Metrics appear in console logs:

โœ… Completed AI use case [MyUseCase = phi3:mini] SUCCESS
   Time: 2.5s | Input: 120 tokens | Output: 85 tokens | Speed: 34.0 tokens/sec

See Performance Monitoring Guide for advanced usage.

๐Ÿ“Š Comprehensive Logging

Multi-level logging with contextual metadata:

import { logger } from 'ollama-middleware';

logger.info('Operation completed', {
  context: 'MyService',
  metadata: { userId: 123, duration: 150 }
});
โš™๏ธ Model Configuration

Flexible model management:

import { getModelConfig } from 'ollama-middleware';

// MODEL1_NAME is required in .env or will throw error
const config = getModelConfig('MODEL1');
console.log(config.name);     // Value from MODEL1_NAME env variable
console.log(config.baseUrl);  // Value from MODEL1_URL or default localhost
๐ŸŽ›๏ธ Parameter Configuration

Ollama-middleware provides fine-grained control over model parameters to optimize output for different use cases:

import { BaseAIUseCase, ModelParameterOverrides } from 'ollama-middleware';

class MyUseCase extends BaseAIUseCase<MyRequest, MyResult> {
  protected getParameterOverrides(): ModelParameterOverrides {
    return {
      temperatureOverride: 0.8,      // Control creativity vs. determinism
      repeatPenalty: 1.3,             // Reduce word repetition
      frequencyPenalty: 0.2,          // Penalize frequent words
      presencePenalty: 0.2,           // Encourage topic diversity
      topP: 0.92,                     // Nucleus sampling threshold
      topK: 60,                       // Vocabulary selection limit
      repeatLastN: 128                // Context window for repetition
    };
  }
}

Parameter Levels:

  • Global defaults: Set in ModelParameterManagerService
  • Use-case level: Override via getParameterOverrides() method
  • Request level: Pass parameters directly in requests

Available Presets:

import { ModelParameterManagerService } from 'ollama-middleware';

// Use curated presets for common use cases
const creativeParams = ModelParameterManagerService.getDefaultParametersForType('creative_writing');
const factualParams = ModelParameterManagerService.getDefaultParametersForType('factual');
const poeticParams = ModelParameterManagerService.getDefaultParametersForType('poetic');
const dialogueParams = ModelParameterManagerService.getDefaultParametersForType('dialogue');
const technicalParams = ModelParameterManagerService.getDefaultParametersForType('technical');
const marketingParams = ModelParameterManagerService.getDefaultParametersForType('marketing');

Presets Include:

  • ๐Ÿ“š Creative Writing: Novels, stories, narrative fiction
  • ๐Ÿ“Š Factual: Reports, documentation, journalism
  • ๐ŸŽญ Poetic: Poetry, lyrics, artistic expression
  • ๐Ÿ’ฌ Dialogue: Character dialogue, conversational content
  • ๐Ÿ”ง Technical: Code documentation, API references
  • ๐Ÿ“ข Marketing: Advertisements, promotional content

For detailed documentation about all parameters, value ranges, and preset configurations, see: Ollama Parameters Guide

๐Ÿค Contributing

We welcome contributions! Please see our Contributing Guidelines for details.

  1. Fork the repository
  2. Create your feature branch (git checkout -b feature/amazing-feature)
  3. Commit your changes (git commit -m 'Add some amazing feature')
  4. Push to the branch (git push origin feature/amazing-feature)
  5. Open a Pull Request

๐Ÿ“„ License

This project is licensed under the MIT License - see the LICENSE file for details.

๐Ÿ™ Acknowledgments

  • Ollama for the amazing local LLM platform
  • The open-source community for inspiration and contributions

Made with โค๏ธ for the AI community

GitHub stars Follow on GitHub