ScribeVerse

AI-Powered Documentation Generation for Codebases

ScribeVerse is a powerful tool that analyzes your codebase and generates comprehensive documentation using advanced AI models. It intelligently parses your code, understands dependencies, and creates structured documentation that helps developers understand and maintain projects more effectively.

🚀 Features

• Multi-Language Support: TypeScript, JavaScript, Python, Go, Rust, Java, C++, SQL, and more
• AI-Powered Analysis: Support for 7 major AI providers with 50+ models
• Intelligent Parsing: Extracts functions, classes, interfaces, types, and dependencies
• Multiple Output Formats: Markdown, HTML, JSON, and custom templates
• Incremental Updates: Only processes changed files for efficiency
• Git Integration: Automatic commits and push support
• Dependency Mapping: Visualizes code relationships and imports
• Customizable Templates: Flexible documentation generation
• 🔧 Function Flow Diagrams: Auto-generated Mermaid.js flowcharts showing function call relationships
• 🛡️ Enterprise-Grade Error Handling: Bulletproof validation, error recovery, and comprehensive logging
• 🎯 AI-Optimized Prompts: Token-efficient documentation generation designed for AI comprehension
• ⚡ Smart Filtering: Advanced file/folder exclusion patterns for optimal performance
• 🧪 130+ Tests: Comprehensive test coverage with zero-bug guarantee

🎯 Supported AI Providers

🔧 Advanced Features

• Function Flow Diagrams: Auto-generated Mermaid.js flowcharts showing function call relationships within files
• Smart Error Recovery: Graceful handling of invalid inputs, network failures, and provider errors with detailed feedback
• AI-Optimized Prompts: Token-efficient documentation designed specifically for AI comprehension
• Comprehensive Validation: Input sanitization, type checking, and security hardening
• Performance Optimized: Advanced filtering excludes 50+ unnecessary file/folder patterns
• 🧠 Smart Automation: Intelligent language detection, auto-commits, and workflow optimization - see Smart Features Guide
• 📈 Streaming Progress: Real-time progress indicators during generation with step-by-step feedback
• 🛡️ Robust Error Handling: Continues processing on failures, provides detailed error reports and suggestions

📦 Installation

npm install -g scribeverse

Or use with npx:

npx scribeverse --help

🏗️ Quick Start

1. Initialize Configuration

scribeverse init

This creates a scribeverse.config.json with default settings.

2. Configure AI Provider

Edit the generated config file:

{
  "sourceDir": "./src",
  "outputDir": "./docs",
  "ai": {
    "provider": "openai",
    "apiKey": "your-api-key",
    "model": "gpt-4o-mini",
    "maxTokens": 4000,
    "temperature": 0.2
  }
}

3. Generate Documentation

# Generate full documentation
scribeverse generate

# Generate for specific files
scribeverse generate --include "src/**/*.ts"

# Watch mode (regenerate on changes)
scribeverse generate --watch

# Generate with streaming progress (default)
scribeverse generate

⚙️ Configuration

Basic Configuration

{
  "sourceDir": "./src",
  "outputDir": "./docs",
  "include": ["**/*.{ts,js,py,go,rs,java,cpp,sql}"],
  "exclude": ["node_modules/**", "dist/**", "*.test.*"],
  "languages": ["typescript", "python", "sql"],
  "ai": {
    "provider": "openai",
    "apiKey": "sk-...",
    "model": "gpt-4o-mini"
  }
}

AI Provider Configuration

OpenAI

{
  "ai": {
    "provider": "openai",
    "apiKey": "sk-...",
    "model": "gpt-4o-mini",
    "baseURL": "https://api.openai.com/v1"
  }
}

Anthropic Claude

{
  "ai": {
    "provider": "anthropic",
    "apiKey": "sk-ant-...",
    "model": "claude-3-5-sonnet-20241022",
    "maxTokens": 4096
  }
}

Google Gemini

{
  "ai": {
    "provider": "google-gemini",
    "apiKey": "AIza...",
    "model": "gemini-2.0-flash-exp",
    "projectId": "your-project-id"
  }
}

GitHub Copilot

{
  "ai": {
    "provider": "github-copilot",
    "githubToken": "ghp_...",
    "model": "gpt-4",
    "copilotAccessMethod": "auto"
  }
}

xAI Grok

{
  "ai": {
    "provider": "grok",
    "apiKey": "xai-...",
    "model": "grok-3-beta",
    "enableSearch": true,
    "reasoningEffort": "medium"
  }
}

Ollama (Local)

{
  "ai": {
    "provider": "ollama",
    "model": "llama3.2",
    "baseURL": "http://localhost:11434"
  }
}

LiteLLM Proxy

{
  "ai": {
    "provider": "litellm",
    "model": "openai/gpt-4o-mini",
    "baseURL": "http://localhost:4000",
    "apiKey": "optional"
  }
}

Git Integration & Smart Commits

{
  "git": {
    "enabled": true,
    "autoPush": false,
    "remote": "origin",
    "branch": "main",
    "commitPrefix": "docs:"
  }
}

Smart Commit Examples

ScribeVerse automatically generates intelligent commit messages based on the changes made:

# Auto-generated commit messages
scribeverse generate
# Creates commit: "docs: Update API documentation and add function flow diagrams"

# Smart commit with analysis
scribeverse smart-commit --add-all
# Analyzes changes and creates: "feat: Add user authentication module with flow diagrams"

# Manual commit with generated message
scribeverse commit-docs
# Generates: "docs: Update module documentation for UserService and AuthController"

Example Generated Commit Messages:

• docs: Update API documentation for authentication module
• feat: Add function flow diagrams for payment processing
• docs: Update architecture documentation with new dependencies
• refactor: Improve module documentation structure and formatting

Metadata & Caching

{
  "metadata": {
    "dir": ".scribeverse",
    "enableCache": true,
    "cacheDir": ".scribeverse/cache"
  }
}

🛡️ Error Handling & Recovery

ScribeVerse includes comprehensive error handling with detailed feedback:

Streaming Progress with Error Reporting

scribeverse generate
# Output with error handling:
[1/6] Generating project overview... (17%) ✅
[2/6] Analyzing architecture... (33%) ✅
[3/6] Creating API reference... (50%) ✅
[4/6] Documenting database schemas... (67%) ⚠️  No database files found
[5/6] Generating module documentation... (83%) ❌ Module auth.js: No extractable code chunks found
[6/6] Writing documentation files... (100%) ✅

✨ Documentation generation complete!
  Files processed: 34
  Chunks analyzed: 128
  Modules with errors: 3

⚠ Warnings:
  - Module auth.js: No extractable code chunks found
  - Module config.js: AI provider timeout, retrying...
  - Database documentation skipped: No SQL files found

Cross-Platform Path Handling

• Windows: Automatically converts docs\README.md to docs/README.md for Git
• macOS/Linux: Native path handling without conversion
• Error Recovery: Continues processing if individual files fail

AI Provider Error Handling

# Graceful fallback on provider errors
[ERROR] OpenAI API rate limit exceeded, retrying in 5s...
[RETRY] Attempting with exponential backoff...
[SUCCESS] Documentation generated successfully

🛠️ CLI Commands

Generate Documentation

scribeverse generate [options]

Options:
  -c, --config <file>     Config file path (default: scribeverse.config.json)
  -w, --watch             Watch for changes and regenerate
  -i, --include <pattern> Include files matching pattern
  -e, --exclude <pattern> Exclude files matching pattern
  --force                 Force regeneration of all files
  --no-git                Disable git operations

Initialize Project

scribeverse init [options]

Options:
  --provider <provider>   AI provider (openai, anthropic, etc.)
  --model <model>        AI model to use
  --output <dir>         Output directory

List Providers & Models

scribeverse providers                 # List available providers
scribeverse models --provider openai  # List models for provider

Token Usage Tracking

scribeverse usage                     # Show usage statistics
scribeverse usage --days 7           # Show last 7 days
scribeverse usage --provider openai   # Filter by provider
scribeverse usage --export           # Export usage data to JSON
scribeverse usage --clear            # Clear all usage history
scribeverse usage --clear 30         # Clear records older than 30 days

Validate Configuration

scribeverse validate                  # Validate config and AI connection

📁 Output Structure

docs/
├── README.md                   # Project overview
├── api/                        # API documentation
│   ├── functions.md
│   ├── classes.md
│   └── interfaces.md
├── architecture/               # High-level documentation
│   ├── overview.md
│   ├── dependencies.md
│   └── diagrams/
├── modules/                    # Module-specific docs
│   ├── auth/
│   ├── database/
│   └── utils/
└── .metadata/                  # ScribeVerse metadata
    ├── chunks.json
    ├── dependencies.json
    └── cache/

🔍 Code Analysis Features

Supported Code Elements

• Functions/Methods: Parameters, return types, documentation
• Classes: Properties, methods, inheritance
• Interfaces/Types: Structure and relationships
• Variables/Constants: Type information and usage
• Imports/Exports: Dependency mapping
• Database Schemas: Tables, columns, relationships

Analysis Types

// Function analysis
export async function authenticateUser(
  credentials: UserCredentials
): Promise<AuthResult> {
  // Implementation
}

// Class analysis
export class UserService {
  private db: Database;

  async createUser(data: CreateUserData): Promise<User> {
    // Implementation
  }
}

// Interface analysis
export interface UserProfile {
  id: string;
  username: string;
  email: string;
  createdAt: Date;
}

🎨 Customization

Custom Templates

Create custom documentation templates:

// templates/function.md.hbs
# {{name}}

**Description**: {{description}}

**Parameters**:
{{#each parameters}}
- `{{name}}`: {{type}} - {{description}}
{{/each}}

**Returns**: {{returnType}}

**Usage**:
```typescript
{{example}}

### Custom Providers

Extend ScribeVerse with custom AI providers:

```typescript
import { BaseAIProvider, AIResponse } from 'scribeverse';

export class CustomProvider extends BaseAIProvider {
  async generateText(prompt: string): Promise<AIResponse> {
    // Custom implementation
  }
}

🔧 Advanced Usage

Token Usage Tracking & Analytics

ScribeVerse automatically tracks token usage and costs for all AI providers with a built-in SQLite database:

Features

• Automatic Tracking: All AI calls are tracked automatically
• Cost Estimation: Real-time cost calculations for all providers
• Usage Analytics: Detailed breakdowns by provider, model, and operation
• Historical Data: Complete usage history with timestamps
• Export/Import: JSON export for external analysis
• Privacy First: All data stored locally in ~/.scribeverse/usage.db

Usage Statistics

# View comprehensive usage statistics
scribeverse usage

# Output example:
📊 Token Usage Statistics (Last 30 days)

📈 Overview:
  Total Tokens: 125,847
  Estimated Cost: $12.45
  Sessions: 23
  Last Usage: 12/24/2025, 3:45:20 PM

🤖 By Provider:
  openai: 89,234 tokens
  anthropic: 36,613 tokens

🎯 By Model:
  gpt-4o-mini: 67,891 tokens
  claude-3-5-sonnet: 36,613 tokens
  gpt-4: 21,343 tokens

⚡ By Operation:
  generateText: 156 calls
  analyzeCode: 89 calls
  summarize: 34 calls

Data Export

# Export usage data for analysis
scribeverse usage --export
# Creates: scribeverse-usage-2025-09-24.json

# Example export structure:
{
  "exportDate": "2025-09-24T10:30:00Z",
  "statistics": { ... },
  "usageHistory": [
    {
      "provider": "openai",
      "model": "gpt-4o-mini",
      "tokensUsed": 150,
      "cost": 0.000375,
      "operation": "generateText",
      "timestamp": "2025-09-24T10:25:00Z"
    }
  ]
}

Programmatic API

import { ScribeVerse } from 'scribeverse';

const docDelta = new ScribeVerse({
  sourceDir: './src',
  outputDir: './docs',
  ai: {
    provider: 'openai',
    apiKey: process.env.OPENAI_API_KEY
  }
});

// Generate documentation
await docDelta.generate();

// Parse specific files
const chunks = await docDelta.parseFile('./src/index.ts');

// Get dependency graph
const dependencies = await docDelta.analyzeDependencies();

Batch Processing

import { ScribeVerseManager } from 'scribeverse';

const manager = new ScribeVerseManager();

// Process multiple projects
await manager.processProjects([
  { name: 'frontend', path: './frontend' },
  { name: 'backend', path: './backend' },
  { name: 'shared', path: './shared' }
]);

Custom Analysis

import { TypeScriptParser } from 'scribeverse/parsers';

const parser = new TypeScriptParser();
const chunks = await parser.parse('./src/app.ts', sourceCode);

// Filter specific chunk types
const functions = chunks.filter(chunk => chunk.type === 'function');
const classes = chunks.filter(chunk => chunk.type === 'class');

🧪 Testing

ScribeVerse includes comprehensive test coverage:

# Run all tests
npm test

# Run specific test suites
npm test -- --testPathPattern=parser
npm test -- --testPathPattern=providers
npm test -- --testPathPattern=generator

# Run with coverage
npm test -- --coverage

🛡️ Environment Variables

# AI Provider API Keys
OPENAI_API_KEY=sk-...
ANTHROPIC_API_KEY=sk-ant-...
GOOGLE_AI_API_KEY=AIza...
XAI_API_KEY=xai-...
GITHUB_TOKEN=ghp_...

# Optional Configuration
DOCDELTA_CONFIG=./custom-config.json
DOCDELTA_OUTPUT_DIR=./documentation
DOCDELTA_LOG_LEVEL=debug

🤝 Contributing

We welcome contributions! Please see CONTRIBUTING.md for details.

Development Setup

# Clone repository
git clone https://github.com/jatin2507/scribeverse.git
cd scribeverse

# Install dependencies
npm install

# Run development build
npm run dev

# Run tests
npm test

📄 License

MIT License - see LICENSE file for details.

🙋 Support

• Issues: GitHub Issues
• Discussions: GitHub Discussions

🗺️ Roadmap

• Language Support: Add PHP, Ruby, C#, Kotlin, Swift
• Output Formats: PDF, Confluence, Notion integration
• Advanced AI Features: Code suggestions, automated refactoring
• Team Collaboration: Multi-user workflows, approval processes
• IDE Integration: VS Code, JetBrains plugins
• Analytics: Documentation usage and effectiveness metrics

ScribeVerse - Intelligent Documentation for Modern Development