MCP Server Tester - Universal Testing Tool for Model Context Protocol

The most comprehensive MCP (Model Context Protocol) server testing tool supporting HTTP REST APIs, Server-Sent Events (SSE), and STDIO process communication. Built for testing AI/LLM tools, Anthropic Claude MCP servers, OpenAI-compatible APIs, and custom implementations.

📑 Table of Contents

• Why This Tool?
• Quick Start
• Installation
• Usage Examples
• Configuration
• Features
• Architecture
• Troubleshooting
• API Reference
• Contributing
• License

🎯 Why This Tool?

The Problem

• 🔴 Self-written test scripts - Every team reinvents the wheel with custom scripts
• 🔴 No unified architecture - Fragmented testing approaches across different transports
• 🔴 Protocol complexity - SSE, HTTP, and STDIO each require different handling
• 🔴 Maintenance nightmare - Custom scripts break with protocol updates
• 🔴 Poor readability - Complex test logic buried in imperative code

Our Solution

• ✅ Professional architecture - Well-designed, extensible testing framework
• ✅ All MCP protocols - Complete support for every existing MCP transport
• ✅ Declarative YAML tests - Describe test scenarios following BDD best practices
• ✅ Scenario-based testing - Write tests as user stories with Given-When-Then structure
• ✅ Real SSE support - Native EventSource protocol, not just HTTP POST
• ✅ TypeScript safety - Catch errors at compile time
• ✅ One-line execution - npx tsx src/cli.ts tools test.yaml --server-config server.json
• ✅ CI/CD ready - JUnit XML output for any pipeline

🚀 Quick Start

1. Create test file (test.yaml)

tools:
  tests:
    - name: Test server status
      tool: get_status
      params: {}
      expect:
        success: true
        result:
          contains: "status"

2. Create server config (server.json)

{
  "mcpServers": {
    "my-server": {
      "transport": "sse",
      "url": "http://localhost:8001/sse"
    }
  }
}

3. Run tests

# Install globally
npm install -g mcp-server-tester-sse-http-stdio

# Run tests (✅ ALWAYS specify --server-name!)
npx mcp-server-tester-sse-http-stdio tools test.yaml --server-config server.json --server-name my-server

✅ Server must be running on specified URL for tests to pass!

Test Any Transport in Seconds

HTTP:

npx mcp-server-tester-sse-http-stdio tools examples/crawl4ai-http-tests.yaml \
  --server-config examples/crawl4ai-http-config.json --server-name crawl4ai

SSE:

npx mcp-server-tester-sse-http-stdio tools examples/graphiti-sse-tests.yaml \
  --server-config examples/graphiti-sse-config.json --server-name graphiti

STDIO:

npx mcp-server-tester-sse-http-stdio tools examples/crawl4ai-stdio-tests.yaml \
  --server-config examples/crawl4ai-stdio-config.json --server-name crawl4ai

🚨 Common Pitfalls

Multiple servers in config

❌ Wrong:

npx mcp-server-tester-sse-http-stdio tools test.yaml --server-config server.json
# Error: Multiple servers found, please specify server name

✅ Correct:

npx mcp-server-tester-sse-http-stdio tools test.yaml --server-config server.json --server-name graphiti

Server config format

❌ Wrong (copying from MCP client configs):

{
  "mcpServers": {
    "graphiti": {
      "transport": "sse",
      "url": "http://localhost:8001/sse",
      "description": "My server",  // ❌ Not supported
      "timeout": 30000             // ❌ Not supported
    }
  }
}

✅ Correct (minimal format):

{
  "mcpServers": {
    "graphiti": {
      "transport": "sse",
      "url": "http://localhost:8001/sse"
    }
  }
}

Test result expectations

❌ Wrong:

expect:
  success: true
  result:
    matches: true  # ❌ Not supported

✅ Correct:

expect:
  success: true
  result:
    contains: "expected text"  # ✅ or use 'equals'

Server not running

❌ Error:

Error: Failed to connect: ECONNREFUSED 127.0.0.1:8001

✅ Solution:

1. Start your MCP server first
2. Verify server is accessible: curl http://localhost:8001/health
3. Check server URL in config matches actual server

📦 Installation

Requirements

• Node.js >= 18.0.0
• npm >= 8.0.0
• Python 3.8+ (optional, for STDIO servers)

From Source

git clone https://github.com/stgmt/mcp-server-tester-sse-http-stdio.git
cd mcp-server-tester-sse-http-stdio
npm install
npm run build

From NPM (when published)

npm install -g mcp-server-tester-sse-http-stdio

📝 Usage Examples

Basic Test Structure

YAML Test File:

tools:
  expected_tool_list:
    - tool_name_1
    - tool_name_2
    
  tests:
    - name: Test scenario description
      tool: tool_name
      params:
        param1: value1
      expect:
        success: true
        result:
          contains: "expected text"

Server Configuration:

{
  "mcpServers": {
    "server_name": {
      "transport": "http|sse|stdio",
      "url": "http://localhost:3000",     // for HTTP/SSE
      "command": "python",                 // for STDIO
      "args": ["server.py", "--stdio"]     // for STDIO
    }
  }
}

⚙️ Configuration

Transport Types

Command Line Options

Options:
  --server-config <file>   MCP server configuration (required)
  --server-name <name>     Specific server from config
  --timeout <ms>           Test timeout (default: 10000)
  --debug                  Enable debug output
  --junit-xml [filename]   Generate JUnit XML report
  -t, --transport <type>   Override transport type
  -u, --url <url>          Override server URL

✨ Features

Core Capabilities

• 🔧 Universal Protocol Support - HTTP, SSE, STDIO in one tool
• 📝 YAML Test Scenarios - Human-readable test definitions
• 🎯 BDD Testing - Given-When-Then test structure
• 🔍 TypeScript - Full type safety and IntelliSense
• 📊 Test Reports - Console, JSON, JUnit XML formats
• 🚀 CI/CD Integration - Jenkins, GitHub Actions, GitLab CI

Test Types

• Tools Testing - Validate tool discovery and execution
• Compliance Testing - Protocol compliance verification
• Eval Testing - LLM-based evaluation tests
• Integration Testing - End-to-end scenarios

🏗️ Architecture

System Overview

┌─────────────────┐
│   YAML Tests    │
└────────┬────────┘
         │
┌────────▼────────┐
│   Test Runner   │
└────────┬────────┘
         │
┌────────▼────────┐
│  MCP Client     │
└────────┬────────┘
         │
    ┌────┴────┬──────────┐
    ▼         ▼          ▼
┌──────┐  ┌─────┐  ┌───────┐
│ HTTP │  │ SSE │  │ STDIO │
└──────┘  └─────┘  └───────┘

Transport Comparison

vs Alternatives

🐛 Troubleshooting

Common Issues

Debug Mode

# Enable detailed logging
npx tsx src/cli.ts tools test.yaml --server-config server.json --debug

📚 API Reference

Test Runner API

import { CapabilitiesTestRunner } from './commands/tools/runner.js';

const runner = new CapabilitiesTestRunner(
  toolsConfig,
  options,
  displayManager
);

const result = await runner.run();

Custom Transport Implementation

class CustomTransport implements McpTransport {
  async connect(): Promise<void> { /* ... */ }
  async sendRequest(method: string, params: any): Promise<any> { /* ... */ }
  async close(): Promise<void> { /* ... */ }
}

🤝 Contributing

We welcome contributions! See CONTRIBUTING.md for:

• Development setup
• Code style guidelines
• Testing requirements
• Pull request process

Development Commands

npm run dev          # Development mode
npm test            # Run tests
npm run lint        # Check code style
npm run build       # Production build

📄 License

MIT License - see LICENSE file for details.

🔗 Links

• 📦 NPM Package
• 🐛 Issue Tracker
• 💬 Telegram Channel
• 📖 MCP Documentation

⭐ Support

If this tool helps your development:

• ⭐ Star this repository
• 🔄 Share with your team
• 🐛 Report issues
• 🤝 Contribute improvements

Every star helps others discover this tool!