RunHuman MCP Server

A Model Context Protocol (MCP) server that allows AI agents to interact with the RunHuman QA testing service.

Overview

This MCP server provides tools for creating and managing human QA jobs through the RunHuman API. AI agents can use this server to:

• Create new QA jobs with custom schemas
• Check the status of running jobs
• Retrieve completed job results

Installation

For Claude Desktop (Recommended)

1. Get your API key at: https://qa-experiment.fly.dev/app.html

2. Add to your Claude Desktop config (~/Library/Application Support/Claude/claude_desktop_config.json on Mac):

{
  "mcpServers": {
    "runhuman": {
      "command": "npx",
      "args": ["-y", "@runhuman/mcp-server", "--api-key=qa_live_xxxxxxxxxxxxx"]
    }
  }
}

3. Restart Claude Desktop

That's it! The server will be automatically downloaded and run by Claude.

For Development

From the monorepo root:

npm install
npm run build --workspace=@runhuman/mcp-server

# Run with API key
node packages/mcp-server/dist/index.js --api-key=qa_live_xxxxx

Available Tools

create_job

Create a new QA job with human testers.

Parameters:

• url (string): The URL to test
• description (string): Instructions for the human tester describing what to test
• schema (object): Expected result schema that the tester response will be extracted into

get_job_status

Get the current status of a QA job.

Parameters:

• jobId (string): The ID of the job to check

get_job_result

Get the results of a completed QA job.

Parameters:

• jobId (string): The ID of the completed job

Configuration

The MCP server needs to be configured with your RunHuman API credentials.

1. Get an API Key

Option A: Via Dashboard

1. Start the API server: npm run dev --workspace=@runhuman/api
2. Open http://localhost:3000/app.html
3. Go to "API Keys" tab
4. Click "Create API Key"
5. Copy the key (starts with qa_live_)

Option B: Use Default Test Key

• For local development, you can use: qa_live_test_key_123
• This key exists in packages/api/data/api-keys.json

2. Configure Environment Variables

Create a .env file in the MCP server directory:

# For local development
RUNHUMAN_API_URL=http://localhost:3000
RUNHUMAN_API_KEY=qa_live_test_key_123

# For production
RUNHUMAN_API_URL=https://api.runhuman.com
RUNHUMAN_API_KEY=qa_live_xxxxxxxxxxxxxxxxxxxxx

Important: Never commit .env files to git! They're already in .gitignore.

3. Verify Configuration

Test your API key works:

curl http://localhost:3000/api/jobs \
  -H "Authorization: Bearer qa_live_test_key_123" \
  -H "Content-Type: application/json" \
  -d '{"url":"https://example.com","description":"test","outputSchema":{}}'

Should return a job ID if authentication works.

For more details, see docs/API-AUTHENTICATION.md

Testing

The MCP server includes automated tests to verify it's working correctly:

# Build first
npm run build --workspace=@runhuman/mcp-server

# Run simple automated test
npm run test --workspace=@runhuman/mcp-server

# Or use the MCP Inspector (interactive testing)
npm run test:inspector --workspace=@runhuman/mcp-server

The test script will:

1. ✅ Initialize a connection to the MCP server
2. ✅ List all available tools (create_job, get_job_status, get_job_result)
3. ✅ Test calling the create_job tool

Expected Test Output

✅ Server initialized successfully
✅ Tools listed: create_job, get_job_status, get_job_result
✅ create_job tool called successfully

Development

# Watch mode (auto-rebuild on changes)
npm run dev --workspace=@runhuman/mcp-server

# Build
npm run build --workspace=@runhuman/mcp-server

# Test after building
npm run test --workspace=@runhuman/mcp-server

Integration with Claude Desktop

To use this MCP server with Claude Desktop, add it to your configuration:

{
  "mcpServers": {
    "runhuman": {
      "command": "node",
      "args": ["/path/to/qa-experiment/packages/mcp-server/dist/index.js"]
    }
  }
}

Example Usage

Once connected to an AI agent (like Claude), the agent can use these tools naturally:

User: "Can someone test my checkout page at https://myapp.com/checkout?"

Agent uses create_job:

✅ Job created successfully!
Job ID: job_abc123
Status: pending
...

Agent polls get_job_status until complete, then calls get_job_result:

✅ Test completed!
Results Summary:
- Checkout Flow: ✅ Working
- Payment Processing: ✅ Successful
...

Developer Documentation

For developers working on this MCP server:

• docs/HOW-AGENTS-USE-MCP.md - How AI agents discover and use MCP servers
• docs/TOOL-RESPONSE-BEST-PRACTICES.md - Best practices for tool responses

Learn More

• Model Context Protocol Documentation
• RunHuman API Documentation