Package Exports

@runhuman/mcp-server
@runhuman/mcp-server/factory
@runhuman/mcp-server/types

Readme

@runhuman/mcp-server

A Model Context Protocol (MCP) server that enables AI agents to orchestrate on-demand human QA testing through the Runhuman service.

Note: This is the MCP server package. For the main CLI tool, see runhuman (coming soon).

What is this?

This MCP server allows AI agents (like Claude, GPT, or any MCP-compatible AI) to request human testing of web applications. The AI defines what to test and what data format it needs back, and real humans perform the testing with structured results extracted automatically.

Perfect for:

AI coding agents that need human verification of their work
Automated workflows requiring human-in-the-loop testing
CI/CD pipelines with human QA gates
Visual/UX testing that requires real human judgment

Quick Start

Claude Desktop

Get your API key at runhuman.com/dashboard
Add to your Claude Desktop config:

macOS: ~/Library/Application Support/Claude/claude_desktop_config.json Windows: %APPDATA%\Claude\claude_desktop_config.json

{
  "mcpServers": {
    "runhuman": {
      "command": "npx",
      "args": ["-y", "@runhuman/mcp-server", "--api-key=rh_xxxxxxxxxxxxx"]
    }
  }
}

Restart Claude Desktop

That's it! Claude can now request human testing:

"Can someone test my checkout page at https://myapp.com/checkout and verify the payment flow works?"

Other MCP Clients

For other MCP clients (VS Code extensions, custom agents, etc.):

# Install globally
npm install -g @runhuman/mcp-server

# Run with your API key
runhuman-mcp --api-key=rh_xxxxxxxxxxxxx

Or use npx without installation:

npx @runhuman/mcp-server --api-key=rh_xxxxxxxxxxxxx

Features

Available Tools

The MCP server exposes a comprehensive set of tools for orchestrating human QA testing. The five most commonly used tools are documented in detail below; the rest are summarized in the Additional Tools section. The full reference (with input schemas and examples) lives at runhuman.com/docs/mcp.

`create_job`

Create a new human QA testing job with custom instructions.

Parameters:

url (string, optional): The URL to test
description (string, required if no template): Natural language instructions for the human tester
template (string, optional): Template name to use as base configuration
outputSchema (object, optional): JSON schema defining the structured output format
targetDurationMinutes (number, optional): Time limit for tester (default: 30, range: 1-60)
Plus additional options (screen size, GitHub repo, etc.)

Example:

{
  url: "https://myapp.com/checkout",
  description: "Test the checkout flow: add item to cart, proceed to checkout, and verify payment options are displayed correctly.",
  outputSchema: {
    type: "object",
    properties: {
      checkout_loaded: { type: "boolean" },
      payment_options_visible: { type: "boolean" },
      any_errors: { type: "string" }
    }
  },
  targetDurationMinutes: 30
}

Returns: jobId for tracking the job

`run_template`

⭐ Templates are important! Create a job from a pre-configured template.

Templates let you reuse test configurations without writing full descriptions every time. Perfect for common test scenarios.

Parameters:

template (string, required): Template name (get from list_templates)
Any parameter from create_job to override template defaults

Example:

{
  template: "checkout-flow",
  url: "https://staging.myapp.com"  // Override template's default URL
}

Returns: jobId for the created job

`wait`

Idiomatic polling - Wait for a job to complete and return results automatically.

Polls job status every 10 seconds until completion, timeout, or failure. Blocks until terminal state is reached (just like CLI wait command).

Parameters:

jobId (string, required): Job ID from create_job or run_template
timeoutSeconds (number, optional): Maximum wait time (default: 600, max: 3600)

Returns when complete:

result: Structured test results matching your schema
status: "completed"
costUsd: Exact cost in USD
testDurationSeconds: Time spent by tester
testerResponse: Raw natural language feedback
testerName/testerAlias: Tester identification
Full job details

Example workflow:

// Create job
const job = await create_job({
  url: "https://myapp.com",
  description: "Test the login page"
});

// Wait automatically polls until completion
const result = await wait({ jobId: job.jobId });
// Returns immediately when done - no manual polling needed!

`get_job`

Get current job status without waiting (quick status check).

Use this for:

Checking if a job is still running
Getting quick updates
Manual polling with your own timing
Checking multiple jobs in parallel

Parameters:

jobId (string, required): Job ID to check

Returns: Current job status and details (if completed, includes results)

`list_templates`

List available templates for your project.

Parameters:

limit (number, optional): Max templates to return (default: 50)

Returns: Array of templates with names, URLs, and descriptions

Additional Tools

These cover job lifecycle, discovery, member management, artifact retrieval, tester knowledge-base notes, enhanced-video rendering, and the full template / schedule CRUD surface. See runhuman.com/docs/mcp for input schemas, example payloads, and role/permission requirements.

Job lifecycle (beyond the five above):

rerun_job — Re-dispatch an existing job (#5933), reusing all its settings; override individual fields (url, commitSha, description, targetDurationMinutes, deviceClass, outputSchema). Returns a new jobId — follow up with wait. The canonical use is re-pointing the same test at a fresh build via url.

Discovery:

list_organizations — List orgs the authenticated user belongs to (no params).
list_projects — List accessible projects, optionally filtered by organizationId.

Organization member management:

list_organization_members — List members of an org with role / status / owner flag / join date. Params: organizationId.
invite_organization_member — Invite by email. Roles: admin / contributor / viewer (default contributor). Requires admin permission.

Artifact retrieval (paid-tier gated, except recording):

get_test_artifacts — Fetch rich session data for a completed job. Artifact types: structured-output, transcript, console-logs, network-requests, conversation, events, key-moments, recording. Use after wait / get_job to investigate why a test passed or failed. wait / get_job now report a Recording: readiness line so you know when the recording artifact is fetchable.

Template CRUD (beyond list_templates):

create_template — Create a reusable template (params: projectId, name, optional testDescription, url, outputSchema, targetDurationMinutes, deviceClass, GitHub repos, autoCreateGithubIssues).
update_template — Update any field on an existing template.
delete_template — Permanently delete a template (existing schedules using it will fail on next execution).

Schedule management:

list_schedules — List recurring schedules for a project.
create_schedule — Create a recurring schedule. Required: projectId, name, templateId, scheduledHour, scheduledMinute, timezone. Frequencies: daily / weekly / biweekly / monthly / once.
update_schedule — Pause/resume (status), retime, swap templates, or change frequency.
delete_schedule — Permanently delete (use update_schedule with status: "paused" to temporarily stop without deleting).

Tester knowledge-base notes:

list_tester_notes — Search/list tester-facing notes (credentials, setup, known issues, …). Filter by organizationId, projectId, search, tags.
create_tester_note — Add a note (params: title, body, organizationId, optional projectId, tags).
update_tester_note — Edit an existing note.
delete_tester_note — Archive (soft-delete) a note.

Enhanced video:

render_baked_demo_video — Trigger the enhanced (baked) demo-video render for a completed job and, by default, wait for it to finish. Returns the video URL and a public share path.

Configuration

API URL (Optional)

By default, the server connects to the production API at https://runhuman.com. For local development or testing:

{
  "mcpServers": {
    "runhuman": {
      "command": "npx",
      "args": [
        "-y",
        "@runhuman/mcp-server",
        "--api-key=rh_xxxxx",
        "--api-url=http://localhost:3000"
      ]
    }
  }
}

Environment Variables

Alternatively, use environment variables:

export RUNHUMAN_API_KEY=rh_xxxxxxxxxxxxx
export RUNHUMAN_API_URL=https://runhuman.com  # optional

Pricing

Runhuman uses pay-per-second pricing:

$0.0085 per second of tester time
Duration is rounded up to the nearest second
Example: 220 seconds of testing = $1.87

Costs are calculated exactly and returned in the API response. No monthly fees, no minimums.

How It Works

AI creates job - Your AI agent calls create_job or run_template with test instructions
Job posted to testers - Request goes to the human tester pool via Slack
Human performs test - A real person tests the application (video/screenshots recorded)
Human reports findings - Tester describes what they observed in natural language
AI extracts data - GPT-4o processes feedback into structured JSON matching your schema
AI gets results - wait automatically polls and returns clean, typed data ready for automation

Example Use Cases

AI Coding Agents

User: "Can you update my checkout page and verify it works?"
Agent: [Updates code, deploys, then uses run_template to verify with checkout template]
Agent: "✅ Updated and verified by human tester. Payment flow works correctly."

CI/CD Pipelines

- name: Human QA Gate
  run: |
    # Deploy preview
    # Call Runhuman API via MCP
    # Fail build if test fails

Visual Testing

Agent: "Please verify the new homepage design looks good on mobile"
[Human tester provides UX feedback]
Agent: "Human feedback: Design looks great, minor spacing issue noted..."

Development

Local Development

# Clone the repo
git clone https://github.com/volter-ai/runhuman.git
cd runhuman

# Install dependencies
npm install

# Build the MCP server
npm run build --workspace=@runhuman/mcp-server

# Run with local API
cd packages/mcp-server
node dist/index.js --api-key=rh_test_key_123 --api-url=http://localhost:3000

Testing

# Run tests
npm run test --workspace=@runhuman/mcp-server

# Use MCP Inspector (interactive testing)
npm run test:inspector --workspace=@runhuman/mcp-server

Project Structure

packages/mcp-server/
├── src/
│   ├── index.ts                 # CLI entry point (stdio)
│   ├── mcp-server-factory.ts    # Core server factory
│   ├── lib.ts                   # Library exports
│   ├── types.ts                 # TypeScript types
│   └── tools/                   # Tool implementations (one *.tool.ts per tool)
│       ├── create-job.tool.ts
│       ├── create-job.schema.ts # create_job schema + contract-drift guard
│       ├── rerun-job.tool.ts
│       ├── wait.tool.ts
│       ├── get-job.tool.ts
│       └── …                     # discovery, templates, schedules, notes, artifacts, video
├── dist/                        # Compiled output
├── tests/                       # Test suite
├── docs/                        # Developer documentation
├── package.json
├── tsconfig.json
└── README.md                    # This file

Documentation

Model Context Protocol - Learn about MCP
Runhuman Website - Product information
API Documentation - REST API reference
GitHub Repository - Full source code

Developer Docs

How Agents Use MCP - Understanding MCP tool discovery
Tool Response Best Practices - Writing effective tool responses
API Authentication - Authentication details

Integration Examples

Claude Desktop Usage

Once installed, Claude can naturally use the tools:

User: "Test my app at https://staging.myapp.com and check if the login works"

Claude:

I'll create a QA job to test your login page.
[Calls create_job tool]
Job created! Waiting for a human tester...
[Calls wait tool - automatically polls until complete]
✅ Test complete! The login page works correctly. The tester was able to log in successfully with test credentials.

Programmatic Usage (Custom MCP Client)

import { createMcpServer } from '@runhuman/mcp-server/factory';

const server = createMcpServer({
  mode: 'direct',
  apiUrl: 'https://runhuman.com',
  apiKey: 'rh_xxxxx'
});

// Use with your MCP client
await server.connect(transport);

Troubleshooting

"Error: API key is required"

Make sure you've:

Created an account at runhuman.com
Generated an API key in the dashboard
Added the key to your config with the correct format (--api-key=rh_...)

"Invalid API key format"

API keys must start with:

rh_ for production
rh_ for testing

Get a valid key at runhuman.com/dashboard

Connection Issues

If you're having trouble connecting:

Check your API URL is correct (default: https://runhuman.com)
Verify your network connection
Test the API directly:

curl https://runhuman.com/api/jobs \
  -H "Authorization: Bearer rh_xxxxx" \
  -H "Content-Type: application/json" \
  -d '{"url":"https://example.com","description":"test","outputSchema":{}}'

Claude Desktop Not Detecting Server

Verify config file location is correct for your OS
Check JSON syntax is valid (use a JSON validator)
Restart Claude Desktop completely
Check Claude's logs for errors

Contributing

This package is part of the Runhuman monorepo. Contributions welcome!

# Clone the full repo
git clone https://github.com/volter-ai/runhuman.git
cd runhuman

# Install dependencies
npm install

# Make changes to packages/mcp-server/

# Build and test
npm run build --workspace=@runhuman/mcp-server
npm run test --workspace=@runhuman/mcp-server

# Submit PR

Support

Email: hey@runhuman.com
Issues: GitHub Issues
Website: runhuman.com

License

ISC License - See LICENSE for details

About Runhuman

Runhuman is an AI-orchestrated human QA testing service. Part of the Volter AI ecosystem.

Key Features:

On-demand human testing (no hiring/managing)
AI-powered orchestration and result extraction
Pay-per-second pricing ($0.0085/sec)
Custom output schemas for structured results
Browser automation with video/screenshot recording
GitHub Actions integration
REST API and MCP server for any workflow

Built with ❤️ by the Volter AI team