playwright-testrail-sync

TypeScript integration to sync Playwright test results with TestRail, featuring automatic screenshot and ideo uploads, flexible configuration, comprehensive error handling, and clean logs.

Installation

# Using npm
npm install playwright-testrail-sync

# Using yarn
yarn add playwright-testrail-sync

Prerequisites

Before getting started, ensure you have the following:

🔑 TestRail Access

• Active TestRail account with API access
• TestRail project and suite IDs
• API key generated from TestRail settings

⚙️ Environment Setup

Configure these environment variables in your system or .env file:

💡 Advanced Configuration: For more detailed settings like logging levels, attachment settings, and validation modes, use the Programmatic Configuration in your playwright.config.ts file.

📋 Getting Your TestRail IDs

1. Project ID: Found in your TestRail URL: /index.php?/projects/overview/1 → ID is 1
2. Suite ID: Navigate to your project → Test Suites → Click on suite → URL shows the ID
3. API Key: Go to TestRail → My Settings → API Keys → Generate new key

Quick Start

1. Configure Playwright

Add the TestRail integration to your playwright.config.ts:

import { defineConfig } from '@playwright/test'

export default defineConfig({
  reporter: [
    ['html'],
    ['playwright-testrail-sync']
  ],
  // ... your other config
})

2. Add Test Case IDs

Include TestRail case IDs in your test titles using the C12345 format:

// Single case ID
test('C12345: User login with valid credentials', async ({ page }) => {
  // Your test code
})

// Multiple case IDs
test('C12345 C12346 C12347: Complex user workflow', async ({ page }) => {
  // Your test code
})

3. Run Your Tests

Execute your tests and watch the magic happen:

# Using npm
npx playwright test

# Using yarn
yarn playwright test

Features

🚀 Automatic Test Run Management

• Creates new TestRail runs automatically
• Links test results to existing runs
• Handles run naming with timestamps

📊 Smart Test Case Matching

• Extracts case IDs from test titles (C12345 format)
• Supports multiple case IDs per test
• Validates case existence before upload

📸 Rich Media Support

• Automatic screenshot capture on failures
• Video recording integration
• Log file attachments
• Custom file uploads

🔍 Advanced Logging

• Clean, structured log output
• Verbose mode for debugging
• Error tracking and reporting
• Performance metrics

⚙️ Flexible Configuration

• Environment variable configuration
• Programmatic options
• Custom run naming
• Attachment filtering

Configuration Options

Programmatic Configuration

// playwright.config.ts
export default defineConfig({
  reporter: [
    ['html'],
    ['playwright-testrail-sync', {
      // TestRail connection settings
      testRail: {
        host: 'https://your-instance.testrail.io',
        username: 'your-email@company.com',
        password: 'your-api-key',
        projectId: 1,
        suiteId: 1,
        testRunName: 'My Test Run', // Optional, otherwise uses a default name: Test Run - Date/Time stamp
        runId: 12345, // Optional: use existing run
        assignedToId: 1, // Optional: assign to user
        milestoneId: 1, // Optional: associate with milestone
        refs: 'v1.2.3', // Optional: test run references
      },
      
      // Logging configuration
      logging: {
        level: 'info', // debug, info, warn, error
        verbose: true,
      },
      
      // Attachment settings
      attachments: {
        enabled: true,
        maxSize: 10, // MB
        screenshotOnFailure: true,
        uploadVideos: true,
        allowedTypes: ['png', 'jpg', 'mp4', 'webm']
      },
      
      
      // Validation settings
      validation: {
        strictMode: true,
        validateCaseIds: true,
        validateRunId: true
      }
    }]
  ]
})

🔄 Configuration Priority

Environment variables always take precedence over configuration file settings:

# Environment variables override config file
export TESTRAIL_PASSWORD="different-password"
export TESTRAIL_VERBOSE_LOGGING="true"

This allows you to:

• Override secrets without changing code
• Use different settings per environment
• Keep sensitive data in environment variables

Test Case ID Format

The integration recognizes TestRail case IDs in test titles using the C prefix:

// ✅ Valid formats
test('C12345: Test description', () => {})
test('C12345 C12346: Multiple cases', () => {})
test('Login C12345 should work', () => {})
test('C12345-C12346-C12347: Complex test', () => {})

// ❌ Invalid formats
test('12345: Missing C prefix', () => {})
test('TC12345: Wrong prefix', () => {})

Output Examples

Successful Test Run

[Playwright-TestRail-Sync] info: Test execution started 
[Playwright-TestRail-Sync] info: Creating new TestRail run... 
[Playwright-TestRail-Sync] info: Created new TestRail run: 12345 
[Playwright-TestRail-Sync] info: Test passed: C12345 should login with valid credentials 
[Playwright-TestRail-Sync] info: Test passed: C12346 should display user dashboard
[Playwright-TestRail-Sync] info: Test passed: C12347 should logout successfully  
[Playwright-TestRail-Sync] info: TestRail sync completed 
[Playwright-TestRail-Sync] info: testRailUrl: https://your-instance.testrail.io/index.php?/runs/view/12345 

Error Handling

The integration provides comprehensive error handling:

• Invalid TestRail credentials → Clear error messages with setup instructions
• Missing test case IDs → Warnings for tests without case IDs
• Network failures → Clear error messages with actionable guidance
• Invalid case IDs → Validation before upload attempts
• Attachment failures → Graceful degradation without stopping tests

Troubleshooting

Common Issues

Issue: Configuration validation failed: TESTRAIL_HOST is required Solution: Ensure all required environment variables are set

Issue: TestRail case not found: C12345 Solution: Verify the case ID exists in your TestRail project

Issue: Failed to upload results: Invalid status Solution: Check that your TestRail instance supports the status being used

Debug Mode

Enable verbose logging to troubleshoot issues:

TESTRAIL_VERBOSE_LOGGING=true npx playwright test

License

This project is licensed under the MIT License - see the LICENSE file for details.