Package Exports

This package does not declare an exports field, so the exports above have been automatically detected and optimized by JSPM instead. If any package subpath is missing, it is recommended to post an issue to the original package (@raxtzu/ai-test) to support the "exports" field. If that is not possible, create a JSPM override to customize the exports field for this package.

Readme

🤖 AI Visual Testing Assistant

🎯 The smartest AI-powered visual testing CLI that automatically analyzes UI failures and provides actionable fixes for web applications.

✅ Production Ready - Comprehensive framework support with intelligent automation

Transform your visual testing workflow with AI-powered analysis. Get specific, implementable solutions for UI/UX issues through intelligent detection, smart recommendations, and multi-provider AI analysis.

🚨 Current Implementation Status

✅ What Actually Works

Core Analysis Engine - Full ai-test run functionality with comprehensive orchestration
Playwright Integration - Complete test execution and screenshot capture
Gemini Provider - Production-ready with real API integration
Template System - Sophisticated prompt engineering with customization
TypeScript Architecture - Robust SOLID principles implementation
CLI Framework - Complete command structure with proper error handling

🚧 In Development (Partial/Mock)

Setup & Init Commands - Basic stubs, not fully functional
Configuration Management - No persistence, hardcoded values
Multiple AI Providers - Claude, OpenAI, Azure return mock responses only
Framework Detection - Returns hardcoded Playwright results

❌ Not Yet Implemented

Ollama (Local LLM) - Placeholder only, non-functional
Interactive Setup Wizard - Planned but not implemented
Cypress/Selenium/Puppeteer - Only Playwright works
Custom Reporter - Planned feature

Bottom Line: This is a sophisticated Playwright + Gemini visual testing tool with extensive architecture for future expansion. Core analysis functionality is production-ready.

✨ Key Features

🎯 Multi-LLM Support

Google Gemini ✅ - Fully implemented with real API integration
Claude 3.5 Sonnet 🚧 - Basic implementation (mock responses)
OpenAI GPT-4o 🚧 - Basic implementation (mock responses)
Azure OpenAI 🚧 - Basic implementation (mock responses)
Local Ollama ❌ - Placeholder only (not functional)

🔍 LLM-Powered Analysis

Visual Regression Detection - LLM analysis of layout and styling issues via screenshots
Accessibility Guidance - AI-generated WCAG 2.1 AA compliance suggestions
Responsive Design Review - LLM analysis of cross-device compatibility
UI/UX Best Practices - AI-powered design standards evaluation
Performance Indicators - LLM assessment of loading states and user experience

Note: Analysis is performed through sophisticated prompt engineering with LLMs, not computer vision algorithms.

🛠️ Framework Integration

Playwright ✅ - Fully implemented with advanced screenshot and trace support
Cypress 🚧 - Planned (not yet implemented)
Selenium 🚧 - Planned (not yet implemented)
Puppeteer 🚧 - Planned (not yet implemented)

📊 Actionable Outputs

Specific CSS Fixes - Exact selectors and property changes
JavaScript Solutions - Framework-specific code improvements
Test Enhancements - Better selectors and strategies
Implementation Guides - Step-by-step fix instructions

🚀 Quick Start

Installation

# Install globally for CLI usage
npm install -g @raxtzu/ai-test

# Or install locally in your project
npm install --save-dev @raxtzu/ai-test

Setup (Basic)

ai-test setup

⚠️ Currently a basic implementation that shows:

Manual API key configuration instructions
Basic provider information
Next steps for getting started

Note: Interactive wizard is planned but not yet implemented.

Initialize Your Project

# Basic project initialization (stub implementation)
ai-test init my-project

# Options available but not fully functional yet
ai-test init --type=vue --provider=claude

⚠️ Currently a stub implementation that shows success message only.

Note: Actual project initialization logic is planned but not yet implemented.

Run Analysis ✅

# Analyze failed test screenshots (fully implemented)
ai-test run

# Continuous improvement mode (works)
ai-test run --continuous --max-iterations=5

# Target specific test patterns (works)
ai-test run --pattern="responsive" --output-format=html

# Additional options available
ai-test run --dry-run --provider=gemini --debug

✅ This is the core implemented feature with full functionality.

📋 Real-World Use Cases

1. Professional Trading Platform Quality Assurance

// Playwright test that fails due to visual issues
test('trading dashboard professional appearance', async ({ page }) => {
  await page.goto('/dashboard');
  
  // Test will fail if layout is broken
  await expect(page.locator('.price-chart')).toBeVisible();
  await expect(page.locator('.trading-controls')).toBeVisible();
  
  // Screenshot automatically captured on failure
});

AI Analysis Output:

🔍 Visual Issues Found:
• Price display uses #666 text on #f8f9fa background (contrast ratio 2.1:1)
• Trading buttons lack visual hierarchy for quick decision-making
• Chart legend overlaps with price data on mobile viewports

💡 Specific Fixes:
.price-display {
  color: #1a1a1a !important;  /* WCAG AA compliant */
  font-weight: 600;
  font-size: 1.25rem;
}

.trading-button.buy {
  background: linear-gradient(135deg, #02c076, #059669);
  box-shadow: 0 2px 4px rgba(2, 192, 118, 0.3);
}

📱 Mobile Fix:
@media (max-width: 768px) {
  .chart-legend {
    position: absolute;
    top: 10px;
    right: 10px;
    background: rgba(255, 255, 255, 0.95);
  }
}

2. E-commerce Conversion Optimization

test('product page conversion elements', async ({ page }) => {
  await page.goto('/product/premium-plan');
  
  // Test pricing visibility and CTA prominence
  await expect(page.locator('[data-test="price"]')).toBeVisible();
  await expect(page.locator('[data-test="cta-button"]')).toBeVisible();
});

AI Analysis Output:

🎯 Conversion Issues Detected:
• CTA button "Start Free Trial" has insufficient visual weight
• Price display lacks urgency indicators
• Trust signals positioned below the fold

🚀 Conversion Optimizations:
.cta-button {
  background: #ff6b35;           /* High-contrast orange */
  font-size: 1.1rem;
  padding: 1rem 2rem;
  box-shadow: 0 4px 12px rgba(255, 107, 53, 0.4);
  animation: pulse 2s infinite;  /* Subtle attention-grabber */
}

.price-container::after {
  content: "Limited Time Offer";
  background: #ff4444;
  color: white;
  padding: 0.25rem 0.5rem;
  border-radius: 4px;
  font-size: 0.75rem;
}

🎛️ Configuration Options

Provider Selection

# Basic configuration (limited implementation)
ai-test config --list
ai-test config --get provider

# Setup command (basic implementation)
ai-test setup

Analysis Customization

{
  "analysisFeatures": [
    "visual-regression",    // Layout and styling issues
    "accessibility",        // WCAG compliance
    "responsive",          // Mobile/tablet compatibility
    "ux-practices",        // Professional design standards
    "performance",         // Loading states and UX
    "branding"            // Brand consistency
  ],
  "analysisDepth": "comprehensive",  // quick | balanced | comprehensive
  "maxIterations": 5,
  "outputFormat": "markdown"         // markdown | html | json | all
}

Quality Standards

{
  "qualityStandards": {
    "contrastRatio": 4.5,        // WCAG AA compliance
    "minTouchTarget": 44,        // Mobile accessibility
    "maxLoadTime": 2000,         // Performance expectations
    "brandColors": ["#0066cc", "#ff6b35"]  // Brand palette validation
  }
}

🏗️ Architecture & Design Principles

SOLID Principles Implementation

Single Responsibility: Each class has one clear purpose
Open/Closed: Extensible provider system without modifying core
Liskov Substitution: All providers implement consistent interfaces
Interface Segregation: Focused interfaces for specific capabilities
Dependency Inversion: High-level modules depend on abstractions

Modular Structure

src/
├── core/                    # Core interfaces and abstractions
│   ├── interfaces.ts        # Contract definitions
│   └── container.ts         # Dependency injection
├── providers/               # AI provider implementations
│   ├── base-provider.ts     # Abstract base class
│   ├── claude-provider.ts   # Anthropic Claude
│   ├── gemini-provider.ts   # Google Gemini
│   └── provider-factory.ts  # Factory pattern
├── services/                # Business logic services
│   ├── logger.ts           # Logging abstraction
│   ├── rate-limiter.ts     # API rate management
│   └── config-manager.ts   # Configuration handling
├── cli/                     # Command-line interface
│   ├── index.ts            # Main CLI application
│   └── commands/           # Command implementations
└── types/                   # TypeScript definitions
    └── index.ts            # Type definitions

🧪 Integration Examples

GitHub Actions CI/CD

name: AI Visual Quality Assurance
on: [push, pull_request]

jobs:
  visual-testing:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      
      - name: Setup Node.js
        uses: actions/setup-node@v4
        with:
          node-version: '18'
          
      - name: Install dependencies
        run: npm ci
        
      - name: Install AI Visual Testing
        run: npm install -g @raxtzu/ai-test
        
      - name: Run Playwright tests
        run: npx playwright test
        continue-on-error: true
        
      - name: AI Visual Analysis
        run: ai-test run --max-iterations=3
        env:
          ANTHROPIC_API_KEY: ${{ secrets.ANTHROPIC_API_KEY }}
          
      - name: Upload Analysis Reports
        uses: actions/upload-artifact@v4
        with:
          name: visual-analysis-reports
          path: .ai-testing/output/reports/

Package.json Scripts

{
  "scripts": {
    "test:visual": "ai-test run --pattern='visual'",
    "test:accessibility": "ai-test run --provider=gemini",
    "test:mobile": "ai-test run --pattern='responsive'",
    "setup:ai": "ai-test setup",
    "qa:full": "npm run test && ai-test run"
  }
}

📊 Cost Optimization

Provider Cost Comparison

Provider	Implementation Status	Cost (per 1M tokens)	Working Status
Gemini 2.0 Flash	✅ Fully Implemented	$1.25 / $5.00	✅ Production Ready
Claude 3.5 Sonnet	🚧 Mock Implementation	$3.00 / $15.00	❌ Returns Mock Data
GPT-4o	🚧 Mock Implementation	Variable by image	❌ Returns Mock Data
Azure OpenAI	🚧 Mock Implementation	Enterprise pricing	❌ Returns Mock Data
Ollama (Local)	❌ Placeholder Only	Infrastructure only	❌ Non-functional

Smart Cost Management

# Basic configuration commands (stub implementations)
ai-test config --set provider=gemini
ai-test config --get provider

# Core analysis command (fully implemented)
ai-test run --provider=gemini --max-iterations=3

🔧 Advanced Usage

Custom Analysis Templates

// .ai-testing/templates/trading-platform.md
You are analyzing a professional trading platform interface.

Focus on:
- Real-time data visibility and clarity
- Quick action button accessibility
- Professional color schemes (Bloomberg Terminal standards)
- Information density vs. readability balance
- Mobile trading workflow optimization

Provide fixes that meet financial industry UI standards.

Programmatic API

import { AnalysisEngine, ProviderFactory } from '@raxtzu/ai-test';

const engine = new AnalysisEngine();
const provider = ProviderFactory.create('claude', { apiKey: 'your-key' });

const result = await engine.analyze(screenshots, {
  provider,
  features: ['accessibility', 'responsive'],
  depth: 'comprehensive'
});

console.log('Issues found:', result.analysis.visualIssues.length);
console.log('CSS fixes:', result.fixes.css);

Playwright Integration

// playwright.config.js
export default defineConfig({
  use: {
    screenshot: 'only-on-failure',    // AI needs failure screenshots
    video: 'retain-on-failure',      // Additional context
    trace: 'retain-on-failure',      // Detailed execution trace
  },
  
  // Custom reporter for AI integration (not yet implemented)
  reporter: [
    ['html'],
    ['@raxtzu/ai-test/reporter']  // Planned feature
  ]
});

🐛 Troubleshooting

Common Issues

No screenshots found:

ai-test doctor  # Basic health check
ai-test status  # Check service status

API issues:

ai-test status  # Check provider status
# Note: Only Gemini provider is fully functional

Basic analysis commands:

ai-test run --debug  # Debug mode
ai-test run --dry-run  # Preview mode

Permission errors:

ai-test doctor  # Basic diagnostics

📚 Learning Resources

Example Projects

Trading Platform Demo - Professional financial interface
E-commerce Store - Conversion optimization focus
SaaS Dashboard - Complex data visualization
Design System - Component library validation

Best Practices

Screenshot Strategy: Focus on failure points, not success states
Prompt Engineering: Use specific, actionable language
Iterative Improvement: Run analysis after each fix cycle
Cost Management: Use appropriate models for analysis depth needed

🤝 Contributing

We welcome contributions! Please see our Contributing Guide for details.

Development Setup

git clone https://github.com/RaxTzu/ai-test.git
cd ai-test
npm install
npm run dev

Running Tests

npm test              # Unit tests
npm run test:integration  # Integration tests
npm run lint          # Code quality

📄 License

MIT License - see LICENSE file for details.

🙋‍♂️ Support & Community

Issues: GitHub Issues
Discussions: GitHub Discussions
Documentation: Full API Reference

Made with ❤️ by Raveen Abeywickrama

Empowering developers to build visually perfect web applications with AI assistance.