Rabbitize (🐰👀)

Point. Click. Rabbitize.

Turn Playwright into a live, stateful REST service - recording video, screenshots, and DOM snapshots for every step—so humans and AI agents can SEE automation, not just hope it worked.

# LFG
npm install rabbitize
sudo npx playwright install-deps # required for PW to work
npx rabbitize # start server for interactive sessions
# http://localhost:3037

Why Rabbitize?

• Visual by default – video + before/after screenshots for every command.
• Stateful sessions – keep browsers warm between API calls.
• AI-ready – deterministic file paths for screenshots & DOM dumps.
• Human-like coordinates – real mouse movement, not synthetic DOM clicks.
• Click-to-code Flow Builder – non-devs can point-and-automate.
• Declarative - just some simple JSON

The Problem

1. Traditional browser automation is blind. You write scripts, run them, and hope they work. When they fail, you're left guessing why.
2. These tools are often geared toward developers and have a high bar for entry for people just looking to locally automate some workflow.

The Solution

Rabbitize changes browser automation from a black box into a visual feedback loop:

# Send a command
curl -X POST localhost:3037/execute -d '{"command": [":move-mouse", ":to", 240, 1230]}'

# Get instant visual feedback
ls rabbitize-runs/session/screenshots/
# before-click.jpg
# after-click.jpg
# zoomed-click.jpg

Every action generates screenshots, videos, a markdown'd DOM, and performance data. Sessions stay alive between commands, enabling true interactive automation.

Two Ways to Use It

For Everyone: Flow Builder & Browser

Point, click, and create automations without writing code:

1. Open http://localhost:3037/flow-builder
2. Click around any website
3. Watch your automation build itself
4. End the session (saves)
5. Export as cURL, CLI, schedule with cron, or go to the Dashboard and re-run it with a single click
6. Browse historical data collected via the UI or your harddrive (no weird formats)

For Developers: REST API

Perfect for AI agents and complex integrations:

# Start session
response = requests.post("http://localhost:3000/start", json={"url": "https://example.com"})

# AI analyzes screenshot, decides next action
response = requests.post("http://localhost:3000/execute", json={"command": [":click", ":at", 400, 300]})

# Session maintains state, ready for next command when you are
# or script CURL commands!

# or run the whole thing in one shot (once your commands are nailed down, just run it - process will end on completion)
rabbitize \
  --stability-detection false \
  --exit-on-end true \
  --process-video true \
  --client-id "test" \
  --port 3000 \
  --test-id "batchtest" \
  --batch-url "https://rvbbit.com" \
  --batch-commands='[
    [":move-mouse", ":to", 1600, 75],
    [":move-mouse", ":to", 1600, 575],
    [":scroll-wheel-down", 3],
    [":wait", 5],
    [":scroll-wheel-up", 3],
    [":move-mouse", ":to", 1600, 75]
  ]'

What Makes It Different

• Mouse actually moves - Watch the cursor travel across the screen
• Real coordinates - Click at (x,y) just like a human would
• Visual feedback - See every action happen in real(ish)-time
• No hidden shortcuts - If a human can't do it, neither should your tests

Real-World Use Cases

• QA Engineers: Record bugs with visual proof, no coding required
• AI Developers: Build web agents that can see and react
• Business Users: Automate daily tasks by showing, not coding
• DevOps: Monitor sites with video evidence of failures

Example: AI Web Agent

// 1. Start browser
await fetch('/start', {
  method: 'POST',
  body: JSON.stringify({ url: 'https://news.ycombinator.com' })
});

// 2. AI reads screenshot from deterministic filepath: rabbitize-runs/.../screenshots/latest.jpg
// 3. AI decides: "Click on the top story"

await fetch('/execute', {
  method: 'POST',
  body: JSON.stringify({ command: [':click', ':at', 150, 200] })
});

// 4. New screenshot appears, AI continues...

// OR scripted as a "tool" for an LLM to use

Boatloads of data at every step

Every session creates:

rabbitize-runs/
└── session-id/
    ├── video.webm          # Full session recording
    ├── screenshots/        # Before/after each action
    ├── commands.json       # Detailed audit trail
    ├── metrics.json        # Performance data
    ├── dom_snapshots/      # Page content as markdown text
    └── dom_coords/         # All useful DOM info in simple JSON

Complete Command Reference

Navigation

[":url", "https://example.com"]         // Navigate to URL
[":back"]                                // Browser back
[":forward"]                             // Browser forward

Mouse Actions

[":click", ":at", 100, 200]             // Click at coordinates
[":move-mouse", ":to", 100, 200]        // Move to position
[":right-click", ":at", 100, 200]       // Right click
[":middle-click", ":at", 100, 200]      // Middle click
[":drag", ":from", 100, 200, ":to", 300, 400]  // Drag from A to B
[":start-drag", ":from", 100, 200]      // Start drag
[":end-drag", ":to", 300, 400]          // End drag
[":click-hold", ":at", 100, 200]        // Mouse down
[":click-release", ":at", 100, 200]     // Mouse up

Scrolling

[":scroll-wheel-up", 3]                  // Scroll up 3 clicks
[":scroll-wheel-down", 5]                // Scroll down 5 clicks

Keyboard Input

[":keypress", "Enter"]                   // Single key
[":keypress", "Control+a"]               // Key combo
[":type", "Hello World"]                 // Type text

File Handling

[":set-upload-file", "/path/to/file.pdf"]        // Prepare file for upload dialogs
[":set-upload-file", "file1.pdf", "file2.jpg"]   // Multiple files
[":set-download-path", "./downloads"]            // Set download directory (default: session dir)

Page Extraction

[":extract", 100, 200, 500, 600]        // Extract text from rectangle
[":extract-page"]                        // Extract all page content

Utility

[":wait", 2]                             // Wait 2 seconds
[":width", 1920]                         // Set viewport width
[":height", 1080]                        // Set viewport height
[":print-pdf"]                           // Save page as PDF

Real coordinates. Real mouse movement. Real results. No javascript required.

Dashboard

Watch automations run live at http://localhost:3037:

License

MIT

Stop writing blind browser automation. Start seeing what actually happens.

Ryan Robitaille 2025