JSPM

  • ESM via JSPM
  • ES Module Entrypoint
  • Export Map
  • Keywords
  • License
  • Repository URL
  • TypeScript Types
  • README
  • Created
  • Published
  • Downloads 27
  • Score
    100M100P100Q56208F
  • License Apache-2.0

Local UI grounding for AI agents. Drop in a screenshot + text target, get back a JSON bbox. Qwen3-VL-2B LoRA. MLX-native on Apple Silicon.

Package Exports

    This package does not declare an exports field, so the exports above have been automatically detected and optimized by JSPM instead. If any package subpath is missing, it is recommended to post an issue to the original package (browserground) to support the "exports" field. If that is not possible, create a JSPM override to customize the exports field for this package.

    Readme

    browserground

    Drop-in local UI grounding for AI agents. Qwen3-VL-2B LoRA. Strict-JSON bbox output. Runs locally on Apple Silicon / CUDA / CPU.

    HF Model GitHub

    What you get

    browserground parse screenshot.png --target "Submit button"
# {"bbox_2d": [344, 612, 478, 658]}

    A single-line strict-JSON bbox of the element to click. No markdown fences. 100% format compliance on the eval set.

    Install

    npm install -g browserground
# or: bun install -g browserground

    Prerequisites

    The CLI shells out to Python for inference. Install once:

    # Option A (recommended): uv
brew install uv     # or: pipx install uv
uv pip install --system 'torch>=2.4' 'transformers>=4.55' 'peft>=0.13' huggingface_hub pillow

# Option B: pip
pip install 'torch>=2.4' 'transformers>=4.55' 'peft>=0.13' huggingface_hub pillow

    On first browserground parse, the model auto-downloads from Hugging Face to ~/.cache/huggingface/:

    • base Qwen/Qwen3-VL-2B-Instruct (~4.3 GB)
    • LoRA adapter renezander030/browserground (~67 MB)

    Usage

    # one-shot (loads model each call — slow first time)
browserground parse screenshot.png --target "Submit button"

# daemon mode (loads once, fast subsequent calls)
browserground serve &
browserground parse a.png --target "Chrome icon"
browserground parse b.png --target "the back arrow"
browserground stop

# raw model output (for debugging)
browserground parse screenshot.png --target "X" --text

# status
browserground status

    Hooks

    Claude Code

    In a repo with .claude/skills/, drop the file at plugins/claude-code/SKILL.md into .claude/skills/browserground/SKILL.md. Claude Code will route screen-grounding prompts to the CLI automatically.

    Codex CLI

    Add to your AGENTS.md per plugins/codex/AGENTS.md.

    browser-use (Python)

    import subprocess, json
def ground(screenshot_path, target):
    out = subprocess.check_output(["browserground", "parse", screenshot_path, "--target", target])
    return json.loads(out)["bbox_2d"]

    Benchmark

    ScreenSpot-v2 point-grounding accuracy (300 items, 100/split: mobile / desktop / web):

    Model Params Overall Mobile
    GPT-4o (cloud) — 18.3% —
    browserground v0.1 2 B 45.3% 64.0%
    SeeClick 9.6 B 55.1% —
    ShowUI-2B 2 B 75.5% —
    UI-TARS-2B-SFT 2 B 89.5% —

    v0.1 is a one-epoch / 12k-example fine-tune intended to validate the recipe. v0.2 (Tier 2) is in progress with target ≥ 60%.

    Limitations

    • v0.1 desktop & web accuracy lag mobile (training mix is mobile-heavy)
    • English-only training data
    • Single-target per call (batch mode in v0.2)
    • No mouse-action prediction (only location)

    License

    Apache 2.0.