verify

Portfolio-wide post-deployment web QA/QC. One manifest. One report. Zero manual checks.

verify is the hard gate between deploy and done. Drop a .verify.json manifest into any project, run verify run, and get back a machine-readable report covering console errors, GA4/Ads firing, broken links, unrendered template tokens, SEO metadata, accessibility, Lighthouse budgets, visual regressions, form submissions, authenticated flows, and more. On failure, verify auto-raises a Vantage incident in the calling project with fingerprint-based deduplication. On the next PASS, it resolves itself.

5-Minute Onboarding

# In any project you want to QA:
cd ~/projects/your-project
npx @eleetai/verify init           # scaffolds .verify.json with sensible defaults
verify run --env prod              # run against production
# → writes runs/<project>/<timestamp>/report.json, exits 0 on PASS

That's it. If anything failed, the report tells you what, where, and how to fix it. A Vantage incident is opened in your project until you redeploy and rerun.

The coherence contract (exit code ↔ status)

Exit code and report.status are the same decision, reported two ways. They cannot disagree.

Consumers check exit code OR report.status, not both. They encode the same gate decision. The runner guarantees coherence: exitCode === f(status) always.

The contract is pinned at two test boundaries: (1) unit tests of deriveStatus / deriveExitCode exhaustively cover the 30-cell (tier × gate) matrix (tests/unit/exit-code-coherence.test.ts); (2) integration tests spawn the compiled CLI and assert the process exit code matches report.status for every contract row (tests/integration/cli-exit-code-coherence.test.ts). The process-boundary pin exists because consumers observe $? after the process exits, not function return values — those are different boundaries and can drift independently.

--fail-on-tier lets you tolerate noise without hiding it. Example: verify run --fail-on-tier P1 means P2/P3/P4 failures log as tolerated warnings but don't trip the gate. Surfaced in the report's summary.tolerated_warnings + summary.gate_tripped. A PASS_WITH_WARNINGS run does NOT raise a Vantage incident — the consumer explicitly opted in to tolerating those failures.

What it checks

Universal (every consumer gets these for free)

• Console errors & pageerror events — with manifest-declared suppressions
• Network 4xx/5xx on main document and same-origin
• Resource 404s — images, CSS, JS, favicons
• Unrendered template tokens — {{var}}, ${...}, literal undefined leaking into visible text
• Mixed content — http:// on https://
• SEO metadata — title, description, canonical, OG tags, JSON-LD schema
• Broken links — same-origin crawl at declared rate limit
• Canonical host — canonical URL host matches manifest allowlist
• CSP violations — surfaced from console
• robots.txt + sitemap.xml — present and well-formed
• Security headers (HSTS, X-Frame-Options, X-Content-Type-Options, Referrer-Policy, Permissions-Policy, CSP) — absence FAILs at P1; version-bearing values in Server / X-Powered-By (e.g., nginx/1.18.0, PHP/8.1.2) FAIL as CVE-mappable fingerprint leaks. Bare vendor identifiers (CloudFront, cloudflare, Kestrel, etc.) surface as warnings — aligned with Mozilla Observatory / OWASP threat models

Expectation-driven (declared per-project in .verify.json)

• GA4 firing — intercepts /collect?v=2, compares tid to manifest
• Google Ads firing — AW- tag verification
• Forbidden text — regex scan for strings that must NOT appear (old brand names, unset template tokens)
• Required text — strings that MUST appear (USPs, compliance disclaimers)
• Form submissions — fill + submit + assert post-submit state
• Authenticated flows — Cognito or form login via Vantage-vault secrets
• Lighthouse budgets — LCP, CLS, TBT, a11y, SEO, performance scores

Orchestrator-level (automatic)

• Multi-viewport — desktop (1280×720) + mobile (390×844) for every route
• Visual regression — .verify/baselines/ comparison per route × viewport, diff PNG on failure
• Regression detection — tags the run regression: true when previous PASS becomes current FAIL
• JUnit XML output — --junit report.xml for CI consumption

Invocation modes

• CLI — verify run [--env prod|staging] [--junit PATH] [--dry-run]
• MCP — drop templates/mcp/verify.mcp.json into any project's .mcp.json; Claude Code agents invoke verify_run, verify_get_latest_report, verify_list_runs, verify_update_baselines
• Scheduled — verify schedule --project NAME --cron "0 2 * * *" registers a Vantage cron-script agent for nightly runs

The contract

Both schemas are frozen at v1. Breaking changes are major-version events.

• Manifest: schemas/manifest.schema.json — the public API
• Report: schemas/report.schema.json — the wire protocol
• Examples: examples/nc.verify.json, examples/eleetai.verify.json

See docs/PLAYBOOKS.md for per-site-type cookbooks (static Next.js, Astro marketing, SPA with Cognito auth, landing pages optimized for ads).

Vantage integration

When .verify.json has vantage.enabled: true and project_id set:

No Vantage code changes required — uses shipped endpoints. See docs/SYSTEM_ARCHITECTURE.md for wire details.

Philosophy

Read ~/projects/zeos-apps/verify/VERIFY_SOUL.md for the full doctrine. In brief:

1. Schema is scripture. The manifest and report are the product.
2. Hard gate, not suggestion. No bypass flags. Suppressions are explicit, dated, auditable.
3. Checks are pure. One file, one function, one result.
4. Evidence or it didn't happen. Every check ships artifacts.
5. AI-native first. The report is designed for agents to parse and act on.
6. Platform, not product. Adding a check benefits every consumer immediately.

Status

Phase 0 — zeos scaffold complete. Schemas locking. Runner in progress.

See docs/MASTER_ROADMAP.md for current phase and open work.

verify is an eleet.ai service offering. Part of the zeos venture factory control plane.