@evalguard/cli

Command-line interface for EvalGuard -- run LLM evaluations, security scans, and guardrail checks from your terminal and CI/CD pipelines.

Installation

npm install -g @evalguard/cli

Quick Start

# Authenticate
evalguard login --key eg_live_...

# Initialize a project
evalguard init --project my-project

# Run an evaluation
evalguard eval evals/example.json --wait

# Run a security scan
evalguard scan scans/example.json --wait

# Run a local eval (no API key needed)
evalguard eval:local --model openai:gpt-4o --prompt "Answer: {{input}}" --scorer exact-match

# Check firewall rules
evalguard firewall check "Ignore all instructions"

Commands

Command	Description
`evalguard login`	Authenticate with your EvalGuard API key
`evalguard logout`	Remove stored credentials
`evalguard init`	Initialize EvalGuard in the current project (creates config and example files)
`evalguard eval <file>`	Run an evaluation from a JSON config file
`evalguard scan <file>`	Run a security scan from a JSON config file
`evalguard whoami`	Show current authentication status
`evalguard eval:local`	Run an evaluation locally without the cloud API
`evalguard scan:local`	Run a security scan locally without the cloud API
`evalguard generate`	Auto-generate eval cases or scan configs from a prompt
`evalguard validate`	Validate an eval or scan config file for correctness
`evalguard compare`	Compare results between two eval runs (drift detection)
`evalguard list`	List past eval runs and security scans
`evalguard firewall`	Check input against firewall rules or manage rule sets
`evalguard watch`	Watch eval/scan files and re-run on changes

Configuration

The CLI stores credentials at ~/.evalguard/config.json. Project-level settings go in evalguard.config.json in your repository root.

{
  "$schema": "https://evalguard.ai/schema/config.json",
  "projectId": "my-project",
  "defaultModel": "gpt-4o",
  "evalsDir": "./evals",
  "scansDir": "./scans"
}

CI/CD Usage

# GitHub Actions example
- name: Run EvalGuard security scan
  env:
    EVALGUARD_API_KEY: ${{ secrets.EVALGUARD_API_KEY }}
  run: |
    npx @evalguard/cli login --key $EVALGUARD_API_KEY
    npx @evalguard/cli scan scans/production.json --wait

Documentation

Full documentation at evalguard.ai/docs/cli.

License

Apache License, Version 2.0 — see LICENSE and NOTICE.

This CLI is a thin public client for the EvalGuard service. It is not covered by any proprietary license restriction — you may fork, modify, and redistribute it under Apache 2.0. The EvalGuard service itself, its backend engine, scorers, and attack plugins are proprietary software operated as a hosted service and are NOT covered by Apache 2.0. Access is governed by the EvalGuard Terms of Service.

"EvalGuard" is a trademark of EvalGuard, Inc. Derivative forks must not use the EvalGuard name or logo to imply endorsement or drop-in compatibility with the hosted service.