skill-cli

CLI for managing and testing Anthropic Agent Skills (e.g. anthropics/skills).

Install (npm)

# Global
npm install -g ancoder-skill-cli

# Or run without installing
npx ancoder-skill-cli --help

The npm package is self-contained and includes prebuilt binaries for:

• macOS arm64
• macOS x64
• Linux arm64
• Linux x64
• Windows x64

After install, the wrapper selects the correct bundled binary for the current platform automatically.

One-line install (Gitee or intranet)

For users who cannot access GitHub, publish the release-only bundle to Gitee or copy these files to an intranet static file server:

• scripts/install.ps1
• scripts/install.sh
• bin/targets/skill-cli-win32-x64.exe
• bin/targets/skill-cli-linux-x64
• bin/targets/skill-cli-linux-arm64
• bin/targets/skill-cli-darwin-arm64
• bin/targets/skill-cli-darwin-x64

Windows users can install the latest Gitee release with PowerShell:

irm https://gitee.com/marvin-dev/skill-cli-release/raw/main/scripts/install.ps1 | iex

macOS/Linux users can install the latest Gitee release with curl:

curl -fsSL https://gitee.com/marvin-dev/skill-cli-release/raw/main/scripts/install.sh | bash

The installer downloads the matching prebuilt binary, installs it as skill-cli, runs skill-cli install, then runs a lite doctor check. No Go toolchain is required.

The raw/main URL is the moving latest channel. Each GitHub release updates Gitee main with the newest install bundle. If you need a reproducible install, pin a specific release tag instead:

irm https://gitee.com/marvin-dev/skill-cli-release/raw/v0.13.33/scripts/install.ps1 | iex

curl -fsSL https://gitee.com/marvin-dev/skill-cli-release/raw/v0.13.33/scripts/install.sh | bash

Before sharing the Gitee command externally, make sure both the Gitee main branch and the release tag have been updated. The installer in main is generated from the latest release bundle.

Gitee release checklist:

1. Run go test ./..., bash scripts/build-all.sh, node scripts/check-bin.js, and npm pack --dry-run.

2. Commit bin/targets/, scripts/install.ps1, scripts/install.sh, scripts/make-install-bundle.js, scripts/check-install-host.js, scripts/check-bin.js, scripts/build-all.sh, package.json, package-lock.json, .gitattributes, and README.md.

3. Push the release tag to GitHub. The GitHub Actions release workflow builds the binaries and syncs a dist-only bundle to git@gitee.com:marvin-dev/skill-cli-release.git.

4. Verify these URLs return 200 before sending the install command:

   • https://gitee.com/marvin-dev/skill-cli-release/raw/main/scripts/install.ps1
   • https://gitee.com/marvin-dev/skill-cli-release/raw/main/scripts/install.sh
   • https://gitee.com/marvin-dev/skill-cli-release/raw/main/bin/targets/skill-cli-win32-x64.exe
   • https://gitee.com/marvin-dev/skill-cli-release/raw/v0.13.33/scripts/install.ps1
   • https://gitee.com/marvin-dev/skill-cli-release/raw/v0.13.33/bin/targets/skill-cli-win32-x64.exe Or run:

   node scripts/check-install-host.js \
     --script-root https://gitee.com/marvin-dev/skill-cli-release/raw/main \
     --base-url https://gitee.com/marvin-dev/skill-cli-release/raw/main/bin/targets

5. Share the raw/main install command only after the check above passes.

GitHub Actions Gitee sync setup:

1. Create a dedicated SSH key for the Gitee release-only repo:

   ssh-keygen -t ed25519 -C "github-actions skill-cli-release" -f ~/.ssh/skill-cli-release-gitee

2. Add the public key ~/.ssh/skill-cli-release-gitee.pub to Gitee with write access to git@gitee.com:marvin-dev/skill-cli-release.git.

3. Add the private key content from ~/.ssh/skill-cli-release-gitee to the GitHub repository secret GITEE_SSH_PRIVATE_KEY.

The workflow only pushes the generated dist/install/ bundle to Gitee, so the source repository remains GitHub-only.

For private intranet hosting, generate a static install bundle and upload the generated dist/install/ directory as-is:

npm run install-bundle -- --root https://intranet.example.com/skill-cli

That directory contains:

• scripts/install.ps1
• scripts/install.sh
• bin/targets/skill-cli-*
• manifest.json
• README.txt

After uploading dist/install/ to https://intranet.example.com/skill-cli/, users can run:

node scripts/check-install-host.js --root https://intranet.example.com/skill-cli --all-binaries

irm https://intranet.example.com/skill-cli/scripts/install.ps1 | iex

curl -fsSL https://intranet.example.com/skill-cli/scripts/install.sh | bash

If binaries are hosted somewhere else, point the installer at the directory containing the skill-cli-* files:

$env:SKILL_CLI_BASE_URL = "https://intranet.example.com/skill-cli/bin/targets"
irm https://intranet.example.com/skill-cli/scripts/install.ps1 | iex

curl -fsSL https://intranet.example.com/skill-cli/scripts/install.sh | \
  bash -s -- --base-url https://intranet.example.com/skill-cli/bin/targets

Useful options:

# Install the binary only, without writing ~/.claude components
$env:SKILL_CLI_NO_INSTALL = "1"
irm https://gitee.com/marvin-dev/skill-cli-release/raw/main/scripts/install.ps1 | iex

# For a clean smoke test against a temporary Claude directory
$env:SKILL_CLI_NO_INSTALL = $null
$env:SKILL_CLI_CLAUDE_DIR = "$PWD\.skill-cli-test-claude"
irm https://gitee.com/marvin-dev/skill-cli-release/raw/main/scripts/install.ps1 | iex

# Install the binary only, without writing ~/.claude components
curl -fsSL https://gitee.com/marvin-dev/skill-cli-release/raw/main/scripts/install.sh | bash -s -- --no-install

# For a clean smoke test against a temporary Claude directory
curl -fsSL https://gitee.com/marvin-dev/skill-cli-release/raw/main/scripts/install.sh | \
  bash -s -- --claude-dir "$PWD/.skill-cli-test-claude"

Build from source (Go)

cd skill-cli
go build -o bin/skill-cli .
# Then run: ./bin/skill-cli --help
# Or via npm: node bin/skill-cli.js --help

Commands

Real-Task-First Generation

skill-cli generate is optimized for completing the user's current concrete medium-complexity task in a real workspace, not for producing a broadly reusable skill for an entire task class. The default OMC pipeline uses deterministic gates as quick feedback, then runs an adaptive real-task execute/audit/repair loop. Simple tasks still exit after their first passing batch; incomplete medium-complexity tasks can persist progress across several batches and receive bounded repair rounds automatically.

Default task-first generation settings:

• --profile smoke
• --max-iterations 12
• --final-gate-repair-cycles 2
• --real-task-rounds 3 for generic tasks (1 for automatically detected heavy agentic/AUTOSAR smoke tasks)
• --real-task-batches 4 for generic tasks (heavy tasks use checkpointed micro-batches)
• --real-task-batch-timeout 300 seconds for generic tasks
• --real-task-timeout 12
• --real-task-pass-score 0.90

Profiles:

When resuming an existing skill with --real-task-only, the CLI reruns validate/test/verify first so an invalid skill cannot be reported as a successful continuation.

The executor maintains an acceptance-checklist.json for the concrete request. The auditor independently decomposes the request into evidence-backed criteria, recomputes their weighted score, requires every critical criterion to pass, and refuses to accept an unsupported scalar score. The run must reach the configured pass score (default 0.90). If the adaptive budget stops below the threshold, the CLI still delivers the best round's artifacts plus a gap report (exit code 2) with failed acceptance criteria so the same workspace can be resumed. Use skill-cli eval init and skill-cli eval run to tighten deterministic evals, --real-task-workdir when auto-detection is ambiguous, and --profile full when the default bounded budget is insufficient.

Context Handoff

Generic real-task batches use a bounded, host-written output/handoff.json by default. Empty mode, auto, and summary-only all consume this structured handoff instead of prior execution transcripts. The handoff captures the request, acceptance criteria, completed evidence, existing artifacts, the last batch outcome, and exactly one next action. It is regenerated from disk before every batch, so resumes and retries do not depend on an opaque Claude session. Use --real-task-context-mode legacy only when deliberately testing the broader legacy prompt.

skill-cli generate report-audit --desc "..."

# Explicit compatibility/benchmark comparison only
skill-cli generate report-audit --desc "..." \
  --real-task-context-mode legacy

Every round writes execution-context-metrics.jsonl and round-metrics.json, recording prompt bytes, execution duration, Read calls, repeated identical read requests, and the independent audit score. The same values are persisted into rounds.json for benchmark analysis.

--real-task-read-only-subagents is an experimental companion flag. It exposes a bounded explorer subagent with only Read, Glob, Grep, and LS; the main executor remains the sole writer. Use it only with independently investigable slices, and compare it against a baseline corpus before making it part of a workflow.

skill-cli generate source-audit --desc "..." \
  --real-task-read-only-subagents

skill-cli real-task-benchmark <workspace-root> --compare-experiments

Use the saved-run benchmark to measure the product claim over a task corpus instead of relying on individual high scores:

skill-cli real-task-benchmark ./task-corpus
skill-cli real-task-benchmark ./task-corpus --json
skill-cli real-task-benchmark ./task-corpus --latest 20
skill-cli real-task-benchmark ./task-corpus --since 2026-07-01
skill-cli real-task-benchmark ./task-corpus --compare-experiments
skill-cli real-task-benchmark ./task-corpus --require-target \
  --target-score 0.90 --target-pass-rate 0.90

The benchmark only counts a case as passing when execution passed, the score reached the per-case target, and the result has verifiable acceptance-criterion evidence paths or strong deterministic/golden coverage. Unsupported historical scalar scores are reported separately.

--compare-experiments groups saved runs by context mode and the read-only-subagent toggle. It reports pass rate, best/audit score, prompt bytes, execution time, and repeated-read count per cohort; historical runs without metrics remain visible but are excluded from cost averages.

The repository also includes a reproducible medium-complexity corpus covering release-readiness synthesis, incident correlation, pricing reconciliation, Python code implementation/testing, and API contract auditing:

go build -o bin/skill-cli .
python3 scripts/run-real-task-corpus.py --all \
  --output-root /tmp/skill-cli-real-task-corpus \
  --require-target

Corpus inputs and task-specific skills live under testdata/real-task-corpus/. Runs are isolated under /tmp by default, completed rounds are resumable, and interrupted current rounds resume their existing acceptance checklist and deliverables instead of restarting.

Skill Self-Iteration Loops

Generated and improved skills now use a loop-engineering operating layer by default. Each skill can carry:

• LOOP.md for cadence, gates, and escalation rules
• STATE.md for durable memory across runs
• loop-budget.md for token budget and kill switch
• loop-run-log.md for append-only cycle history

Use skill-cli loop skill audit <skill> to inspect readiness, skill-cli loop skill init <skill> to add the files to an older skill, and skill-cli loop skill run <skill> --logs ./logs to run evidence-backed self-improvement cycles.

Machine-Readable Skill Layout

Task-oriented skills can now include a deterministic verification harness:

my-skill/
├── SKILL.md
├── skill.contract.yaml
├── evals/
│   └── smoke.yaml
├── fixtures/
└── scripts/

• skill.contract.yaml defines the executable contract: entrypoint, inputs, outputs, invariants, and datasets.
• evals/*.yaml defines runnable verification suites with deterministic checks like file existence, required content, and JSON assertions.
• skill-cli verify materializes fixture data into a temp workspace, runs the skill entrypoint, and enforces the declared checks.

skill-cli verify executes local code declared by the skill contract, so only run it against trusted skills and repositories.

Generation failure logs

Every skill-cli generate run writes a local JSONL session log under:

<skills-dir>/.skill-cli/logs/generate/<skill-name>-<timestamp>.jsonl

Every generation run prints the exact log path. The log records major stages such as Claude binary resolution, scaffold creation, brainstorm probe, loop start/ticks, deterministic gates, repair attempts, and real-task execution/audit/gap-report artifacts, including captured command output and error tails for troubleshooting.

Publish to npm

Releases are published by GitHub Actions, not by running npm publish from a developer machine. The workflow in .github/workflows/release.yml runs when a v* tag is pushed; it builds the release binaries, creates the GitHub Release, syncs a release-only install bundle to Gitee, prepares bin/targets/ for the npm package, and publishes with the repository NPM_TOKEN secret.

1. Bump package.json and package-lock.json to the intended version.

2. Run local verification:

   go test ./...
   bash scripts/build-all.sh
   node scripts/check-bin.js
   npm pack --dry-run

3. Commit the source changes, version files, and regenerated binaries under bin/targets/.

4. Create and push the release tag:

   git tag v0.13.10
   git push origin v0.13.10

The CI release publishes binaries named:

• skill-cli-darwin-arm64, skill-cli-darwin-x64
• skill-cli-linux-x64, skill-cli-linux-arm64
• skill-cli-win32-x64.exe

Users who npm install -g ancoder-skill-cli get a fully bundled package. No extra binary download is required during install.

Optional Task-Specific Test Skill Mode

skill-cli generate --test-skill remains available as an experimental feedback mode. It creates a task-specific test skill that compares expected and actual outputs for the current task, then uses the structured diff feedback to improve the main skill. This mode is a fast feedback helper; it is not the default product bet and should not be used to chase a broadly reusable evaluator for a whole task class.

The default real-task execution/audit/repair loop still runs after deterministic gates and remains the final value gate. A run only passes when the concrete real task reaches the configured auditor pass score, default 0.90.

Structured Diff Feedback

Task-specific test skills output structured diff reports instead of simple pass/fail:

diffs:
  - location: "page 3, paragraph 2"
    type: "content_loss"
    severity: "critical"
    expected: "table with 3 columns and 5 rows"
    actual: "table missing entirely"
  - location: "page 5, heading"
    type: "format_drift"
    severity: "warning"
    expected: "## Second-level heading"
    actual: "### Third-level heading"

This structured feedback is injected back into the main skill's improvement loop, enabling targeted fixes rather than blind retries.

QA Cycling with Early Exit

• Test skill finds issues -> structured diagnosis -> main skill fixes -> retest -> loop
• Same error appearing 3 times triggers early exit (avoids infinite compute burn)
• Maximum 5 QA cycles per iteration

Task-Specific Acceptance Criteria

Test skills should define concrete, current-task acceptance criteria (not vague "implementation is complete"):

Bad:  "PDF conversion works correctly"
Good: "All tables with merged cells are preserved as HTML <table> blocks
       with correct colspan/rowspan attributes"

skill_eval Check Type

The verify system supports a skill_eval check type that invokes a test skill as a verification oracle:

checks:
  - id: quality-check
    type: skill_eval
    skill: pdf-to-md-test
    config:
      threshold: 0.95
      output_format: structured_diff

Verify Phase: Independent Executor

During the loop's verify phase, a separate Claude executor is spawned to run the test skill. This executor:

• Has no shared context with the main skill's executor
• Produces an objective evaluation report
• Returns structured diff feedback that feeds into the next iteration

This mirrors OMC's principle: "Keep authoring and review as separate passes."

oh-my-claudecode (OMC) Integration

skill-cli embeds the full oh-my-claudecode multi-agent orchestration bundle (synced from GitHub release v4.13.6) and installs it into ~/.claude/omc/ by default. This gives any skill-cli user a single-command path to OMC's agents, skills, hooks, and runtime scripts without needing to clone the OMC repo or configure the plugin marketplace separately.

What gets installed

When you run skill-cli install, OMC is installed alongside ECC components:

Flags

# Default — installs ECC + OMC
skill-cli install

# Skip OMC entirely (opt-out)
skill-cli install --no-omc

# Install only the OMC bundle
skill-cli install --component omc

# Preview without writing files
skill-cli install --dry-run

Browse embedded OMC content

skill-cli list --type omc       # list embedded OMC agents and skills
skill-cli info autopilot        # show the autopilot skill content
skill-cli doctor                # verify OMC install health and version

Why the hook rewrite matters

OMC hooks are authored for the Claude Code plugin system and reference scripts via $CLAUDE_PLUGIN_ROOT/scripts/.... Because skill-cli installs OMC as a plain directory (not as a marketplace plugin), the installer rewrites $CLAUDE_PLUGIN_ROOT → ${claudeDir}/omc at merge time so hooks resolve correctly without the plugin loader.

If you already have OMC installed via the Claude Code plugin marketplace, the skill-cli install places a separate self-contained copy under ~/.claude/omc/ and will not touch the marketplace install. The two copies can coexist; hooks from both sources will simply fire in sequence.

Upgrading OMC

The embedded OMC version is pinned to the release tagged in embedded/omc/VERSION. To bump it, re-run the sync workflow that downloads a fresh GitHub release tarball into embedded/omc/ and rebuild.

Meta-Harness (experimental)

meta-harness/ is a Python sub-project that implements the outer-loop harness optimizer from arXiv:2603.28052 (Stanford, 2026).

Architecture

meta-harness search   ←  outer loop (Python, Claude Code proposer)
      │
      └─ skill-cli eval validate / run / ls / diff   ←  evaluator backend (Go)
                │
                └─ harness.py (user-supplied Python)  ←  inner execution layer

Two independent binaries — intentionally decoupled:

• skill-cli knows nothing about meta-harness; it only runs harness candidates and emits scores/traces.
• meta-harness knows nothing about OMC internals; it calls skill-cli via CLI contract only.

Quick start

# Build skill-cli
go build -o bin/skill-cli .

# Install meta-harness
cd meta-harness
python3 -m venv .venv && source .venv/bin/activate
pip install -e ".[dev]"

# Run smoke test (no API key needed)
cd ..
bash scripts/meta-harness-smoke.sh

# Real search (requires ANTHROPIC_API_KEY + claude CLI)
meta-harness search \
  --suite meta-harness/domains/text_classification/suite.yaml \
  --out search-runs/run-01 \
  --max-iter 5 \
  --k 2 \
  --seed meta-harness/domains/text_classification/seeds/zero_shot.py \
  --seed meta-harness/domains/text_classification/seeds/few_shot.py \
  --skill-cli bin/skill-cli \
  --samples 20

CLI contract (skill-cli eval)

Tuning

The meta-harness/src/meta_harness/skill.md file is the most important lever on search quality. Per Appendix D of the paper: run 3–5 short iterations (--max-iter 3) specifically to debug and refine it before committing to a full run.

License

MIT