Package Exports

deflaky-cli
deflaky-cli/dist/index.js

This package does not declare an exports field, so the exports above have been automatically detected and optimized by JSPM instead. If any package subpath is missing, it is recommended to post an issue to the original package (deflaky-cli) to support the "exports" field. If that is not possible, create a JSPM override to customize the exports field for this package.

Readme

DeFlaky

Detect flaky tests by running your test suite multiple times.

DeFlaky wraps your existing test command, runs it N times, and identifies which tests are flaky, always-failing, or stable. It calculates a FlakeScore and optionally pushes results to the DeFlaky Dashboard.

Works with Playwright, Selenium, Cypress, Jest, Pytest, Mocha and any framework that outputs JUnit XML or JSON reports.

Install

npm install -g deflaky-cli

Or use without installing:

npx deflaky-cli --help

Quick Start

# Run your test suite 5 times and detect flaky tests
deflaky run -c "npx playwright test" -r 5

# Run 10 times with verbose output
deflaky run -c "npx jest --ci" -r 10 --verbose

# Push results to DeFlaky dashboard
deflaky run -c "npx playwright test" -r 5 --push --token YOUR_TOKEN

# Fail CI if FlakeScore drops below 90%
deflaky run -c "npx pytest" -r 3 --fail-threshold 90

CLI Reference

Flag	Alias	Description	Default
`--command <cmd>`	`-c`	Test command to run	required
`--runs <number>`	`-r`	Number of iterations	`5`
`--push`		Send results to DeFlaky dashboard	`false`
`--token <token>`	`-t`	API token (or `DEFLAKY_TOKEN` env)
`--format <format>`		Report format: `junit`, `json`, `auto`	`auto`
`--fail-threshold <n>`		Fail if FlakeScore is below this %
`--verbose`		Show detailed output	`false`

What is FlakeScore?

FlakeScore = (stable tests / total tests) x 100

100% = All tests are stable (pass or fail consistently)
< 100% = Some tests are flaky (sometimes pass, sometimes fail)
A test is "flaky" if it passes in some runs and fails in others

GitHub Actions

Run DeFlaky automatically on every push and PR. Add this workflow to .github/workflows/deflaky.yml in your repository:

name: DeFlaky - Flaky Test Detection

on:
  push:
    branches: [main]
  pull_request:
    branches: [main]
  schedule:
    - cron: '0 2 * * 1'  # Weekly on Monday at 2am

jobs:
  deflaky:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-node@v4
        with:
          node-version: '20'
      - run: npm ci
      - run: npm install -g deflaky-cli
      - name: Run DeFlaky
        run: deflaky run -c "npx playwright test" -r 3 --push --token ${{ secrets.DEFLAKY_TOKEN }}
        env:
          DEFLAKY_TOKEN: ${{ secrets.DEFLAKY_TOKEN }}

Add your DeFlaky token as a GitHub repository secret named DEFLAKY_TOKEN (Settings > Secrets and variables > Actions). Get your token at deflaky.com/dashboard.

For the full setup guide with PR comments, framework examples (Playwright, Cypress, Jest, Pytest, Selenium), and troubleshooting, see the GitHub Actions documentation.

License

MIT

deflaky-cli