tokenlean

Lean CLI tools for AI agents — maximum insight, minimum tokens

Installation • Quick Start • Tools • AI Integration • Configuration

Why tokenlean?

AI coding assistants are powerful, but they have a fundamental constraint: context windows. Every file read, every search result consumes tokens. This matters because:

tokenlean provides 32 specialized CLI tools that give you exactly the information needed — no more, no less.

Instead of reading a 500-line file    →  tl-exports (~50 tokens)
Instead of reading all type files     →  tl-types (~100 tokens)
Instead of guessing impact            →  tl-impact (know for sure)

Installation

npm install -g tokenlean

git clone https://github.com/edimuj/tokenlean.git
cd tokenlean
npm link

Requirements

• Node.js >= 18.0.0
• ripgrep (rg) — for search-based tools
• git — for history/blame/diff tools

Quick Start

# Get project overview
tl-structure

# Check file sizes before reading
tl-context src/api/

# Get function signatures without bodies
tl-symbols src/utils.ts

# Understand what a module exports
tl-exports src/lib/

# Check what would break if you change a file
tl-impact src/core/auth.ts

AI Agent Integration

Add tokenlean instructions to your AI tool's config:

The --minimal flag produces a compact version that uses fewer tokens.

Tools Reference

Before Reading Files

Understand code structure without reading full implementations.

Before Modifying Files

Understand dependencies and impact before making changes.

Understanding History

Track changes and authorship efficiently.

Finding Things

Search and discover code patterns.

Utilities

Common Options

All tools support these flags:

-l N, --max-lines N    Limit output to N lines
-t N, --max-tokens N   Limit output to ~N tokens
-j, --json             Output as JSON (for piping)
-q, --quiet            Minimal output (no headers/stats)
-h, --help             Show help

Configuration

Create .tokenleanrc.json in your project root or ~/.tokenleanrc.json globally:

{
  "output": {
    "maxLines": 100,
    "maxTokens": null
  },
  "skipDirs": [
    "generated",
    "vendor"
  ],
  "skipExtensions": [
    ".gen.ts"
  ],
  "importantDirs": [
    "domain",
    "core"
  ],
  "importantFiles": [
    "ARCHITECTURE.md"
  ],
  "searchPatterns": {
    "hooks": {
      "description": "Find React hooks",
      "pattern": "use[A-Z]\\w+",
      "glob": "**/*.{ts,tsx}"
    }
  }
}

{
  "output": {
    "maxLines": 100,
    "maxTokens": null
  },
  "skipDirs": [
    "generated",
    "vendor"
  ],
  "skipExtensions": [
    ".gen.ts"
  ],
  "importantDirs": [
    "domain",
    "core"
  ],
  "importantFiles": [
    "ARCHITECTURE.md"
  ],
  "searchPatterns": {
    "hooks": {
      "description": "Find React hooks",
      "pattern": "use[A-Z]\\w+",
      "glob": "**/*.{ts,tsx}"
    }
  },
  "hotspots": {
    "days": 90
  },
  "structure": {
    "depth": 3
  },
  "cache": {
    "enabled": true,
    "ttl": 300,
    "maxSize": "100MB",
    "location": null
  }
}

Caching

tokenlean caches expensive ripgrep operations with git-based invalidation — automatically invalidates on commits or file changes.

tl-cache stats      # View cache statistics
tl-cache clear      # Clear cache for current project
tl-cache clear-all  # Clear all cached data

Disable with TOKENLEAN_CACHE=0 or in config: {"cache":{"enabled":false}}

Example Workflows

tl-structure                    # Get the lay of the land
tl-entry                        # Find entry points
tl-exports src/lib/             # Understand the public API
tl-docs src/utils/              # Read documentation, not code
tl-types src/types/             # Understand data shapes
tl-schema                       # Understand the database

tl-impact src/core/auth.ts      # What would break?
tl-deps src/core/auth.ts        # What does it depend on?
tl-related src/core/auth.ts     # Find the tests
tl-coverage src/core/auth.ts    # Is it well tested?
tl-complexity src/core/auth.ts  # How complex is it?

tl-component src/Button.tsx     # Props, hooks, dependencies
tl-symbols src/Button.tsx       # Function signatures
tl-history src/Button.tsx       # Recent changes
tl-blame src/Button.tsx         # Who wrote what

tl-complexity src/ --threshold 15  # Complex functions
tl-unused src/                     # Dead code
tl-todo                            # Outstanding TODOs
tl-hotspots                        # Frequently changed (unstable?)

tl-secrets                         # Scan for hardcoded secrets
tl-secrets --staged                # Only check staged files
tl-secrets --min-severity high     # Only high severity issues

tl-pr feature-branch               # Summary of branch changes
tl-pr 123                          # GitHub PR #123 (needs gh CLI)
tl-pr --full                       # Include files, stats, commits

tl-changelog --unreleased          # What's new since last tag
tl-changelog v0.1.0..v0.2.0        # Between versions
tl-changelog --format compact      # Quick summary

tl-name coolproject awesomelib     # Check npm, GitHub, domains
tl-name myapp -s                   # Suggest variations if taken
tl-name myapp --tld io             # Check .io domain

Design Principles

1. Single purpose — Each tool does one thing well
2. Minimal output — Show only what's needed
3. Token-conscious — Every tool saves context tokens
4. Composable — Tools work together with JSON output for piping
5. Fast — No heavy parsing or external services
6. Universal — Works with JS/TS projects, most tools support Python/Go too

Other tools for Claude Code

Contributing

Contributions are welcome! Please feel free to submit a Pull Request.

License

MIT © Edin Mujkanovic