pi-knowledge-search

Semantic search over local files for pi. Indexes directories of text/markdown files using vector embeddings, watches for changes in real-time, and exposes a knowledge_search tool the LLM can call.

Install

Recommended: Install pi-total-recall to get the complete context stack — persistent memory, session history search, and local knowledge search in one package:

pi install pi-total-recall

Or install pi-knowledge-search standalone:

pi install git:github.com/samfoy/pi-knowledge-search

Or try without installing:

pi -e git:github.com/samfoy/pi-knowledge-search

Setup

Run the interactive setup command inside pi:

/knowledge-search-setup

This walks you through:

1. Directories to index (comma-separated paths)
2. File extensions to include (default: .md, .txt)
3. Directories to exclude (default: node_modules, .git, .obsidian, .trash)
4. Embedding provider — OpenAI, OpenAI-compatible (local/self-hosted), AWS Bedrock, or Ollama

Config is saved to ~/.pi/knowledge-search.json. Run /reload to activate.

Config file

You can also edit the config file directly:

{
  "dirs": ["~/notes", "~/docs"],
  "fileExtensions": [".md", ".txt"],
  "excludeDirs": ["node_modules", ".git", ".obsidian", ".trash"],
  "provider": {
    "type": "openai",
    "model": "text-embedding-3-small"
  }
}

The API key for OpenAI can be set in the config file ("apiKey": "sk-...") or via the OPENAI_API_KEY environment variable.

{
  "dirs": ["~/vault"],
  "provider": {
    "type": "bedrock",
    "profile": "my-aws-profile",
    "region": "us-west-2",
    "model": "amazon.titan-embed-text-v2:0"
  }
}

{
  "dirs": ["~/notes"],
  "provider": {
    "type": "ollama",
    "url": "http://localhost:11434",
    "model": "nomic-embed-text"
  }
}

ollama serve
ollama pull nomic-embed-text

{
  "dirs": ["~/notes"],
  "provider": {
    "type": "openai-compatible",
    "baseUrl": "http://127.0.0.1:8080",
    "apiKey": "your-local-key",
    "model": "qwen3-embeddings"
  }
}

python -m llama_cpp.server --model ./models/qwen3-embedding.gguf --port 8080

Bedrock Knowledge Bases

You can add Amazon Bedrock Knowledge Bases as additional search sources. These are managed RAG services — Amazon handles chunking, embedding, and vector storage. pi-knowledge-search queries them at search time and merges results with local file results.

Add via command:

/knowledge-add-kb

Or add directly to the config file:

{
  "dirs": ["~/notes"],
  "provider": { "type": "openai" },
  "knowledgeBases": [
    {
      "id": "XXXXXXXXXX",
      "region": "us-east-1",
      "profile": "default",
      "label": "Team docs"
    }
  ]
}

You can use Knowledge Bases alongside local file indexing, or on their own (omit dirs and provider for KB-only mode).

KB-only config:

{
  "knowledgeBases": [
    {
      "id": "XXXXXXXXXX",
      "region": "us-east-1",
      "profile": "my-work-profile",
      "label": "Engineering wiki"
    }
  ]
}

Requires the AWS SDK and valid credentials with bedrock:Retrieve permissions.

Environment variable overrides

Every config field can be overridden via environment variables. This is useful for CI or when you want different settings per shell session. See env-vars.md for the full list.

How it works

1. On session start, loads the index from disk and incrementally syncs — only re-embeds new or modified files
2. Starts a file watcher for real-time updates (debounced, 2s)
3. Registers a knowledge_search tool the LLM calls with natural language queries
4. Returns ranked results with file paths, relevance scores, and content excerpts

The index is stored at ~/.pi/knowledge-search/index.json.

Commands

Performance

Typical numbers for 500 markdown files (20MB):

License

MIT