@signalwire/docusaurus-plugin-llms-txt

📣 Version 2.0 Documentation This documentation is for version 2.0, which includes breaking API changes. If you're using version 1.x, please refer to the v1.2.2 documentation on npm.

A Docusaurus plugin that transforms your documentation into AI-friendly formats. It automatically converts your site's rendered HTML pages into clean markdown files and generates an llms.txt index file, making your documentation easily consumable by Large Language Models while preserving the human-readable experience.

Perfect for: API documentation, internal knowledge bases, developer resources, and any documentation that you want to make accessible to AI assistants, chatbots, or LLM-powered tools.

How It Works

This plugin processes your final HTML output after Docusaurus builds your site, not your source MDX/MD files. This approach captures fully rendered components, resolved data, and processed content that only exists after build time. The HTML is then converted through a sophisticated pipeline that extracts clean content, processes it through rehype/remark transformations, and outputs pristine markdown optimized for AI consumption.

Features

• 🔄 HTML to Markdown Conversion: Automatically converts your Docusaurus HTML pages to clean markdown files
• 📝 llms.txt Generation: Creates a comprehensive index file with links to all your documentation
• 🗂️ Section-Based Organization: Intuitive section-based organization with route precedence logic
• ⚡ Smart Caching: Efficient caching system for fast incremental builds
• 🎯 Content Filtering: Flexible filtering by content type (docs, blog, pages) and custom patterns
• 📎 File Attachments: Include local files (OpenAPI specs, schemas, guides) with YAML/JSON formatting preservation
• 🔗 External Links: Organize external URLs within sections or optional sections
• 💻 CLI Commands: Standalone CLI for generation and cleanup operations
• 🎨 Customizable Content Extraction: Configurable CSS selectors for precise content extraction
• 🔗 Link Management: Smart internal link processing with relative/absolute path options

Core Concepts

Output Files

• llms.txt - Hierarchical index file with links to all your documentation, organized by sections
• Individual markdown files - Clean .md versions of each page, mirroring your route structure
• llms-full.txt - Optional single file containing all content, useful for complete exports

Section Organization

Content is organized into logical sections that help AI systems understand documentation structure. You can define sections in two ways:

Manual Sections

Define sections explicitly with custom names, descriptions, and route patterns:

sections: [
  {
    id: 'api-docs',
    name: 'API Reference',
    routes: [{ route: '/api/**' }],
  },
];

Auto-Generated Sections

For routes not matching any manual section, the plugin auto-generates sections based on URL path segments. Use autoSectionDepth to control which path level becomes top-level sections:

With autoSectionDepth: 1 (group by first segment):

• /blog/post-1.md → "Blog" section
• /blog/post-2.md → "Blog" section
• /docs/intro.md → "Docs" section

With autoSectionDepth: 2 (group by second segment):

• /docs/advanced/plugin.md → "Advanced" section
• /docs/tutorial-basics/intro.md → "Tutorial Basics" section
• /blog/post-1.md → "Post 1" section (falls back to depth 1)

Routes shallower than autoSectionDepth automatically fall back to their actual depth, ensuring all content is included.

Documents within each section are sorted by path hierarchy (depth-first, then lexicographic), ensuring related content stays grouped together (e.g., /api/methods/* before /api/guides/*).

Content Processing Pipeline

HTML → Content Extraction (CSS selectors) → HTML Processing (rehype) → Markdown Conversion (remark) → Clean Output

Route Patterns

Use glob patterns like /docs/** or /api/* to filter and organize content. Routes determine both what gets processed and how it's organized in sections.

Default Excluded Routes

The plugin automatically excludes common Docusaurus-generated pages from processing. These defaults apply to all three excludeRoutes options (markdown, llmsTxt, and ui.copyPageContent.display):

• /search - Search page
• /404.html - 404 error page
• /tags - Global tags index
• /tags/** - Individual tag pages
• /blog/tags - Blog tags index
• /blog/tags/** - Individual blog tag pages
• /blog/archive - Blog archive page
• /blog/authors - Blog authors index
• /blog/authors/** - Individual author pages

You can add your own patterns to any excludeRoutes array, which will be merged with these defaults:

{
  markdown: {
    excludeRoutes: ['/admin/**', '/internal/**'], // Merged with defaults
  },
}

Installation

npm install @signalwire/docusaurus-plugin-llms-txt
# or
yarn add @signalwire/docusaurus-plugin-llms-txt

Quick Start

Basic Setup

Add the plugin to your docusaurus.config.ts:

import type { Config } from '@docusaurus/types';
import type { PluginOptions } from '@signalwire/docusaurus-plugin-llms-txt/public';

const config: Config = {
  plugins: [
    [
      '@signalwire/docusaurus-plugin-llms-txt',
      {
        // Enable with defaults
        generate: {
          enableMarkdownFiles: true,
          enableLlmsFullTxt: false,
        },
        include: {
          includeBlog: false,
          includePages: false,
          includeDocs: true,
        },
      } satisfies PluginOptions,
    ],
  ],
};

export default config;

Build your site and the plugin will automatically generate:

• build/llms.txt - Hierarchical index of your documentation
• build/**/*.md - Individual markdown files for each page (mirrors your route structure)

Advanced Configuration

import type { Config } from '@docusaurus/types';
import type { PluginOptions } from '@signalwire/docusaurus-plugin-llms-txt/public';

const config: Config = {
  plugins: [
    [
      '@signalwire/docusaurus-plugin-llms-txt',
      {
        // Markdown file generation options
        markdown: {
          enableFiles: true,
          relativePaths: true,
          includeBlog: true,
          includePages: true,
          includeDocs: true,
          includeVersionedDocs: true,
          excludeRoutes: ['/admin/**', '/internal/**'],
        },

        // llms.txt index file configuration
        llmsTxt: {
          enableLlmsFullTxt: true,
          includeBlog: true,
          includePages: true,
          includeDocs: true,
          excludeRoutes: ['/admin/**'],

          // Site metadata
          siteTitle: 'My Documentation',
          siteDescription: 'Comprehensive documentation for developers',

          // Auto-section organization
          autoSectionDepth: 2, // Group by 2nd path segment (/docs/api/* → "Api" section)
          autoSectionPosition: 10, // Auto-sections appear after positioned manual sections

          // Manual section organization
          sections: [
            {
              id: 'getting-started',
              name: 'Getting Started',
              description: 'Quick start guides and tutorials',
              position: 1,
              routes: [{ route: '/docs/intro/**' }],
            },
            {
              id: 'api-reference',
              name: 'API Reference',
              description: 'Complete API documentation',
              position: 2,
              routes: [{ route: '/docs/api/**' }],
              attachments: [
                {
                  source: './api/openapi.yaml',
                  title: 'OpenAPI Specification',
                  description: 'Complete API specification in OpenAPI 3.0 format',
                },
              ],
            },
          ],
        },

        // UI features (requires theme package)
        ui: {
          copyPageContent: {
            buttonLabel: 'Copy Page',
            display: {
              docs: true,
              excludeRoutes: ['/admin/**'],
            },
          },
        },
      } satisfies PluginOptions,
    ],
  ],
};

export default config;

API Reference

The plugin configuration is organized into three main areas: markdown file generation, llms.txt index creation, and UI features.

Top-Level Options

These options control plugin behavior and error handling.

MarkdownOptions

Generate individual .md files for each page.

LlmsTxtOptions

Generate and configure the llms.txt index file.

UiOptions

Enable UI features on your documentation pages.

Complex Types Reference

These types are used in the configuration options above.

SectionDefinition

Organize content into logical sections in llms.txt.

{
  id: 'api-docs',                    // Unique kebab-case ID
  name: 'API Documentation',         // Display name
  description: 'Complete API docs',  // Optional context
  position: 1,                       // Sort order (lower = earlier)
  routes: [{ route: '/api/**' }],    // Which pages belong here
  subsections: [],                   // Nested sections
  attachments: [],                   // Section-specific files
  optionalLinks: []                  // Section-specific external links
}

SectionRoute

Assign routes to sections using glob patterns.

{
  route: '/api/**',                  // Match all /api/* routes
  contentSelectors: ['.api-content'] // Optional: custom selectors for these pages
}

RouteRule

Customize content extraction for specific routes (separate from section assignment).

{
  route: '/api/**',
  contentSelectors: ['.api-content', 'article']
}

Note: Use RouteRule (in markdown.routeRules) for processing customization. Use SectionRoute (in sections[].routes) for section assignment.

AttachmentFile

Include external text files in your output.

{
  source: './specs/openapi.yaml',
  title: 'API Specification',
  description: 'Complete OpenAPI 3.0 spec',
  fileName: 'api-spec',  // Custom output filename (prevents collisions)
  includeInFullTxt: true
}

OptionalLink

Link to external resources.

{
  title: 'API Status Page',
  url: 'https://status.example.com',
  description: 'Real-time API status'
}

CopyPageContentOptions

Configure the copy button feature (requires theme package).

{
  buttonLabel: 'Copy Page',
  display: {
    docs: true,
    excludeRoutes: ['/admin/**']
  },
  contentStrategy: 'prefer-markdown',
  actions: {
    viewMarkdown: true,
    ai: {
      chatGPT: true,
      claude: { prompt: 'Help me understand this:' }
    }
  }
}

MIT © SignalWire