@signalwire/docusaurus-plugin-llms-txt

📣 Version 2.0 Documentation This documentation is for version 2.0, which includes breaking API changes. If you're using version 1.x, please refer to the v1.2.2 documentation on npm.

A Docusaurus plugin that transforms your documentation into AI-friendly formats. It automatically converts your site's rendered HTML pages into clean markdown files and generates an llms.txt index file, making your documentation easily consumable by Large Language Models while preserving the human-readable experience.

Perfect for: API documentation, internal knowledge bases, developer resources, and any documentation that you want to make accessible to AI assistants, chatbots, or LLM-powered tools.

How It Works

This plugin processes your final HTML output after Docusaurus builds your site, not your source MDX/MD files. This approach captures fully rendered components, resolved data, and processed content that only exists after build time. The HTML is then converted through a sophisticated pipeline that extracts clean content, processes it through rehype/remark transformations, and outputs pristine markdown optimized for AI consumption.

Features

• 🔄 HTML to Markdown Conversion: Automatically converts your Docusaurus HTML pages to clean markdown files
• 📝 llms.txt Generation: Creates a comprehensive index file with links to all your documentation
• 🗂️ Section-Based Organization: Intuitive section-based organization with route precedence logic
• ⚡ Smart Caching: Efficient caching system for fast incremental builds
• 🎯 Content Filtering: Flexible filtering by content type (docs, blog, pages) and custom patterns
• 📎 File Attachments: Include local files (OpenAPI specs, schemas, guides) with YAML/JSON formatting preservation
• 🔗 External Links: Organize external URLs within sections or optional sections
• 💻 CLI Commands: Standalone CLI for generation and cleanup operations
• 🎨 Customizable Content Extraction: Configurable CSS selectors for precise content extraction
• 🔗 Link Management: Smart internal link processing with relative/absolute path options

Core Concepts

Output Files

• llms.txt - Hierarchical index file with links to all your documentation, organized by sections
• Individual markdown files - Clean .md versions of each page, mirroring your route structure
• llms-full.txt - Optional single file containing all content, useful for complete exports

Section Organization

Content is organized into logical sections that help AI systems understand documentation structure. Sections can have custom names, descriptions, and route patterns for precise content grouping.

Content Processing Pipeline

HTML → Content Extraction (CSS selectors) → HTML Processing (rehype) → Markdown Conversion (remark) → Clean Output

Route Patterns

Use glob patterns like /docs/** or /api/* to filter and organize content. Routes determine both what gets processed and how it's organized in sections.

Installation

npm install @signalwire/docusaurus-plugin-llms-txt
# or
yarn add @signalwire/docusaurus-plugin-llms-txt

Quick Start

Basic Setup

Add the plugin to your docusaurus.config.ts:

import type { Config } from '@docusaurus/types';
import type { PluginOptions } from '@signalwire/docusaurus-plugin-llms-txt/public';

const config: Config = {
  plugins: [
    [
      '@signalwire/docusaurus-plugin-llms-txt',
      {
        // Enable with defaults
        generate: {
          enableMarkdownFiles: true,
          enableLlmsFullTxt: false,
        },
        include: {
          includeBlog: false,
          includePages: false,
          includeDocs: true,
        },
      } satisfies PluginOptions,
    ],
  ],
};

export default config;

Build your site and the plugin will automatically generate:

• build/llms.txt - Hierarchical index of your documentation
• build/**/*.md - Individual markdown files for each page (mirrors your route structure)

Advanced Configuration

import type { Config } from '@docusaurus/types';
import type { PluginOptions } from '@signalwire/docusaurus-plugin-llms-txt/public';

const config: Config = {
  plugins: [
    [
      '@signalwire/docusaurus-plugin-llms-txt',
      {
        // Generation options
        generate: {
          enableMarkdownFiles: true,
          enableLlmsFullTxt: true,
          relativePaths: true,
        },

        // Content inclusion
        include: {
          includeBlog: true,
          includePages: true,
          excludeRoutes: ['/admin/**', '/internal/**'],
        },

        // Content organization
        structure: {
          sections: [
            {
              id: 'getting-started',
              name: 'Getting Started',
              position: 1,
              routes: [{ route: '/docs/intro/**' }],
            },
            {
              id: 'api-reference',
              name: 'API Reference',
              position: 2,
              routes: [{ route: '/docs/api/**' }],
            },
          ],
          siteTitle: 'My Documentation',
          siteDescription: 'Comprehensive documentation for developers',
        },

        // File attachments
        processing: {
          attachments: [
            {
              source: './api/openapi.yaml',
              title: 'OpenAPI Specification',
              sectionId: 'api-reference',
            },
          ],
        },

        // UI features (requires theme package)
        ui: {
          copyPageContent: true,
        },
      } satisfies PluginOptions,
    ],
  ],
};

export default config;

API Reference

Main Configuration Options

GenerateOptions

Controls what files the plugin generates and how it formats output paths.

IncludeOptions

Specifies which content types from your Docusaurus site should be processed and included in the generated output.

StructureOptions

Defines how your content is organized into logical sections in the llms.txt output, along with site metadata.

ProcessingOptions

Controls how HTML content is extracted and converted to markdown, plus additional content processing features.

UiOptions

Settings for UI features that integrate with your Docusaurus theme.

Complex Types

SectionDefinition

Defines a logical section in your documentation for organizing content in llms.txt.

SectionRoute

Configuration for route patterns within a section definition. Use this to assign specific routes to sections.

RouteRule

Global rules for customizing content extraction for specific routes. Use this for processing customization, not section assignment.

AttachmentFile

Configuration for including external files in the llms.txt output.

OptionalLink

External URLs to include in the llms.txt index for referencing external resources.

CopyPageContentOptions

Configuration for the copy page content button feature when using object configuration.

Troubleshooting

Common Issues

"No cached routes found" - Run npm run build first; the plugin needs route cache from build Empty or minimal content - Check contentSelectors match your theme; use logLevel: 3 for debugging Sections not organizing correctly - Verify route patterns don't overlap; first matching section wins Missing pages - Check excludeRoutes patterns and ensure correct include options are set Build performance issues - Use excludeRoutes to skip large sections; disable enableLlmsFullTxt if not needed

Advanced Usage

Custom Remark/Rehype Plugins

import remarkCustomPlugin from './my-remark-plugin';

const pluginConfig: PluginOptions = {
  processing: {
    beforeDefaultRemarkPlugins: [[remarkCustomPlugin, { option: 'value' }]],
  },
};

Section Hierarchies

import type { PluginOptions } from '@signalwire/docusaurus-plugin-llms-txt/public';

const pluginConfig: PluginOptions = {
  structure: {
    sections: [
      {
        id: 'api',
        name: 'API Reference',
        subsections: [
          { id: 'rest', name: 'REST API', routes: [{ route: '/api/rest/**' }] },
          { id: 'graphql', name: 'GraphQL', routes: [{ route: '/api/graphql/**' }] },
        ],
      },
    ],
  },
};

Performance Optimization

• Use excludeRoutes to skip unnecessary pages
• Set includeVersionedDocs: false to process only current version
• Enable logLevel: 0 for production builds to reduce console output
• Use route-specific contentSelectors for pages with different structures

CLI Commands

Generate Command

npx docusaurus llms-txt [siteDir]

Generates llms.txt and markdown files using cached routes from a previous build.

Prerequisites: You must run npm run build first to create the route cache.

Clean Command

npx docusaurus llms-txt-clean [siteDir] [options]

Removes all generated files:

• Deletes build/llms.txt, build/llms-full.txt, and all generated markdown files
• Use --clear-cache to also clear the .docusaurus/llms-txt-plugin cache directory
• Useful for forcing full regeneration or cleaning up after plugin removal

Copy Page Content Feature

Add a copy button to your documentation pages with AI integration:

# Install both packages
npm install @signalwire/docusaurus-plugin-llms-txt @signalwire/docusaurus-theme-llms-txt

import type { Config } from '@docusaurus/types';
import type { PluginOptions } from '@signalwire/docusaurus-plugin-llms-txt/public';

const config: Config = {
  themes: ['@signalwire/docusaurus-theme-llms-txt'],
  plugins: [
    [
      '@signalwire/docusaurus-plugin-llms-txt',
      {
        ui: {
          copyPageContent: true, // Enable with defaults
        },
      } satisfies PluginOptions,
    ],
  ],
};

export default config;

Important Notes:

• Currently only supports documentation pages (not blog or custom pages)
• Requires both the plugin AND theme package to be installed
• Button appears in the doc page header with dropdown for multiple actions

License

MIT © SignalWire