@mdxdb/payload

A Git-backed filesystem database adapter for Payload CMS. Store your content as human-readable .mdx files, version everything with Git, and query at scale with optional chdb (embedded ClickHouse) acceleration.

Features

• Human-readable storage - Documents stored as .mdx files with YAML frontmatter
• Git-native versioning - Full history, branching, and collaboration via Git
• GitHub-friendly - Edit content directly on GitHub, review changes in PRs
• Fast queries - Optional chdb integration for complex WHERE clauses and JOINs
• Relationship support - hasOne/hasMany relationships stored as markdown tables
• Code block fields - Store code snippets with syntax highlighting metadata
• MongoDB-style migrations - Data transformation migrations with up/down functions
• CLI tools - Sync, watch, and rebuild indexes from the command line
• Zero lock-in - Plain files mean you can always read/write content directly

Installation

npm install @mdxdb/payload
# or
pnpm add @mdxdb/payload
# or
yarn add @mdxdb/payload

Optional: Install chdb for query acceleration

npm install chdb

Quick Start

// payload.config.ts
import { buildConfig } from 'payload'
import { mdxdb } from '@mdxdb/payload'

export default buildConfig({
  db: mdxdb({
    basePath: './content',
  }),
  collections: [
    {
      slug: 'posts',
      fields: [
        { name: 'title', type: 'text', required: true },
        { name: 'status', type: 'select', options: ['draft', 'published'] },
        { name: 'content', type: 'richText' },
      ],
    },
  ],
})

This creates documents like:

content/
  posts/
    my-first-post.mdx
    another-post.mdx

Configuration

mdxdb({
  // Required: Base directory for content files
  basePath: './content',

  // Optional: Per-collection configuration
  collections: {
    posts: {
      // Custom path pattern (supports {id}, {slug}, etc.)
      pathPattern: '{year}/{month}/{slug}',
      // Custom MDX template
      template: '...',
    },
  },

  // Optional: Git integration settings
  git: {
    // Auto-commit changes (default: false)
    autoCommit: true,
    // Debounce time for batching commits (default: 5000ms)
    debounceMs: 5000,
    // Commit message template
    messageTemplate: '{action}({collection}): {id}',
  },

  // Optional: chdb session for query acceleration
  session: chdbSession,
})

MDX File Format

Documents are stored as MDX files with YAML frontmatter:

---
id: my-post
status: published
publishedAt: 2024-01-15
views: 1234
---

# My Post Title

This is the main content of the post. It supports **markdown** formatting.

## Author

| key  | value            |
| ---- | ---------------- |
| id   | nathan           |
| name | Nathan Clevenger |

## Tags

| id  | label | color |
| --- | ----- | ----- |
| ai  | AI    | blue  |
| web | Web   | green |

## Code

```typescript
export function hello() {
  console.log('Hello, world!')
}
```

### Field Mapping

| Payload Field Type | MDX Representation |
|-------------------|-------------------|
| text, number, date, etc. | YAML frontmatter |
| richText | Markdown content after frontmatter |
| relationship (hasOne) | Key-value table |
| relationship (hasMany) | Row-based table |
| code, json | Fenced code block |

## Relationships

### hasOne Relationship

Stored as a two-column key-value table:

```mdx
## Author

| key | value |
| --- | --- |
| id | user-123 |
| name | John Doe |
| email | john@example.com |

hasMany Relationship

Stored as a multi-column table with one row per related document:

## Tags

| id       | label      | color |
| -------- | ---------- | ----- |
| tech     | Technology | blue  |
| news     | News       | red   |
| featured | Featured   | gold  |

Query Acceleration with chdb

For collections with many documents or complex queries, enable chdb:

import { createSession } from 'chdb'

const session = await createSession()

export default buildConfig({
  db: mdxdb({
    basePath: './content',
    session,
  }),
  // ...
})

chdb provides:

• Fast WHERE queries - Full SQL query capability
• Relationship JOINs - Efficient population of related documents
• Aggregations - COUNT, SUM, AVG, etc.

Without chdb, the adapter falls back to file scanning (works fine for smaller collections).

Query Translation

Payload queries are translated to ClickHouse SQL:

// Payload query
await payload.find({
  collection: 'posts',
  where: {
    status: { equals: 'published' },
    views: { greater_than: 100 },
  },
  sort: '-publishedAt',
  limit: 10,
})

// Translates to:
// SELECT * FROM documents
// WHERE collection = 'posts'
//   AND data.status = 'published'
//   AND data.views > 100
// ORDER BY data.publishedAt DESC
// LIMIT 10

Migrations

Create data transformation migrations:

// migrations/20240115_add_default_status.ts
import type { MdxdbMigration } from '@mdxdb/payload'

export const migration: MdxdbMigration = {
  name: '20240115_add_default_status',

  async up({ payload }) {
    const posts = await payload.find({
      collection: 'posts',
      where: { status: { exists: false } },
      limit: 0,
    })

    for (const post of posts.docs) {
      await payload.update({
        collection: 'posts',
        id: post.id,
        data: { status: 'draft' },
      })
    }
  },

  async down({ payload }) {
    // Reverse the migration if needed
  },
}

Run migrations:

# Run pending migrations
payload migrate

# Roll back last batch
payload migrate:down

# Check migration status
payload migrate:status

# Reset and re-run all migrations
payload migrate:fresh

CLI Commands

The package includes a CLI for managing the chdb index:

# Sync files to chdb index
npx db-gitfs sync

# Watch for changes and auto-sync
npx db-gitfs watch

# Rebuild index from scratch
npx db-gitfs rebuild

Directory Structure

By default, collections are organized by slug:

content/
  posts/
    hello-world.mdx
    my-second-post.mdx
  pages/
    about.mdx
    contact.mdx
  site-settings.mdx      # Global

With admin groups:

content/
  blog/                   # admin.group: 'blog'
    posts/
      hello-world.mdx
    categories/
      tech.mdx
  settings/               # admin.group: 'settings'
    navigation.mdx

Git Integration

Auto-commit

Enable automatic commits on content changes:

mdxdb({
  basePath: './content',
  git: {
    autoCommit: true,
    debounceMs: 5000, // Batch changes within 5 seconds
    messageTemplate: '{action}({collection}): {id}',
  },
})

Git Metadata

Access git history for any document:

import { getGitMetadata } from '@mdxdb/payload/git/metadata'

const metadata = await getGitMetadata('/path/to/document.mdx')
// {
//   hash: 'abc123',
//   author: 'Nathan',
//   email: 'nathan@example.com',
//   date: Date,
//   message: 'update(posts): hello-world'
// }

API Reference

mdxdb(options)

Creates the database adapter.

Exported Types

import type {
  MdxdbAdapter,
  MdxdbAdapterArgs,
  MdxdbMigration,
  MigrateUpArgs,
  MigrateDownArgs,
} from '@mdxdb/payload'

Limitations

• No transactions - Git doesn't support atomic multi-file transactions
• No document versions - Use Git history instead of Payload's version system
• Query performance - Without chdb, large collections may be slow
• File conflicts - Concurrent edits may cause Git merge conflicts

When to Use

Good fit:

• Content-heavy sites (blogs, docs, marketing)
• Projects needing Git-based workflows
• Teams comfortable with Git/GitHub
• Content that benefits from human-readable storage

Consider alternatives:

• High-frequency writes (use db-mongodb or db-postgres)
• Complex transactions required
• Real-time collaboration needs

Development

# Install dependencies
pnpm install

# Run tests
pnpm test

# Build
pnpm build

# Lint
pnpm lint

License

MIT