🌐 Website to Markdown MCP Server

A powerful Model Context Protocol (MCP) server designed for fetching website content and converting it to Markdown format, making it easier for AI to understand and process website information.

✨ Key Features

🚀 Quick Start

📦 Installation

Install the package globally via npm:

npm install -g website-to-markdown-mcp

Or use it directly with npx:

npx website-to-markdown-mcp

🎯 One-Click Configuration (Recommended)

💡 Best Practice: Use external configuration files for easier management!

Step 1: Create Configuration File 📄

Create a my-websites.json file:

{
  "websites": [
    {
      "name": "your_website",
      "url": "https://your-website.com",
      "description": "Your Project Website"
    },
    {
      "name": "api_docs",
      "url": "https://api.example.com/openapi.json",
      "description": "Your API Specification"
    }
  ]
}

Step 2: Configure MCP Server ⚙️

Add to .cursor/mcp.json:

{
  "mcpServers": {
    "website-to-markdown": {
      "command": "npx",
      "args": ["-y", "website-to-markdown-mcp"],
      "disabled": false,
      "env": {
        "WEBSITES_CONFIG_PATH": "./my-websites.json"
      }
    }
  }
}

Alternative: Local Installation

If you prefer to install locally:

{
  "mcpServers": {
    "website-to-markdown": {
      "command": "node",
      "args": ["./node_modules/website-to-markdown-mcp/dist/cli.js"],
      "disabled": false,
      "env": {
        "WEBSITES_CONFIG_PATH": "./my-websites.json"
      }
    }
  }
}

Step 3: Restart and Test 🔄

1. Restart Cursor
2. Open Chat and use Agent mode
3. Test command: Please list all configured websites

🆕 Complete OpenAPI/Swagger Support

🌟 Pre-configured Example Websites

{
  "websites": [
    {
      "name": "petstore_openapi",
      "url": "https://petstore3.swagger.io/api/v3/openapi.json",
      "description": "🐕 Swagger Petstore OpenAPI 3.0 Spec (Demo)"
    },
    {
      "name": "petstore_swagger",
      "url": "https://petstore.swagger.io/v2/swagger.json",
      "description": "🐱 Swagger Petstore Swagger 2.0 Spec (Demo)"
    },
    {
      "name": "github_api",
      "url": "https://raw.githubusercontent.com/github/rest-api-description/main/descriptions/api.github.com/api.github.com.json",
      "description": "🐙 GitHub REST API OpenAPI Spec"
    }
  ]
}

📦 Installation & Setup

🛠️ System Requirements

• Node.js 18+
• npm or yarn
• Cursor Editor

⚡ Quick Installation

Option 1: Global Installation (Recommended)

# Install globally
npm install -g website-to-markdown-mcp

# Verify installation
website-to-markdown-mcp --help

Option 2: Project-specific Installation

# In your project directory
npm install website-to-markdown-mcp

# Use via npx
npx website-to-markdown-mcp

Option 3: Development Setup

# Clone the repository
git clone https://github.com/your-repo/website-to-markdown-mcp.git
cd website-to-markdown-mcp

# Install dependencies
npm install

# Build project
npm run build

# Run locally
npm start

🎛️ Advanced Configuration Options

graph TD
    A[🔍 Check Environment Variable<br/>WEBSITES_CONFIG_PATH] --> B{File exists?}
    B -->|Yes| C[✅ Load External Config File]
    B -->|No| D[🔍 Check Environment Variable<br/>WEBSITES_CONFIG]
    D --> E{Valid JSON?}
    E -->|Yes| F[✅ Load Embedded Config]
    E -->|No| G[🔍 Check config.json]
    G --> H{File exists?}
    H -->|Yes| I[✅ Load Local Config]
    H -->|No| J[🔧 Use Default Config]

🎨 Configuration Method Details

📋 Method 1: External Configuration File (🌟 Recommended)

💡 Advantages: Easy to edit, syntax highlighting, version control friendly

{
  "websites": [
    {
         "name": "my_docs",
      "url": "https://docs.example.com",
         "description": "📚 My Documentation Website"
    }
  ]
}

{
      "env": {
       "WEBSITES_CONFIG_PATH": "./my-api-configs.json"
  }
}

📋 Method 2: NPX with Configuration

{
  "mcpServers": {
    "website-to-markdown": {
      "command": "npx",
      "args": ["-y", "website-to-markdown-mcp"],
      "disabled": false,
      "env": {
        "WEBSITES_CONFIG": "{\"websites\":[{\"name\":\"example\",\"url\":\"https://example.com\",\"description\":\"Example Website\"}]}"
      }
    }
  }
}

📋 Method 3: Global Installation

{
  "mcpServers": {
    "website-to-markdown": {
      "command": "website-to-markdown-mcp",
      "disabled": false,
      "env": {
        "WEBSITES_CONFIG_PATH": "./my-websites.json"
      }
    }
  }
}

🔧 Available Tools

🌐 General Tools

🎯 Dedicated Tools

Each configured website automatically generates corresponding dedicated tools:

• fetch_petstore_openapi - Fetch Petstore OpenAPI 3.0 spec
• fetch_petstore_swagger - Fetch Petstore Swagger 2.0 spec
• fetch_github_api - Fetch GitHub API spec
• fetch_tailwind_css - Fetch Tailwind CSS documentation

📊 Output Format Examples

🌐 General Website Content

# Website Title

**Source**: https://example.com
**Website**: example_site - Example Website

---

[Converted Markdown content...]

📋 OpenAPI 3.x Specification File

# 🚀 Example API (v2.1.0)

**Source**: https://api.example.com/openapi.json
**OpenAPI Version**: 3.0.3
**Validation Status**: ✅ Valid

---

## 📋 API Basic Information

- **API Name**: Example API
- **Version**: 2.1.0
- **OpenAPI Version**: 3.0.3
- **Description**: A powerful example API

## 🌐 Servers

1. **https://api.example.com**
   - 🏢 Production server
2. **https://staging-api.example.com**
   - 🧪 Testing server

## 🛠️ API Endpoints

Total of **15** endpoints:

### 👥 `/users`
- **GET**: Get user list
- **POST**: Create new user

### 🔍 `/users/{id}`
- **GET**: Get specific user
- **PUT**: Update user information
- **DELETE**: Delete user

## 🧩 Components

- **Schemas**: 8 data models
- **Parameters**: 5 reusable parameters  
- **Responses**: 12 reusable responses
- **Security Schemes**: 2 security mechanisms

🎯 Usage Examples

💻 Basic Usage

Please fetch the content from https://docs.example.com and convert to markdown

🔍 OpenAPI Specification Fetching

Please use the fetch_petstore_openapi tool to fetch Petstore OpenAPI specification

📚 Documentation Website Fetching

Please fetch React official documentation content

🔍 Search Functionality

Please search for "authentication" in all configured websites

🚨 Troubleshooting

❓ Common Issues

🔍 Debug Mode

Detailed logs are output to stderr at startup:

# View debug messages (global installation)
website-to-markdown-mcp 2> debug.log

# View debug messages (npx)
npx website-to-markdown-mcp 2> debug.log

📈 Performance & Optimization

⚡ Performance Tips

• 🚀 Batch Fetching: Configure multiple websites at once
• 💾 Caching Mechanism: Repeated requests are faster
• 🎯 Precise Selectors: Improve content extraction accuracy

🛡️ Security Considerations

• 🔒 HTTPS websites only (recommended)
• 🛠️ Auto filter malicious scripts
• 📝 Limit output content length

📦 Dependencies

📝 Changelog

🎉 v1.0.1 (Latest)

• 📦 Published to NPM registry
• ✨ Added Global CLI command support
• ✨ Added NPX support for easy usage
• 🔧 Improved Installation and setup process
• 📝 Updated Documentation with installation methods

🎉 v1.1.0

• ✨ Added Full OpenAPI 3.x/Swagger 2.0 support
• ✨ Added JSON/YAML format auto-detection
• ✨ Added Professional-grade spec validation and reference resolution
• ✨ Added Version auto-adaptation mechanism
• ✨ Added Structured API documentation summary
• 🔧 Pre-configured Multiple OpenAPI/Swagger examples

🎯 v1.0.0 (Stable)

• 🎉 Initial Release
• 🌐 Basic Functions Website content fetching
• 📝 Core Functions Markdown conversion
• ⚙️ Configuration Support Multi-website management

🤝 Contributing

💡 How to Contribute

1. 🍴 Fork this project
2. 🌟 Create feature branch (git checkout -b feature/AmazingFeature)
3. 📝 Commit changes (git commit -m 'Add some AmazingFeature')
4. 📤 Push to branch (git push origin feature/AmazingFeature)
5. 🔄 Open Pull Request

🐛 Issue Reporting

Report issues on the Issues page, please include:

• 🔍 Issue Description
• 🔄 Reproduction Steps
• 💻 Environment Information
• 📸 Screenshots or Logs

📄 License

This project is licensed under the MIT License - see the LICENSE file for details.