JSPM

fetch-arwaky

1.1.6
  • ESM via JSPM
  • ES Module Entrypoint
  • Export Map
  • Keywords
  • License
  • Repository URL
  • TypeScript Types
  • README
  • Created
  • Published
  • Downloads 64
  • Score
    100M100P100Q60001F
  • License MIT

MCP server and CLI for fetching web content as HTML, Markdown, plain text, JSON, or YouTube transcripts

Package Exports

  • fetch-arwaky
  • fetch-arwaky/dist/index.js

This package does not declare an exports field, so the exports above have been automatically detected and optimized by JSPM instead. If any package subpath is missing, it is recommended to post an issue to the original package (fetch-arwaky) to support the "exports" field. If that is not possible, create a JSPM override to customize the exports field for this package.

Readme

Fetch Arwaky Server

fetch mcp logo

npm version

An MCP server for fetching web content in multiple formats — HTML, JSON, plain text, Markdown, readable article content, and YouTube transcripts.

Fetch Server MCP server

Tools

All tools accept the following common parameters:

Parameter Type Required Description
url string Yes URL to fetch
headers object No Custom headers to include in the request
max_length number No Maximum characters to return (default: 5000)
start_index number No Start from this character index (default: 0)
proxy string No Proxy URL (e.g. http://proxy:8080)
  • fetch_html — Fetch a website and return its raw HTML content.

  • fetch_markdown — Fetch a website and return its content converted to Markdown.

  • fetch_txt — Fetch a website and return plain text with HTML tags, scripts, and styles removed.

  • fetch_json — Fetch a URL and return the JSON response.

  • fetch_readable — Fetch a website and extract the main article content using Mozilla Readability, returned as Markdown. Strips navigation, ads, and boilerplate. Ideal for articles and blog posts.

  • fetch_youtube_transcript — Fetch a YouTube video's captions/transcript. Uses yt-dlp if available, otherwise extracts directly from the page. Accepts an additional lang parameter (default: "en") to select the caption language.

Installation

As an MCP server

Add to your MCP client configuration:

{
  "mcpServers": {
    "fetch": {
      "command": "npx",
      "args": ["fetch-arwaky"]
    }
  }
}

As a CLI

npx fetch-arwaky <command> <url> [flags]

Or install globally:

npm install -g fetch-arwaky
fetch-arwaky <command> <url> [flags]

CLI Usage

fetch-arwaky <command> <url> [flags]

Commands

Command Description
html Fetch a URL and return raw HTML
markdown Fetch a URL and return Markdown
readable Fetch a URL and return article content as Markdown (via Readability)
txt Fetch a URL and return plain text
json Fetch a URL and return JSON
youtube Fetch a YouTube video transcript

Flags

Flag Description
--max-length <N> Maximum characters to return
--start-index <N> Start from this character index
--proxy <URL> Proxy URL
--lang <code> Language code for YouTube transcripts (default: en)
--help Show help message
--version Show version

Examples

# Fetch a page as markdown
fetch-arwaky markdown https://example.com

# Extract article content without boilerplate
fetch-arwaky readable https://example.com/blog/post

# Get a YouTube transcript in Spanish
fetch-arwaky youtube https://www.youtube.com/watch?v=dQw4w9WgXcQ --lang es

# Fetch with a length limit
fetch-arwaky html https://example.com --max-length 10000

# Fetch through a proxy
fetch-arwaky json https://api.example.com/data --proxy http://proxy:8080

Environment Variables

Variable Description
DEFAULT_LIMIT Default character limit for responses (default: 5000, set to 0 for no limit)
MAX_RESPONSE_BYTES Maximum response body size in bytes (default: 10485760 / 10 MB)

Example with a custom limit:

{
  "mcpServers": {
    "fetch": {
      "command": "npx",
      "args": ["fetch-arwaky"],
      "env": {
        "DEFAULT_LIMIT": "50000"
      }
    }
  }
}

Features

  • Fetch web content as HTML, JSON, plain text, or Markdown
  • Extract article content with Mozilla Readability (strips ads, nav, boilerplate)
  • Extract YouTube video transcripts (via yt-dlp or direct extraction)
  • Proxy support for requests behind firewalls
  • Pagination with max_length and start_index
  • Custom request headers
  • SSRF protection (blocks private/localhost addresses and DNS rebinding)
  • Response size limits to prevent memory exhaustion

Development

bun install
bun run dev     # start with watch mode
bun test        # run tests
bun run build   # build for production

License

This project is licensed under the MIT License.