JSPM

A fast and stable DHT crawler.

To scrap the content from the web site

A util tool

Parkour the web like a yamakazi

Website schema based crawler

User Agent

Powerful Scraping and Crawling library with anti-scraping, scalability, storage, static/dynamic contents, monitoring UI and more. Ready to deploy on cloud instances or serverless.

Nexstack Nodejs library that provides an Api for obtaining the movies information website.

HTTP request module customized for crawlers.

x-ray's crawler

Get information using the string of the specified rule

crawler

基于 MCP 的网页爬取服务器，内置 Puppeteer 无头浏览器支持

A powerful and flexible web scraper library built with TypeScript

Generate LLM-friendly text files from Next.js applications by crawling sitemaps and extracting content

A web crawler. Automatically crawls websites. Define custom handlers to parse content. Obeys robots.txt, rate limits and concurrency limits.

Templates for the crawlee projects

The scalable web crawling and scraping library for JavaScript/Node.js. Enables development of data extraction and web automation jobs (not only) with headless Chrome and Puppeteer.

Crawl an instagram profile by id

The unofficial HLTV Node.js API

Provides APIs by simple configuration.

Just Another Sitemap Generator

High-performance web crawler implemented in Go with JavaScript bindings

a module used for scraping with either an electron browser (webview, BrowserWindow, BrowserView) or http requests (axios)

Fetch the pre-rendered content, meta, links and Open Graph of a webpage, especially Single-Page Application (SPA)

A lightweight and simple API for web crawling built on chromium puppeteer

Convert a website to static markdown.

A tool to allow for quick running of JSON-based scrapers using request-promise and jsonframe-cheerio.

A search and crawler for Wikipedia articles

A SoundCloud Downloader made with Node and Typescript

Verify that a request is from Google crawlers using Google's DNS verification steps

Verify that a request is from Baidu crawlers using Baidu's DNS verification

Distributed web crawler powered by Headless Chrome

Memory efficient and synchronous downloader of map tiles. Allows for a fast and easy approach to make map tiles (from a WMS) available offline.

Webcrawler for data mining and unification purposes

模仿scrapy的node爬虫框架

Functional web scraping in typescript

A twitter client for agents

过早客论坛信息获取 MCP (Model Context Protocol) 服务器