JSPM

Some tools to help you to render your application as a static web site using the crawlable module.

Simple website crawler and scraper

PhantomJS and JSDOM based crawling tool. Used PhantomJS for full load of asynchronously-loaded resources and JSDOM for quick crawls. Allows custom [tough-cookie](https://www.npmjs.com/package/tough-cookie) insertion. Refer to [cheerio](https://www.npmj

A lightweight and modular web crawling framework built with Puppeteer.

This extracts the top five news metadata from NAVER headlines.

Single Page App SER

Lightweight crawler written in TypeScript using ES6 generators.

fork from headless-chrome-crawler and update puppeteer to the latest version

Net Crawler is a web spider written with Nodejs

Crawler is a web spider written with Nodejs. It gives you the full power of jQuery on the server to parse a big number of pages as they are downloaded, asynchronously. Scraping should be simple and fun!

Web scraping/crawling framework built on top of headless Chrome

DCrawler is a distribited web spider written in Nodejs and queued with Mongodb. It gives you the full power of jQuery to parse big pages as they are downloaded, asynchronously. Simplifying distributed crawler!

Fast and lightweight web crawler with built-in cheerio, xml and json parser.

a headless browser automation library with easy-use API

Helper to extract confessions from webpages

Collection of patches for puppeteer and playwright to avoid automation detection and leaks. Helps to avoid Cloudflare and DataDome CAPTCHA pages. Easy to patch/unpatch, can be enabled/disabled on demand.

A small package to crawl a site and return a redirect template. This is helpful for migration from one to another website with different url schemes.

Crawler made simple

Distributed web crawler powered by Headless Chrome

SoongSil UniverSity U-saint Score Crawling

Environment for Goose Parser which allows to run it using JsDOM

Daily use crawling methods for puppeteer

Datasco API SDK for Node.js to collect any data from any website

proxidoor helps you make HTTP requests through a rotating proxy, you can use it for services such as web scraping, web crawling and more.

Distributed web crawler powered by Headless Chrome

Environment for Goose parser which allows to run it in commmon Browser

A Node.js scraping framework built on puppeteer-extra (to use a headless Chrome/Chromium browser). Has the ability to solve reCaptcha

web scraper for album reviews from pitchfork

Crawler is a web spider written with Nodejs. It gives you the full power of jQuery on the server to parse a big number of pages as they are downloaded, asynchronously. Scraping should be simple and fun!

Transform your text with dynamic typing animations! crawling-typer lets you display an array of strings one at a time, each with its own color. Customize typing speed, delete speed, and pauses between strings. Enjoy full control with loop counts, post-loo

Official JavaScript/TypeScript SDK for the Friday API

An API to get data off of IMDB using Puppeteer.

A Node.js scraping framework built on puppeteer-core (to use a headless Chrome/Chromium browser). The core module without browser installation

An API to get magnet links using Puppeteer.

A Node.js scraping framework built on puppeteer-extra (to use a headless Chrome/Chromium browser). Has the ability to solve reCaptcha. The core module without browser installation

Distributed web crawler powered by Headless Chrome

Tem o objetivo de executar rotinas de CRAWLING a partir de um arquivo JSON utilizando xpath mas aceitando para cada passo uma função callback que recebe o valor e pode passar esse valor para um próximo passo.

A straightforward sitemap generator written in TypeScript.

robin web crawling engine with nodejs

NodeJS Crawler for Twitter

The most advanced web crawler for JavaScript

StackSleuth in-house browser automation agent for debugging and user simulation

Crawler Second-system effect,the second development

keyword mention 크롤러

billboard chart crawling module

Package to find style links from the site you want

Set of utils and queues to make web scraping easy.

The error crawler that powers http://plucky.io/

A Simple Job Manager

based on node-crawler

Simple Instagram Crawling without using public API

Harvesting data at the <html> mine.

Easily scrap web pages by providing json recipes

Web crawler

Node.js web scraping utility powered by puppeteer pool

NodeJs crawling & scraping framework heavily inspired by Scrapy (Pyhton)

A simple command0line tool to crawl and test your website

make web scraping easy

A Wight backend for fetching static web pages

spamlet is an efficient and simple crawler for playwright

Easily create a scraper api with the @web/scrapper library, which includes a scraper and advanced events for your website.

This is the React Component for Detect Crawling

Minimalist Node.js web scraper and crawler working with under-the-hood JSDOM

A web-crawler and scraper that extracts data from a family of nested dynamic webpages with added enhancements to assist in knowledge mining applications.

A set of shared utilities that can be used by crawlers

Web crawler for Node.js

Easy To Use Web Crawler

Providers are the core of applications, where the subtitles are collected. Each provider exports a unique strategy for gathering data. From legendastv's web scraping from opensubtitle API usage, you can collect subtitles from your favorite tv shows and mo

Moving or backing up your Wordpress site to Blogger

Parkour the web like a yamakazi

scrap and caching by use a redis from instagram

n8n node for Firecrawl v2 API - Web scraping, crawling, and data extraction tool for workflows and AI agents

A tool to get sitemaps from websites and crawl them

naver stock data crawler

A lightweight and simple API for web crawling built on chromium puppeteer

Distributed web crawler powered by Headless Chrome