JSPM

🤖/👨‍🦰 Recognise bots/crawlers/spiders using the user agent string.

Verify that a request is from Google using Google's recommended DNS verification steps

Keep Bots Away From Your Express App

koa detect googlebot, facebookbot, ...

A simple redis primitives to incr() and top() user agents

Lightweight bot detection middleware for tracking AI crawler visits (OpenAI, ChatGPT, etc.) with API support and analytics

A lightweight, zero-touch sitemap generator for any nextjs project

Generate an SEO website and sitemap for google escaped fragments

isbot for nodejs, Contains most of the world's bot or spider

Express HTML snapshots middleware

express.js middleware for generating web page html snapshots for web crawlers

Generates a sitemap for nextjs apps at runtime, using a crawl of the local filesystem

Pluggable, flexible and agnostic prerenderer for sites and SPAs, optimized for SEO. No code changes required.

Keep Bots Away From Your Express App!

koa detect googlebot, facebookbot ...

Express middleware that informs if request originates from a Google Bot or a Google Crawler

Magnetic is a tool that makes it easy to fetch web pages that may have dynamic content and render them to static HTML.

Simple node user-agent inspector that detects bot requests

isbot Bundle UMD

🤖 detect bots/crawlers/spiders via the user agent.

Magnetic example - Magnetic is an Express middleware that makes it easy for you to add automatic server-side static rendering to your dynamic single-page app.

Magnetic Express is an Express middleware for Magnetic - a tool that makes it easy for you to add automatic server-side static rendering to your dynamic single-page app.

Handle http request from crawler (Something like Googlebot)

detect crawler

detects bots/crawlers/spiders via the user agent.