isbot
🤖/👨🦰 Recognise bots/crawlers/spiders using the user agent string.
Found 32 results for crawlers
@crawlee/utils
A set of shared utilities that can be used by crawlers
sitemapper
Parser for XML Sitemaps to be used with Robots.txt and web crawlers
robot-directives
Parse robot directives within HTML meta and/or HTTP headers.
parallaxapis-sdk-ts
ParallaxAPIs SDK
@context-labs/sitemapper
Parser for XML Sitemaps to be used with Robots.txt and web crawlers
parallaxapis-sdk-playwright
ParallaxAPIs SDK
@microlink/ua
A simple redis primitives to incr() and top() user agents
sitemapper_mos
Parser for XML Sitemaps to be used with Robots.txt and web crawlers
crawler-js
Opensource Framework Crawler in Node.js
@dataxquare/sitemapper
Parser for XML Sitemaps to be used with Robots.txt and web crawlers
bot-detector
It uses the user-agents.org xml file for detecting bots.
@drorgl/sitemapper
Parser for XML Sitemaps to be used with Robots.txt and web crawlers
@mastixmc/sitemapper
Parser for XML Sitemaps to be used with Robots.txt and web crawlers. (Extended version by mastixmc)
@cabbiepete/sitemapper
Parser for XML Sitemaps to be used with Robots.txt and web crawlers
@yeskiy/sitemapper
Parser for XML Sitemaps to be used with Robots.txt and web crawlers
@vladfrangu-dev/crawlee-utils
A set of shared utilities that can be used by crawlers
@qbbsh/sitemapper
Parser for XML Sitemaps to be used with Robots.txt and web crawlers
isbot-bundle
isbot Bundle UMD
robots-txt-component
Lightweight robots.txt parsing component without any external dependencies for Node.js.
@jarronb/site-mapper
Site mapper utilizing puppeteer to generate a xml sitemap
@javivelasco/isbot
🤖 detect bots/crawlers/spiders via the user agent.
adibenmat-sitemapper
Parser for XML Sitemaps to be used with Robots.txt and web crawlers
@public-square/sitemapper
Parser for XML Sitemaps to be used with Robots.txt and web crawlers
aliyu-crawlers
crawlers
simplecrawling
Crawler made simple
hidemyemail
A jQuery plugin that helps you to hide your email on your page and prevent crawlers to get it!
@myagizmaktav/sitemapper
Parser for XML Sitemaps to be used with Robots.txt and web crawlers
@raing3/sitemapper
Parser for XML Sitemaps to be used with Robots.txt and web crawlers
@xianyukeji/isbot
detects bots/crawlers/spiders via the user agent.
planisphere
A straightforward sitemap generator written in TypeScript.
@wmod/sitemapper
Parser for XML Sitemaps to be used with Robots.txt and web crawlers