@aeolens/middleware

Zero-latency AI crawler tracking for Next.js. Detects visits from GPT, Claude, Perplexity, Gemini, and 20+ other AI crawlers and reports them to AEOlens — without adding any latency to your site.

How it works

The proxy runs on the server, checks the User-Agent header on every request, and fires a fire-and-forget beacon to AEOlens if an AI crawler is detected. Your users never wait for the beacon — it runs in the background after the response is sent.

Quick start

npx @aeolens/middleware init

That's it. The CLI creates proxy.ts (or middleware.ts for Next.js < 16) and writes your API key to .env.local. Deploy and AI crawler visits appear in your AEOlens dashboard.

Get your API key from AEOlens → Settings → Integrations.

Need to change your API key? Run:

npx @aeolens/middleware init --reset-key

Manual setup

If you prefer to set things up yourself:

npm install @aeolens/middleware

1. Add your API key to .env.local:

AEOLENS_KEY=aa_your_key_here

2. Create proxy.ts in your project root (Next.js 16+):

import { withAEOlens } from '@aeolens/middleware'

export const proxy = withAEOlens()

export const config = {
  matcher: ['/((?!_next/static|_next/image|favicon.ico).*)'],
}

For Next.js 13–15, use middleware.ts with export const middleware = withAEOlens() instead.

Composing with existing proxy

If you already have a proxy.ts, call withAEOlens() alongside your existing logic:

import { withAEOlens } from '@aeolens/middleware'
import { auth } from './lib/auth'
import type { NextRequest } from 'next/server'

const tracking = withAEOlens()

export async function proxy(req: NextRequest) {
  // Tracking runs first — non-blocking, never delays auth
  await tracking(req)
  return auth(req)
}

export const config = {
  matcher: ['/((?!_next/static|_next/image|favicon.ico).*)'],
}

Options

withAEOlens({
  // Explicit key — overrides AEOLENS_KEY env var
  key: 'aa_your_key_here',

  // Called when a bot is detected. Return false to skip the beacon.
  onDetect({ bot, url, userAgent }) {
    console.log(`${bot} visited ${url}`)
    // return false  ← suppress beacon for this request
  },
})

Detected crawlers

GPTBot · ChatGPT-User · OAI-SearchBot · ClaudeBot · Claude-Web · anthropic-ai · PerplexityBot · Googlebot · Google-Extended · bingbot · Bytespider · DuckAssistBot · CCBot · Amazonbot · Applebot · cohere-ai · meta-externalagent · LinkedInBot · Twitterbot · and more.

The list is kept in sync with the AEOlens backend. You can also export KNOWN_AI_BOTS and detectBot from this package if you need them elsewhere.

License

MIT