The site-owner worker

An optional edge worker for verified domain owners. It reports AI agent requests from your CDN edge to Lyrenth, so your dashboard sees the full picture of who is reading your site, not just agents that route through our API. Two templates: Cloudflare and Vercel.

Reads

Known AI agents

Matches incoming User-Agents against the list of AI crawlers and assistants in the templates below. We keep this list up to date, so copy it again now and then.

Reports

Edge, not origin

Fires a non-blocking beacon from the edge. Zero added latency for your visitors.

Never

No user PII

Only the bot User-Agent, path, time bucket, and count. No human visitor data is collected.

Verify your domain and get the ingest token

You must have a verified domain first. Verify at /sites/add, then generate the ingest token in your site's settings. It starts with aiwt_ and only works for your own domain, so a leaked token cannot report for anyone else.

Cloudflare Worker

Deploy this Worker route in front of your zone, with your domain in the URL and the token in the LYRENTH_INGEST_TOKEN secret. It reports AI-agent requests to Lyrenth and passes everything else straight through.

The Worker only sees requests when your domain's traffic goes through Cloudflare (the proxied, orange-cloud setting). If the domain is set to DNS only, use the Vercel template below, or send the same POST from your own server.

worker.js

// The AI agents to report. Lyrenth names each one on its side.
const AI_AGENTS = [
  "GPTBot", "OAI-SearchBot", "ChatGPT-User", "ClaudeBot", "Claude-User",
  "Claude-SearchBot", "Claude-Web", "anthropic-ai", "PerplexityBot",
  "Perplexity-User", "Google-CloudVertexBot", "Google-Extended",
  "GoogleOther", "Applebot-Extended", "meta-externalagent",
  "meta-externalfetcher", "meta-webindexer", "FacebookBot", "Amazonbot",
  "Amzn-SearchBot", "AzureAI-SearchBot", "Bytespider", "TikTokSpider",
  "MistralAI-User", "cohere-ai", "DuckAssistBot", "xAI-SearchBot",
  "ExaSearchBot", "LinkupBot", "ShapBot", "Shap-User", "YouBot", "AI2Bot",
  "Diffbot", "CCBot", "AIWebIndex",
];
const AI_AGENT_RE = new RegExp("\\b(?:" + AI_AGENTS.join("|") + ")\\b", "i");

export default {
  async fetch(req, env, ctx) {
    const ua = req.headers.get("user-agent") || "";
    if (AI_AGENT_RE.test(ua)) {
      // The time, rounded down to 30 seconds as the ingest API asks.
      const bucket = new Date(Math.floor(Date.now() / 30000) * 30000);
      ctx.waitUntil(fetch("https://api.lyrenth.com/v1/ingest/bot/YOUR-DOMAIN.com", {
        method: "POST",
        headers: {
          "Authorization": `Bearer ${env.LYRENTH_INGEST_TOKEN}`,
          "Content-Type": "application/json",
        },
        body: JSON.stringify({
          events: [{
            raw_ua: ua,
            path: new URL(req.url).pathname,
            bucket_at: bucket.toISOString(),
            count: 1,
          }],
        }),
      }));
    }
    return fetch(req); // pass through, never block
  },
};

Vercel (Next.js proxy)

From Next.js 16 the file is proxy.ts and the function is proxy. On earlier versions, name the file middleware.ts and the function middleware; the rest is the same. Put the file at your project root (or in src/) and set LYRENTH_INGEST_TOKEN in your Vercel environment variables.

The report goes through event.waitUntil(...), which lets it finish after the page is sent. A plain fetch that nothing waits for can be cut off when the response ends. If you already have this file, add the event argument and the if block to the start of your function.

proxy.ts

import { NextResponse } from "next/server";
import type { NextFetchEvent, NextRequest } from "next/server";

// The AI agents to report. Lyrenth names each one on its side.
const AI_AGENTS = [
  "GPTBot", "OAI-SearchBot", "ChatGPT-User", "ClaudeBot", "Claude-User",
  "Claude-SearchBot", "Claude-Web", "anthropic-ai", "PerplexityBot",
  "Perplexity-User", "Google-CloudVertexBot", "Google-Extended",
  "GoogleOther", "Applebot-Extended", "meta-externalagent",
  "meta-externalfetcher", "meta-webindexer", "FacebookBot", "Amazonbot",
  "Amzn-SearchBot", "AzureAI-SearchBot", "Bytespider", "TikTokSpider",
  "MistralAI-User", "cohere-ai", "DuckAssistBot", "xAI-SearchBot",
  "ExaSearchBot", "LinkupBot", "ShapBot", "Shap-User", "YouBot", "AI2Bot",
  "Diffbot", "CCBot", "AIWebIndex",
];
const AI_AGENT_RE = new RegExp("\\b(?:" + AI_AGENTS.join("|") + ")\\b", "i");

export function proxy(req: NextRequest, event: NextFetchEvent) {
  const ua = req.headers.get("user-agent") || "";
  const token = process.env.LYRENTH_INGEST_TOKEN;
  if (token && AI_AGENT_RE.test(ua)) {
    // The time, rounded down to 30 seconds as the ingest API asks.
    const bucket = new Date(Math.floor(Date.now() / 30000) * 30000);
    // waitUntil lets the report finish after the response is sent.
    event.waitUntil(
      fetch("https://api.lyrenth.com/v1/ingest/bot/YOUR-DOMAIN.com", {
        method: "POST",
        headers: {
          Authorization: `Bearer ${token}`,
          "Content-Type": "application/json",
        },
        body: JSON.stringify({
          events: [{
            raw_ua: ua,
            path: req.nextUrl.pathname,
            bucket_at: bucket.toISOString(),
            count: 1,
          }],
        }),
        signal: AbortSignal.timeout(2000),
      }).catch(() => {}), // a failed report never affects your site
    );
  }
  return NextResponse.next();
}

Watch the data land

Within a minute, the requests appear on your site's page in the dashboard: by agent under AI agent traffic, by day under AI agent requests by day, and by path under Paths AI agents request most. Each request an AI agent makes to your site counts once. Pair it with the index-side reads Lyrenth already tracks for the complete view. High-traffic sites can batch: one POST accepts up to 1,000 events, so a worker may collect requests for around 30 seconds and send them together.

Go to your sites →Back to docs