AutomationFlowsWeb Scraping › Send Weekly AI Crawler Visibility Reports with Cloudflare, Claude, and…

Send Weekly AI Crawler Visibility Reports with Cloudflare, Claude, and…

Original n8n title: Send Weekly AI Crawler Visibility Reports with Cloudflare, Claude, and Telegram/email

ByMeridian IA @meridianiainc on n8n.io

This workflow runs on a weekly schedule, queries Cloudflare GraphQL Analytics for requests matching known AI crawler user-agents, aggregates request counts per bot, generates a short summary with Anthropic Claude, and optionally delivers the report via SMTP email and Telegram.…

Cron / scheduled trigger★★★★☆ complexityAI-powered15 nodesHTTP RequestAnthropicEmail SendTelegram
Web Scraping Trigger: Cron / scheduled Nodes: 15 Complexity: ★★★★☆ AI nodes: yes Added:

This workflow corresponds to n8n.io template #17635 — we link there as the canonical source.

This workflow follows the Emailsend → HTTP Request recipe pattern — see all workflows that pair these two integrations.

The workflow JSON

Copy or download the full n8n JSON below. Paste it into a new n8n workflow, add your credentials, activate. Full import guide →

Download .json
{
  "id": "6h574kOdFBELd0nb",
  "name": "AI Crawler Visibility Report [TEMPLATE]",
  "tags": [],
  "nodes": [
    {
      "id": "45021d61-80fa-4505-ae50-88cc5a542cf8",
      "name": "Sticky Note - Overview",
      "type": "n8n-nodes-base.stickyNote",
      "position": [
        -240,
        -160
      ],
      "parameters": {
        "width": 560,
        "height": 700,
        "content": "## \ud83e\udd16 AI Crawler Visibility Report\n\n### How it works\nRuns on a schedule and queries Cloudflare's Analytics API for AI crawler traffic on your domain (GPTBot, ClaudeBot, PerplexityBot, and others), by matching known user-agents. It totals requests per bot, writes a plain-language summary, and sends it by email and/or Telegram -- so you know which AI engines are actually visiting your content, without digging through Cloudflare's dashboard.\n\n### Setup steps\n- Edit CONFIG: your Cloudflare Zone ID, site name, lookback window, alert email, and Telegram chat ID.\n- Create a Cloudflare API Token (My Profile -> API Tokens -> Create Token -> Zone > Analytics > Read, scoped to your specific zone) and add it as a Header Auth credential (Name: Authorization, Value: Bearer YOUR_TOKEN) on the 'Query Cloudflare Analytics' node.\n- Add your Anthropic API credential (used to write the plain-language summary).\n- Add SMTP and/or Telegram credentials for delivery.\n\n### Customization\nAdd or remove bots to track in CONFIG's AI_BOTS_TO_TRACK field. Note: on Cloudflare's free plan, analytics data is only retained for 24 hours -- keep LOOKBACK_DAYS at 1 unless you're on a paid plan."
      },
      "typeVersion": 1
    },
    {
      "id": "dd3f7374-73ad-4594-b7e8-32fd04d6cc0e",
      "name": "Sticky Note - Fetch",
      "type": "n8n-nodes-base.stickyNote",
      "position": [
        384,
        160
      ],
      "parameters": {
        "color": 7,
        "width": 852,
        "height": 380,
        "content": "## Load config & fetch AI crawler data\n\nLoads CONFIG, builds a GraphQL query for the known AI bots you're tracking, and queries Cloudflare's Analytics API for matching requests in your lookback window."
      },
      "typeVersion": 1
    },
    {
      "id": "5bbf14b3-95e5-43f4-9953-f50a52738135",
      "name": "Sticky Note - Summarize",
      "type": "n8n-nodes-base.stickyNote",
      "position": [
        1280,
        160
      ],
      "parameters": {
        "color": 7,
        "width": 728,
        "height": 380,
        "content": "## Aggregate & summarize\n\nTotals requests per bot (exact counts, computed in code -- not estimated by the AI), then has Claude write a short plain-language report."
      },
      "typeVersion": 1
    },
    {
      "id": "29fa0d67-f6a5-4251-bd79-251846db2b82",
      "name": "Sticky Note - Notify",
      "type": "n8n-nodes-base.stickyNote",
      "position": [
        2064,
        32
      ],
      "parameters": {
        "color": 7,
        "width": 544,
        "height": 648,
        "content": "## Send report\n\nSends the report through email and/or Telegram, independently toggled via CONFIG."
      },
      "typeVersion": 1
    },
    {
      "id": "503f2c0c-505b-4912-b796-8c5359e5b887",
      "name": "Weekly Schedule",
      "type": "n8n-nodes-base.scheduleTrigger",
      "position": [
        432,
        320
      ],
      "parameters": {
        "rule": {
          "interval": [
            {
              "triggerAtHour": 8
            }
          ]
        }
      },
      "typeVersion": 1.3
    },
    {
      "id": "2e1af1c7-15f3-4e72-851a-1183f86db87c",
      "name": "CONFIG - Edit Me First",
      "type": "n8n-nodes-base.set",
      "position": [
        656,
        320
      ],
      "parameters": {
        "options": {},
        "assignments": {
          "assignments": [
            {
              "id": "cfg-zone-id",
              "name": "CLOUDFLARE_ZONE_ID",
              "type": "string",
              "value": "your-cloudflare-zone-id"
            },
            {
              "id": "cfg-site-name",
              "name": "SITE_NAME",
              "type": "string",
              "value": "your-site.com"
            },
            {
              "id": "cfg-days",
              "name": "LOOKBACK_DAYS",
              "type": "number",
              "value": 1
            },
            {
              "id": "cfg-bots",
              "name": "AI_BOTS_TO_TRACK",
              "type": "string",
              "value": "GPTBot,ChatGPT-User,OAI-SearchBot,ClaudeBot,Claude-User,PerplexityBot,Perplexity-User,Google-CloudVertexBot,Bytespider,CCBot,Amazonbot,Meta-ExternalAgent,Applebot"
            },
            {
              "id": "cfg-language",
              "name": "LANGUAGE",
              "type": "string",
              "value": "en"
            },
            {
              "id": "cfg-enable-email",
              "name": "ENABLE_EMAIL",
              "type": "boolean",
              "value": true
            },
            {
              "id": "cfg-enable-telegram",
              "name": "ENABLE_TELEGRAM",
              "type": "boolean",
              "value": true
            },
            {
              "id": "cfg-alert-email",
              "name": "ALERT_EMAIL",
              "type": "string",
              "value": "user@example.com"
            },
            {
              "id": "cfg-telegram-chat",
              "name": "TELEGRAM_CHAT_ID",
              "type": "string",
              "value": "your-telegram-chat-id"
            }
          ]
        }
      },
      "typeVersion": 3.4
    },
    {
      "id": "9f83670d-b86a-4fdb-b579-0c60f3bf4c1e",
      "name": "Build GraphQL Query",
      "type": "n8n-nodes-base.code",
      "position": [
        880,
        320
      ],
      "parameters": {
        "jsCode": "const cfg = $input.first().json;\nconst now = new Date();\nconst since = new Date(now.getTime() - cfg.LOOKBACK_DAYS * 24 * 60 * 60 * 1000);\n\nconst bots = cfg.AI_BOTS_TO_TRACK.split(',').map(b => b.trim()).filter(Boolean);\nconst orFilters = bots.map(b => `{ userAgent_like: \"%${b}%\" }`).join(',\\n            ');\n\nconst query = `query GetAiCrawlerRequests($zoneTag: string, $since: string, $until: string) {\n  viewer {\n    zones(filter: { zoneTag: $zoneTag }) {\n      httpRequestsAdaptiveGroups(\n        filter: {\n          datetime_geq: $since,\n          datetime_leq: $until,\n          requestSource: \"eyeball\",\n          OR: [\n            ${orFilters}\n          ]\n        },\n        limit: 5000,\n        orderBy: [count_DESC]\n      ) {\n        count\n        dimensions {\n          userAgent\n          edgeResponseStatus\n        }\n      }\n    }\n  }\n}`;\n\nreturn [{\n  json: {\n    query,\n    variables: {\n      zoneTag: cfg.CLOUDFLARE_ZONE_ID,\n      since: since.toISOString(),\n      until: now.toISOString()\n    }\n  }\n}];"
      },
      "typeVersion": 2
    },
    {
      "id": "b8e80466-d778-479a-b464-39a52e351f03",
      "name": "Query Cloudflare Analytics",
      "type": "n8n-nodes-base.httpRequest",
      "position": [
        1104,
        320
      ],
      "parameters": {
        "url": "https://api.cloudflare.com/client/v4/graphql",
        "method": "POST",
        "options": {},
        "jsonBody": "={{ JSON.stringify({ query: $json.query, variables: $json.variables }) }}",
        "sendBody": true,
        "specifyBody": "json",
        "authentication": "genericCredentialType",
        "genericAuthType": "httpHeaderAuth"
      },
      "typeVersion": 4.4
    },
    {
      "id": "f9ac4eaf-122d-4870-9217-f57faf9aff60",
      "name": "Aggregate Crawler Activity",
      "type": "n8n-nodes-base.code",
      "position": [
        1328,
        320
      ],
      "parameters": {
        "jsCode": "const cfg = $('CONFIG - Edit Me First').first().json;\nconst groups = $input.first().json.data?.viewer?.zones?.[0]?.httpRequestsAdaptiveGroups || [];\nconst knownBots = cfg.AI_BOTS_TO_TRACK.split(',').map(b => b.trim()).filter(Boolean);\n\nfunction extractBotName(ua) {\n  for (const bot of knownBots) {\n    if (ua.includes(bot)) return bot;\n  }\n  return 'Other';\n}\n\nconst activity = {};\nfor (const g of groups) {\n  const ua = g.dimensions.userAgent || '';\n  const bot = extractBotName(ua);\n  activity[bot] = (activity[bot] || 0) + g.count;\n}\n\nreturn [{\n  json: {\n    siteName: cfg.SITE_NAME,\n    lookbackDays: cfg.LOOKBACK_DAYS,\n    activity,\n    language: cfg.LANGUAGE,\n    enableEmail: cfg.ENABLE_EMAIL,\n    enableTelegram: cfg.ENABLE_TELEGRAM,\n    alertEmail: cfg.ALERT_EMAIL,\n    telegramChatId: cfg.TELEGRAM_CHAT_ID\n  }\n}];"
      },
      "typeVersion": 2
    },
    {
      "id": "d50ff392-c81d-42c1-85e8-8267f263df0b",
      "name": "Generate Plain-Language Summary",
      "type": "@n8n/n8n-nodes-langchain.anthropic",
      "position": [
        1520,
        320
      ],
      "parameters": {
        "modelId": {
          "__rl": true,
          "mode": "list",
          "value": "claude-sonnet-4-6",
          "cachedResultName": "claude-sonnet-4-6"
        },
        "options": {
          "maxTokens": 400
        },
        "messages": {
          "values": [
            {
              "content": "=Write a short, plain-language weekly AI crawler activity report in {{ $json.language === 'fr' ? 'French' : 'English' }} for {{ $json.siteName }}, covering the last {{ $json.lookbackDays }} day(s).\n\nRequest counts by AI crawler user-agent: {{ JSON.stringify($json.activity) }}\n\nExplain clearly which AI engines are visiting. IMPORTANT: mention once, briefly, that this counts self-reported user-agents and technically could include spoofed requests (full cryptographic verification requires Cloudflare's paid Bot Management) -- but keep this caveat to one short sentence, don't dwell on it. Keep the whole report under 150 words, friendly and direct, no markdown headers."
            }
          ]
        }
      },
      "typeVersion": 1
    },
    {
      "id": "72f65a48-5c51-4472-a390-738f61f9662d",
      "name": "Prepare Report Message",
      "type": "n8n-nodes-base.code",
      "position": [
        1856,
        320
      ],
      "parameters": {
        "jsCode": "const agg = $('Aggregate Crawler Activity').first().json;\nconst summary = $input.first().json.content[0].text;\nreturn [{\n  json: {\n    subject: `AI Crawler Visibility Report \u2014 ${agg.siteName}`,\n    message: summary,\n    enableEmail: agg.enableEmail,\n    enableTelegram: agg.enableTelegram,\n    alertEmail: agg.alertEmail,\n    telegramChatId: agg.telegramChatId\n  }\n}];"
      },
      "typeVersion": 2
    },
    {
      "id": "2e44baca-fedd-4102-9309-0f3a659234a9",
      "name": "IF Email Enabled",
      "type": "n8n-nodes-base.if",
      "position": [
        2176,
        224
      ],
      "parameters": {
        "options": {},
        "conditions": {
          "options": {
            "version": 1,
            "leftValue": "",
            "caseSensitive": true,
            "typeValidation": "strict"
          },
          "combinator": "and",
          "conditions": [
            {
              "operator": {
                "type": "boolean",
                "operation": "true"
              },
              "leftValue": "={{ $json.enableEmail }}",
              "rightValue": true
            }
          ]
        }
      },
      "typeVersion": 2.2
    },
    {
      "id": "e7d1e292-ae4e-4f84-80ac-aed41c29930d",
      "name": "IF Telegram Enabled",
      "type": "n8n-nodes-base.if",
      "position": [
        2176,
        448
      ],
      "parameters": {
        "options": {},
        "conditions": {
          "options": {
            "version": 1,
            "leftValue": "",
            "caseSensitive": true,
            "typeValidation": "strict"
          },
          "combinator": "and",
          "conditions": [
            {
              "operator": {
                "type": "boolean",
                "operation": "true"
              },
              "leftValue": "={{ $json.enableTelegram }}",
              "rightValue": true
            }
          ]
        }
      },
      "typeVersion": 2.2
    },
    {
      "id": "8dc93f51-0634-475c-a66f-69aaf4fd7a40",
      "name": "Send Email Report",
      "type": "n8n-nodes-base.emailSend",
      "position": [
        2400,
        176
      ],
      "parameters": {
        "text": "={{ $json.message }}",
        "options": {},
        "subject": "={{ $json.subject }}",
        "toEmail": "={{ $json.alertEmail }}",
        "fromEmail": "={{ $json.alertEmail }}",
        "emailFormat": "text"
      },
      "typeVersion": 2.1
    },
    {
      "id": "869839a4-a754-4104-83d8-449ef1220842",
      "name": "Send Telegram Report",
      "type": "n8n-nodes-base.telegram",
      "position": [
        2400,
        496
      ],
      "parameters": {
        "text": "={{ $json.subject }}\n\n{{ $json.message }}",
        "chatId": "={{ $json.telegramChatId }}",
        "additionalFields": {}
      },
      "typeVersion": 1.2
    }
  ],
  "active": false,
  "settings": {
    "binaryMode": "separate",
    "executionOrder": "v1"
  },
  "versionId": "7634dbed-ed0c-4f7e-bb49-299fb44a33e2",
  "nodeGroups": [],
  "connections": {
    "Weekly Schedule": {
      "main": [
        [
          {
            "node": "CONFIG - Edit Me First",
            "type": "main",
            "index": 0
          }
        ]
      ]
    },
    "IF Email Enabled": {
      "main": [
        [
          {
            "node": "Send Email Report",
            "type": "main",
            "index": 0
          }
        ]
      ]
    },
    "Build GraphQL Query": {
      "main": [
        [
          {
            "node": "Query Cloudflare Analytics",
            "type": "main",
            "index": 0
          }
        ]
      ]
    },
    "IF Telegram Enabled": {
      "main": [
        [
          {
            "node": "Send Telegram Report",
            "type": "main",
            "index": 0
          }
        ]
      ]
    },
    "CONFIG - Edit Me First": {
      "main": [
        [
          {
            "node": "Build GraphQL Query",
            "type": "main",
            "index": 0
          }
        ]
      ]
    },
    "Prepare Report Message": {
      "main": [
        [
          {
            "node": "IF Email Enabled",
            "type": "main",
            "index": 0
          },
          {
            "node": "IF Telegram Enabled",
            "type": "main",
            "index": 0
          }
        ]
      ]
    },
    "Aggregate Crawler Activity": {
      "main": [
        [
          {
            "node": "Generate Plain-Language Summary",
            "type": "main",
            "index": 0
          }
        ]
      ]
    },
    "Query Cloudflare Analytics": {
      "main": [
        [
          {
            "node": "Aggregate Crawler Activity",
            "type": "main",
            "index": 0
          }
        ]
      ]
    },
    "Generate Plain-Language Summary": {
      "main": [
        [
          {
            "node": "Prepare Report Message",
            "type": "main",
            "index": 0
          }
        ]
      ]
    }
  }
}
Pro

For the full experience including quality scoring and batch install features for each workflow upgrade to Pro

About this workflow

This workflow runs on a weekly schedule, queries Cloudflare GraphQL Analytics for requests matching known AI crawler user-agents, aggregates request counts per bot, generates a short summary with Anthropic Claude, and optionally delivers the report via SMTP email and Telegram.…

Source: https://n8n.io/workflows/17635/ — original creator credit. Request a take-down →

More Web Scraping workflows → · Browse all categories →

Related workflows

Workflows that share integrations, category, or trigger type with this one. All free to copy and import.

Web Scraping

Automate price monitoring for e-commerce competitors—ideal for retailers, analysts, and pricing teams. Scrapes competitor sites, extracts pricing/stock data via AI, detects changes, and sends instant

@Mendable/N8N Nodes Firecrawl, Perplexity, HTTP Request +3
Web Scraping

Women creators, homemakers-turned-entrepreneurs, and feminine lifestyle brands who want a graceful, low-lift way to keep an eye on competitor content and spark weekly ideas.

HTTP Request, Google Sheets, Email Send +1
Web Scraping

Turn any Amazon India product URL into a fully-edited 10-second lifestyle video and auto-publish it to Instagram, Facebook, X (Twitter), LinkedIn, YouTube, and Threads — with platform-optimized captio

HTTP Request, @Apify/N8N Nodes Apify, OpenRouter Chat +3
Web Scraping

Schedule Trigger runs every 6 hours (customizable) Apify Scraper fetches Upwork jobs matching your criteria Deduplication filters out jobs you've already seen AI Scoring (GPT-4) evaluates fit, client

HTTP Request, Google Sheets, OpenAI +2
Web Scraping

Turn your n8n workflow into an automated competitive intelligence unit. This template monitors competitor activities across blog feeds and YouTube channels to detect strategic shifts. Instead of simpl

YouTube, @Apify/N8N Nodes Apify, RSS Feed Read +4