Why does Trakkr say my site is blocking AI crawlers when I can see GPTBot traffic getting through in my own logs?
Updated June 30, 2026 · Reviewed by [email protected]
Trakkr's check and your own logs can disagree because they're measuring different things. Trakkr's crawl check sends its own request and looks at what comes back: a 403, a CAPTCHA, or a WAF challenge page means "blocked" from Trakkr's side, even if the real GPTBot is getting through on a separate path.
This happens most often on Cloudflare and similar WAFs, which can allow-list the real AI bot by verified IP while still challenging other automated traffic, including Trakkr's own crawl check. So your logs showing real OpenAI/Anthropic/Perplexity traffic getting through doesn't automatically mean Trakkr's check will pass.
If you want both to agree, add an explicit Allow rule in your WAF for the AI bot user agents (GPTBot, OAI-SearchBot, ChatGPT-User, ClaudeBot, Claude-Web, PerplexityBot, Google-Extended) rather than relying only on automatic bot verification.
Related questions
- I got a robots.txt or crawler-blocking alert from Trakkr that I think is wrong. How do I check it myself?
- Crawler tracking shows visits from some AI bots (like Meta or Mistral) but not others (like ChatGPT or Perplexity), even though I know those engines have visited. Is tracking broken?
- Which AI bot user agents should I allow in my firewall or WAF so Trakkr can track them?
- Can I install crawler tracking on a site that isn't Cloudflare, Vercel, Netlify, or WordPress (like Wix, Framer, or a custom PHP site)?
- Cloudflare says it denied analytics access when I try to connect my site to Trakkr. What permissions does the token need?