Not sure where to start? Take a guided tour.
AI Site Grader
AI Query Scanner
llms.txt GeneratorAnthropicLive fetcher
Claude-Web
Claude-Web is a reported Anthropic-related token seen in public crawler registries and Trakkr detection, but absent from Anthropic’s current bot documentation.
Anthropic's current documentation does not list Claude-Web. Do not treat a matching string as proof of Anthropic traffic.
- ClaudeBotTrainingNot checked
- Claude-SearchBotSearchNot checked
- Claude-UserUser fetchNot checked
- Claude-WebUser fetchNot checked
Reads your live robots.txt and applies the same rules these crawlers use. Nothing is stored.
- robots.txt token
Claude-Web- Follows robots.txt
- Not documented
- How to verify it
- Not published
- Seen on
- Not measured
- Last checked
- 21 July 2026
Should you block Claude-Web?
There is little to gain. Anthropic does not document Claude-Web, so a rule naming it controls none of its documented crawlers. Write rules for ClaudeBot, Claude-SearchBot and Claude-User instead.
| Anthropic bot | Job | If you block it |
|---|---|---|
| ClaudeBot | Training | Disallowing ClaudeBot expresses a training-data opt-out for future compliant requests. A crawler log cannot confirm whether content entered a training dataset. |
| Claude-SearchBot | Search | Disallowing it can reduce Claude search discovery. Allowing it does not guarantee indexing, ranking, citation, or referral traffic. |
| Claude-User | User fetch | Disallowing it asks the documented fetcher not to retrieve covered paths. A matching request does not prove a page was cited in a Claude answer. |
| Claude-WebThis page | User fetch | A matching user-agent is not enough to prove Anthropic origin or infer training, search, or retrieval purpose. Use Anthropic’s documented bot tokens for policy decisions. |
About robots.txtAnthropic has not documented whether Claude-Web follows robots.txt, so a rule is a request, not a guarantee.
User-agent: Claude-Web
Disallow: /Asks Claude-Web not to crawl any page. Other Anthropic bots are unaffected. Add it to the robots.txt file at the root of your site, above any wider rules.
Claude-Web fires when a real user's question pulls in your page. To know how often Claude reaches for your brand at all, track your brand mentions in Claude.
Spot it in your logs.
Match the name, then check where the request came from. Anyone can copy a user-agent.
Claude-WebThis string comes from a public crawler registry or observed naming, not current operator documentation.
- 1
Match the name
Search your server or CDN log for
Claude-Web, case-insensitive. A match is a clue, not proof. - 2
Check the source
Anthropic publishes no IP ranges, reverse DNS or signatures for Claude-Web in the sources here, so it cannot be verified. Treat matches as unconfirmed.
Serve the content in HTML
The cited operator material does not verify JavaScript rendering. Serve useful HTML before client-side JavaScript where possible.
Anthropic’s current public documentation lists ClaudeBot, Claude-SearchBot, and Claude-User, not Claude-Web.
Questions, answered.
Claude-Web is a reported Anthropic-related token seen in public crawler registries and Trakkr detection, but absent from Anthropic's current bot documentation. This record has no current Anthropic source, so treat a matching request as unverified.
Sources.
Reported, not operator verified. Operator facts and Trakkr’s measurements keep separate dates.
- [1]Reported crawler reference
Reported, not operator verified · checked 21 July 2026
github.com/ai-robots-txt/ai.robots.txt/blob/main/table-of-bot-metrics.md - [2]Anthropic current crawler documentation
Officially documented · checked 18 August 2026
support.claude.com/en/articles/8896518-does-anthropic-crawl-data-from-the-web-and-how-can-site-owners-block-the-crawler
Last source check 21 July 2026.
Crawlers like Claude-Web.
All AI crawlersSee which AI crawlers read your site.
Trakkr shows which bots, Claude-Web included, visit which pages each day, and whether the answers they feed mention you.
14-day free trial. Cancel anytime.