# What is Claude-Web? AI crawler guide

Canonical URL: https://trakkr.ai/bots/claude-web
Published: 2026-06-11
Last updated: 2026-06-11

Learn what Claude-Web is, who operates it, its verified user-agent, robots.txt posture, and how blocking it can affect AI search, citations, training, or agent visibility.

Anthropic-related web agent token reported in crawler registries and Trakkr backend detection.

## What is Claude-Web?

Claude-Web is a web crawler user-agent token associated with Anthropic. It appears in public crawler registries and has been detected in backend monitoring, but Anthropic does not currently document it on its official help page. The company's public guidance focuses on ClaudeBot, Claude-SearchBot, and Claude-User, leaving the exact role of Claude-Web unverified. Site owners may encounter it as a lightweight retrieval agent, though its behavior and policy compliance are not formally defined by Anthropic.

## What it's for

If you see Claude-Web in your logs, it likely represents some form of Claude-related retrieval traffic. However, because Anthropic's explicit training and search policies are tied to ClaudeBot and Claude-SearchBot, you should not assume Claude-Web follows the same rules. Its presence may indicate experimental or auxiliary fetching, but without official documentation, its impact on your site's visibility in AI products is uncertain.

## How to handle Claude-Web

Since the token is unverified and undocumented by Anthropic, the safest approach is to block it via robots.txt. The page already renders a disallow snippet for Claude-Web. If you want to permit known Anthropic bots while restricting unknown ones, keep the Claude-Web block in place and separately allow ClaudeBot and Claude-SearchBot according to your preferences.

## robots.txt rule

User-agent: Claude-Web
Disallow: /

## Blocking cost

Blocking Claude-Web is unlikely to affect your site's appearance in Claude's answers, search features, or training data, because those functions are governed by the documented ClaudeBot and Claude-SearchBot tokens.

## Examples

- A site owner notices Claude-Web in server logs but finds no mention of it on Anthropic's official crawler documentation.
- A webmaster adds a blanket block for Claude-Web in robots.txt while still allowing ClaudeBot for AI training opt-in.
- A developer checks public crawler registries and sees Claude-Web listed alongside other Anthropic tokens, but with no usage details.

## Related bots

- Google-Gemini-CLI: Also tracked as a interaction crawler.
- Claude-Code: Another Anthropic interaction crawler to compare.
- cohere-ai: Also tracked as a interaction crawler.
- AmazonBuyForMe: Also tracked as a interaction crawler.
- Google-Agent: Also tracked as a interaction crawler.
- Claude-User: Another Anthropic interaction crawler to compare.
- Gemini-Deep-Research: Also tracked as a interaction crawler.
- Google-NotebookLM: Also tracked as a interaction crawler.
- GoogleAgent-Mariner: Also tracked as a interaction crawler.
- Anthropic-AI: Claude-Web connects this operator term to its crawler behavior.
- AI Crawlers: Claude-Web is a concrete crawler example for this concept.

## Frequently Asked Questions

### Is Claude-Web an official Anthropic crawler?

Anthropic has not published documentation for Claude-Web. Its current public guidance covers ClaudeBot, Claude-SearchBot, and Claude-User, so Claude-Web's status is unverified.

### Does Claude-Web follow robots.txt?

There is no official statement from Anthropic about Claude-Web's robots.txt compliance. Because the token is unverified, its behavior cannot be guaranteed.

### Will blocking Claude-Web affect my site in Claude's AI products?

It is unlikely, because Anthropic ties its training and search crawling to ClaudeBot and Claude-SearchBot. Blocking Claude-Web should not impact those documented use cases.

### How is Claude-Web different from ClaudeBot?

ClaudeBot is a documented crawler used for AI training and product improvement, with clear policy guidance. Claude-Web is an unverified token with no official description, so its purpose and behavior are unknown.

### Should I block Claude-Web?

Given the lack of documentation and unverified posture, blocking Claude-Web is a cautious choice. You can still allow known Anthropic bots separately if you wish to permit their specific functions.

## Data And Sources

- [Claude-Web source reference](https://github.com/ai-robots-txt/ai.robots.txt/blob/main/table-of-bot-metrics.md) - Source used to verify Claude-Web.