What is Bravebot? AI crawler guide

Bravebot by Brave: Brave Search crawler used to discover and index pages, including AI-search-adjacent retrieval. Check its reported user-agent, robots.txt behavior, source, and verification guidance.

Brave Search crawler used to discover and index pages, including AI-search-adjacent retrieval.

What is Bravebot?

Bravebot is the web crawler operated by Brave for its Brave Search engine. It systematically discovers and indexes publicly available web pages so they can appear in Brave Search results. The crawler also supports AI-search-adjacent retrieval, meaning it helps power Brave's AI-driven answer features that rely on the search index. Bravebot identifies itself with the user-agent token Bravebot and respects the Robots Exclusion Protocol, obeying directives in robots.txt files. It is a focused, search-oriented bot that does not handle other Brave services like the browser or ads.

What it's for

For a site owner, Bravebot's visits mean your content may become discoverable in Brave Search and could be used in Brave's AI answer features. Allowing Bravebot can increase your site's visibility to users of Brave's privacy-focused search engine. If you block it, your pages will not appear in Brave Search results, and they will not be available for any AI-generated answers that Brave builds on top of its search index.

Allowing a search crawler only creates the possibility of retrieval. To find out whether your pages are actually selected, track citation gaps across AI answer engines using a stable query set.

How to handle Bravebot

To manage Bravebot's access, add a rule in your robots.txt file targeting the user-agent token Bravebot. You can disallow specific paths or the entire site. The crawler honors standard robots.txt directives, so no additional steps are needed. If you want full visibility in Brave Search, allow Bravebot to crawl your content. If you prefer to opt out, blocking it will prevent indexing and inclusion in Brave's AI answer features.

robots.txt rule

User-agent: Bravebot Disallow: /

Blocking cost

Blocking Bravebot will prevent your site from appearing in Brave Search results and may exclude it from any AI answer features that rely on Brave's search index.

Examples

• A news website allows Bravebot to crawl all articles, leading to their stories appearing in Brave Search results and potentially in Brave's AI-generated answers.
• An e-commerce site blocks Bravebot via robots.txt, and its product pages no longer show up in Brave Search, reducing traffic from Brave users.
• A blog disallows Bravebot from a private directory, and those pages are excluded from Brave's index while the rest of the site remains searchable.

Related bots

• Googlebot: Also tracked as a search crawler.
• YandexAdditional: Also tracked as a search crawler.
• PetalBot: Also tracked as a search crawler.
• Kagibot: Also tracked as a search crawler.
• Bingbot: Also tracked as a search crawler.
• xAI-SearchBot: Also tracked as a search crawler.
• Amzn-SearchBot: Also tracked as a search crawler.
• Applebot: Also tracked as a search crawler.
• Claude-SearchBot: Also tracked as a search crawler.
• Robots.txt: Robots.txt is the control file used to allow or block Bravebot.
• AI Crawlers: Bravebot is a concrete crawler example for this concept.