What is Bingbot? AI crawler guide
Bingbot by Microsoft: Microsoft Bing crawler used to crawl and index pages for Bing and Microsoft search-powered experiences. Check its reported user-agent, robots.txt behavior, source, and verification guidance.
Microsoft Bing crawler used to crawl and index pages for Bing and Microsoft search-powered experiences.
What is Bingbot?
Bingbot is Microsoft's web crawler that scans public websites to build the Bing search index. It identifies itself with the user-agent token Bingbot and respects the Robots Exclusion Protocol, including robots.txt directives. The crawler fetches pages so they can appear in Bing search results and in Microsoft search-powered experiences. It is operated by Microsoft and is classified as a search crawler. Site owners can control its access through standard robots.txt rules, and it does not require a separate verification token beyond the user-agent string.
What it's for
For a site owner, Bingbot's crawling determines whether your pages appear in Bing search results. If Bingbot cannot access your site, your content may not be indexed, which could reduce organic traffic from Bing. Additionally, because Microsoft Copilot relies on Bing search for some answers, blocking Bingbot may also limit your visibility in Copilot's responses, potentially affecting referral traffic and brand exposure from AI-assisted search experiences.
Whether to allow Bingbot is a policy call. Either way, check which AI platforms actually send you human visitors before assuming crawl volume equals value.
How to handle Bingbot
To manage Bingbot, add a rule in your robots.txt file targeting the user-agent Bingbot. You can disallow specific paths or the entire site. The crawler honors the Robots Exclusion Protocol, so standard directives work. If you want to allow full access, you can omit a disallow rule or explicitly allow it. There is no separate verification token needed; control is solely through robots.txt.
robots.txt rule
User-agent: Bingbot Disallow: /
Blocking cost
Blocking Bingbot can prevent your pages from being indexed in Bing and may reduce visibility in Microsoft Copilot answers that draw from Bing search results.
Examples
- Bingbot crawls a news article, and the page later appears in Bing search results for relevant queries.
- A product page crawled by Bingbot is surfaced in Bing Shopping results, driving traffic to the online store.
- After Bingbot indexes a how-to guide, the content may be used by Microsoft Copilot to answer user questions, citing the source.
Related bots
- Googlebot: Also tracked as a search crawler.
- KlaviyoAIBot: Also tracked as a search crawler.
- Amzn-SearchBot: Also tracked as a search crawler.
- Bravebot: Also tracked as a search crawler.
- Kagibot: Also tracked as a search crawler.
- YouBot: Also tracked as a search crawler.
- Applebot: Also tracked as a search crawler.
- atlassian-bot: Also tracked as a search crawler.
- Kimi-SearchBot: Also tracked as a search crawler.
- Microsoft Copilot: Bingbot connects this operator term to its crawler behavior.
- Robots.txt: Robots.txt is the control file used to allow or block Bingbot.
- IbouBot: Also tracked as a search crawler.
Frequently Asked Questions
What does Bingbot do?
Bingbot is Microsoft's crawler that scans public web pages to include them in the Bing search index and other Microsoft search-powered experiences.
Does Bingbot obey robots.txt?
Yes, Bingbot honors the Robots Exclusion Protocol and follows directives set in a site's robots.txt file.
Will blocking Bingbot affect my site's visibility in Microsoft Copilot?
Blocking Bingbot may indirectly affect visibility in Microsoft Copilot, as Copilot can rely on Bing search results to generate answers.
How can I allow Bingbot to crawl my entire site?
You can allow Bingbot full access by not including a disallow rule for its user-agent token in your robots.txt file, or by explicitly allowing the root path.
Is there a separate token or verification needed to control Bingbot?
No, control is based solely on the user-agent token Bingbot in your robots.txt file; no additional verification token is required.
Data & Sources
- Microsoft documentation - Primary source for Bingbot crawler details.
- Bingbot source reference - Source used to verify Bingbot.