# What is Meta-ExternalFetcher? AI crawler guide Canonical URL: https://trakkr.ai/bots/meta-externalfetcher Published: 2026-06-11 Last updated: 2026-06-11 Learn what Meta-ExternalFetcher is, who operates it, its verified user-agent, robots.txt posture, and how blocking it can affect AI search, citations, training, or agent visibility. Meta user-requested fetcher for AI and link features across Meta products. ## What is Meta-ExternalFetcher? Meta-ExternalFetcher is a web crawler operated by Meta. It fetches specific URLs on behalf of Meta users, rather than performing broad, autonomous crawling. When a user takes an action within a Meta product-such as sharing a link, asking an AI assistant to retrieve information, or using a feature that needs to preview a webpage-this bot may be dispatched to fetch that exact URL. It is documented by Meta as a user-requested fetcher, meaning its visits are typically tied to direct user intent. The bot identifies itself with the user-agent token Meta-ExternalFetcher and is part of Meta's suite of crawlers that support link sharing, AI features, and other interactive services across platforms like Facebook, Instagram, and WhatsApp. ## What it's for For site owners, a visit from Meta-ExternalFetcher usually means a Meta user has triggered a fetch of a specific page on your site. This could happen when someone shares your link in a Meta product, asks a Meta AI assistant about your content, or uses a feature that generates a link preview. Unlike broad crawlers that scan many pages, this bot is on-demand and targeted. Its activity can drive referral traffic and increase visibility within Meta's ecosystem, but it may also consume server resources for individual requests. Understanding this bot helps you distinguish user-driven fetches from automated indexing crawls. ## How to handle Meta-ExternalFetcher You can block Meta-ExternalFetcher using robots.txt, but Meta notes that because it performs user-requested fetches, it may bypass robots.txt restrictions. If you want to prevent Meta from fetching your content in response to user actions, a robots.txt disallow rule may not be fully effective. If you are comfortable with user-driven fetches, no action is needed; the bot will fetch URLs only when a user action triggers it. ## robots.txt rule User-agent: Meta-ExternalFetcher Disallow: / ## Blocking cost Blocking Meta-ExternalFetcher may prevent your content from appearing in user-requested link previews, AI responses, or other interactive features within Meta products, potentially reducing referral traffic and visibility. ## Examples - A user shares a link to a news article in a Facebook post, and Meta-ExternalFetcher retrieves the page to generate a preview with title, image, and description. - Someone asks a Meta AI assistant for the latest updates from a specific blog, prompting the bot to fetch the blog's homepage to extract relevant information. - A WhatsApp user sends a link to a product page, and the bot fetches the URL to display a rich preview in the chat. ## Related bots - MistralAI-User: Also tracked as a interaction crawler. - ChatGPT-User: Also tracked as a interaction crawler. - Claude-Code: Also tracked as a interaction crawler. - Google-NotebookLM: Also tracked as a interaction crawler. - cohere-ai: Also tracked as a interaction crawler. - Google-Gemini-CLI: Also tracked as a interaction crawler. - ChatGPT Agent: Also tracked as a interaction crawler. - Perplexity-User: Also tracked as a interaction crawler. - GoogleAgent-Mariner: Also tracked as a interaction crawler. - Meta AI: Meta-ExternalFetcher connects this operator term to its crawler behavior. - Noindex: Meta-ExternalFetcher gives crawler context for Noindex. ## Frequently Asked Questions ### Does Meta-ExternalFetcher obey robots.txt? Meta documents that Meta-ExternalFetcher may bypass robots.txt because it performs user-requested fetches. A disallow rule might not prevent it from accessing your site. ### How is Meta-ExternalFetcher different from other Meta crawlers? Unlike broad crawlers that scan many pages for indexing, Meta-ExternalFetcher fetches specific URLs only when a user action triggers it, such as sharing a link or using an AI feature. ### Will blocking Meta-ExternalFetcher affect my site's search ranking? Blocking this bot does not directly impact search engine rankings, but it may prevent your content from appearing in user-requested link previews or AI responses within Meta products. ### Can I verify that a visit is really from Meta-ExternalFetcher? You can check the user-agent string and perform reverse DNS lookups to confirm the request originates from Meta's IP ranges, as documented in their official web crawler information. ### Does Meta-ExternalFetcher cache fetched content? Meta may temporarily cache fetched content to improve performance for link previews and other features, but the specifics of caching are not detailed in their public documentation. ## Data And Sources - [Meta documentation](https://developers.facebook.com/docs/sharing/webmasters/web-crawlers/) - Primary source for Meta-ExternalFetcher crawler details. - [Meta-ExternalFetcher live crawler data](https://trakkr.ai/data/crawlers/meta-externalfetcher) - Trakkr crawler telemetry for this user agent.