What is facebookexternalhit? AI crawler guide

facebookexternalhit by Meta: Meta link preview crawler used when content is shared on Meta family apps. Check its reported user-agent, robots.txt behavior, source, and verification guidance.

Meta link preview crawler used when content is shared on Meta family apps.

What is facebookexternalhit?

facebookexternalhit is Meta's web crawler that fetches pages when someone shares a link on Facebook, Messenger, Instagram, or other Meta family apps. It reads Open Graph tags and page metadata to build link previews with titles, descriptions, and images. The crawler identifies itself with the user-agent token facebookexternalhit and operates from Meta's infrastructure. It does not index pages for search or train AI models; its sole job is to generate rich previews at the moment a link is shared. Site owners can control its access through robots.txt, but blocking it will prevent previews from appearing correctly on Meta platforms.

What it's for

For site owners, facebookexternalhit is what makes shared links look engaging on Meta platforms. When it can access a page, it pulls the title, description, and image specified in Open Graph meta tags, which can increase click-through rates from social media. If the crawler is blocked, shared links may appear as plain URLs without any preview, which can reduce visibility and traffic from Facebook, Messenger, and Instagram. Allowing it ensures that content shared by users or pages looks professional and informative, supporting social media marketing and organic reach.

Allowing facebookexternalhit creates the possibility of retrieval. To find out whether Meta AI actually cites or recommends you, track your Meta AI mentions over time.

How to handle facebookexternalhit

To allow facebookexternalhit, ensure your site serves valid Open Graph meta tags and does not block the crawler in robots.txt. If you need to restrict access to certain paths, you can add a Disallow rule for the facebookexternalhit user-agent token, but be aware that this will prevent link previews for those URLs. The crawler respects robots.txt directives, so you can selectively allow or block it as needed. There is no need to verify the crawler via DNS or IP ranges; simply configure your robots.txt and meta tags accordingly.

robots.txt rule

User-agent: facebookexternalhit Disallow: /

Blocking cost

Blocking facebookexternalhit will prevent rich link previews from appearing when your content is shared on Meta platforms, which can reduce click-through rates and social traffic.

Examples

• A news article shared on Facebook shows a headline, summary, and lead image because facebookexternalhit successfully fetched the page's Open Graph tags.
• A product page shared in Messenger displays a blank preview because the site's robots.txt blocks facebookexternalhit, preventing the crawler from reading the metadata.
• An Instagram user shares a blog post, and the preview appears with the correct title and image after the site owner added Open Graph tags and allowed facebookexternalhit in robots.txt.

Related bots

• FacebookBot: Another Meta social crawler to compare.
• TikTokSpider: Also tracked as a social crawler.
• Meta-WebIndexer: Also operated by Meta.
• GPTBot: Shares crawler-policy context with facebookexternalhit.
• meta-externalads: Also operated by Meta.
• Meta-ExternalAgent: Also operated by Meta.
• AI2Bot: Shares crawler-policy context with facebookexternalhit.
• Applebot: Shares crawler-policy context with facebookexternalhit.
• atlassian-bot: Shares crawler-policy context with facebookexternalhit.
• Meta AI: facebookexternalhit connects this operator term to its crawler behavior.
• AI Training Opt-Out: facebookexternalhit gives crawler context for AI Training Opt-Out.