User-triggered fetchers · Meta

Meta-ExternalFetcher

Fetches a page when a user of Meta AI asks about it. Meta says it may bypass robots.txt because a person asked.

Operator: Meta
Purpose: Live user request. Fetchers that load a page because a person asked an assistant about it. Blocking one means the assistant can’t open your pages on request.
robots.txt token: Meta-ExternalFetcher
User agent: meta-externalfetcher/1.1
Documentation: developers.facebook.com/docs/sharing/webmasters/web-crawlers

Block Meta-ExternalFetcher

Add a group naming it to /robots.txt. Compliant crawlers stop before requesting any page. A firewall rule on the user agent is the only way to enforce it against crawlers that ignore robots.txt.

User-agent: Meta-ExternalFetcher
Disallow: /

Allow Meta-ExternalFetcher when everything else is blocked

A group naming the bot wins over User-agent: *, so an explicit allow lets it in while your catch-all rules stay in place.

User-agent: Meta-ExternalFetcher
Allow: /

User-agent: *
Disallow: /

Does Meta-ExternalFetcher reach your site?

The checker applies your robots.txt the way Meta-ExternalFetcher does, then requests your page with the user agent above, alongside 29 other bots.

Also run by Meta: Meta-ExternalAgent.

Other user-triggered fetchers: ChatGPT-User, Claude-User, Perplexity-User, MistralAI-User.

Build a complete file with the robots.txt generator.