Bot directory · AI training

AI training · Apple

Applebot-Extended

Applebot-Extended isn’t a crawler. It is a name you put in robots.txt to tell Apple whether pages its crawler, Applebot, fetched may be used to train Apple’s foundation models. Disallowing it doesn’t remove your site from Siri, Spotlight or Safari.

Checked against Apple’s documentation on 19 October 2026

Operator
Apple
Purpose
Model training. Crawlers that collect pages to train models. Blocking them does not affect search or AI answers.
robots.txt token
Applebot-Extended
We test with
None. This is a robots.txt token, not a crawler: the operator’s normal crawler fetches the page and this name only governs how it may be used.
Documentation
support.apple.com/en-us/119829

What Applebot-Extended does

Applebot-Extended does not crawl webpages. Webpages that disallow Applebot-Extended can still be included in search results. Applebot-Extended is only used to determine how to use the data crawled by the Applebot user agent.

support.apple.com

The data crawled by Applebot may also be used to help train Apple foundation models powering generative AI features across Apple products, including Apple Intelligence, Services, and Developer Tools. Web publishers can opt-out from having their content used to train generative foundation models by disallowing Applebot-Extended in the robots.txt file.

support.apple.com

Apple also says rules for Applebot-Extended aren’t considered in ranking for search.

The Applebot-Extended user agent

None. No request ever arrives as Applebot-Extended: Applebot fetches the pages, and this token only governs what Apple may do with them. You won’t see it in your logs.

How Applebot-Extended reads robots.txt

Add a group naming it:

User-agent: Applebot-Extended
Disallow: /

Applebot-Extended IP addresses

Applebot-Extended has no addresses of its own, because it sends no requests. Applebot’s addresses are on the Applebot page, with a verifier.

Can you block Applebot-Extended at a firewall?

No. A firewall sees requests, and none carry this name. Blocking Applebot at the firewall would also remove you from Siri, Spotlight and Safari. Use robots.txt.

Block Applebot-Extended

Add a group naming it to /robots.txt. Nothing stops fetching; the operator’s crawler still reads the page for its normal purpose, but may not use it this way. Applebot keeps crawling for Siri, Spotlight and Safari; Apple just won’t use the pages to train its models.

User-agent: Applebot-Extended
Disallow: /

Allow Applebot-Extended when everything else is blocked

A group naming the bot wins over User-agent: *, so an explicit allow lets it in while your catch-all rules stay in place.

User-agent: Applebot-Extended
Allow: /

User-agent: *
Disallow: /

Does Applebot-Extended reach your site?

The checker applies your robots.txt the way Applebot-Extended does, alongside 29 other bots, and opens the report on Applebot-Extended.

Questions about Applebot-Extended

What is Applebot-Extended?

A robots.txt token Apple uses to decide whether pages Applebot crawled may be used to train its generative foundation models. Apple says Applebot-Extended doesn’t crawl webpages itself.

Does disallowing Applebot-Extended remove my site from Siri or Spotlight?

No. Apple says pages that disallow Applebot-Extended can still be included in search results, and its rules aren’t considered in search ranking.

Can I block Applebot-Extended at my firewall?

No requests use that name, so there is nothing to block. The only control is a robots.txt group naming it.

Also run by Apple: Applebot.

Other ai training: GPTBot, ClaudeBot, Google-Extended, CCBot, GoogleOther, Meta-ExternalAgent, Bytespider.

Build a complete file with the robots.txt generator.