Bot directory · AI training

AI training · Google

Google-Extended

Google-Extended is a robots.txt token, not a crawler. Disallowing it tells Google not to use pages it crawls from your site to train future Gemini models or to ground Gemini’s answers. Google says it doesn’t affect your inclusion or ranking in Google Search.

Checked against Google’s documentation on 19 October 2026

Operator
Google
Purpose
Model training. Crawlers that collect pages to train models. Blocking them does not affect search or AI answers.
robots.txt token
Google-Extended
We test with
None. This is a robots.txt token, not a crawler: the operator’s normal crawler fetches the page and this name only governs how it may be used.
Documentation
developers.google.com/crawling/docs/crawlers-fetchers/google-common-crawlers#google-extended

What Google-Extended does

Google-Extended is a standalone product token that web publishers can use to manage whether content Google crawls from their sites may be used for training future generations of Gemini models that power Gemini Apps and Vertex AI API for Gemini and for grounding (providing content from the Google Search index to the model at prompt time to improve factuality and relevancy) in Gemini Apps and Grounding with Google Search on Vertex AI.

developers.google.com

Google-Extended does not impact a site’s inclusion in Google Search nor is it used as a ranking signal in Google Search.

developers.google.com

AI Overviews and AI Mode in Google Search are part of Search, which Googlebot crawls for; Google-Extended governs Gemini Apps and Vertex AI.

The Google-Extended user agent

None. Google says Google-Extended has no user agent string of its own: Google’s existing crawlers fetch the pages, and the token is only read from robots.txt. You won’t see it in your logs.

How Google-Extended reads robots.txt

User-agent: Google-Extended
Disallow: /

Google-Extended IP addresses

Google-Extended has no addresses of its own. Google says its common crawlers generally crawl from the ranges in common-crawlers.json; to check one, use the Googlebot verifier.

Can you block Google-Extended at a firewall?

No. No request carries the name, and blocking Google’s crawlers at the firewall would remove you from Google Search. robots.txt is the only control.

Block Google-Extended

Add a group naming it to /robots.txt. Nothing stops fetching; the operator’s crawler still reads the page for its normal purpose, but may not use it this way. Googlebot keeps crawling and indexing your pages for Google Search; Google just won’t use them for Gemini training or grounding.

User-agent: Google-Extended
Disallow: /

Allow Google-Extended when everything else is blocked

A group naming the bot wins over User-agent: *, so an explicit allow lets it in while your catch-all rules stay in place.

User-agent: Google-Extended
Allow: /

User-agent: *
Disallow: /

Does Google-Extended reach your site?

The checker applies your robots.txt the way Google-Extended does, alongside 29 other bots, and opens the report on Google-Extended.

Questions about Google-Extended

What is Google-Extended?

A robots.txt token Google reads to decide whether content it crawls from your site may be used to train future Gemini models and to ground answers in Gemini Apps and Vertex AI. It isn’t a separate crawler.

Does blocking Google-Extended affect my Google rankings?

No. Google says Google-Extended does not affect a site’s inclusion in Google Search and isn’t used as a ranking signal.

Does Google-Extended stop AI Overviews using my site?

No. AI Overviews are part of Google Search, which Googlebot crawls for. Google-Extended covers Gemini Apps and Vertex AI.

Why don’t I see Google-Extended in my logs?

It has no user agent. Google’s regular crawlers fetch pages and the token is only read from robots.txt.

Also run by Google: Googlebot, GoogleOther.

Other ai training: GPTBot, ClaudeBot, CCBot, Applebot-Extended, GoogleOther, Meta-ExternalAgent, Bytespider.

Build a complete file with the robots.txt generator.