GPTBot
GPTBot is the crawler OpenAI uses to collect pages that may train its models. Blocking it is the documented way to keep your content out of OpenAI’s training data, and it doesn’t affect ChatGPT search, which uses a different crawler. Here is its user agent, its official IP ranges, and how to block or allow it.
- Operator
- OpenAI
- Purpose
- Model training. Crawlers that collect pages to train models. Blocking them does not affect search or AI answers.
- robots.txt token
GPTBot- We test with
Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko); compatible; GPTBot/1.4; +https://openai.com/gptbot
OpenAI’s example string, above.- Documentation
- developers.openai.com/api/docs/bots
What GPTBot does
It is used to crawl content that may be used in training our generative AI foundation models. Disallowing GPTBot indicates a site’s content should not be used in training generative AI foundation models.
developers.openai.com
OpenAI runs three bots, each with its own robots.txt token, and treats your choice for each separately:
| Bot | What it does | Model training |
|---|---|---|
GPTBot | Collects pages that may be used to train OpenAI’s models | Yes |
OAI-SearchBot | Indexes pages for ChatGPT search answers and links | No training sentence from OpenAI; it is the search control |
ChatGPT-User | Opens a page when a ChatGPT user or a custom GPT asks for it | No |
So you can block GPTBot and still appear in ChatGPT search, or the other way round. OpenAI also runs OAI-AdsBot, which checks pages submitted as ads and isn’t used for training.
The GPTBot user agent
| Request | User agent string |
|---|---|
| Pages | Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko); compatible; GPTBot/1.4; +https://openai.com/gptbot |
OpenAI labels this an example and says the version number may change, and that it may add a robots.txt marker to the string when it fetches robots.txt. Match on GPTBot, never the whole string.
How GPTBot reads robots.txt
- The token is
GPTBot. A group naming it overridesUser-agent: *for GPTBot. - It is independent of OpenAI’s other bots. Disallowing GPTBot doesn’t remove you from ChatGPT search; that is OAI-SearchBot.
- It is about training. OpenAI describes a disallow as indicating your content “should not be used in training generative AI foundation models”.
Each setting is independent of the others – for example, a webmaster can allow OAI-SearchBot in order to appear in search results while disallowing GPTBot to indicate that crawled content should not be used for training OpenAI’s generative AI foundation models. If your site has allowed both bots, we may use the results from just one crawl for both use cases to avoid duplicative crawling. For search results, please note it can take ~24 hours from a site’s robots.txt update for our systems to adjust.
developers.openai.com
To stay in ChatGPT search but out of training:
User-agent: GPTBot
Disallow: /
User-agent: OAI-SearchBot
Allow: /
GPTBot IP addresses and how to verify them
OpenAI publishes the addresses each of its bots crawls from as JSON, in the same format Google uses. It documents no reverse DNS names (the addresses we looked up had none), so the list is the way to verify GPTBot. Every list is IPv4 only.
| List | Covers | IPv4 | IPv6 | Dated |
|---|---|---|---|---|
| openai.com/gptbot.json | GPTBot | 18 | 0 | 22 September 2026 |
| openai.com/searchbot.json | OAI-SearchBot | 39 | 0 | 2 January 2026 |
| openai.com/chatgpt-user.json | ChatGPT-User | 234 | 0 | 7 October 2026 |
Fetch the lists automatically rather than copying them: OpenAI adds and removes ranges, and an allowlist built from an old copy turns real requests away.
Check an IP now
Paste an address from your logs. We match it against OpenAI’s current lists, along with the lists of the other operators that publish them.
Block GPTBot at your firewall or CDN
robots.txt asks; a firewall enforces. A rule on the user agent refuses GPTBot even if it doesn’t honour robots.txt. Match the name case-insensitively and anywhere in the string, never a full string or a version.
Cloudflare
In Security → WAF → Custom rules, create a rule with this expression and the action Block. Cloudflare’s contains is case-sensitive, hence lower().
(lower(http.user_agent) contains "gptbot")
Cloudflare also has one-click controls: AI bot policies, which block AI crawlers by what they do (search, agent or training), and AI Crawl Control, which lets you allow or block crawlers one by one and shows which of them fetch your pages.
nginx
# in the http block
map $http_user_agent $blocked_bot {
default 0;
~*gptbot 1;
}
# in the server block
if ($blocked_bot) { return 403; }
Block GPTBot
Add a group naming it to /robots.txt. Compliant crawlers stop before requesting any page. A firewall rule on the user agent is the only way to enforce it against crawlers that ignore robots.txt. This keeps your pages out of OpenAI’s training crawls. It doesn’t affect ChatGPT search (OAI-SearchBot) or pages ChatGPT opens for its users (ChatGPT-User).
User-agent: GPTBot
Disallow: /
Allow GPTBot when everything else is blocked
A group naming the bot wins over User-agent: *, so an explicit allow lets it in while your catch-all rules stay in place.
User-agent: GPTBot
Allow: /
User-agent: *
Disallow: /
Does GPTBot reach your site?
The checker applies your robots.txt the way GPTBot does, then requests your page with the user agent above, alongside 29 other bots, and opens the report on GPTBot.
Questions about GPTBot
What is GPTBot?
GPTBot is OpenAI’s web crawler for training data. OpenAI says it crawls content that may be used to train its generative AI foundation models, and that disallowing it in robots.txt indicates your content should not be used for that training.
Does blocking GPTBot remove my site from ChatGPT?
Not from ChatGPT search. Search uses OAI-SearchBot, which you allow or block separately, and pages ChatGPT opens for a user come from ChatGPT-User. OpenAI says each setting is independent.
What is the GPTBot user agent string?
OpenAI gives “Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko); compatible; GPTBot/1.4; +https://openai.com/gptbot” as an example and says the version may change. Match on GPTBot, case-insensitively.
What IP addresses does GPTBot use?
OpenAI publishes them at openai.com/gptbot.json. In October 2026 the list held 18 IPv4 ranges and no IPv6. Fetch it automatically rather than copying it, because the ranges change.
How do I know a request claiming to be GPTBot is real?
Check its IP against openai.com/gptbot.json. OpenAI documents no reverse DNS names for GPTBot, so the list is the way to verify it. The verifier on this page does it for you.
How long does it take GPTBot to notice a robots.txt change?
OpenAI only gives a figure for search: about 24 hours from a robots.txt update for its systems to adjust. Expect GPTBot to read a changed file on its next visit.
Also run by OpenAI: OAI-SearchBot, ChatGPT-User.
Other ai training: ClaudeBot, Google-Extended, CCBot, Applebot-Extended, GoogleOther, Meta-ExternalAgent, Bytespider.
Build a complete file with the robots.txt generator.