Bot directory · AI training

AI training · OpenAI

GPTBot

GPTBot is the crawler OpenAI uses to collect pages that may train its models. Blocking it is the documented way to keep your content out of OpenAI’s training data, and it doesn’t affect ChatGPT search, which uses a different crawler. Here is its user agent, its official IP ranges, and how to block or allow it.

Checked against OpenAI’s documentation on 19 October 2026

Operator
OpenAI
Purpose
Model training. Crawlers that collect pages to train models. Blocking them does not affect search or AI answers.
robots.txt token
GPTBot
We test with
Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko); compatible; GPTBot/1.4; +https://openai.com/gptbot
OpenAI’s example string, above.
Documentation
developers.openai.com/api/docs/bots

What GPTBot does

It is used to crawl content that may be used in training our generative AI foundation models. Disallowing GPTBot indicates a site’s content should not be used in training generative AI foundation models.

developers.openai.com

OpenAI runs three bots, each with its own robots.txt token, and treats your choice for each separately:

BotWhat it doesModel training
GPTBotCollects pages that may be used to train OpenAI’s modelsYes
OAI-SearchBotIndexes pages for ChatGPT search answers and linksNo training sentence from OpenAI; it is the search control
ChatGPT-UserOpens a page when a ChatGPT user or a custom GPT asks for itNo

So you can block GPTBot and still appear in ChatGPT search, or the other way round. OpenAI also runs OAI-AdsBot, which checks pages submitted as ads and isn’t used for training.

The GPTBot user agent

RequestUser agent string
PagesMozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko); compatible; GPTBot/1.4; +https://openai.com/gptbot

OpenAI labels this an example and says the version number may change, and that it may add a robots.txt marker to the string when it fetches robots.txt. Match on GPTBot, never the whole string.

How GPTBot reads robots.txt

Each setting is independent of the others – for example, a webmaster can allow OAI-SearchBot in order to appear in search results while disallowing GPTBot to indicate that crawled content should not be used for training OpenAI’s generative AI foundation models. If your site has allowed both bots, we may use the results from just one crawl for both use cases to avoid duplicative crawling. For search results, please note it can take ~24 hours from a site’s robots.txt update for our systems to adjust.

developers.openai.com

To stay in ChatGPT search but out of training:

User-agent: GPTBot
Disallow: /

User-agent: OAI-SearchBot
Allow: /

GPTBot IP addresses and how to verify them

OpenAI publishes the addresses each of its bots crawls from as JSON, in the same format Google uses. It documents no reverse DNS names (the addresses we looked up had none), so the list is the way to verify GPTBot. Every list is IPv4 only.

ListCoversIPv4IPv6Dated
openai.com/gptbot.jsonGPTBot18022 September 2026
openai.com/searchbot.jsonOAI-SearchBot3902 January 2026
openai.com/chatgpt-user.jsonChatGPT-User23407 October 2026

Fetch the lists automatically rather than copying them: OpenAI adds and removes ranges, and an allowlist built from an old copy turns real requests away.

Check an IP now

Paste an address from your logs. We match it against OpenAI’s current lists, along with the lists of the other operators that publish them.

Block GPTBot at your firewall or CDN

robots.txt asks; a firewall enforces. A rule on the user agent refuses GPTBot even if it doesn’t honour robots.txt. Match the name case-insensitively and anywhere in the string, never a full string or a version.

Cloudflare

In Security → WAF → Custom rules, create a rule with this expression and the action Block. Cloudflare’s contains is case-sensitive, hence lower().

(lower(http.user_agent) contains "gptbot")

Cloudflare also has one-click controls: AI bot policies, which block AI crawlers by what they do (search, agent or training), and AI Crawl Control, which lets you allow or block crawlers one by one and shows which of them fetch your pages.

nginx

# in the http block
map $http_user_agent $blocked_bot {
    default 0;
    ~*gptbot 1;
}

# in the server block
if ($blocked_bot) { return 403; }

Block GPTBot

Add a group naming it to /robots.txt. Compliant crawlers stop before requesting any page. A firewall rule on the user agent is the only way to enforce it against crawlers that ignore robots.txt. This keeps your pages out of OpenAI’s training crawls. It doesn’t affect ChatGPT search (OAI-SearchBot) or pages ChatGPT opens for its users (ChatGPT-User).

User-agent: GPTBot
Disallow: /

Allow GPTBot when everything else is blocked

A group naming the bot wins over User-agent: *, so an explicit allow lets it in while your catch-all rules stay in place.

User-agent: GPTBot
Allow: /

User-agent: *
Disallow: /

Does GPTBot reach your site?

The checker applies your robots.txt the way GPTBot does, then requests your page with the user agent above, alongside 29 other bots, and opens the report on GPTBot.

Questions about GPTBot

What is GPTBot?

GPTBot is OpenAI’s web crawler for training data. OpenAI says it crawls content that may be used to train its generative AI foundation models, and that disallowing it in robots.txt indicates your content should not be used for that training.

Does blocking GPTBot remove my site from ChatGPT?

Not from ChatGPT search. Search uses OAI-SearchBot, which you allow or block separately, and pages ChatGPT opens for a user come from ChatGPT-User. OpenAI says each setting is independent.

What is the GPTBot user agent string?

OpenAI gives “Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko); compatible; GPTBot/1.4; +https://openai.com/gptbot” as an example and says the version may change. Match on GPTBot, case-insensitively.

What IP addresses does GPTBot use?

OpenAI publishes them at openai.com/gptbot.json. In October 2026 the list held 18 IPv4 ranges and no IPv6. Fetch it automatically rather than copying it, because the ranges change.

How do I know a request claiming to be GPTBot is real?

Check its IP against openai.com/gptbot.json. OpenAI documents no reverse DNS names for GPTBot, so the list is the way to verify it. The verifier on this page does it for you.

How long does it take GPTBot to notice a robots.txt change?

OpenAI only gives a figure for search: about 24 hours from a robots.txt update for its systems to adjust. Expect GPTBot to read a changed file on its next visit.

Also run by OpenAI: OAI-SearchBot, ChatGPT-User.

Other ai training: ClaudeBot, Google-Extended, CCBot, Applebot-Extended, GoogleOther, Meta-ExternalAgent, Bytespider.

Build a complete file with the robots.txt generator.