Who blocks AI · #450 in the Majestic Million

robots.txt · checked 9 Nov 2026

Does naver.com block AI crawlers?

Yes. naver.com’s robots.txt blocks all 19 AI crawlers we track, from GPTBot and ClaudeBot to PerplexityBot. It also blocks Googlebot, Bingbot and 5 more among search engines.

GPTBotBlockedOAI-SearchBotBlockedClaudeBotBlockedPerplexityBotBlockedGoogle-ExtendedBlockedCCBotBlocked
robots.txt
www.naver.com/robots.txt · 4 lines
llms.txt
none
Checked
9 November 2026, as canaicrawl. We check again every week.

Every bot, and the line that decides

Search engines

BotOperatorrobots.txtDeciding line
GooglebotGoogleBlockedline 2 · Disallow: / under User-agent: *
BingbotMicrosoftBlockedline 2 · Disallow: / under User-agent: *
DuckDuckBotDuckDuckGoBlockedline 2 · Disallow: / under User-agent: *
ApplebotAppleBlockedline 2 · Disallow: / under User-agent: *
PetalBotHuaweiBlockedline 2 · Disallow: / under User-agent: *
YandexBotYandexBlockedline 2 · Disallow: / under User-agent: *
BaiduspiderBaiduBlockedline 2 · Disallow: / under User-agent: *

AI search

BotOperatorrobots.txtDeciding line
OAI-SearchBotOpenAIBlockedline 2 · Disallow: / under User-agent: *
PerplexityBotPerplexityBlockedline 2 · Disallow: / under User-agent: *
Claude-SearchBotAnthropicBlockedline 2 · Disallow: / under User-agent: *
DuckAssistBotDuckDuckGoBlockedline 2 · Disallow: / under User-agent: *
AmazonbotAmazonBlockedline 2 · Disallow: / under User-agent: *
YouBotYou.comBlockedline 2 · Disallow: / under User-agent: *

AI training

BotOperatorrobots.txtDeciding line
GPTBotOpenAIBlockedline 2 · Disallow: / under User-agent: *
ClaudeBotAnthropicBlockedline 2 · Disallow: / under User-agent: *
Google-ExtendedGoogleBlockedline 2 · Disallow: / under User-agent: *
CCBotCommon CrawlBlockedline 2 · Disallow: / under User-agent: *
Applebot-ExtendedAppleBlockedline 2 · Disallow: / under User-agent: *
GoogleOtherGoogleBlockedline 2 · Disallow: / under User-agent: *
Meta-ExternalAgentMetaBlockedline 2 · Disallow: / under User-agent: *
BytespiderByteDanceBlockedline 2 · Disallow: / under User-agent: *

User-triggered fetchers

BotOperatorrobots.txtDeciding line
ChatGPT-UserOpenAIBlockedline 2 · Disallow: / under User-agent: *
Claude-UserAnthropicBlockedline 2 · Disallow: / under User-agent: *
Perplexity-UserPerplexityBlockedline 2 · Disallow: / under User-agent: *
MistralAI-UserMistralBlockedline 2 · Disallow: / under User-agent: *
Meta-ExternalFetcherMetaBlockedline 2 · Disallow: / under User-agent: *

SEO tools

BotOperatorrobots.txtDeciding line
AhrefsBotAhrefsBlockedline 2 · Disallow: / under User-agent: *
SemrushBotSemrushBlockedline 2 · Disallow: / under User-agent: *
MJ12botMajesticBlockedline 2 · Disallow: / under User-agent: *
DotBotMozBlockedline 2 · Disallow: / under User-agent: *
View www.naver.com/robots.txt · 4 lines
1User-agent: *
2Disallow: /
3Allow : /$
4Allow : /.well-known/privacy-sandbox-attestations.json

What do the bots actually get?

This page reads robots.txt, which states what naver.com asks of crawlers. A firewall or CDN can still refuse a bot that robots.txt allows. The live check requests the page as each bot and shows the status it gets.

Run a live check on naver.com Test a path in the robots.txt tester

History

Questions

Does naver.com block GPTBot?

Yes. naver.com’s robots.txt disallows GPTBot from the whole site (line 2: Disallow: /). GPTBot is OpenAI’s crawler for training data; blocking it doesn’t affect ChatGPT search, which uses OAI-SearchBot. Checked 9 November 2026.

Does naver.com block ClaudeBot?

Yes. naver.com’s robots.txt disallows ClaudeBot from the whole site (line 2: Disallow: /). ClaudeBot is Anthropic’s crawler for training data. Checked 9 November 2026.

Does naver.com block Google-Extended?

Yes. naver.com’s robots.txt disallows Google-Extended from the whole site (line 2: Disallow: /). Google-Extended is the token that controls whether Google may use pages for Gemini training and grounding; it doesn’t affect Google Search. Checked 9 November 2026.

Does naver.com block PerplexityBot?

Yes. naver.com’s robots.txt disallows PerplexityBot from the whole site (line 2: Disallow: /). PerplexityBot is the crawler that indexes pages for Perplexity’s answers. Checked 9 November 2026.

Can ChatGPT search find naver.com?

Not through its own crawler. naver.com’s robots.txt disallows OAI-SearchBot from the whole site (line 2: Disallow: /). OAI-SearchBot is the crawler behind ChatGPT search; blocking it keeps the site out of ChatGPT’s search answers. Checked 9 November 2026.