Who blocks AI · #899 in the Majestic Million

robots.txt · checked 9 Nov 2026

Does prezi.com block AI crawlers?

Yes. prezi.com’s robots.txt blocks 6 of the 19 AI crawlers we track, including GPTBot, ClaudeBot, CCBot and 3 more. Search engines such as Googlebot and Bingbot may crawl it.

GPTBotBlocked OAI-SearchBotPartly blocked ClaudeBotBlocked PerplexityBotAllowed Google-ExtendedAllowed CCBotBlocked

robots.txt: prezi.com/robots.txt · 66 lines
llms.txt: found
Checked: 9 November 2026, as canaicrawl. We check again every week.

Every bot, and the line that decides

Search engines

Bot	Operator	robots.txt	Deciding line
`Googlebot`	Google	Allowed	no rule matches the home page
`Bingbot`	Microsoft	Allowed	no rule matches the home page
`DuckDuckBot`	DuckDuckGo	Allowed	no rule matches the home page
`Applebot`	Apple	Allowed	no rule matches the home page
`PetalBot`	Huawei	Allowed	no rule matches the home page
`YandexBot`	Yandex	Allowed	no rule matches the home page
`Baiduspider`	Baidu	Allowed	no rule matches the home page

AI search

Bot	Operator	robots.txt	Deciding line
`OAI-SearchBot`	OpenAI	Partly blocked	line 23 · `Disallow: /cms/` line 24 · `Disallow: /explore/search/` and 2 more
`PerplexityBot`	Perplexity	Allowed	no rule matches the home page
`Claude-SearchBot`	Anthropic	Partly blocked	line 23 · `Disallow: /cms/` line 24 · `Disallow: /explore/search/` and 2 more
`DuckAssistBot`	DuckDuckGo	Allowed	no rule matches the home page
`Amazonbot`	Amazon	Allowed	no rule matches the home page
`YouBot`	You.com	Allowed	no rule matches the home page

AI training

Bot	Operator	robots.txt	Deciding line
`GPTBot`	OpenAI	Blocked	line 11 · `Disallow: /` under `User-agent: GPTBot`
`ClaudeBot`	Anthropic	Blocked	line 8 · `Disallow: /` under `User-agent: ClaudeBot`
`Google-Extended`	Google	Allowed	no rule matches the home page
`CCBot`	Common Crawl	Blocked	line 18 · `Disallow: /` under `User-agent: CCBot`
`Applebot-Extended`	Apple	Blocked	line 18 · `Disallow: /` under `User-agent: Applebot-Extended`
`GoogleOther`	Google	Allowed	no rule matches the home page
`Meta-ExternalAgent`	Meta	Blocked	line 18 · `Disallow: /` under `User-agent: meta-externalagent`
`Bytespider`	ByteDance	Blocked	line 18 · `Disallow: /` under `User-agent: Bytespider`

User-triggered fetchers

Bot	Operator	robots.txt	Deciding line
`ChatGPT-User`	OpenAI	Allowed	no rule matches the home page
`Claude-User`	Anthropic	Partly blocked	line 30 · `Disallow: /cms/` line 31 · `Disallow: /explore/search/` and 2 more
`Perplexity-User`	Perplexity	Allowed	no rule matches the home page
`MistralAI-User`	Mistral	Allowed	no rule matches the home page
`Meta-ExternalFetcher`	Meta	Allowed	no rule matches the home page

SEO tools

Bot	Operator	robots.txt	Deciding line
`AhrefsBot`	Ahrefs	Allowed	no rule matches the home page
`SemrushBot`	Semrush	Allowed	no rule matches the home page
`MJ12bot`	Majestic	Allowed	no rule matches the home page
`DotBot`	Moz	Allowed	no rule matches the home page

View prezi.com/robots.txt · 66 lines

1User-agent: *

2Disallow: /cms/

3Disallow: /explore/search/

4Disallow: /invoice/download/

5Disallow: /login/

7User-agent: ClaudeBot

8Disallow: /

10User-agent: GPTBot

11Disallow: /

13User-agent: anthropic-ai

14User-agent: Applebot-Extended

15User-agent: Bytespider

16User-agent: CCBot

17User-agent: meta-externalagent

18Disallow: /

20User-agent: Claude-SearchBot

21User-agent: OAI-SearchBot

22Allow: /

23Disallow: /cms/

24Disallow: /explore/search/

25Disallow: /invoice/download/

26Disallow: /login/

28User-agent: Claude-User

29Allow: /

30Disallow: /cms/

31Disallow: /explore/search/

32Disallow: /invoice/download/

33Disallow: /login/

35User-agent: OAI-AdsBot

36Allow: /

37Disallow: /cms/

38Disallow: /explore/search/

39Disallow: /invoice/download/

40Disallow: /login/

42User-agent: AdsBot-Google

43User-agent: AdsBot-Google-Mobile

44Allow: /

45Disallow: /cms/

46Disallow: /explore/search/

47Disallow: /invoice/download/

48Disallow: /login/

50Sitemap: https://prezi.com/sitemap_seo_landing.xml

51Sitemap: https://prezi.com/sitemap_next-prezis_index.xml

52Sitemap: https://prezi.com/sitemap_classic-prezis_index.xml

53Sitemap: https://prezi.com/sitemap_videos_index.xml

54Sitemap: https://prezi.com/sitemap_marketing_index.xml

55Sitemap: https://prezi.com/sitemap_design_index.xml

56Sitemap: https://prezi.com/sitemap_craft-pages_index.xml

57Sitemap: https://prezi.com/sitemap_blog.xml

58Sitemap: https://prezi.com/sitemap_craft-pages_20260324_0.xml

59Sitemap: https://prezi.com/sitemap_craft-pages_20260424_0.xml

60Sitemap: https://prezi.com/sitemap_craft-pages_20260525_0.xml

61Sitemap: https://prezi.com/sitemap_craft-pages_20260625_0.xml

62Sitemap: https://prezi.com/sitemap_craft-pages_20260727_0.xml

63Sitemap: https://prezi.com/sitemap_craft-pages_20260827_0.xml

64Sitemap: https://prezi.com/sitemap_craft-pages_20260927_0.xml

65Sitemap: https://prezi.com/sitemap_craft-pages_20261028_0.xml

What do the bots actually get?

This page reads robots.txt, which states what prezi.com asks of crawlers. A firewall or CDN can still refuse a bot that robots.txt allows. The live check requests the page as each bot and shows the status it gets.

Run a live check on prezi.com Test a path in the robots.txt tester

History

19 October 2026: first checked.
2 November 2026: now blocks part of the site to OAI-SearchBot; now blocks part of the site to Claude-SearchBot; started blocking CCBot; started blocking Applebot-Extended; started blocking Meta-ExternalAgent; started blocking Bytespider; now blocks part of the site to Claude-User.

Questions

Does prezi.com block GPTBot?

Yes. prezi.com’s robots.txt disallows GPTBot from the whole site (line 11: Disallow: /). GPTBot is OpenAI’s crawler for training data; blocking it doesn’t affect ChatGPT search, which uses OAI-SearchBot. Checked 9 November 2026.

Does prezi.com block ClaudeBot?

Yes. prezi.com’s robots.txt disallows ClaudeBot from the whole site (line 8: Disallow: /). ClaudeBot is Anthropic’s crawler for training data. Checked 9 November 2026.

Does prezi.com block Google-Extended?

No. prezi.com’s robots.txt lets Google-Extended crawl the site. Google-Extended is the token that controls whether Google may use pages for Gemini training and grounding; it doesn’t affect Google Search. Checked 9 November 2026.

Does prezi.com block PerplexityBot?

No. prezi.com’s robots.txt lets PerplexityBot crawl the site. PerplexityBot is the crawler that indexes pages for Perplexity’s answers. Checked 9 November 2026.

Can ChatGPT search find prezi.com?

Partly. prezi.com’s robots.txt keeps OAI-SearchBot out of some paths, such as /cms/, and lets it crawl the rest. OAI-SearchBot is the crawler behind ChatGPT search; blocking it keeps the site out of ChatGPT’s search answers. Checked 9 November 2026.