Who blocks AI · #281 in the Majestic Million
robots.txt · checked 9 Nov 2026
Does netflix.com block AI crawlers?
Yes. netflix.com’s robots.txt blocks 11 of the 19 AI crawlers we track, including CCBot, Meta-ExternalAgent, Applebot-Extended and 8 more. It also blocks DuckDuckBot and PetalBot among search engines.
GPTBotPartly blockedOAI-SearchBotPartly blockedClaudeBotPartly blockedPerplexityBotPartly blockedGoogle-ExtendedPartly blockedCCBotBlocked- robots.txt
- www.netflix.com/robots.txt · 181 lines
- llms.txt
- none
- Checked
- 9 November 2026, as canaicrawl. We check again every week.
Every bot, and the line that decides
Search engines
| Bot | Operator | robots.txt | Deciding line |
|---|---|---|---|
Googlebot | Partly blocked | line 34 · Disallow: /accountstatus line 35 · Disallow: /AccountStatus and 129 more | |
Bingbot | Microsoft | Partly blocked | line 34 · Disallow: /accountstatus line 35 · Disallow: /AccountStatus and 129 more |
DuckDuckBot | DuckDuckGo | Blocked | line 2 · Disallow: / under User-agent: * |
Applebot | Apple | Partly blocked | line 34 · Disallow: /accountstatus line 35 · Disallow: /AccountStatus and 129 more |
PetalBot | Huawei | Blocked | line 2 · Disallow: / under User-agent: * |
YandexBot | Yandex | Partly blocked | line 34 · Disallow: /accountstatus line 35 · Disallow: /AccountStatus and 129 more |
Baiduspider | Baidu | Partly blocked | line 34 · Disallow: /accountstatus line 35 · Disallow: /AccountStatus and 129 more |
AI search
| Bot | Operator | robots.txt | Deciding line |
|---|---|---|---|
OAI-SearchBot | OpenAI | Partly blocked | line 34 · Disallow: /accountstatus line 35 · Disallow: /AccountStatus and 129 more |
PerplexityBot | Perplexity | Partly blocked | line 34 · Disallow: /accountstatus line 35 · Disallow: /AccountStatus and 129 more |
Claude-SearchBot | Anthropic | Partly blocked | line 34 · Disallow: /accountstatus line 35 · Disallow: /AccountStatus and 129 more |
DuckAssistBot | DuckDuckGo | Blocked | line 2 · Disallow: / under User-agent: * |
Amazonbot | Amazon | Blocked | line 2 · Disallow: / under User-agent: * |
YouBot | You.com | Blocked | line 2 · Disallow: / under User-agent: * |
AI training
| Bot | Operator | robots.txt | Deciding line |
|---|---|---|---|
GPTBot | OpenAI | Partly blocked | line 34 · Disallow: /accountstatus line 35 · Disallow: /AccountStatus and 129 more |
ClaudeBot | Anthropic | Partly blocked | line 34 · Disallow: /accountstatus line 35 · Disallow: /AccountStatus and 129 more |
Google-Extended | Partly blocked | line 34 · Disallow: /accountstatus line 35 · Disallow: /AccountStatus and 129 more | |
CCBot | Common Crawl | Blocked | line 2 · Disallow: / under User-agent: * |
Applebot-Extended | Apple | Blocked | line 2 · Disallow: / under User-agent: * |
GoogleOther | Blocked | line 2 · Disallow: / under User-agent: * | |
Meta-ExternalAgent | Meta | Blocked | line 2 · Disallow: / under User-agent: * |
Bytespider | ByteDance | Blocked | line 2 · Disallow: / under User-agent: * |
User-triggered fetchers
| Bot | Operator | robots.txt | Deciding line |
|---|---|---|---|
ChatGPT-User | OpenAI | Partly blocked | line 34 · Disallow: /accountstatus line 35 · Disallow: /AccountStatus and 129 more |
Claude-User | Anthropic | Partly blocked | line 34 · Disallow: /accountstatus line 35 · Disallow: /AccountStatus and 129 more |
Perplexity-User | Perplexity | Blocked | line 2 · Disallow: / under User-agent: * |
MistralAI-User | Mistral | Blocked | line 2 · Disallow: / under User-agent: * |
Meta-ExternalFetcher | Meta | Blocked | line 2 · Disallow: / under User-agent: * |
SEO tools
| Bot | Operator | robots.txt | Deciding line |
|---|---|---|---|
AhrefsBot | Ahrefs | Blocked | line 2 · Disallow: / under User-agent: * |
SemrushBot | Semrush | Blocked | line 2 · Disallow: / under User-agent: * |
MJ12bot | Majestic | Blocked | line 2 · Disallow: / under User-agent: * |
DotBot | Moz | Blocked | line 2 · Disallow: / under User-agent: * |
View www.netflix.com/robots.txt · 181 lines
1User-agent: *
2Disallow: /
3
4User-agent: googlebot
5User-agent: Googlebot-Video
6User-agent: Applebot
7User-agent: bingbot
8User-agent: Baiduspider
9User-agent: Baiduspider-mobile
10User-agent: Baiduspider-video
11User-agent: Baiduspider-image
12User-agent: NaverBot
13User-agent: Y!J-WSC
14User-agent: YJ-WSC
15User-agent: Yeti
16User-agent: Yandex
17User-agent: YandexBot
18User-agent: YandexMobileBot
19User-agent: YandexVideo
20User-agent: YandexWebmaster
21User-agent: YandexSitelinks
22User-agent: SeznamBot
23User-agent: facebookexternalhit
24User-agent: GPTBot
25User-agent: ChatGPT-User
26User-agent: OAI-SearchBot
27User-agent: Google-Extended
28User-agent: ClaudeBot
29User-agent: Claude-User
30User-agent: Claude-SearchBot
31User-agent: PerplexityBot
32Allow: /
33
34Disallow: /accountstatus
35Disallow: /AccountStatus
36Disallow: /aui/inbound
37Disallow: /authenticate
38Disallow: /autologin
39Disallow: /clearcookies
40Disallow: /companies
41Disallow: /editpayment
42Disallow: /emailunsubscribe
43Disallow: /error
44Disallow: /eula
45Disallow: /geooverride
46Disallow: /help
47Disallow: /imagelibrary
48Disallow: /learnmorelayer
49Disallow: /learnmorelayertv
50Disallow: /login
51Disallow: /loginhelp
52Disallow: /loginhelp/lookup
53Disallow: /loginhelpsucess
54Disallow: /LoginHelp
55Disallow: /password
56Disallow: /logout
57Disallow: /Logout
58Disallow: /mcd
59Disallow: /modernizr
60Disallow: /n/
61Disallow: /notamember
62Disallow: /notfound
63Disallow: /notices
64Disallow: /nrdapp
65Disallow: /optout
66Disallow: /overviewblockseeother
67Disallow: /popup/codewhatisthis
68Disallow: /popupdetails
69Disallow: /PopupDetails
70Disallow: /popupprivacypolicy
71Disallow: /privacypolicychanges
72Disallow: /registration
73Disallow: /rememberme
74Disallow: /signout
75Disallow: /signurl
76Disallow: /subscriptioncancel
77Disallow: /tastesurvey
78Disallow: /termsofusechanges
79Disallow: /tvsignup
80Disallow: /upcomingevents
81Disallow: /verifyidentity
82Disallow: /whysecure
83
84Disallow: /arabic
85Disallow: /Arabic
86Disallow: /chinese
87Disallow: /Chinese
88Disallow: /korean
89Disallow: /Korean
90
91Disallow: /airtel
92Disallow: /anan
93Disallow: /bouyguestelecom
94Disallow: /britishairways
95Disallow: /brutus
96Disallow: /comhem
97Disallow: /courts
98Disallow: /csl
99Disallow: /elisa
100Disallow: /entertain
101Disallow: /FireTV
102Disallow: /firetv
103Disallow: /freemonth
104Disallow: /kpn
105Disallow: /lg
106Disallow: /maxis
107Disallow: /Maxis
108Disallow: /meo
109Disallow: /Meo
110Disallow: /orangefrance
111Disallow: /Panasonic
112Disallow: /panasonic
113Disallow: /playstation
114Disallow: /proximus
115Disallow: /qantas
116Disallow: /samsung
117Disallow: /Sony
118Disallow: /sony
119Disallow: /talktalk
120Disallow: /tdc
121Disallow: /telenor
122Disallow: /telfort
123Disallow: /tim
124Disallow: /virginaustralia
125Disallow: /vodafone
126Disallow: /vodafonedemobilelaunch
127Disallow: /xboxone
128Disallow: /xfinity
129Disallow: /xs4all
130Disallow: /ziggo
131
132Disallow: /accountaccess
133Disallow: /AccountAccess
134Disallow: /BillingActivity
135Disallow: /browse
136Disallow: /browse/*
137Allow: /browse/genre/*
138Disallow: /CancelPlan
139Disallow: /ChangePlan
140Disallow: /changeplan
141Disallow: /deviceManagement
142Disallow: /DoNotTest
143Disallow: /EditProfiles
144Disallow: /email
145Disallow: /EmailPreferences
146Disallow: /entrytrap
147Disallow: /HdToggle
148Disallow: /LanguagePreferences
149Disallow: /ManageDevices
150Disallow: /ManageProfiles
151Disallow: /MoviesYouveSeen
152Disallow: /NewWatchInstantlyRSS
153Disallow: /NewWatchInstantlyRSS/*
154Disallow: /payment
155Disallow: /Payment
156Disallow: /phonenumber
157Disallow: /pin
158Disallow: /profiles
159Disallow: /profiles/*
160Disallow: /ProfilesGate
161Disallow: /search
162Disallow: /search/*
163Disallow: /viewingactivity
164Disallow: /WiViewingActivity
165Disallow: /yourAccount
166Disallow: /youraccount
167Disallow: /YourAccount
168Disallow: /YourAccountPayment
169
170User-agent: AdsBot-Google
171User-agent: Twitterbot
172User-agent: Adidxbot
173Allow: /
174
175User-agent: Yahoo Pipes 1.0
176User-agent: Facebot
177Disallow: /
178Allow: /tudum
179
180Sitemap: https://www.netflix.com/sitemap/index
181
What do the bots actually get?
This page reads robots.txt, which states what netflix.com asks of crawlers. A firewall or CDN can still refuse a bot that robots.txt allows. The live check requests the page as each bot and shows the status it gets.
Run a live check on netflix.com Test a path in the robots.txt tester
History
- 19 October 2026: first checked.
Questions
Does netflix.com block GPTBot?
Partly. netflix.com’s robots.txt keeps GPTBot out of some paths, such as /accountstatus, and lets it crawl the rest. GPTBot is OpenAI’s crawler for training data; blocking it doesn’t affect ChatGPT search, which uses OAI-SearchBot. Checked 9 November 2026.
Does netflix.com block ClaudeBot?
Partly. netflix.com’s robots.txt keeps ClaudeBot out of some paths, such as /accountstatus, and lets it crawl the rest. ClaudeBot is Anthropic’s crawler for training data. Checked 9 November 2026.
Does netflix.com block Google-Extended?
Partly. netflix.com’s robots.txt keeps Google-Extended out of some paths, such as /accountstatus, and lets it crawl the rest. Google-Extended is the token that controls whether Google may use pages for Gemini training and grounding; it doesn’t affect Google Search. Checked 9 November 2026.
Does netflix.com block PerplexityBot?
Partly. netflix.com’s robots.txt keeps PerplexityBot out of some paths, such as /accountstatus, and lets it crawl the rest. PerplexityBot is the crawler that indexes pages for Perplexity’s answers. Checked 9 November 2026.
Can ChatGPT search find netflix.com?
Partly. netflix.com’s robots.txt keeps OAI-SearchBot out of some paths, such as /accountstatus, and lets it crawl the rest. OAI-SearchBot is the crawler behind ChatGPT search; blocking it keeps the site out of ChatGPT’s search answers. Checked 9 November 2026.