Who blocks AI · #701 in the Majestic Million
robots.txt · checked 9 Nov 2026
Does yandex.com block AI crawlers?
No. yandex.com’s robots.txt lets all 19 AI crawlers we track crawl the site. Search engines such as Googlebot and Bingbot may crawl it.
GPTBotAllowedOAI-SearchBotAllowedClaudeBotAllowedPerplexityBotAllowedGoogle-ExtendedAllowedCCBotAllowed- robots.txt
- yandex.com/robots.txt · 843 lines
- llms.txt
- none
- Checked
- 9 November 2026, as canaicrawl. We check again every week.
Every bot, and the line that decides
Search engines
| Bot | Operator | robots.txt | Deciding line |
|---|---|---|---|
Googlebot | Allowed | no rule matches the home page | |
Bingbot | Microsoft | Allowed | no rule matches the home page |
DuckDuckBot | DuckDuckGo | Allowed | no rule matches the home page |
Applebot | Apple | Partly blocked | line 596 · Disallow: /? line 597 · Disallow: /403.html and 172 more |
PetalBot | Huawei | Allowed | no rule matches the home page |
YandexBot | Yandex | Allowed | no rule matches the home page |
Baiduspider | Baidu | Allowed | no rule matches the home page |
AI search
| Bot | Operator | robots.txt | Deciding line |
|---|---|---|---|
OAI-SearchBot | OpenAI | Allowed | no rule matches the home page |
PerplexityBot | Perplexity | Allowed | no rule matches the home page |
Claude-SearchBot | Anthropic | Allowed | no rule matches the home page |
DuckAssistBot | DuckDuckGo | Allowed | no rule matches the home page |
Amazonbot | Amazon | Allowed | no rule matches the home page |
YouBot | You.com | Allowed | no rule matches the home page |
AI training
| Bot | Operator | robots.txt | Deciding line |
|---|---|---|---|
GPTBot | OpenAI | Allowed | no rule matches the home page |
ClaudeBot | Anthropic | Allowed | no rule matches the home page |
Google-Extended | Allowed | no rule matches the home page | |
CCBot | Common Crawl | Allowed | no rule matches the home page |
Applebot-Extended | Apple | Allowed | no rule matches the home page |
GoogleOther | Allowed | no rule matches the home page | |
Meta-ExternalAgent | Meta | Allowed | no rule matches the home page |
Bytespider | ByteDance | Allowed | no rule matches the home page |
User-triggered fetchers
| Bot | Operator | robots.txt | Deciding line |
|---|---|---|---|
ChatGPT-User | OpenAI | Allowed | no rule matches the home page |
Claude-User | Anthropic | Allowed | no rule matches the home page |
Perplexity-User | Perplexity | Allowed | no rule matches the home page |
MistralAI-User | Mistral | Allowed | no rule matches the home page |
Meta-ExternalFetcher | Meta | Allowed | no rule matches the home page |
SEO tools
| Bot | Operator | robots.txt | Deciding line |
|---|---|---|---|
AhrefsBot | Ahrefs | Allowed | no rule matches the home page |
SemrushBot | Semrush | Allowed | no rule matches the home page |
MJ12bot | Majestic | Allowed | no rule matches the home page |
DotBot | Moz | Allowed | no rule matches the home page |
View yandex.com/robots.txt · 843 lines
1# yandex.com
2User-agent: *
3Disallow: /?
4Disallow: /403.html
5Disallow: /404.html
6Disallow: /500.html
7Disallow: /about.html
8Disallow: /adddata
9Disallow: /adresa-segmentator
10Disallow: /advanced_engl.html
11Disallow: /advertising
12Disallow: /ads/
13Disallow: /adfox/
14Disallow: /an/
15Disallow: /alice/chat/
16Disallow: /all-supported-params
17Disallow: /articles
18
19Disallow: /blog/*?text=
20Disallow: /blog/*?tag=
21Disallow: /blog/*/*/*
22Disallow: /blog/punto
23
24Disallow: /activity/
25Disallow: /business/widget
26
27Disallow: /MAPS/
28Disallow: /MaPS/
29Disallow: /MapS/
30Disallow: /Maps/
31Disallow: /mApS/
32Disallow: /harita/business/
33Disallow: /maps/business/
34Disallow: /maps-api/docs/js-api/reference-guide/instruction-gxs.html
35Disallow: /maps-api/docs/js-api/reference-guide/instruction-rs.html
36Disallow: /blogs*
37Allow: /blogs/$
38Allow: /blogs/pad/$
39Allow: /blogs/pad$
40
41Disallow: /masters
42Disallow: /services
43
44Disallow: /catalog/?text=
45Disallow: /cgi-bin/
46Disallow: /cgi/
47Disallow: /chisla.html
48Disallow: /clck
49
50Disallow: /news
51Disallow: /news/
52
53Disallow: /collections/
54
55Disallow: /company/*.rss
56Disallow: /company/search
57
58Disallow: /comments/*
59
60Disallow: /conflagexp
61Disallow: /cy
62Disallow: /cycounter
63Disallow: /dzen
64Disallow: /edu/ping
65Disallow: /edu/tasks
66Disallow: /edu/teachers
67Disallow: /edu/test
68Disallow: /experiments.xml
69Disallow: /efir
70Disallow: /formfeedback
71
72Disallow: /gorsel/*
73Allow: /gorsel/$
74Allow: /gorsel/?*
75Allow: /gorsel/smart/$
76Allow: /gorsel/touch/$
77Allow: /gorsel/touch/?*
78
79Disallow: /gundem
80
81Disallow: /goto_issue/
82Disallow: /goto_rubric/
83Disallow: /i/
84
85Disallow: /images-data
86Disallow: /images.html
87Disallow: /images/*
88Disallow: /images-apphost/*
89
90Allow: /images/$
91Allow: /images/?*
92Allow: /images/smart/$
93Allow: /images/touch/$
94Allow: /images/touch/?
95
96Disallow: /index_m
97Disallow: /infected
98Disallow: /iznanka/
99Disallow: /int_search_summary
100Disallow: /delivery_tracking
101Disallow: /keyboard_qwerty.html
102Disallow: /logotypes
103
104Disallow: /map-constructor/loader*
105Disallow: /more_samples
106Disallow: /msearch
107Disallow: /msearchpart
108Disallow: /maps/print/*
109Allow: /maps/*?lang=kk$
110Allow: /maps/*?lang=uz$
111Disallow: /nmaps/*?
112Disallow: /mapeditor/*?
113
114Disallow: /metro/*?from
115Disallow: /neuralsearch/api
116Disallow: /norobot
117Disallow: /opensearch.xml
118Disallow: /padsearch
119Disallow: /people*
120Disallow: /person
121Disallow: /podpiska/login.pl
122Disallow: /polling
123Disallow: /portal/set/
124Disallow: /portal/tune/
125Disallow: /promo/diskelement
126Disallow: /promo/*goodbye*
127Disallow: /promo/skype*?*
128Disallow: /promo/skype/*/*
129Disallow: /promo/launcher/feedback
130Disallow: /promo/launcher/mgoodbye/*
131Disallow: /promo/*welcome*
132Disallow: /promo/yobject/changelog*
133Disallow: /soft/bm/goodbye
134Disallow: /soft/chrome/ext-install/*
135Disallow: /soft/chrome/searchline-install/*
136Disallow: /soft/*goodbye*
137Disallow: /soft/punto/mac/uninstall/*
138Disallow: /soft/punto/win/uninstall/*
139Disallow: /soft/*welcome*
140Disallow: /soft/win/?*
141Disallow: /quotes
142Disallow: /redir
143Disallow: /region_map
144Disallow: /regions_list.xml
145Disallow: /regions.html?
146Disallow: /rubric2sport
147Disallow: /s/
148Disallow: /save
149Disallow: /safety/?
150Disallow: /safety?
151Disallow: /search
152Disallow: /setup
153Disallow: /showcaptcha
154Disallow: /sitesearch
155Disallow: /skazki
156Disallow: /sl/*.html
157Disallow: /soft/extensions/goodbye
158Disallow: /sportagent
159Disallow: /storeclick
160Disallow: /storerequest
161Disallow: /subscribe/confirm.pl
162Disallow: /subscribe/view.pl
163
164Disallow: /support/direct-images
165Disallow: /support/direct-tooltips
166Disallow: /support/distr
167Disallow: /support/dsp
168Disallow: /support/fe4be44a295cc679e19bf0b8f133083d
169Disallow: /support/maps-beta
170Disallow: /support/market-images
171Disallow: /support/security
172Disallow: /support/webmaster-images
173Disallow: /support/*zout_
174Disallow: /support/search-results/
175Disallow: /support/praktikum/flow.html
176Disallow: /support/praktikum-english/
177
178Disallow: /telsearch
179Disallow: /themes
180Disallow: /toggle-experiment
181Disallow: /touchsearch
182Disallow: /tune*retpath=
183Disallow: /versions
184
185Disallow: /v$
186Disallow: /viewconfig$
187Disallow: /video/v$
188Disallow: /video/viewconfig$
189Disallow: /images/v$
190Disallow: /images/viewconfig$
191
192Allow: /uslugi/$
193Disallow: /uslugi/*
194
195Disallow: /video/*
196Disallow: /video/*filmId=*
197Disallow: /video/search
198Disallow: /video/preview
199Disallow: /video/*/search
200Disallow: /video/*/preview
201Allow: /video/$
202Allow: /video/?
203Allow: /video/sitemap
204Allow: /video/dizi-izle/
205Allow: /video/dizi-izle/?
206Allow: /video/yerli-dizi-izle/
207Allow: /video/yabanci-dizi-izle/
208Allow: /video/touch/
209Allow: /video/pad/
210
211Disallow: /xmlsearch
212Disallow: /yaca
213Disallow: /yandsearch
214Disallow: /yca/cy
215
216Disallow: /soft/distribution
217Disallow: /soft/?*
218Disallow: /promo/launcher/?*
219Disallow: /opera/?*
220Disallow: /firefox/?*
221Disallow: /soft/bm/?*
222Disallow: /soft/browsers/?*
223Disallow: /soft/punto/?*
224Disallow: /ie/?*
225Disallow: /element/?*
226Disallow: /element/*goodbye*
227Disallow: /soft/*goodbye*
228Disallow: /adult
229
230Disallow: /sport*?*parent-reqid*
231Disallow: /mirror*?*parent-reqid*
232Disallow: /turbo*?*parent-reqid*
233Disallow: /mirror/hide
234
235Disallow: /pogoda/0
236
237Disallow: /turbo*?*ajax=1
238Disallow: /sport*?*ajax=1
239Disallow: /mirror*?*ajax=1
240
241Disallow: /games/preview/
242Disallow: /games/install
243Disallow: /games/*?*reqId*
244Disallow: /games/*?*pageId*
245Disallow: /games/*?experimental=
246Disallow: /games/*?*app-id=
247Disallow: /games/*?*game-id=
248Disallow: /games/*?*div-rtx-reqid=
249Disallow: /games/*?*header=
250Disallow: /games/*?*from=
251Disallow: /games/*?*clid=
252Disallow: /games/*?*deviceid=
253Disallow: /games/*?*flags=
254Disallow: /games/*?*inapp_id=
255Disallow: /games/*?*k50id=
256Disallow: /games/*?*lang=
257Disallow: /games/*?*payload=
258Disallow: /games/*?*pfid=
259Disallow: /games/*?*promo_id=
260Disallow: /games/*?*promo_intent=
261Disallow: /games/*?*referrer=
262Disallow: /games/*?*utm_
263Disallow: /games/*?*ypr=
264
265Disallow: /weather/*
266Disallow: /weather*
267Disallow: /pogoda/*
268Disallow: /pogod*
269Disallow: /hava/*
270Disallow: /hava*
271Disallow: *maps/covid19*
272Disallow: /support2/*zout_
273Disallow: /sport
274Disallow: /support2/sport*
275Disallow: /video/%D0%B7%D0%B0%D0%BF%D1%80%D0%BE%D1%81/*
276Disallow: /video/touch/%D0%B7%D0%B0%D0%BF%D1%80%D0%BE%D1%81/*
277
278Disallow: /medicine
279Allow: /medicine/doctors
280Allow: /medicine/clinics
281Disallow: /finance
282
283Clean-Param: ncrnd&redircnt&clid&_&win /
284Clean-Param: random_cgi&sign /turbo*
285Clean-Param: ll&z&sll /maps
286Clean-Param: appsearch_header /pogoda
287
288Sitemap: https://yandex.com/support/sitemap.xml
289Sitemap: https://yandex.com/blog/sitemap.xml
290Sitemap: https://yandex.com/turbo/public/sitemap.xml
291Sitemap: https://yandex.com/games/sitemaps/sitemap.index.xml
292
293User-agent: yandex
294Disallow: /?
295Disallow: /403.html
296Disallow: /404.html
297Disallow: /500.html
298Disallow: /about.html
299Disallow: /adddata
300Disallow: /adresa-segmentator
301Disallow: /advanced_engl.html
302Disallow: /advertising
303Disallow: /alice/chat/
304Disallow: /all-supported-params
305Disallow: /articles
306
307Disallow: /activity/
308Disallow: /blog/*?text=
309Disallow: /blog/*?tag=
310Disallow: /blog/*/*/*
311Disallow: /blog/punto
312
313Disallow: /business/widget
314
315Disallow: /MAPS/
316Disallow: /MaPS/
317Disallow: /MapS/
318Disallow: /Maps/
319Disallow: /mApS/
320Disallow: /harita/business/
321Disallow: /maps/business/
322Disallow: /maps-api/docs/js-api/reference-guide/instruction-gxs.html
323Disallow: /maps-api/docs/js-api/reference-guide/instruction-rs.html
324Disallow: /blogs*
325Allow: /blogs/$
326Allow: /blogs/pad/$
327Allow: /blogs/pad$
328
329Disallow: /masters
330Disallow: /services
331
332Disallow: /catalog/?text=
333Disallow: /chat
334Disallow: /cgi-bin/
335Disallow: /cgi/
336Disallow: /chisla.html
337Disallow: /clck
338
339Disallow: /news
340Disallow: /news/
341
342Disallow: /collections/
343
344Disallow: /company/*.rss
345Disallow: /company/search
346
347Disallow: /comments/*
348
349Disallow: /conflagexp
350Disallow: /cy
351Disallow: /cycounter
352Disallow: /dzen
353Disallow: /edu/ping
354Disallow: /edu/tasks
355Disallow: /edu/teachers
356Disallow: /edu/test
357Disallow: /experiments.xml
358Disallow: /efir
359Disallow: /formfeedback
360
361Disallow: /games/preview/
362Disallow: /games/install
363Disallow: /games/*?experimental=
364
365Disallow: /gorsel/*
366Allow: /gorsel/$
367Allow: /gorsel/?*
368Allow: /gorsel/smart/$
369Allow: /gorsel/touch/$
370Allow: /gorsel/touch/?*
371
372Disallow: /goto_issue/
373Disallow: /goto_rubric/
374Disallow: /i/
375
376Disallow: /gundem
377
378Disallow: /images-data
379Disallow: /images.html
380Disallow: /images/*
381Disallow: /images-apphost/*
382
383Allow: /images/$
384Allow: /images/?*
385Allow: /images/smart/$
386Allow: /images/touch/$
387Allow: /images/touch/?
388
389Disallow: /index_m
390Disallow: /infected
391Disallow: /iznanka/
392Disallow: /int_search_summary
393Disallow: /delivery_tracking
394Disallow: /keyboard_qwerty.html
395Disallow: /logotypes
396
397Disallow: /map-constructor/loader*
398Disallow: /more_samples
399Disallow: /msearch
400Disallow: /msearchpart
401Disallow: /maps/print/*
402Allow: /maps/*?lang=kk$
403Allow: /maps/*?lang=uz$
404Disallow: /nmaps/*?
405Disallow: /mapeditor/*?
406
407Disallow: /metro/*?from
408Disallow: /neuralsearch/api
409Disallow: /norobot
410Disallow: /opensearch.xml
411Disallow: /padsearch
412Disallow: /people*
413Disallow: /person
414Disallow: /podpiska/login.pl
415Disallow: /polling
416Disallow: /portal/set/
417Disallow: /portal/tune/
418
419Disallow: /pogoda/0
420
421Disallow: /promo/diskelement
422Disallow: /promo/*goodbye*
423Disallow: /promo/skype*?*
424Disallow: /promo/skype/*/*
425Disallow: /promo/launcher/feedback
426Disallow: /promo/launcher/mgoodbye/*
427Disallow: /promo/*welcome*
428Disallow: /promo/yobject/changelog*
429Disallow: /soft/bm/goodbye
430Disallow: /soft/chrome/ext-install/*
431Disallow: /soft/chrome/searchline-install/*
432Disallow: /soft/*goodbye*
433Disallow: /soft/punto/mac/uninstall/*
434Disallow: /soft/punto/win/uninstall/*
435Disallow: /soft/*welcome*
436Disallow: /soft/win/?*
437Disallow: /quotes
438Disallow: /redir
439Disallow: /region_map
440Disallow: /regions_list.xml
441Disallow: /regions.html?
442Disallow: /rubric2sport
443Disallow: /s/
444Disallow: /save
445Disallow: /safety/?*
446Disallow: /search
447Disallow: /setup
448Disallow: /showcaptcha
449Disallow: /sitesearch
450Disallow: /skazki
451Disallow: /sl/*.html
452Disallow: /soft/extensions/goodbye
453Disallow: /sportagent
454Disallow: /storeclick
455Disallow: /storerequest
456Disallow: /subscribe/confirm.pl
457Disallow: /subscribe/view.pl
458
459Disallow: /support/direct-images
460Disallow: /support/direct-tooltips
461Disallow: /support/distr
462Disallow: /support/dsp
463Disallow: /support/fe4be44a295cc679e19bf0b8f133083d
464Disallow: /support/maps-beta
465Disallow: /support/market-images
466Disallow: /support/security
467Disallow: /support/webmaster-images
468Disallow: /support/*zout_
469Disallow: /support/search-results/
470Disallow: /support/praktikum/flow.html
471Disallow: /support/praktikum-english/
472
473Disallow: /telsearch
474Disallow: /themes
475Disallow: /toggle-experiment
476Disallow: /touchsearch
477Disallow: /tune*retpath=
478Disallow: /versions
479
480Disallow: /v$
481Disallow: /viewconfig$
482Disallow: /video/v$
483Disallow: /video/viewconfig$
484Disallow: /images/v$
485Disallow: /images/viewconfig$
486
487Allow: /uslugi/$
488Disallow: /uslugi/*
489
490Disallow: /video/*
491Disallow: /video/*filmId=*
492Disallow: /video/search
493Disallow: /video/preview
494Disallow: /video/*/search
495Disallow: /video/*/preview
496Disallow: /video/touch
497Allow: /video/$
498Allow: /video/?
499Allow: /video/sitemap
500Allow: /video/dizi-izle/
501Allow: /video/dizi-izle/?
502Allow: /video/yerli-dizi-izle/
503Allow: /video/yabanci-dizi-izle/
504Allow: /video/pad/
505Allow: /video/search?text=video$
506
507Disallow: /xmlsearch
508Disallow: /yaca
509Disallow: /yandsearch
510Disallow: /yca/cy
511
512Disallow: /soft/distribution
513Disallow: /soft/?*
514Disallow: /promo/launcher/?*
515Disallow: /opera/?*
516Disallow: /firefox/?*
517Disallow: /soft/bm/?*
518Disallow: /soft/browsers/?*
519Disallow: /soft/punto/?*
520Disallow: /ie/?*
521Disallow: /element/?*
522Disallow: /element/*goodbye*
523Disallow: /soft/*goodbye*
524Disallow: /adult
525
526Disallow: /sport*?*parent-reqid*
527Disallow: /mirror*?*parent-reqid*
528Disallow: /turbo*?*parent-reqid*
529Disallow: /mirror/hide
530
531Disallow: /turbo*?*ajax=1*
532Disallow: /sport*?*ajax=1*
533Disallow: /mirror*?*ajax=1*
534
535Disallow: /weather/*
536Disallow: /weather*
537Disallow: /pogoda/*
538Disallow: /pogod*
539Disallow: /hava/*
540Disallow: /hava*
541Disallow: *maps/covid19*
542Disallow: /support2/*zout_
543Disallow: /sport
544Disallow: /support2/sport*
545Disallow: /video/%D0%B7%D0%B0%D0%BF%D1%80%D0%BE%D1%81/*
546Disallow: /video/touch/%D0%B7%D0%B0%D0%BF%D1%80%D0%BE%D1%81/*
547
548Disallow: /medicine
549Allow: /medicine/doctors
550Allow: /medicine/clinics
551Disallow: /finance
552
553User-Agent: Twitterbot
554Allow: /images
555Allow: /gorsel
556Allow: /video
557Disallow: /adult
558
559Disallow: /activity/
560Disallow: /news
561Disallow: /news/
562
563Disallow: /games/*?*reqId
564Disallow: /games/*?*pageId
565Disallow: /games/preview/
566Disallow: /games/install
567Disallow: /games/*?experimental=
568
569Disallow: /gundem
570
571Disallow: /weather/*
572Disallow: /weather*
573Disallow: /pogoda/*
574Disallow: /pogod*
575Disallow: /hava/*
576Disallow: /hava*
577Disallow: *maps/covid19*
578Disallow: /support2/*zout_
579Disallow: /sport
580Disallow: /support2/sport*
581Disallow: /video/%D0%B7%D0%B0%D0%BF%D1%80%D0%BE%D1%81/*
582Disallow: /video/touch/%D0%B7%D0%B0%D0%BF%D1%80%D0%BE%D1%81/*
583
584Disallow: /medicine
585Allow: /medicine/doctors
586Allow: /medicine/clinics
587Disallow: /finance
588
589Sitemap: https://yandex.com/support/sitemap.xml
590Sitemap: https://yandex.com/blog/sitemap.xml
591Sitemap: https://yandex.com/turbo/public/sitemap.xml
592Sitemap: https://yandex.com/games/sitemaps/sitemap.index.xml
593
594
595User-agent: Applebot
596Disallow: /?
597Disallow: /403.html
598Disallow: /404.html
599Disallow: /500.html
600Disallow: /about.html
601Disallow: /adddata
602Disallow: /adresa-segmentator
603Disallow: /advanced_engl.html
604Disallow: /advertising
605Disallow: /all-supported-params
606Disallow: /articles
607
608Disallow: /blog/*?text=
609Disallow: /blog/*?tag=
610Disallow: /blog/*/*/*
611Disallow: /blog/punto
612
613Disallow: /activity/
614Disallow: /news
615Disallow: /news/
616
617Disallow: /blogs*
618Allow: /blogs/$
619Allow: /blogs/pad/$
620Allow: /blogs/pad$
621
622Disallow: /catalog/?text=
623Disallow: /chat
624Disallow: /cgi-bin/
625Disallow: /cgi/
626Disallow: /chisla.html
627Disallow: /clck
628
629Disallow: /collections/
630
631Disallow: /company/*.rss
632Disallow: /company/search
633
634Disallow: /comments/*
635
636Disallow: /conflagexp
637Disallow: /cy
638Disallow: /cycounter
639Disallow: /dzen
640Disallow: /edu/ping
641Disallow: /edu/tasks
642Disallow: /edu/teachers
643Disallow: /edu/test
644Disallow: /experiments.xml
645Disallow: /efir
646Disallow: /formfeedback
647
648Disallow: /gorsel/*
649Allow: /gorsel/$
650Allow: /gorsel/?*
651Allow: /gorsel/smart/$
652Allow: /gorsel/touch/$
653Allow: /gorsel/touch/?*
654
655Disallow: /gundem
656
657Disallow: /goto_issue/
658Disallow: /goto_rubric/
659Disallow: /i/
660
661Disallow: /images-data
662Disallow: /images.html
663Disallow: /images/*
664Allow: /images/$
665Allow: /images/?*
666Allow: /images/smart/$
667Allow: /images/touch/$
668Allow: /images/touch/?
669
670Disallow: /index_m
671Disallow: /infected
672Disallow: /iznanka/
673Disallow: /keyboard_qwerty.html
674Disallow: /logotypes
675
676Disallow: /map-constructor/loader*
677Disallow: /more_samples
678Disallow: /msearch
679Disallow: /msearchpart
680Disallow: /nmaps/*?
681Disallow: /maps/print/*
682Allow: /maps/*?ll=*
683Allow: /maps/org/
684Disallow: /mapeditor/*?
685
686Disallow: /metro/*?from
687
688Disallow: /norobot
689Disallow: /opensearch.xml
690Disallow: /padsearch
691Disallow: /people*
692Disallow: /person
693Disallow: /podpiska/login.pl
694Disallow: /polling
695Disallow: /promo/diskelement
696Disallow: /promo/*goodbye*
697Disallow: /promo/skype*?*
698Disallow: /promo/skype/*/*
699Disallow: /promo/launcher/feedback
700Disallow: /promo/launcher/mgoodbye/*
701Disallow: /promo/*welcome*
702Disallow: /promo/yobject/changelog*
703Disallow: /soft/bm/goodbye
704Disallow: /soft/chrome/ext-install/*
705Disallow: /soft/chrome/searchline-install/*
706Disallow: /soft/*goodbye*
707Disallow: /soft/punto/mac/uninstall/*
708Disallow: /soft/punto/win/uninstall/*
709Disallow: /soft/*welcome*
710Disallow: /soft/win/?*
711Disallow: /quotes
712Disallow: /redir
713Disallow: /region_map
714Disallow: /regions_list.xml
715Disallow: /regions.html?
716Disallow: /rubric2sport
717Disallow: /s/
718Disallow: /save
719Disallow: /safety/?*
720Disallow: /search
721Disallow: /setup
722Disallow: /showcaptcha
723Disallow: /sitesearch
724Disallow: /skazki
725Disallow: /sl/*.html
726Disallow: /soft/extensions/goodbye
727Disallow: /sportagent
728Disallow: /storeclick
729Disallow: /storerequest
730Disallow: /subscribe/confirm.pl
731Disallow: /subscribe/view.pl
732
733Disallow: /support/direct-images
734Disallow: /support/direct-tooltips
735Disallow: /support/distr
736Disallow: /support/dsp
737Disallow: /support/fe4be44a295cc679e19bf0b8f133083d
738Disallow: /support/maps-beta
739Disallow: /support/market-images
740Disallow: /support/security
741Disallow: /support/webmaster-images
742Disallow: /support/*zout_
743Disallow: /support/search-results/
744Disallow: /support/praktikum/flow.html
745Disallow: /support/praktikum-english/
746
747Disallow: /telsearch
748Disallow: /themes
749Disallow: /toggle-experiment
750Disallow: /touchsearch
751Disallow: /tune*retpath=
752Disallow: /versions
753
754Disallow: /v$
755Disallow: /viewconfig$
756Disallow: /video/v$
757Disallow: /video/viewconfig$
758Disallow: /images/v$
759Disallow: /images/viewconfig$
760
761Allow: /uslugi/$
762Disallow: /uslugi/*
763
764Allow: /tutor
765Disallow: /tutor/admin
766Disallow: /tutor/search/problems
767
768Disallow: /video/*
769Disallow: /video/*filmId=*
770Allow: /video/$
771Allow: /video/?
772Allow: /video/sitemap
773Allow: /video/dizi-izle/
774Allow: /video/dizi-izle/?
775Allow: /video/yerli-dizi-izle/
776Allow: /video/yabanci-dizi-izle/
777Allow: /video/touch/
778Allow: /video/pad/
779
780Disallow: /xmlsearch
781Disallow: /yaca
782Disallow: /yandsearch
783Disallow: /yca/cy
784
785Disallow: /soft/distribution
786Disallow: /soft/?*
787Disallow: /promo/launcher/?*
788Disallow: /opera/?*
789Disallow: /firefox/?*
790Disallow: /soft/bm/?*
791Disallow: /soft/browsers/?*
792Disallow: /soft/punto/?*
793Disallow: /ie/?*
794Disallow: /element/?*
795Disallow: /element/*goodbye*
796Disallow: /soft/*goodbye*
797Disallow: /adult
798
799Disallow: /games/*?*reqId
800Disallow: /games/*?*pageId
801Disallow: /games/preview/
802Disallow: /games/install
803Disallow: /games/*?experimental=
804Disallow: /games/*?*app-id=
805Disallow: /games/*?*game-id=
806Disallow: /games/*?*div-rtx-reqid=
807Disallow: /games/*?*header=
808Disallow: /games/*?*from=
809
810Disallow: /weather/*
811Disallow: /weather*
812Disallow: /pogoda/*
813Disallow: /pogod*
814Disallow: /hava/*
815Disallow: /hava*
816Disallow: *maps/covid19*
817Disallow: /support2/*zout_
818Disallow: /sport
819Disallow: /support2/sport*
820Disallow: /video/%D0%B7%D0%B0%D0%BF%D1%80%D0%BE%D1%81/*
821Disallow: /video/touch/%D0%B7%D0%B0%D0%BF%D1%80%D0%BE%D1%81/*
822
823Clean-Param: ncrnd&redircnt&clid&_&win /
824Clean-Param: random_cgi&sign /turbo*
825Clean-Param: app-id&game-id&div-rtx-reqid&lang&platform&header&from&backQueryParams /games/
826Clean-Param: ll&z&sll /maps
827Clean-param: oll&rtext&pt&source&via&from&origin&um&utm_medium&utm_source&portal&panorama%5Bspan%5D&panorama%5Bpoint%5D&panorama%5Bfull%5D&panorama%5Bdirection%5D&panorama%5Bair%5D&masstransit%5BstopId%5D&ol&poi%5Bpoint%5D&whatshere%5Bpoint%5D&whatshere%5Bzoom%5D&ouri&ruri&l&mode&tab&rtt&rtm&display-text&text&indoorLevel&trfm&trfst&no-distribution&distance&slices&scale&descy&descx&mapY&mapX&mapID&sctx&lay&where&results&map&id&spn&sspn&rtn&ost&profile-mode /maps/
828Clean-param: z&ll&mapTypeRoute&slug&uuid&app&is_user_location&app_with_layers&app_soyuz&from&dateRoute&clid&via&city&&topic&lat&lon&lang&le_Lightning&from&utm_source&utm_campaign&utm_medium&utm_content&utm_term /pogoda/
829Clean-param: z&ll&mapTypeRoute&slug&uuid&app&is_user_location&app_with_layers&app_soyuz&from&dateRoute&clid&via&city&&topic&lat&lon&lang&le_Lightning&from&utm_source&utm_campaign&utm_medium&utm_content&utm_term /hava/
830Clean-param: z&ll&mapTypeRoute&slug&uuid&app&is_user_location&app_with_layers&app_soyuz&from&dateRoute&clid&via&city&&topic&lat&lon&lang&le_Lightning&from&utm_source&utm_campaign&utm_medium&utm_content&utm_term /weather/
831Clean-param: from_pumpkin&le_WindParticles&nowcast&yw_preset&gen_time&for_date&morda_geolocation&retpath&acc_maps&le_TemperatureBalloons&request&le_Borders&elmt&may&source&ids&june&july&referrer&PAGEN_1&from_render_fallback&preset&to&august&appsearch_ys®ion&tm&banner-test-tags /pogoda/
832Clean-param: stat-id&format-type&actual-format&april&test-tag&adgroup&campaign_id&c&nowcast_header&detailsType&utm_from&bar_exp&march&november&bounds&game&text&city_id&february&september&january&no_turbo&october&ref&december&weather_url&build&lp_source&tab&geoid&cameras&day&thndr_onbrd /pogoda/
833Clean-param: from_pumpkin&le_WindParticles&nowcast&yw_preset&gen_time&for_date&morda_geolocation&retpath&acc_maps&le_TemperatureBalloons&request&le_Borders&elmt&may&source&ids&june&july&referrer&PAGEN_1&from_render_fallback&preset&to&august&appsearch_ys®ion&tm&banner-test-tags&stat-id /hava/
834Clean-param: format-type&actual-format&april&test-tag&adgroup&campaign_id&c&nowcast_header&detailsType&utm_from&bar_exp&march&november&bounds&game&text&city_id&february&september&january&no_turbo&october&ref&december&weather_url&build&lp_source&tab&geoid&cameras&day&thndr_onbrd /hava/
835Clean-param: from_pumpkin&le_WindParticles&nowcast&yw_preset&gen_time&for_date&morda_geolocation&retpath&acc_maps&le_TemperatureBalloons&request&le_Borders&elmt&may&source&ids&june&july&referrer&PAGEN_1&from_render_fallback&preset&to&august&appsearch_ys®ion&tm&banner-test-tags&stat-id /weather/
836Clean-param: format-type&actual-format&april&test-tag&adgroup&campaign_id&c&nowcast_header&detailsType&utm_from&bar_exp&march&november&bounds&game&text&city_id&february&september&january&no_turbo&october&ref&december&weather_url&build&lp_source&tab&geoid&cameras&day&thndr_onbrd /weather/
837
838
839Sitemap: https://yandex.com/support/sitemap.xml
840Sitemap: https://yandex.com/blog/sitemap.xml
841Sitemap: https://yandex.com/turbo/public/sitemap.xml
842Sitemap: https://yandex.com/games/sitemaps/sitemap.index.xml
843
What do the bots actually get?
This page reads robots.txt, which states what yandex.com asks of crawlers. A firewall or CDN can still refuse a bot that robots.txt allows. The live check requests the page as each bot and shows the status it gets.
Run a live check on yandex.com Test a path in the robots.txt tester
History
- 19 October 2026: first checked.
Questions
Does yandex.com block GPTBot?
No. yandex.com’s robots.txt lets GPTBot crawl the site. GPTBot is OpenAI’s crawler for training data; blocking it doesn’t affect ChatGPT search, which uses OAI-SearchBot. Checked 9 November 2026.
Does yandex.com block ClaudeBot?
No. yandex.com’s robots.txt lets ClaudeBot crawl the site. ClaudeBot is Anthropic’s crawler for training data. Checked 9 November 2026.
Does yandex.com block Google-Extended?
No. yandex.com’s robots.txt lets Google-Extended crawl the site. Google-Extended is the token that controls whether Google may use pages for Gemini training and grounding; it doesn’t affect Google Search. Checked 9 November 2026.
Does yandex.com block PerplexityBot?
No. yandex.com’s robots.txt lets PerplexityBot crawl the site. PerplexityBot is the crawler that indexes pages for Perplexity’s answers. Checked 9 November 2026.
Can ChatGPT search find yandex.com?
Yes. yandex.com’s robots.txt lets OAI-SearchBot crawl the site. OAI-SearchBot is the crawler behind ChatGPT search; blocking it keeps the site out of ChatGPT’s search answers. Checked 9 November 2026.