Who blocks AI · #191 in the Majestic Million
robots.txt · checked 9 Nov 2026
Does unesco.org block AI crawlers?
Yes. unesco.org’s robots.txt blocks 7 of the 19 AI crawlers we track, including GPTBot, ClaudeBot, CCBot and 4 more. Search engines such as Googlebot and Bingbot may crawl it.
GPTBotBlockedOAI-SearchBotAllowedClaudeBotBlockedPerplexityBotAllowedGoogle-ExtendedAllowedCCBotBlocked- robots.txt
- www.unesco.org/robots.txt · 1,627 lines
- llms.txt
- found
- Checked
- 9 November 2026, as canaicrawl. We check again every week.
Every bot, and the line that decides
Search engines
| Bot | Operator | robots.txt | Deciding line |
|---|---|---|---|
Googlebot | Allowed | no rule matches the home page | |
Bingbot | Microsoft | Allowed | no rule matches the home page |
DuckDuckBot | DuckDuckGo | Allowed | no rule matches the home page |
Applebot | Apple | Allowed | no rule matches the home page |
PetalBot | Huawei | Allowed | no rule matches the home page |
YandexBot | Yandex | Allowed | no rule matches the home page |
Baiduspider | Baidu | Allowed | no rule matches the home page |
AI search
| Bot | Operator | robots.txt | Deciding line |
|---|---|---|---|
OAI-SearchBot | OpenAI | Allowed | no rule matches the home page |
PerplexityBot | Perplexity | Allowed | no rule matches the home page |
Claude-SearchBot | Anthropic | Allowed | no rule matches the home page |
DuckAssistBot | DuckDuckGo | Allowed | no rule matches the home page |
Amazonbot | Amazon | Blocked | line 59 · Disallow: / under User-agent: Amazonbot |
YouBot | You.com | Allowed | no rule matches the home page |
AI training
| Bot | Operator | robots.txt | Deciding line |
|---|---|---|---|
GPTBot | OpenAI | Blocked | line 59 · Disallow: / under User-agent: GPTBot |
ClaudeBot | Anthropic | Blocked | line 59 · Disallow: / under User-agent: ClaudeBot |
Google-Extended | Allowed | no rule matches the home page | |
CCBot | Common Crawl | Blocked | line 59 · Disallow: / under User-agent: CCBot |
Applebot-Extended | Apple | Blocked | line 59 · Disallow: / under User-agent: Applebot-Extended |
GoogleOther | Allowed | no rule matches the home page | |
Meta-ExternalAgent | Meta | Blocked | line 59 · Disallow: / under User-agent: Meta-ExternalAgent |
Bytespider | ByteDance | Blocked | line 59 · Disallow: / under User-agent: Bytespider |
User-triggered fetchers
| Bot | Operator | robots.txt | Deciding line |
|---|---|---|---|
ChatGPT-User | OpenAI | Allowed | no rule matches the home page |
Claude-User | Anthropic | Allowed | no rule matches the home page |
Perplexity-User | Perplexity | Allowed | no rule matches the home page |
MistralAI-User | Mistral | Allowed | no rule matches the home page |
Meta-ExternalFetcher | Meta | Allowed | no rule matches the home page |
SEO tools
| Bot | Operator | robots.txt | Deciding line |
|---|---|---|---|
AhrefsBot | Ahrefs | Allowed | no rule matches the home page |
SemrushBot | Semrush | Allowed | no rule matches the home page |
MJ12bot | Majestic | Allowed | no rule matches the home page |
DotBot | Moz | Allowed | no rule matches the home page |
View www.unesco.org/robots.txt · 1,627 lines
1# robots.txt - www.unesco.org
2# Last updated: 2026-10-13 (corrected structure)
3# Contact: digital.com@unesco.org
4#
5# As a condition of accessing this website, you agree to abide by the following
6# content signals:
7# (a) If a Content-Signal = yes, you may collect content for the corresponding use.
8# (b) If a Content-Signal = no, you may not collect content for the corresponding use.
9# (c) If the website operator does not include a Content-Signal for a
10# corresponding use, the website operator neither grants nor restricts
11# permission via Content-Signal with respect to the corresponding use.
12# The content signals and their meanings are:
13# - search: building a search index and providing search results (e.g., returning
14# hyperlinks and short excerpts from your website's contents). Search does not
15# include providing AI-generated search summaries.
16# - ai-input: inputting content into one or more AI models (e.g., retrieval
17# augmented generation, grounding, or other real-time taking of content for
18# generative AI search answers).
19# - ai-train: training or fine-tuning AI models.
20# - use: how AI systems may consume the content (immediate, reference, or full).
21# ANY RESTRICTIONS EXPRESSED VIA CONTENT SIGNALS ARE EXPRESS RESERVATIONS OF
22# RIGHTS BY UNESCO IN RESPECT OF THE MATERIALS MADE AVAILABLE ON THIS SITE,
23# INCLUDING ITS RIGHTS AS MAKER OF ITS DATABASES. THEY APPLY WHETHER OR NOT
24# UNESCO'S TERMS OF USE HAVE BEEN ACCEPTED. NOTHING IN THIS FILE CONSTITUTES
25# A WAIVER, EXPRESS OR IMPLIED, OF THE PRIVILEGES AND IMMUNITIES OF UNESCO.
26#
27# UNESCO content may not be used to train, pre-train, fine-tune or calibrate
28# AI models without prior authorisation from UNESCO.
29# Requests: digital.com@unesco.org
30
31# =============================================================================
32# SECTION 1: AI TRAINING BOTS - FULLY BLOCKED
33# These bots are explicitly blocked from all crawling for AI training purposes.
34# =============================================================================
35
36User-agent: Amazonbot
37User-agent: Applebot-Extended
38User-agent: Bytespider
39User-agent: CCBot
40User-agent: ClaudeBot
41User-agent: FacebookBot
42User-agent: GPTBot
43User-agent: Meta-ExternalAgent
44User-agent: omgili
45User-agent: omgilibot
46User-agent: DeepSeekBot
47User-agent: ERNIEBot
48User-agent: DoubaoBot
49User-agent: ChatGLM-Spider
50User-agent: cohere-training-data-crawler
51User-agent: MistralAI-Training
52User-agent: AI2Bot
53
54# NOTE: FacebookBot is NOT facebookexternalhit.
55# facebookexternalhit generates link previews when UNESCO
56# content is shared on Facebook, Instagram or WhatsApp.
57# It must never be blocked.
58
59Disallow: /
60
61# =============================================================================
62# SECTION 2: ALL OTHER AGENTS
63# Single group for all remaining crawlers including AI search bots.
64# No separate groups needed - all follow the same rules below.
65# =============================================================================
66
67User-agent: *
68Content-Signal: search=yes,ai-input=yes,ai-train=no,use=reference
69License: https://www.unesco.org/en/terms-use
70
71# Allow static assets (CSS, JS, images) for proper rendering
72Allow: /core/*.css$
73Allow: /core/*.css?
74Allow: /core/*.js$
75Allow: /core/*.js?
76Allow: /core/*.gif
77Allow: /core/*.jpg
78Allow: /core/*.jpeg
79Allow: /core/*.png
80Allow: /core/*.svg
81Allow: /profiles/*.css$
82Allow: /profiles/*.css?
83Allow: /profiles/*.js$
84Allow: /profiles/*.js?
85Allow: /profiles/*.gif
86Allow: /profiles/*.jpg
87Allow: /profiles/*.jpeg
88Allow: /profiles/*.png
89Allow: /profiles/*.svg
90
91# Block system directories and files
92Disallow: /core/
93Disallow: /profiles/
94Disallow: /README.txt
95Disallow: /web.config
96
97# Block generated CSS/JS, temp and private uploads (root + all sub-sites)
98# Public content in /files/ (images, PDFs, documents) remains allowed.
99Disallow: /files/css/
100Disallow: /files/js/
101Disallow: /files/tmp/
102Disallow: /files/private/
103Disallow: /*/files/css/
104Disallow: /*/files/js/
105Disallow: /*/files/tmp/
106Disallow: /*/files/private/
107
108# Block maps and grid views to prevent duplicate content
109Disallow: /map
110Disallow: /grid
111Disallow: /*/map
112Disallow: /*/grid
113
114# Block admin and user account pages (clean URLs)
115Disallow: /admin/
116Disallow: /comment/reply/
117Disallow: /filter/tips
118Disallow: /node/add/
119Disallow: /user/register
120Disallow: /user/password
121Disallow: /user/login
122Disallow: /user/logout
123Disallow: /media/oembed
124Disallow: /*/media/oembed
125
126# Block admin and user account pages (with index.php)
127Disallow: /index.php/admin/
128Disallow: /index.php/comment/reply/
129Disallow: /index.php/filter/tips
130Disallow: /index.php/node/add/
131Disallow: /index.php/search/
132Disallow: /index.php/user/password
133Disallow: /index.php/user/register
134Disallow: /index.php/user/login
135Disallow: /index.php/user/logout
136Disallow: /index.php/media/oembed
137Disallow: /index.php/*/media/oembed
138
139# Block search and explore pages (all levels via wildcards)
140# Wildcards cover root + all language subfolders (en, fr, es, ru, ar, zh, pt, ja, uz, id, uk, th, it, vi, ...)
141# No need to add new language prefixes when adding subfolder sites.
142Disallow: /search
143Disallow: /search/
144Disallow: /search?
145Disallow: /explore/
146Disallow: /*/search
147Disallow: /*/search/*
148Disallow: /*/search?
149Disallow: /*/explore/*
150
151# Block search via index.php (query parameter variant)
152Disallow: /index.php/search?
153Disallow: /index.php/*/search?
154
155# Block event iCal feeds (root level)
156Disallow: */event/*/ical
157Disallow: /event/*/ical
158Disallow: /en/event/*/ical
159
160# Block query parameters that create duplicate content
161Disallow: /*?*f[0]=*
162Disallow: /*?*f0=*
163Disallow: /*?*date_from=*
164Disallow: /*?f[*
165Disallow: /*&f[*
166Disallow: /*?f%5B0*
167Disallow: /*&f%5B1*
168Disallow: *?hub=*
169Disallow: *?page*
170Disallow: *?fq*
171Disallow: *?country*
172
173# Block TSPD paths (restricted to directory structure only)
174Disallow: /TSPD/
175Disallow: /*/TSPD/
176
177# Block lightbox paths
178Disallow: */lightbox*
179
180# Sitemap location
181Sitemap: https://www.unesco.org/sitemap.xml
182Disallow: /*?f[*
183Disallow: /*&f[*
184Disallow: /*?f%5B0*
185Disallow: /*&f%5B1*
186Disallow: /search
187Disallow: /en/search
188Disallow: /en/search/*
189Disallow: /en/explore/*
190Disallow: /event/*/ical
191Disallow: /en/event/*/ical
192Disallow: /biennaleluanda/2021/*?f[*
193Disallow: /biennaleluanda/2021/*&f[*
194Disallow: /biennaleluanda/2021/*?f%5B0*
195Disallow: /biennaleluanda/2021/*&f%5B1*
196Disallow: /biennaleluanda/2021/search
197Disallow: /biennaleluanda/2021/en/search
198Disallow: /biennaleluanda/2021/en/search/*
199Disallow: /biennaleluanda/2021/en/explore/*
200Disallow: /biennaleluanda/2021/event/*/ical
201Disallow: /biennaleluanda/2021/en/event/*/ical
202Disallow: /fr/search
203Disallow: /fr/search/*
204Disallow: /fr/explore/*
205Disallow: /fr/event/*/ical
206Disallow: /biennaleluanda/2021/fr/search
207Disallow: /biennaleluanda/2021/fr/search/*
208Disallow: /biennaleluanda/2021/fr/explore/*
209Disallow: /biennaleluanda/2021/fr/event/*/ical
210Disallow: /es/search
211Disallow: /es/search/*
212Disallow: /es/explore/*
213Disallow: /es/event/*/ical
214Disallow: /biennaleluanda/2021/es/search
215Disallow: /biennaleluanda/2021/es/search/*
216Disallow: /biennaleluanda/2021/es/explore/*
217Disallow: /biennaleluanda/2021/es/event/*/ical
218Disallow: /ru/search
219Disallow: /ru/search/*
220Disallow: /ru/explore/*
221Disallow: /ru/event/*/ical
222Disallow: /biennaleluanda/2021/ru/search
223Disallow: /biennaleluanda/2021/ru/search/*
224Disallow: /biennaleluanda/2021/ru/explore/*
225Disallow: /biennaleluanda/2021/ru/event/*/ical
226Disallow: /ar/search
227Disallow: /ar/search/*
228Disallow: /ar/explore/*
229Disallow: /ar/event/*/ical
230Disallow: /biennaleluanda/2021/ar/search
231Disallow: /biennaleluanda/2021/ar/search/*
232Disallow: /biennaleluanda/2021/ar/explore/*
233Disallow: /biennaleluanda/2021/ar/event/*/ical
234Disallow: /zh/search
235Disallow: /zh/search/*
236Disallow: /zh/explore/*
237Disallow: /zh/event/*/ical
238Disallow: /biennaleluanda/2021/zh/search
239Disallow: /biennaleluanda/2021/zh/search/*
240Disallow: /biennaleluanda/2021/zh/explore/*
241Disallow: /biennaleluanda/2021/zh/event/*/ical
242Disallow: /pt/search
243Disallow: /pt/search/*
244Disallow: /pt/explore/*
245Disallow: /pt/event/*/ical
246Disallow: /biennaleluanda/2021/pt/search
247Disallow: /biennaleluanda/2021/pt/search/*
248Disallow: /biennaleluanda/2021/pt/explore/*
249Disallow: /biennaleluanda/2021/pt/event/*/ical
250Disallow: /ja/search
251Disallow: /ja/search/*
252Disallow: /ja/explore/*
253Disallow: /ja/event/*/ical
254Disallow: /biennaleluanda/2021/ja/search
255Disallow: /biennaleluanda/2021/ja/search/*
256Disallow: /biennaleluanda/2021/ja/explore/*
257Disallow: /biennaleluanda/2021/ja/event/*/ical
258Disallow: /id/search
259Disallow: /id/search/*
260Disallow: /id/explore/*
261Disallow: /id/event/*/ical
262Disallow: /biennaleluanda/2021/id/search
263Disallow: /biennaleluanda/2021/id/search/*
264Disallow: /biennaleluanda/2021/id/explore/*
265Disallow: /biennaleluanda/2021/id/event/*/ical
266Disallow: /uk/search
267Disallow: /uk/search/*
268Disallow: /uk/explore/*
269Disallow: /uk/event/*/ical
270Disallow: /biennaleluanda/2021/uk/search
271Disallow: /biennaleluanda/2021/uk/search/*
272Disallow: /biennaleluanda/2021/uk/explore/*
273Disallow: /biennaleluanda/2021/uk/event/*/ical
274Disallow: /uz/search
275Disallow: /uz/search/*
276Disallow: /uz/explore/*
277Disallow: /uz/event/*/ical
278Disallow: /biennaleluanda/2021/uz/search
279Disallow: /biennaleluanda/2021/uz/search/*
280Disallow: /biennaleluanda/2021/uz/explore/*
281Disallow: /biennaleluanda/2021/uz/event/*/ical
282Disallow: /th/search
283Disallow: /th/search/*
284Disallow: /th/explore/*
285Disallow: /th/event/*/ical
286Disallow: /biennaleluanda/2021/th/search
287Disallow: /biennaleluanda/2021/th/search/*
288Disallow: /biennaleluanda/2021/th/explore/*
289Disallow: /biennaleluanda/2021/th/event/*/ical
290Disallow: /it/search
291Disallow: /it/search/*
292Disallow: /it/explore/*
293Disallow: /it/event/*/ical
294Disallow: /biennaleluanda/2021/it/search
295Disallow: /biennaleluanda/2021/it/search/*
296Disallow: /biennaleluanda/2021/it/explore/*
297Disallow: /biennaleluanda/2021/it/event/*/ical
298Disallow: /vi/search
299Disallow: /vi/search/*
300Disallow: /vi/explore/*
301Disallow: /vi/event/*/ical
302Disallow: /biennaleluanda/2021/vi/search
303Disallow: /biennaleluanda/2021/vi/search/*
304Disallow: /biennaleluanda/2021/vi/explore/*
305Disallow: /biennaleluanda/2021/vi/event/*/ical
306Disallow: /biodiversity/portal/*?f[*
307Disallow: /biodiversity/portal/*&f[*
308Disallow: /biodiversity/portal/*?f%5B0*
309Disallow: /biodiversity/portal/*&f%5B1*
310Disallow: /biodiversity/portal/search
311Disallow: /biodiversity/portal/en/search
312Disallow: /biodiversity/portal/en/search/*
313Disallow: /biodiversity/portal/en/explore/*
314Disallow: /biodiversity/portal/event/*/ical
315Disallow: /biodiversity/portal/en/event/*/ical
316Disallow: /biodiversity/portal/fr/search
317Disallow: /biodiversity/portal/fr/search/*
318Disallow: /biodiversity/portal/fr/explore/*
319Disallow: /biodiversity/portal/fr/event/*/ical
320Disallow: /biodiversity/portal/es/search
321Disallow: /biodiversity/portal/es/search/*
322Disallow: /biodiversity/portal/es/explore/*
323Disallow: /biodiversity/portal/es/event/*/ical
324Disallow: /biodiversity/portal/ru/search
325Disallow: /biodiversity/portal/ru/search/*
326Disallow: /biodiversity/portal/ru/explore/*
327Disallow: /biodiversity/portal/ru/event/*/ical
328Disallow: /biodiversity/portal/ar/search
329Disallow: /biodiversity/portal/ar/search/*
330Disallow: /biodiversity/portal/ar/explore/*
331Disallow: /biodiversity/portal/ar/event/*/ical
332Disallow: /biodiversity/portal/zh/search
333Disallow: /biodiversity/portal/zh/search/*
334Disallow: /biodiversity/portal/zh/explore/*
335Disallow: /biodiversity/portal/zh/event/*/ical
336Disallow: /biodiversity/portal/pt/search
337Disallow: /biodiversity/portal/pt/search/*
338Disallow: /biodiversity/portal/pt/explore/*
339Disallow: /biodiversity/portal/pt/event/*/ical
340Disallow: /biodiversity/portal/ja/search
341Disallow: /biodiversity/portal/ja/search/*
342Disallow: /biodiversity/portal/ja/explore/*
343Disallow: /biodiversity/portal/ja/event/*/ical
344Disallow: /biodiversity/portal/id/search
345Disallow: /biodiversity/portal/id/search/*
346Disallow: /biodiversity/portal/id/explore/*
347Disallow: /biodiversity/portal/id/event/*/ical
348Disallow: /biodiversity/portal/uk/search
349Disallow: /biodiversity/portal/uk/search/*
350Disallow: /biodiversity/portal/uk/explore/*
351Disallow: /biodiversity/portal/uk/event/*/ical
352Disallow: /biodiversity/portal/uz/search
353Disallow: /biodiversity/portal/uz/search/*
354Disallow: /biodiversity/portal/uz/explore/*
355Disallow: /biodiversity/portal/uz/event/*/ical
356Disallow: /biodiversity/portal/th/search
357Disallow: /biodiversity/portal/th/search/*
358Disallow: /biodiversity/portal/th/explore/*
359Disallow: /biodiversity/portal/th/event/*/ical
360Disallow: /biodiversity/portal/it/search
361Disallow: /biodiversity/portal/it/search/*
362Disallow: /biodiversity/portal/it/explore/*
363Disallow: /biodiversity/portal/it/event/*/ical
364Disallow: /biodiversity/portal/vi/search
365Disallow: /biodiversity/portal/vi/search/*
366Disallow: /biodiversity/portal/vi/explore/*
367Disallow: /biodiversity/portal/vi/event/*/ical
368Disallow: /creativity/*?f[*
369Disallow: /creativity/*&f[*
370Disallow: /creativity/*?f%5B0*
371Disallow: /creativity/*&f%5B1*
372Disallow: /creativity/search
373Disallow: /creativity/en/search
374Disallow: /creativity/en/search/*
375Disallow: /creativity/en/explore/*
376Disallow: /creativity/event/*/ical
377Disallow: /creativity/en/event/*/ical
378Disallow: /creativity/fr/search
379Disallow: /creativity/fr/search/*
380Disallow: /creativity/fr/explore/*
381Disallow: /creativity/fr/event/*/ical
382Disallow: /creativity/es/search
383Disallow: /creativity/es/search/*
384Disallow: /creativity/es/explore/*
385Disallow: /creativity/es/event/*/ical
386Disallow: /creativity/ru/search
387Disallow: /creativity/ru/search/*
388Disallow: /creativity/ru/explore/*
389Disallow: /creativity/ru/event/*/ical
390Disallow: /creativity/ar/search
391Disallow: /creativity/ar/search/*
392Disallow: /creativity/ar/explore/*
393Disallow: /creativity/ar/event/*/ical
394Disallow: /creativity/zh/search
395Disallow: /creativity/zh/search/*
396Disallow: /creativity/zh/explore/*
397Disallow: /creativity/zh/event/*/ical
398Disallow: /creativity/pt/search
399Disallow: /creativity/pt/search/*
400Disallow: /creativity/pt/explore/*
401Disallow: /creativity/pt/event/*/ical
402Disallow: /creativity/ja/search
403Disallow: /creativity/ja/search/*
404Disallow: /creativity/ja/explore/*
405Disallow: /creativity/ja/event/*/ical
406Disallow: /creativity/id/search
407Disallow: /creativity/id/search/*
408Disallow: /creativity/id/explore/*
409Disallow: /creativity/id/event/*/ical
410Disallow: /creativity/uk/search
411Disallow: /creativity/uk/search/*
412Disallow: /creativity/uk/explore/*
413Disallow: /creativity/uk/event/*/ical
414Disallow: /creativity/uz/search
415Disallow: /creativity/uz/search/*
416Disallow: /creativity/uz/explore/*
417Disallow: /creativity/uz/event/*/ical
418Disallow: /creativity/th/search
419Disallow: /creativity/th/search/*
420Disallow: /creativity/th/explore/*
421Disallow: /creativity/th/event/*/ical
422Disallow: /creativity/it/search
423Disallow: /creativity/it/search/*
424Disallow: /creativity/it/explore/*
425Disallow: /creativity/it/event/*/ical
426Disallow: /creativity/vi/search
427Disallow: /creativity/vi/search/*
428Disallow: /creativity/vi/explore/*
429Disallow: /creativity/vi/event/*/ical
430Disallow: /interculturaldialogu/*?f[*
431Disallow: /interculturaldialogu/*&f[*
432Disallow: /interculturaldialogu/*?f%5B0*
433Disallow: /interculturaldialogu/*&f%5B1*
434Disallow: /interculturaldialogu/search
435Disallow: /interculturaldialogu/en/search
436Disallow: /interculturaldialogu/en/search/*
437Disallow: /interculturaldialogu/en/explore/*
438Disallow: /interculturaldialogu/event/*/ical
439Disallow: /interculturaldialogu/en/event/*/ical
440Disallow: /interculturaldialogu/fr/search
441Disallow: /interculturaldialogu/fr/search/*
442Disallow: /interculturaldialogu/fr/explore/*
443Disallow: /interculturaldialogu/fr/event/*/ical
444Disallow: /interculturaldialogu/es/search
445Disallow: /interculturaldialogu/es/search/*
446Disallow: /interculturaldialogu/es/explore/*
447Disallow: /interculturaldialogu/es/event/*/ical
448Disallow: /interculturaldialogu/ru/search
449Disallow: /interculturaldialogu/ru/search/*
450Disallow: /interculturaldialogu/ru/explore/*
451Disallow: /interculturaldialogu/ru/event/*/ical
452Disallow: /interculturaldialogu/ar/search
453Disallow: /interculturaldialogu/ar/search/*
454Disallow: /interculturaldialogu/ar/explore/*
455Disallow: /interculturaldialogu/ar/event/*/ical
456Disallow: /interculturaldialogu/zh/search
457Disallow: /interculturaldialogu/zh/search/*
458Disallow: /interculturaldialogu/zh/explore/*
459Disallow: /interculturaldialogu/zh/event/*/ical
460Disallow: /interculturaldialogu/pt/search
461Disallow: /interculturaldialogu/pt/search/*
462Disallow: /interculturaldialogu/pt/explore/*
463Disallow: /interculturaldialogu/pt/event/*/ical
464Disallow: /interculturaldialogu/ja/search
465Disallow: /interculturaldialogu/ja/search/*
466Disallow: /interculturaldialogu/ja/explore/*
467Disallow: /interculturaldialogu/ja/event/*/ical
468Disallow: /interculturaldialogu/id/search
469Disallow: /interculturaldialogu/id/search/*
470Disallow: /interculturaldialogu/id/explore/*
471Disallow: /interculturaldialogu/id/event/*/ical
472Disallow: /interculturaldialogu/uk/search
473Disallow: /interculturaldialogu/uk/search/*
474Disallow: /interculturaldialogu/uk/explore/*
475Disallow: /interculturaldialogu/uk/event/*/ical
476Disallow: /interculturaldialogu/uz/search
477Disallow: /interculturaldialogu/uz/search/*
478Disallow: /interculturaldialogu/uz/explore/*
479Disallow: /interculturaldialogu/uz/event/*/ical
480Disallow: /interculturaldialogu/th/search
481Disallow: /interculturaldialogu/th/search/*
482Disallow: /interculturaldialogu/th/explore/*
483Disallow: /interculturaldialogu/th/event/*/ical
484Disallow: /interculturaldialogu/it/search
485Disallow: /interculturaldialogu/it/search/*
486Disallow: /interculturaldialogu/it/explore/*
487Disallow: /interculturaldialogu/it/event/*/ical
488Disallow: /interculturaldialogu/vi/search
489Disallow: /interculturaldialogu/vi/search/*
490Disallow: /interculturaldialogu/vi/explore/*
491Disallow: /interculturaldialogu/vi/event/*/ical
492Disallow: /reports/gem-report/*?f[*
493Disallow: /reports/gem-report/*&f[*
494Disallow: /reports/gem-report/*?f%5B0*
495Disallow: /reports/gem-report/*&f%5B1*
496Disallow: /reports/gem-report/search
497Disallow: /reports/gem-report/en/search
498Disallow: /reports/gem-report/en/search/*
499Disallow: /reports/gem-report/en/explore/*
500Disallow: /reports/gem-report/event/*/ical
501Disallow: /reports/gem-report/en/event/*/ical
502Disallow: /reports/gem-report/fr/search
503Disallow: /reports/gem-report/fr/search/*
504Disallow: /reports/gem-report/fr/explore/*
505Disallow: /reports/gem-report/fr/event/*/ical
506Disallow: /reports/gem-report/es/search
507Disallow: /reports/gem-report/es/search/*
508Disallow: /reports/gem-report/es/explore/*
509Disallow: /reports/gem-report/es/event/*/ical
510Disallow: /reports/gem-report/ru/search
511Disallow: /reports/gem-report/ru/search/*
512Disallow: /reports/gem-report/ru/explore/*
513Disallow: /reports/gem-report/ru/event/*/ical
514Disallow: /reports/gem-report/ar/search
515Disallow: /reports/gem-report/ar/search/*
516Disallow: /reports/gem-report/ar/explore/*
517Disallow: /reports/gem-report/ar/event/*/ical
518Disallow: /reports/gem-report/zh/search
519Disallow: /reports/gem-report/zh/search/*
520Disallow: /reports/gem-report/zh/explore/*
521Disallow: /reports/gem-report/zh/event/*/ical
522Disallow: /reports/gem-report/pt/search
523Disallow: /reports/gem-report/pt/search/*
524Disallow: /reports/gem-report/pt/explore/*
525Disallow: /reports/gem-report/pt/event/*/ical
526Disallow: /reports/gem-report/ja/search
527Disallow: /reports/gem-report/ja/search/*
528Disallow: /reports/gem-report/ja/explore/*
529Disallow: /reports/gem-report/ja/event/*/ical
530Disallow: /reports/gem-report/id/search
531Disallow: /reports/gem-report/id/search/*
532Disallow: /reports/gem-report/id/explore/*
533Disallow: /reports/gem-report/id/event/*/ical
534Disallow: /reports/gem-report/uk/search
535Disallow: /reports/gem-report/uk/search/*
536Disallow: /reports/gem-report/uk/explore/*
537Disallow: /reports/gem-report/uk/event/*/ical
538Disallow: /reports/gem-report/uz/search
539Disallow: /reports/gem-report/uz/search/*
540Disallow: /reports/gem-report/uz/explore/*
541Disallow: /reports/gem-report/uz/event/*/ical
542Disallow: /reports/gem-report/th/search
543Disallow: /reports/gem-report/th/search/*
544Disallow: /reports/gem-report/th/explore/*
545Disallow: /reports/gem-report/th/event/*/ical
546Disallow: /reports/gem-report/it/search
547Disallow: /reports/gem-report/it/search/*
548Disallow: /reports/gem-report/it/explore/*
549Disallow: /reports/gem-report/it/event/*/ical
550Disallow: /reports/gem-report/vi/search
551Disallow: /reports/gem-report/vi/search/*
552Disallow: /reports/gem-report/vi/explore/*
553Disallow: /reports/gem-report/vi/event/*/ical
554Disallow: /sdg4education2030/*?f[*
555Disallow: /sdg4education2030/*&f[*
556Disallow: /sdg4education2030/*?f%5B0*
557Disallow: /sdg4education2030/*&f%5B1*
558Disallow: /sdg4education2030/search
559Disallow: /sdg4education2030/en/search
560Disallow: /sdg4education2030/en/search/*
561Disallow: /sdg4education2030/en/explore/*
562Disallow: /sdg4education2030/event/*/ical
563Disallow: /sdg4education2030/en/event/*/ical
564Disallow: /sdg4education2030/fr/search
565Disallow: /sdg4education2030/fr/search/*
566Disallow: /sdg4education2030/fr/explore/*
567Disallow: /sdg4education2030/fr/event/*/ical
568Disallow: /sdg4education2030/es/search
569Disallow: /sdg4education2030/es/search/*
570Disallow: /sdg4education2030/es/explore/*
571Disallow: /sdg4education2030/es/event/*/ical
572Disallow: /sdg4education2030/ru/search
573Disallow: /sdg4education2030/ru/search/*
574Disallow: /sdg4education2030/ru/explore/*
575Disallow: /sdg4education2030/ru/event/*/ical
576Disallow: /sdg4education2030/ar/search
577Disallow: /sdg4education2030/ar/search/*
578Disallow: /sdg4education2030/ar/explore/*
579Disallow: /sdg4education2030/ar/event/*/ical
580Disallow: /sdg4education2030/zh/search
581Disallow: /sdg4education2030/zh/search/*
582Disallow: /sdg4education2030/zh/explore/*
583Disallow: /sdg4education2030/zh/event/*/ical
584Disallow: /sdg4education2030/pt/search
585Disallow: /sdg4education2030/pt/search/*
586Disallow: /sdg4education2030/pt/explore/*
587Disallow: /sdg4education2030/pt/event/*/ical
588Disallow: /sdg4education2030/ja/search
589Disallow: /sdg4education2030/ja/search/*
590Disallow: /sdg4education2030/ja/explore/*
591Disallow: /sdg4education2030/ja/event/*/ical
592Disallow: /sdg4education2030/id/search
593Disallow: /sdg4education2030/id/search/*
594Disallow: /sdg4education2030/id/explore/*
595Disallow: /sdg4education2030/id/event/*/ical
596Disallow: /sdg4education2030/uk/search
597Disallow: /sdg4education2030/uk/search/*
598Disallow: /sdg4education2030/uk/explore/*
599Disallow: /sdg4education2030/uk/event/*/ical
600Disallow: /sdg4education2030/uz/search
601Disallow: /sdg4education2030/uz/search/*
602Disallow: /sdg4education2030/uz/explore/*
603Disallow: /sdg4education2030/uz/event/*/ical
604Disallow: /sdg4education2030/th/search
605Disallow: /sdg4education2030/th/search/*
606Disallow: /sdg4education2030/th/explore/*
607Disallow: /sdg4education2030/th/event/*/ical
608Disallow: /sdg4education2030/it/search
609Disallow: /sdg4education2030/it/search/*
610Disallow: /sdg4education2030/it/explore/*
611Disallow: /sdg4education2030/it/event/*/ical
612Disallow: /sdg4education2030/vi/search
613Disallow: /sdg4education2030/vi/search/*
614Disallow: /sdg4education2030/vi/explore/*
615Disallow: /sdg4education2030/vi/event/*/ical
616Disallow: /gem-report/*?f[*
617Disallow: /gem-report/*&f[*
618Disallow: /gem-report/*?f%5B0*
619Disallow: /gem-report/*&f%5B1*
620Disallow: /gem-report/search
621Disallow: /gem-report/en/search
622Disallow: /gem-report/en/search/*
623Disallow: /gem-report/en/explore/*
624Disallow: /gem-report/event/*/ical
625Disallow: /gem-report/en/event/*/ical
626Disallow: /gem-report/fr/search
627Disallow: /gem-report/fr/search/*
628Disallow: /gem-report/fr/explore/*
629Disallow: /gem-report/fr/event/*/ical
630Disallow: /gem-report/es/search
631Disallow: /gem-report/es/search/*
632Disallow: /gem-report/es/explore/*
633Disallow: /gem-report/es/event/*/ical
634Disallow: /gem-report/ru/search
635Disallow: /gem-report/ru/search/*
636Disallow: /gem-report/ru/explore/*
637Disallow: /gem-report/ru/event/*/ical
638Disallow: /gem-report/ar/search
639Disallow: /gem-report/ar/search/*
640Disallow: /gem-report/ar/explore/*
641Disallow: /gem-report/ar/event/*/ical
642Disallow: /gem-report/zh/search
643Disallow: /gem-report/zh/search/*
644Disallow: /gem-report/zh/explore/*
645Disallow: /gem-report/zh/event/*/ical
646Disallow: /gem-report/pt/search
647Disallow: /gem-report/pt/search/*
648Disallow: /gem-report/pt/explore/*
649Disallow: /gem-report/pt/event/*/ical
650Disallow: /gem-report/ja/search
651Disallow: /gem-report/ja/search/*
652Disallow: /gem-report/ja/explore/*
653Disallow: /gem-report/ja/event/*/ical
654Disallow: /gem-report/id/search
655Disallow: /gem-report/id/search/*
656Disallow: /gem-report/id/explore/*
657Disallow: /gem-report/id/event/*/ical
658Disallow: /gem-report/uk/search
659Disallow: /gem-report/uk/search/*
660Disallow: /gem-report/uk/explore/*
661Disallow: /gem-report/uk/event/*/ical
662Disallow: /gem-report/uz/search
663Disallow: /gem-report/uz/search/*
664Disallow: /gem-report/uz/explore/*
665Disallow: /gem-report/uz/event/*/ical
666Disallow: /gem-report/th/search
667Disallow: /gem-report/th/search/*
668Disallow: /gem-report/th/explore/*
669Disallow: /gem-report/th/event/*/ical
670Disallow: /gem-report/it/search
671Disallow: /gem-report/it/search/*
672Disallow: /gem-report/it/explore/*
673Disallow: /gem-report/it/event/*/ical
674Disallow: /gem-report/vi/search
675Disallow: /gem-report/vi/search/*
676Disallow: /gem-report/vi/explore/*
677Disallow: /gem-report/vi/event/*/ical
678Disallow: /mab/50anniversary/*?f[*
679Disallow: /mab/50anniversary/*&f[*
680Disallow: /mab/50anniversary/*?f%5B0*
681Disallow: /mab/50anniversary/*&f%5B1*
682Disallow: /mab/50anniversary/search
683Disallow: /mab/50anniversary/en/search
684Disallow: /mab/50anniversary/en/search/*
685Disallow: /mab/50anniversary/en/explore/*
686Disallow: /mab/50anniversary/event/*/ical
687Disallow: /mab/50anniversary/en/event/*/ical
688Disallow: /mab/50anniversary/fr/search
689Disallow: /mab/50anniversary/fr/search/*
690Disallow: /mab/50anniversary/fr/explore/*
691Disallow: /mab/50anniversary/fr/event/*/ical
692Disallow: /mab/50anniversary/es/search
693Disallow: /mab/50anniversary/es/search/*
694Disallow: /mab/50anniversary/es/explore/*
695Disallow: /mab/50anniversary/es/event/*/ical
696Disallow: /mab/50anniversary/ru/search
697Disallow: /mab/50anniversary/ru/search/*
698Disallow: /mab/50anniversary/ru/explore/*
699Disallow: /mab/50anniversary/ru/event/*/ical
700Disallow: /mab/50anniversary/ar/search
701Disallow: /mab/50anniversary/ar/search/*
702Disallow: /mab/50anniversary/ar/explore/*
703Disallow: /mab/50anniversary/ar/event/*/ical
704Disallow: /mab/50anniversary/zh/search
705Disallow: /mab/50anniversary/zh/search/*
706Disallow: /mab/50anniversary/zh/explore/*
707Disallow: /mab/50anniversary/zh/event/*/ical
708Disallow: /mab/50anniversary/pt/search
709Disallow: /mab/50anniversary/pt/search/*
710Disallow: /mab/50anniversary/pt/explore/*
711Disallow: /mab/50anniversary/pt/event/*/ical
712Disallow: /mab/50anniversary/ja/search
713Disallow: /mab/50anniversary/ja/search/*
714Disallow: /mab/50anniversary/ja/explore/*
715Disallow: /mab/50anniversary/ja/event/*/ical
716Disallow: /mab/50anniversary/id/search
717Disallow: /mab/50anniversary/id/search/*
718Disallow: /mab/50anniversary/id/explore/*
719Disallow: /mab/50anniversary/id/event/*/ical
720Disallow: /mab/50anniversary/uk/search
721Disallow: /mab/50anniversary/uk/search/*
722Disallow: /mab/50anniversary/uk/explore/*
723Disallow: /mab/50anniversary/uk/event/*/ical
724Disallow: /mab/50anniversary/uz/search
725Disallow: /mab/50anniversary/uz/search/*
726Disallow: /mab/50anniversary/uz/explore/*
727Disallow: /mab/50anniversary/uz/event/*/ical
728Disallow: /mab/50anniversary/th/search
729Disallow: /mab/50anniversary/th/search/*
730Disallow: /mab/50anniversary/th/explore/*
731Disallow: /mab/50anniversary/th/event/*/ical
732Disallow: /mab/50anniversary/it/search
733Disallow: /mab/50anniversary/it/search/*
734Disallow: /mab/50anniversary/it/explore/*
735Disallow: /mab/50anniversary/it/event/*/ical
736Disallow: /mab/50anniversary/vi/search
737Disallow: /mab/50anniversary/vi/search/*
738Disallow: /mab/50anniversary/vi/explore/*
739Disallow: /mab/50anniversary/vi/event/*/ical
740Disallow: /mil4teachers/*?f[*
741Disallow: /mil4teachers/*&f[*
742Disallow: /mil4teachers/*?f%5B0*
743Disallow: /mil4teachers/*&f%5B1*
744Disallow: /mil4teachers/search
745Disallow: /mil4teachers/en/search
746Disallow: /mil4teachers/en/search/*
747Disallow: /mil4teachers/en/explore/*
748Disallow: /mil4teachers/event/*/ical
749Disallow: /mil4teachers/en/event/*/ical
750Disallow: /mil4teachers/fr/search
751Disallow: /mil4teachers/fr/search/*
752Disallow: /mil4teachers/fr/explore/*
753Disallow: /mil4teachers/fr/event/*/ical
754Disallow: /mil4teachers/es/search
755Disallow: /mil4teachers/es/search/*
756Disallow: /mil4teachers/es/explore/*
757Disallow: /mil4teachers/es/event/*/ical
758Disallow: /mil4teachers/ru/search
759Disallow: /mil4teachers/ru/search/*
760Disallow: /mil4teachers/ru/explore/*
761Disallow: /mil4teachers/ru/event/*/ical
762Disallow: /mil4teachers/ar/search
763Disallow: /mil4teachers/ar/search/*
764Disallow: /mil4teachers/ar/explore/*
765Disallow: /mil4teachers/ar/event/*/ical
766Disallow: /mil4teachers/zh/search
767Disallow: /mil4teachers/zh/search/*
768Disallow: /mil4teachers/zh/explore/*
769Disallow: /mil4teachers/zh/event/*/ical
770Disallow: /mil4teachers/pt/search
771Disallow: /mil4teachers/pt/search/*
772Disallow: /mil4teachers/pt/explore/*
773Disallow: /mil4teachers/pt/event/*/ical
774Disallow: /mil4teachers/ja/search
775Disallow: /mil4teachers/ja/search/*
776Disallow: /mil4teachers/ja/explore/*
777Disallow: /mil4teachers/ja/event/*/ical
778Disallow: /mil4teachers/id/search
779Disallow: /mil4teachers/id/search/*
780Disallow: /mil4teachers/id/explore/*
781Disallow: /mil4teachers/id/event/*/ical
782Disallow: /mil4teachers/uk/search
783Disallow: /mil4teachers/uk/search/*
784Disallow: /mil4teachers/uk/explore/*
785Disallow: /mil4teachers/uk/event/*/ical
786Disallow: /mil4teachers/uz/search
787Disallow: /mil4teachers/uz/search/*
788Disallow: /mil4teachers/uz/explore/*
789Disallow: /mil4teachers/uz/event/*/ical
790Disallow: /mil4teachers/th/search
791Disallow: /mil4teachers/th/search/*
792Disallow: /mil4teachers/th/explore/*
793Disallow: /mil4teachers/th/event/*/ical
794Disallow: /mil4teachers/it/search
795Disallow: /mil4teachers/it/search/*
796Disallow: /mil4teachers/it/explore/*
797Disallow: /mil4teachers/it/event/*/ical
798Disallow: /mil4teachers/vi/search
799Disallow: /mil4teachers/vi/search/*
800Disallow: /mil4teachers/vi/explore/*
801Disallow: /mil4teachers/vi/event/*/ical
802Disallow: /thics-ai/*?f[*
803Disallow: /thics-ai/*&f[*
804Disallow: /thics-ai/*?f%5B0*
805Disallow: /thics-ai/*&f%5B1*
806Disallow: /thics-ai/search
807Disallow: /thics-ai/en/search
808Disallow: /thics-ai/en/search/*
809Disallow: /thics-ai/en/explore/*
810Disallow: /thics-ai/event/*/ical
811Disallow: /thics-ai/en/event/*/ical
812Disallow: /thics-ai/fr/search
813Disallow: /thics-ai/fr/search/*
814Disallow: /thics-ai/fr/explore/*
815Disallow: /thics-ai/fr/event/*/ical
816Disallow: /thics-ai/es/search
817Disallow: /thics-ai/es/search/*
818Disallow: /thics-ai/es/explore/*
819Disallow: /thics-ai/es/event/*/ical
820Disallow: /thics-ai/ru/search
821Disallow: /thics-ai/ru/search/*
822Disallow: /thics-ai/ru/explore/*
823Disallow: /thics-ai/ru/event/*/ical
824Disallow: /thics-ai/ar/search
825Disallow: /thics-ai/ar/search/*
826Disallow: /thics-ai/ar/explore/*
827Disallow: /thics-ai/ar/event/*/ical
828Disallow: /thics-ai/zh/search
829Disallow: /thics-ai/zh/search/*
830Disallow: /thics-ai/zh/explore/*
831Disallow: /thics-ai/zh/event/*/ical
832Disallow: /thics-ai/pt/search
833Disallow: /thics-ai/pt/search/*
834Disallow: /thics-ai/pt/explore/*
835Disallow: /thics-ai/pt/event/*/ical
836Disallow: /thics-ai/ja/search
837Disallow: /thics-ai/ja/search/*
838Disallow: /thics-ai/ja/explore/*
839Disallow: /thics-ai/ja/event/*/ical
840Disallow: /thics-ai/id/search
841Disallow: /thics-ai/id/search/*
842Disallow: /thics-ai/id/explore/*
843Disallow: /thics-ai/id/event/*/ical
844Disallow: /thics-ai/uk/search
845Disallow: /thics-ai/uk/search/*
846Disallow: /thics-ai/uk/explore/*
847Disallow: /thics-ai/uk/event/*/ical
848Disallow: /thics-ai/uz/search
849Disallow: /thics-ai/uz/search/*
850Disallow: /thics-ai/uz/explore/*
851Disallow: /thics-ai/uz/event/*/ical
852Disallow: /thics-ai/th/search
853Disallow: /thics-ai/th/search/*
854Disallow: /thics-ai/th/explore/*
855Disallow: /thics-ai/th/event/*/ical
856Disallow: /thics-ai/it/search
857Disallow: /thics-ai/it/search/*
858Disallow: /thics-ai/it/explore/*
859Disallow: /thics-ai/it/event/*/ical
860Disallow: /thics-ai/vi/search
861Disallow: /thics-ai/vi/search/*
862Disallow: /thics-ai/vi/explore/*
863Disallow: /thics-ai/vi/event/*/ical
864Disallow: /reports/access-to-information/2021/*?f[*
865Disallow: /reports/access-to-information/2021/*&f[*
866Disallow: /reports/access-to-information/2021/*?f%5B0*
867Disallow: /reports/access-to-information/2021/*&f%5B1*
868Disallow: /reports/access-to-information/2021/search
869Disallow: /reports/access-to-information/2021/en/search
870Disallow: /reports/access-to-information/2021/en/search/*
871Disallow: /reports/access-to-information/2021/en/explore/*
872Disallow: /reports/access-to-information/2021/event/*/ical
873Disallow: /reports/access-to-information/2021/en/event/*/ical
874Disallow: /reports/access-to-information/2021/fr/search
875Disallow: /reports/access-to-information/2021/fr/search/*
876Disallow: /reports/access-to-information/2021/fr/explore/*
877Disallow: /reports/access-to-information/2021/fr/event/*/ical
878Disallow: /reports/access-to-information/2021/es/search
879Disallow: /reports/access-to-information/2021/es/search/*
880Disallow: /reports/access-to-information/2021/es/explore/*
881Disallow: /reports/access-to-information/2021/es/event/*/ical
882Disallow: /reports/access-to-information/2021/ru/search
883Disallow: /reports/access-to-information/2021/ru/search/*
884Disallow: /reports/access-to-information/2021/ru/explore/*
885Disallow: /reports/access-to-information/2021/ru/event/*/ical
886Disallow: /reports/access-to-information/2021/ar/search
887Disallow: /reports/access-to-information/2021/ar/search/*
888Disallow: /reports/access-to-information/2021/ar/explore/*
889Disallow: /reports/access-to-information/2021/ar/event/*/ical
890Disallow: /reports/access-to-information/2021/zh/search
891Disallow: /reports/access-to-information/2021/zh/search/*
892Disallow: /reports/access-to-information/2021/zh/explore/*
893Disallow: /reports/access-to-information/2021/zh/event/*/ical
894Disallow: /reports/access-to-information/2021/pt/search
895Disallow: /reports/access-to-information/2021/pt/search/*
896Disallow: /reports/access-to-information/2021/pt/explore/*
897Disallow: /reports/access-to-information/2021/pt/event/*/ical
898Disallow: /reports/access-to-information/2021/ja/search
899Disallow: /reports/access-to-information/2021/ja/search/*
900Disallow: /reports/access-to-information/2021/ja/explore/*
901Disallow: /reports/access-to-information/2021/ja/event/*/ical
902Disallow: /reports/access-to-information/2021/id/search
903Disallow: /reports/access-to-information/2021/id/search/*
904Disallow: /reports/access-to-information/2021/id/explore/*
905Disallow: /reports/access-to-information/2021/id/event/*/ical
906Disallow: /reports/access-to-information/2021/uk/search
907Disallow: /reports/access-to-information/2021/uk/search/*
908Disallow: /reports/access-to-information/2021/uk/explore/*
909Disallow: /reports/access-to-information/2021/uk/event/*/ical
910Disallow: /reports/access-to-information/2021/uz/search
911Disallow: /reports/access-to-information/2021/uz/search/*
912Disallow: /reports/access-to-information/2021/uz/explore/*
913Disallow: /reports/access-to-information/2021/uz/event/*/ical
914Disallow: /reports/access-to-information/2021/th/search
915Disallow: /reports/access-to-information/2021/th/search/*
916Disallow: /reports/access-to-information/2021/th/explore/*
917Disallow: /reports/access-to-information/2021/th/event/*/ical
918Disallow: /reports/access-to-information/2021/it/search
919Disallow: /reports/access-to-information/2021/it/search/*
920Disallow: /reports/access-to-information/2021/it/explore/*
921Disallow: /reports/access-to-information/2021/it/event/*/ical
922Disallow: /reports/access-to-information/2021/vi/search
923Disallow: /reports/access-to-information/2021/vi/search/*
924Disallow: /reports/access-to-information/2021/vi/explore/*
925Disallow: /reports/access-to-information/2021/vi/event/*/ical
926Disallow: /reports/access-to-information/2022/*?f[*
927Disallow: /reports/access-to-information/2022/*&f[*
928Disallow: /reports/access-to-information/2022/*?f%5B0*
929Disallow: /reports/access-to-information/2022/*&f%5B1*
930Disallow: /reports/access-to-information/2022/search
931Disallow: /reports/access-to-information/2022/en/search
932Disallow: /reports/access-to-information/2022/en/search/*
933Disallow: /reports/access-to-information/2022/en/explore/*
934Disallow: /reports/access-to-information/2022/event/*/ical
935Disallow: /reports/access-to-information/2022/en/event/*/ical
936Disallow: /reports/access-to-information/2022/fr/search
937Disallow: /reports/access-to-information/2022/fr/search/*
938Disallow: /reports/access-to-information/2022/fr/explore/*
939Disallow: /reports/access-to-information/2022/fr/event/*/ical
940Disallow: /reports/access-to-information/2022/es/search
941Disallow: /reports/access-to-information/2022/es/search/*
942Disallow: /reports/access-to-information/2022/es/explore/*
943Disallow: /reports/access-to-information/2022/es/event/*/ical
944Disallow: /reports/access-to-information/2022/ru/search
945Disallow: /reports/access-to-information/2022/ru/search/*
946Disallow: /reports/access-to-information/2022/ru/explore/*
947Disallow: /reports/access-to-information/2022/ru/event/*/ical
948Disallow: /reports/access-to-information/2022/ar/search
949Disallow: /reports/access-to-information/2022/ar/search/*
950Disallow: /reports/access-to-information/2022/ar/explore/*
951Disallow: /reports/access-to-information/2022/ar/event/*/ical
952Disallow: /reports/access-to-information/2022/zh/search
953Disallow: /reports/access-to-information/2022/zh/search/*
954Disallow: /reports/access-to-information/2022/zh/explore/*
955Disallow: /reports/access-to-information/2022/zh/event/*/ical
956Disallow: /reports/access-to-information/2022/pt/search
957Disallow: /reports/access-to-information/2022/pt/search/*
958Disallow: /reports/access-to-information/2022/pt/explore/*
959Disallow: /reports/access-to-information/2022/pt/event/*/ical
960Disallow: /reports/access-to-information/2022/ja/search
961Disallow: /reports/access-to-information/2022/ja/search/*
962Disallow: /reports/access-to-information/2022/ja/explore/*
963Disallow: /reports/access-to-information/2022/ja/event/*/ical
964Disallow: /reports/access-to-information/2022/id/search
965Disallow: /reports/access-to-information/2022/id/search/*
966Disallow: /reports/access-to-information/2022/id/explore/*
967Disallow: /reports/access-to-information/2022/id/event/*/ical
968Disallow: /reports/access-to-information/2022/uk/search
969Disallow: /reports/access-to-information/2022/uk/search/*
970Disallow: /reports/access-to-information/2022/uk/explore/*
971Disallow: /reports/access-to-information/2022/uk/event/*/ical
972Disallow: /reports/access-to-information/2022/uz/search
973Disallow: /reports/access-to-information/2022/uz/search/*
974Disallow: /reports/access-to-information/2022/uz/explore/*
975Disallow: /reports/access-to-information/2022/uz/event/*/ical
976Disallow: /reports/access-to-information/2022/th/search
977Disallow: /reports/access-to-information/2022/th/search/*
978Disallow: /reports/access-to-information/2022/th/explore/*
979Disallow: /reports/access-to-information/2022/th/event/*/ical
980Disallow: /reports/access-to-information/2022/it/search
981Disallow: /reports/access-to-information/2022/it/search/*
982Disallow: /reports/access-to-information/2022/it/explore/*
983Disallow: /reports/access-to-information/2022/it/event/*/ical
984Disallow: /reports/access-to-information/2022/vi/search
985Disallow: /reports/access-to-information/2022/vi/search/*
986Disallow: /reports/access-to-information/2022/vi/explore/*
987Disallow: /reports/access-to-information/2022/vi/event/*/ical
988Disallow: /reports/reshaping-creativity/2022/*?f[*
989Disallow: /reports/reshaping-creativity/2022/*&f[*
990Disallow: /reports/reshaping-creativity/2022/*?f%5B0*
991Disallow: /reports/reshaping-creativity/2022/*&f%5B1*
992Disallow: /reports/reshaping-creativity/2022/search
993Disallow: /reports/reshaping-creativity/2022/en/search
994Disallow: /reports/reshaping-creativity/2022/en/search/*
995Disallow: /reports/reshaping-creativity/2022/en/explore/*
996Disallow: /reports/reshaping-creativity/2022/event/*/ical
997Disallow: /reports/reshaping-creativity/2022/en/event/*/ical
998Disallow: /reports/reshaping-creativity/2022/fr/search
999Disallow: /reports/reshaping-creativity/2022/fr/search/*
1000Disallow: /reports/reshaping-creativity/2022/fr/explore/*
1001Disallow: /reports/reshaping-creativity/2022/fr/event/*/ical
1002Disallow: /reports/reshaping-creativity/2022/es/search
1003Disallow: /reports/reshaping-creativity/2022/es/search/*
1004Disallow: /reports/reshaping-creativity/2022/es/explore/*
1005Disallow: /reports/reshaping-creativity/2022/es/event/*/ical
1006Disallow: /reports/reshaping-creativity/2022/ru/search
1007Disallow: /reports/reshaping-creativity/2022/ru/search/*
1008Disallow: /reports/reshaping-creativity/2022/ru/explore/*
1009Disallow: /reports/reshaping-creativity/2022/ru/event/*/ical
1010Disallow: /reports/reshaping-creativity/2022/ar/search
1011Disallow: /reports/reshaping-creativity/2022/ar/search/*
1012Disallow: /reports/reshaping-creativity/2022/ar/explore/*
1013Disallow: /reports/reshaping-creativity/2022/ar/event/*/ical
1014Disallow: /reports/reshaping-creativity/2022/zh/search
1015Disallow: /reports/reshaping-creativity/2022/zh/search/*
1016Disallow: /reports/reshaping-creativity/2022/zh/explore/*
1017Disallow: /reports/reshaping-creativity/2022/zh/event/*/ical
1018Disallow: /reports/reshaping-creativity/2022/pt/search
1019Disallow: /reports/reshaping-creativity/2022/pt/search/*
1020Disallow: /reports/reshaping-creativity/2022/pt/explore/*
1021Disallow: /reports/reshaping-creativity/2022/pt/event/*/ical
1022Disallow: /reports/reshaping-creativity/2022/ja/search
1023Disallow: /reports/reshaping-creativity/2022/ja/search/*
1024Disallow: /reports/reshaping-creativity/2022/ja/explore/*
1025Disallow: /reports/reshaping-creativity/2022/ja/event/*/ical
1026Disallow: /reports/reshaping-creativity/2022/id/search
1027Disallow: /reports/reshaping-creativity/2022/id/search/*
1028Disallow: /reports/reshaping-creativity/2022/id/explore/*
1029Disallow: /reports/reshaping-creativity/2022/id/event/*/ical
1030Disallow: /reports/reshaping-creativity/2022/uk/search
1031Disallow: /reports/reshaping-creativity/2022/uk/search/*
1032Disallow: /reports/reshaping-creativity/2022/uk/explore/*
1033Disallow: /reports/reshaping-creativity/2022/uk/event/*/ical
1034Disallow: /reports/reshaping-creativity/2022/uz/search
1035Disallow: /reports/reshaping-creativity/2022/uz/search/*
1036Disallow: /reports/reshaping-creativity/2022/uz/explore/*
1037Disallow: /reports/reshaping-creativity/2022/uz/event/*/ical
1038Disallow: /reports/reshaping-creativity/2022/th/search
1039Disallow: /reports/reshaping-creativity/2022/th/search/*
1040Disallow: /reports/reshaping-creativity/2022/th/explore/*
1041Disallow: /reports/reshaping-creativity/2022/th/event/*/ical
1042Disallow: /reports/reshaping-creativity/2022/it/search
1043Disallow: /reports/reshaping-creativity/2022/it/search/*
1044Disallow: /reports/reshaping-creativity/2022/it/explore/*
1045Disallow: /reports/reshaping-creativity/2022/it/event/*/ical
1046Disallow: /reports/reshaping-creativity/2022/vi/search
1047Disallow: /reports/reshaping-creativity/2022/vi/search/*
1048Disallow: /reports/reshaping-creativity/2022/vi/explore/*
1049Disallow: /reports/reshaping-creativity/2022/vi/event/*/ical
1050Disallow: /reports/science/2021/*?f[*
1051Disallow: /reports/science/2021/*&f[*
1052Disallow: /reports/science/2021/*?f%5B0*
1053Disallow: /reports/science/2021/*&f%5B1*
1054Disallow: /reports/science/2021/search
1055Disallow: /reports/science/2021/en/search
1056Disallow: /reports/science/2021/en/search/*
1057Disallow: /reports/science/2021/en/explore/*
1058Disallow: /reports/science/2021/event/*/ical
1059Disallow: /reports/science/2021/en/event/*/ical
1060Disallow: /reports/science/2021/fr/search
1061Disallow: /reports/science/2021/fr/search/*
1062Disallow: /reports/science/2021/fr/explore/*
1063Disallow: /reports/science/2021/fr/event/*/ical
1064Disallow: /reports/science/2021/es/search
1065Disallow: /reports/science/2021/es/search/*
1066Disallow: /reports/science/2021/es/explore/*
1067Disallow: /reports/science/2021/es/event/*/ical
1068Disallow: /reports/science/2021/ru/search
1069Disallow: /reports/science/2021/ru/search/*
1070Disallow: /reports/science/2021/ru/explore/*
1071Disallow: /reports/science/2021/ru/event/*/ical
1072Disallow: /reports/science/2021/ar/search
1073Disallow: /reports/science/2021/ar/search/*
1074Disallow: /reports/science/2021/ar/explore/*
1075Disallow: /reports/science/2021/ar/event/*/ical
1076Disallow: /reports/science/2021/zh/search
1077Disallow: /reports/science/2021/zh/search/*
1078Disallow: /reports/science/2021/zh/explore/*
1079Disallow: /reports/science/2021/zh/event/*/ical
1080Disallow: /reports/science/2021/pt/search
1081Disallow: /reports/science/2021/pt/search/*
1082Disallow: /reports/science/2021/pt/explore/*
1083Disallow: /reports/science/2021/pt/event/*/ical
1084Disallow: /reports/science/2021/ja/search
1085Disallow: /reports/science/2021/ja/search/*
1086Disallow: /reports/science/2021/ja/explore/*
1087Disallow: /reports/science/2021/ja/event/*/ical
1088Disallow: /reports/science/2021/id/search
1089Disallow: /reports/science/2021/id/search/*
1090Disallow: /reports/science/2021/id/explore/*
1091Disallow: /reports/science/2021/id/event/*/ical
1092Disallow: /reports/science/2021/uk/search
1093Disallow: /reports/science/2021/uk/search/*
1094Disallow: /reports/science/2021/uk/explore/*
1095Disallow: /reports/science/2021/uk/event/*/ical
1096Disallow: /reports/science/2021/uz/search
1097Disallow: /reports/science/2021/uz/search/*
1098Disallow: /reports/science/2021/uz/explore/*
1099Disallow: /reports/science/2021/uz/event/*/ical
1100Disallow: /reports/science/2021/th/search
1101Disallow: /reports/science/2021/th/search/*
1102Disallow: /reports/science/2021/th/explore/*
1103Disallow: /reports/science/2021/th/event/*/ical
1104Disallow: /reports/science/2021/it/search
1105Disallow: /reports/science/2021/it/search/*
1106Disallow: /reports/science/2021/it/explore/*
1107Disallow: /reports/science/2021/it/event/*/ical
1108Disallow: /reports/science/2021/vi/search
1109Disallow: /reports/science/2021/vi/search/*
1110Disallow: /reports/science/2021/vi/explore/*
1111Disallow: /reports/science/2021/vi/event/*/ical
1112Disallow: /tich4sd/*?f[*
1113Disallow: /tich4sd/*&f[*
1114Disallow: /tich4sd/*?f%5B0*
1115Disallow: /tich4sd/*&f%5B1*
1116Disallow: /tich4sd/search
1117Disallow: /tich4sd/en/search
1118Disallow: /tich4sd/en/search/*
1119Disallow: /tich4sd/en/explore/*
1120Disallow: /tich4sd/event/*/ical
1121Disallow: /tich4sd/en/event/*/ical
1122Disallow: /tich4sd/fr/search
1123Disallow: /tich4sd/fr/search/*
1124Disallow: /tich4sd/fr/explore/*
1125Disallow: /tich4sd/fr/event/*/ical
1126Disallow: /tich4sd/es/search
1127Disallow: /tich4sd/es/search/*
1128Disallow: /tich4sd/es/explore/*
1129Disallow: /tich4sd/es/event/*/ical
1130Disallow: /tich4sd/ru/search
1131Disallow: /tich4sd/ru/search/*
1132Disallow: /tich4sd/ru/explore/*
1133Disallow: /tich4sd/ru/event/*/ical
1134Disallow: /tich4sd/ar/search
1135Disallow: /tich4sd/ar/search/*
1136Disallow: /tich4sd/ar/explore/*
1137Disallow: /tich4sd/ar/event/*/ical
1138Disallow: /tich4sd/zh/search
1139Disallow: /tich4sd/zh/search/*
1140Disallow: /tich4sd/zh/explore/*
1141Disallow: /tich4sd/zh/event/*/ical
1142Disallow: /tich4sd/pt/search
1143Disallow: /tich4sd/pt/search/*
1144Disallow: /tich4sd/pt/explore/*
1145Disallow: /tich4sd/pt/event/*/ical
1146Disallow: /tich4sd/ja/search
1147Disallow: /tich4sd/ja/search/*
1148Disallow: /tich4sd/ja/explore/*
1149Disallow: /tich4sd/ja/event/*/ical
1150Disallow: /tich4sd/id/search
1151Disallow: /tich4sd/id/search/*
1152Disallow: /tich4sd/id/explore/*
1153Disallow: /tich4sd/id/event/*/ical
1154Disallow: /tich4sd/uk/search
1155Disallow: /tich4sd/uk/search/*
1156Disallow: /tich4sd/uk/explore/*
1157Disallow: /tich4sd/uk/event/*/ical
1158Disallow: /tich4sd/uz/search
1159Disallow: /tich4sd/uz/search/*
1160Disallow: /tich4sd/uz/explore/*
1161Disallow: /tich4sd/uz/event/*/ical
1162Disallow: /tich4sd/th/search
1163Disallow: /tich4sd/th/search/*
1164Disallow: /tich4sd/th/explore/*
1165Disallow: /tich4sd/th/event/*/ical
1166Disallow: /tich4sd/it/search
1167Disallow: /tich4sd/it/search/*
1168Disallow: /tich4sd/it/explore/*
1169Disallow: /tich4sd/it/event/*/ical
1170Disallow: /tich4sd/vi/search
1171Disallow: /tich4sd/vi/search/*
1172Disallow: /tich4sd/vi/explore/*
1173Disallow: /tich4sd/vi/event/*/ical
1174Disallow: /reports/sjdi/2020/*?f[*
1175Disallow: /reports/sjdi/2020/*&f[*
1176Disallow: /reports/sjdi/2020/*?f%5B0*
1177Disallow: /reports/sjdi/2020/*&f%5B1*
1178Disallow: /reports/sjdi/2020/search
1179Disallow: /reports/sjdi/2020/en/search
1180Disallow: /reports/sjdi/2020/en/search/*
1181Disallow: /reports/sjdi/2020/en/explore/*
1182Disallow: /reports/sjdi/2020/event/*/ical
1183Disallow: /reports/sjdi/2020/en/event/*/ical
1184Disallow: /reports/sjdi/2020/fr/search
1185Disallow: /reports/sjdi/2020/fr/search/*
1186Disallow: /reports/sjdi/2020/fr/explore/*
1187Disallow: /reports/sjdi/2020/fr/event/*/ical
1188Disallow: /reports/sjdi/2020/es/search
1189Disallow: /reports/sjdi/2020/es/search/*
1190Disallow: /reports/sjdi/2020/es/explore/*
1191Disallow: /reports/sjdi/2020/es/event/*/ical
1192Disallow: /reports/sjdi/2020/ru/search
1193Disallow: /reports/sjdi/2020/ru/search/*
1194Disallow: /reports/sjdi/2020/ru/explore/*
1195Disallow: /reports/sjdi/2020/ru/event/*/ical
1196Disallow: /reports/sjdi/2020/ar/search
1197Disallow: /reports/sjdi/2020/ar/search/*
1198Disallow: /reports/sjdi/2020/ar/explore/*
1199Disallow: /reports/sjdi/2020/ar/event/*/ical
1200Disallow: /reports/sjdi/2020/zh/search
1201Disallow: /reports/sjdi/2020/zh/search/*
1202Disallow: /reports/sjdi/2020/zh/explore/*
1203Disallow: /reports/sjdi/2020/zh/event/*/ical
1204Disallow: /reports/sjdi/2020/pt/search
1205Disallow: /reports/sjdi/2020/pt/search/*
1206Disallow: /reports/sjdi/2020/pt/explore/*
1207Disallow: /reports/sjdi/2020/pt/event/*/ical
1208Disallow: /reports/sjdi/2020/ja/search
1209Disallow: /reports/sjdi/2020/ja/search/*
1210Disallow: /reports/sjdi/2020/ja/explore/*
1211Disallow: /reports/sjdi/2020/ja/event/*/ical
1212Disallow: /reports/sjdi/2020/id/search
1213Disallow: /reports/sjdi/2020/id/search/*
1214Disallow: /reports/sjdi/2020/id/explore/*
1215Disallow: /reports/sjdi/2020/id/event/*/ical
1216Disallow: /reports/sjdi/2020/uk/search
1217Disallow: /reports/sjdi/2020/uk/search/*
1218Disallow: /reports/sjdi/2020/uk/explore/*
1219Disallow: /reports/sjdi/2020/uk/event/*/ical
1220Disallow: /reports/sjdi/2020/uz/search
1221Disallow: /reports/sjdi/2020/uz/search/*
1222Disallow: /reports/sjdi/2020/uz/explore/*
1223Disallow: /reports/sjdi/2020/uz/event/*/ical
1224Disallow: /reports/sjdi/2020/th/search
1225Disallow: /reports/sjdi/2020/th/search/*
1226Disallow: /reports/sjdi/2020/th/explore/*
1227Disallow: /reports/sjdi/2020/th/event/*/ical
1228Disallow: /reports/sjdi/2020/it/search
1229Disallow: /reports/sjdi/2020/it/search/*
1230Disallow: /reports/sjdi/2020/it/explore/*
1231Disallow: /reports/sjdi/2020/it/event/*/ical
1232Disallow: /reports/sjdi/2020/vi/search
1233Disallow: /reports/sjdi/2020/vi/search/*
1234Disallow: /reports/sjdi/2020/vi/explore/*
1235Disallow: /reports/sjdi/2020/vi/event/*/ical
1236Disallow: /reports/safety-journalists/2022/*?f[*
1237Disallow: /reports/safety-journalists/2022/*&f[*
1238Disallow: /reports/safety-journalists/2022/*?f%5B0*
1239Disallow: /reports/safety-journalists/2022/*&f%5B1*
1240Disallow: /reports/safety-journalists/2022/search
1241Disallow: /reports/safety-journalists/2022/en/search
1242Disallow: /reports/safety-journalists/2022/en/search/*
1243Disallow: /reports/safety-journalists/2022/en/explore/*
1244Disallow: /reports/safety-journalists/2022/event/*/ical
1245Disallow: /reports/safety-journalists/2022/en/event/*/ical
1246Disallow: /reports/safety-journalists/2022/fr/search
1247Disallow: /reports/safety-journalists/2022/fr/search/*
1248Disallow: /reports/safety-journalists/2022/fr/explore/*
1249Disallow: /reports/safety-journalists/2022/fr/event/*/ical
1250Disallow: /reports/safety-journalists/2022/es/search
1251Disallow: /reports/safety-journalists/2022/es/search/*
1252Disallow: /reports/safety-journalists/2022/es/explore/*
1253Disallow: /reports/safety-journalists/2022/es/event/*/ical
1254Disallow: /reports/safety-journalists/2022/ru/search
1255Disallow: /reports/safety-journalists/2022/ru/search/*
1256Disallow: /reports/safety-journalists/2022/ru/explore/*
1257Disallow: /reports/safety-journalists/2022/ru/event/*/ical
1258Disallow: /reports/safety-journalists/2022/ar/search
1259Disallow: /reports/safety-journalists/2022/ar/search/*
1260Disallow: /reports/safety-journalists/2022/ar/explore/*
1261Disallow: /reports/safety-journalists/2022/ar/event/*/ical
1262Disallow: /reports/safety-journalists/2022/zh/search
1263Disallow: /reports/safety-journalists/2022/zh/search/*
1264Disallow: /reports/safety-journalists/2022/zh/explore/*
1265Disallow: /reports/safety-journalists/2022/zh/event/*/ical
1266Disallow: /reports/safety-journalists/2022/pt/search
1267Disallow: /reports/safety-journalists/2022/pt/search/*
1268Disallow: /reports/safety-journalists/2022/pt/explore/*
1269Disallow: /reports/safety-journalists/2022/pt/event/*/ical
1270Disallow: /reports/safety-journalists/2022/ja/search
1271Disallow: /reports/safety-journalists/2022/ja/search/*
1272Disallow: /reports/safety-journalists/2022/ja/explore/*
1273Disallow: /reports/safety-journalists/2022/ja/event/*/ical
1274Disallow: /reports/safety-journalists/2022/id/search
1275Disallow: /reports/safety-journalists/2022/id/search/*
1276Disallow: /reports/safety-journalists/2022/id/explore/*
1277Disallow: /reports/safety-journalists/2022/id/event/*/ical
1278Disallow: /reports/safety-journalists/2022/uk/search
1279Disallow: /reports/safety-journalists/2022/uk/search/*
1280Disallow: /reports/safety-journalists/2022/uk/explore/*
1281Disallow: /reports/safety-journalists/2022/uk/event/*/ical
1282Disallow: /reports/safety-journalists/2022/uz/search
1283Disallow: /reports/safety-journalists/2022/uz/search/*
1284Disallow: /reports/safety-journalists/2022/uz/explore/*
1285Disallow: /reports/safety-journalists/2022/uz/event/*/ical
1286Disallow: /reports/safety-journalists/2022/th/search
1287Disallow: /reports/safety-journalists/2022/th/search/*
1288Disallow: /reports/safety-journalists/2022/th/explore/*
1289Disallow: /reports/safety-journalists/2022/th/event/*/ical
1290Disallow: /reports/safety-journalists/2022/it/search
1291Disallow: /reports/safety-journalists/2022/it/search/*
1292Disallow: /reports/safety-journalists/2022/it/explore/*
1293Disallow: /reports/safety-journalists/2022/it/event/*/ical
1294Disallow: /reports/safety-journalists/2022/vi/search
1295Disallow: /reports/safety-journalists/2022/vi/search/*
1296Disallow: /reports/safety-journalists/2022/vi/explore/*
1297Disallow: /reports/safety-journalists/2022/vi/event/*/ical
1298Disallow: /webguidelines/*?f[*
1299Disallow: /webguidelines/*&f[*
1300Disallow: /webguidelines/*?f%5B0*
1301Disallow: /webguidelines/*&f%5B1*
1302Disallow: /webguidelines/search
1303Disallow: /webguidelines/en/search
1304Disallow: /webguidelines/en/search/*
1305Disallow: /webguidelines/en/explore/*
1306Disallow: /webguidelines/event/*/ical
1307Disallow: /webguidelines/en/event/*/ical
1308Disallow: /webguidelines/fr/search
1309Disallow: /webguidelines/fr/search/*
1310Disallow: /webguidelines/fr/explore/*
1311Disallow: /webguidelines/fr/event/*/ical
1312Disallow: /webguidelines/es/search
1313Disallow: /webguidelines/es/search/*
1314Disallow: /webguidelines/es/explore/*
1315Disallow: /webguidelines/es/event/*/ical
1316Disallow: /webguidelines/ru/search
1317Disallow: /webguidelines/ru/search/*
1318Disallow: /webguidelines/ru/explore/*
1319Disallow: /webguidelines/ru/event/*/ical
1320Disallow: /webguidelines/ar/search
1321Disallow: /webguidelines/ar/search/*
1322Disallow: /webguidelines/ar/explore/*
1323Disallow: /webguidelines/ar/event/*/ical
1324Disallow: /webguidelines/zh/search
1325Disallow: /webguidelines/zh/search/*
1326Disallow: /webguidelines/zh/explore/*
1327Disallow: /webguidelines/zh/event/*/ical
1328Disallow: /webguidelines/pt/search
1329Disallow: /webguidelines/pt/search/*
1330Disallow: /webguidelines/pt/explore/*
1331Disallow: /webguidelines/pt/event/*/ical
1332Disallow: /webguidelines/ja/search
1333Disallow: /webguidelines/ja/search/*
1334Disallow: /webguidelines/ja/explore/*
1335Disallow: /webguidelines/ja/event/*/ical
1336Disallow: /webguidelines/id/search
1337Disallow: /webguidelines/id/search/*
1338Disallow: /webguidelines/id/explore/*
1339Disallow: /webguidelines/id/event/*/ical
1340Disallow: /webguidelines/uk/search
1341Disallow: /webguidelines/uk/search/*
1342Disallow: /webguidelines/uk/explore/*
1343Disallow: /webguidelines/uk/event/*/ical
1344Disallow: /webguidelines/uz/search
1345Disallow: /webguidelines/uz/search/*
1346Disallow: /webguidelines/uz/explore/*
1347Disallow: /webguidelines/uz/event/*/ical
1348Disallow: /webguidelines/th/search
1349Disallow: /webguidelines/th/search/*
1350Disallow: /webguidelines/th/explore/*
1351Disallow: /webguidelines/th/event/*/ical
1352Disallow: /webguidelines/it/search
1353Disallow: /webguidelines/it/search/*
1354Disallow: /webguidelines/it/explore/*
1355Disallow: /webguidelines/it/event/*/ical
1356Disallow: /webguidelines/vi/search
1357Disallow: /webguidelines/vi/search/*
1358Disallow: /webguidelines/vi/explore/*
1359Disallow: /webguidelines/vi/event/*/ical
1360Disallow: /reports/world-media-trends/2021/*?f[*
1361Disallow: /reports/world-media-trends/2021/*&f[*
1362Disallow: /reports/world-media-trends/2021/*?f%5B0*
1363Disallow: /reports/world-media-trends/2021/*&f%5B1*
1364Disallow: /reports/world-media-trends/2021/search
1365Disallow: /reports/world-media-trends/2021/en/search
1366Disallow: /reports/world-media-trends/2021/en/search/*
1367Disallow: /reports/world-media-trends/2021/en/explore/*
1368Disallow: /reports/world-media-trends/2021/event/*/ical
1369Disallow: /reports/world-media-trends/2021/en/event/*/ical
1370Disallow: /reports/world-media-trends/2021/fr/search
1371Disallow: /reports/world-media-trends/2021/fr/search/*
1372Disallow: /reports/world-media-trends/2021/fr/explore/*
1373Disallow: /reports/world-media-trends/2021/fr/event/*/ical
1374Disallow: /reports/world-media-trends/2021/es/search
1375Disallow: /reports/world-media-trends/2021/es/search/*
1376Disallow: /reports/world-media-trends/2021/es/explore/*
1377Disallow: /reports/world-media-trends/2021/es/event/*/ical
1378Disallow: /reports/world-media-trends/2021/ru/search
1379Disallow: /reports/world-media-trends/2021/ru/search/*
1380Disallow: /reports/world-media-trends/2021/ru/explore/*
1381Disallow: /reports/world-media-trends/2021/ru/event/*/ical
1382Disallow: /reports/world-media-trends/2021/ar/search
1383Disallow: /reports/world-media-trends/2021/ar/search/*
1384Disallow: /reports/world-media-trends/2021/ar/explore/*
1385Disallow: /reports/world-media-trends/2021/ar/event/*/ical
1386Disallow: /reports/world-media-trends/2021/zh/search
1387Disallow: /reports/world-media-trends/2021/zh/search/*
1388Disallow: /reports/world-media-trends/2021/zh/explore/*
1389Disallow: /reports/world-media-trends/2021/zh/event/*/ical
1390Disallow: /reports/world-media-trends/2021/pt/search
1391Disallow: /reports/world-media-trends/2021/pt/search/*
1392Disallow: /reports/world-media-trends/2021/pt/explore/*
1393Disallow: /reports/world-media-trends/2021/pt/event/*/ical
1394Disallow: /reports/world-media-trends/2021/ja/search
1395Disallow: /reports/world-media-trends/2021/ja/search/*
1396Disallow: /reports/world-media-trends/2021/ja/explore/*
1397Disallow: /reports/world-media-trends/2021/ja/event/*/ical
1398Disallow: /reports/world-media-trends/2021/id/search
1399Disallow: /reports/world-media-trends/2021/id/search/*
1400Disallow: /reports/world-media-trends/2021/id/explore/*
1401Disallow: /reports/world-media-trends/2021/id/event/*/ical
1402Disallow: /reports/world-media-trends/2021/uk/search
1403Disallow: /reports/world-media-trends/2021/uk/search/*
1404Disallow: /reports/world-media-trends/2021/uk/explore/*
1405Disallow: /reports/world-media-trends/2021/uk/event/*/ical
1406Disallow: /reports/world-media-trends/2021/uz/search
1407Disallow: /reports/world-media-trends/2021/uz/search/*
1408Disallow: /reports/world-media-trends/2021/uz/explore/*
1409Disallow: /reports/world-media-trends/2021/uz/event/*/ical
1410Disallow: /reports/world-media-trends/2021/th/search
1411Disallow: /reports/world-media-trends/2021/th/search/*
1412Disallow: /reports/world-media-trends/2021/th/explore/*
1413Disallow: /reports/world-media-trends/2021/th/event/*/ical
1414Disallow: /reports/world-media-trends/2021/it/search
1415Disallow: /reports/world-media-trends/2021/it/search/*
1416Disallow: /reports/world-media-trends/2021/it/explore/*
1417Disallow: /reports/world-media-trends/2021/it/event/*/ical
1418Disallow: /reports/world-media-trends/2021/vi/search
1419Disallow: /reports/world-media-trends/2021/vi/search/*
1420Disallow: /reports/world-media-trends/2021/vi/explore/*
1421Disallow: /reports/world-media-trends/2021/vi/event/*/ical
1422Disallow: /reports/wwdr/*?f[*
1423Disallow: /reports/wwdr/*&f[*
1424Disallow: /reports/wwdr/*?f%5B0*
1425Disallow: /reports/wwdr/*&f%5B1*
1426Disallow: /reports/wwdr/search
1427Disallow: /reports/wwdr/en/search
1428Disallow: /reports/wwdr/en/search/*
1429Disallow: /reports/wwdr/en/explore/*
1430Disallow: /reports/wwdr/event/*/ical
1431Disallow: /reports/wwdr/en/event/*/ical
1432Disallow: /reports/wwdr/fr/search
1433Disallow: /reports/wwdr/fr/search/*
1434Disallow: /reports/wwdr/fr/explore/*
1435Disallow: /reports/wwdr/fr/event/*/ical
1436Disallow: /reports/wwdr/es/search
1437Disallow: /reports/wwdr/es/search/*
1438Disallow: /reports/wwdr/es/explore/*
1439Disallow: /reports/wwdr/es/event/*/ical
1440Disallow: /reports/wwdr/ru/search
1441Disallow: /reports/wwdr/ru/search/*
1442Disallow: /reports/wwdr/ru/explore/*
1443Disallow: /reports/wwdr/ru/event/*/ical
1444Disallow: /reports/wwdr/ar/search
1445Disallow: /reports/wwdr/ar/search/*
1446Disallow: /reports/wwdr/ar/explore/*
1447Disallow: /reports/wwdr/ar/event/*/ical
1448Disallow: /reports/wwdr/zh/search
1449Disallow: /reports/wwdr/zh/search/*
1450Disallow: /reports/wwdr/zh/explore/*
1451Disallow: /reports/wwdr/zh/event/*/ical
1452Disallow: /reports/wwdr/pt/search
1453Disallow: /reports/wwdr/pt/search/*
1454Disallow: /reports/wwdr/pt/explore/*
1455Disallow: /reports/wwdr/pt/event/*/ical
1456Disallow: /reports/wwdr/ja/search
1457Disallow: /reports/wwdr/ja/search/*
1458Disallow: /reports/wwdr/ja/explore/*
1459Disallow: /reports/wwdr/ja/event/*/ical
1460Disallow: /reports/wwdr/id/search
1461Disallow: /reports/wwdr/id/search/*
1462Disallow: /reports/wwdr/id/explore/*
1463Disallow: /reports/wwdr/id/event/*/ical
1464Disallow: /reports/wwdr/uk/search
1465Disallow: /reports/wwdr/uk/search/*
1466Disallow: /reports/wwdr/uk/explore/*
1467Disallow: /reports/wwdr/uk/event/*/ical
1468Disallow: /reports/wwdr/uz/search
1469Disallow: /reports/wwdr/uz/search/*
1470Disallow: /reports/wwdr/uz/explore/*
1471Disallow: /reports/wwdr/uz/event/*/ical
1472Disallow: /reports/wwdr/th/search
1473Disallow: /reports/wwdr/th/search/*
1474Disallow: /reports/wwdr/th/explore/*
1475Disallow: /reports/wwdr/th/event/*/ical
1476Disallow: /reports/wwdr/it/search
1477Disallow: /reports/wwdr/it/search/*
1478Disallow: /reports/wwdr/it/explore/*
1479Disallow: /reports/wwdr/it/event/*/ical
1480Disallow: /reports/wwdr/vi/search
1481Disallow: /reports/wwdr/vi/search/*
1482Disallow: /reports/wwdr/vi/explore/*
1483Disallow: /reports/wwdr/vi/event/*/ical
1484Disallow: /reports/wwdr/2022/*?f[*
1485Disallow: /reports/wwdr/2022/*&f[*
1486Disallow: /reports/wwdr/2022/*?f%5B0*
1487Disallow: /reports/wwdr/2022/*&f%5B1*
1488Disallow: /reports/wwdr/2022/search
1489Disallow: /reports/wwdr/2022/en/search
1490Disallow: /reports/wwdr/2022/en/search/*
1491Disallow: /reports/wwdr/2022/en/explore/*
1492Disallow: /reports/wwdr/2022/event/*/ical
1493Disallow: /reports/wwdr/2022/en/event/*/ical
1494Disallow: /reports/wwdr/2022/fr/search
1495Disallow: /reports/wwdr/2022/fr/search/*
1496Disallow: /reports/wwdr/2022/fr/explore/*
1497Disallow: /reports/wwdr/2022/fr/event/*/ical
1498Disallow: /reports/wwdr/2022/es/search
1499Disallow: /reports/wwdr/2022/es/search/*
1500Disallow: /reports/wwdr/2022/es/explore/*
1501Disallow: /reports/wwdr/2022/es/event/*/ical
1502Disallow: /reports/wwdr/2022/ru/search
1503Disallow: /reports/wwdr/2022/ru/search/*
1504Disallow: /reports/wwdr/2022/ru/explore/*
1505Disallow: /reports/wwdr/2022/ru/event/*/ical
1506Disallow: /reports/wwdr/2022/ar/search
1507Disallow: /reports/wwdr/2022/ar/search/*
1508Disallow: /reports/wwdr/2022/ar/explore/*
1509Disallow: /reports/wwdr/2022/ar/event/*/ical
1510Disallow: /reports/wwdr/2022/zh/search
1511Disallow: /reports/wwdr/2022/zh/search/*
1512Disallow: /reports/wwdr/2022/zh/explore/*
1513Disallow: /reports/wwdr/2022/zh/event/*/ical
1514Disallow: /reports/wwdr/2022/pt/search
1515Disallow: /reports/wwdr/2022/pt/search/*
1516Disallow: /reports/wwdr/2022/pt/explore/*
1517Disallow: /reports/wwdr/2022/pt/event/*/ical
1518Disallow: /reports/wwdr/2022/ja/search
1519Disallow: /reports/wwdr/2022/ja/search/*
1520Disallow: /reports/wwdr/2022/ja/explore/*
1521Disallow: /reports/wwdr/2022/ja/event/*/ical
1522Disallow: /reports/wwdr/2022/id/search
1523Disallow: /reports/wwdr/2022/id/search/*
1524Disallow: /reports/wwdr/2022/id/explore/*
1525Disallow: /reports/wwdr/2022/id/event/*/ical
1526Disallow: /reports/wwdr/2022/uk/search
1527Disallow: /reports/wwdr/2022/uk/search/*
1528Disallow: /reports/wwdr/2022/uk/explore/*
1529Disallow: /reports/wwdr/2022/uk/event/*/ical
1530Disallow: /reports/wwdr/2022/uz/search
1531Disallow: /reports/wwdr/2022/uz/search/*
1532Disallow: /reports/wwdr/2022/uz/explore/*
1533Disallow: /reports/wwdr/2022/uz/event/*/ical
1534Disallow: /reports/wwdr/2022/th/search
1535Disallow: /reports/wwdr/2022/th/search/*
1536Disallow: /reports/wwdr/2022/th/explore/*
1537Disallow: /reports/wwdr/2022/th/event/*/ical
1538Disallow: /reports/wwdr/2022/it/search
1539Disallow: /reports/wwdr/2022/it/search/*
1540Disallow: /reports/wwdr/2022/it/explore/*
1541Disallow: /reports/wwdr/2022/it/event/*/ical
1542Disallow: /reports/wwdr/2022/vi/search
1543Disallow: /reports/wwdr/2022/vi/search/*
1544Disallow: /reports/wwdr/2022/vi/explore/*
1545Disallow: /reports/wwdr/2022/vi/event/*/ical
1546Disallow: /reports/wwdr/2023/*?f[*
1547Disallow: /reports/wwdr/2023/*&f[*
1548Disallow: /reports/wwdr/2023/*?f%5B0*
1549Disallow: /reports/wwdr/2023/*&f%5B1*
1550Disallow: /reports/wwdr/2023/search
1551Disallow: /reports/wwdr/2023/en/search
1552Disallow: /reports/wwdr/2023/en/search/*
1553Disallow: /reports/wwdr/2023/en/explore/*
1554Disallow: /reports/wwdr/2023/event/*/ical
1555Disallow: /reports/wwdr/2023/en/event/*/ical
1556Disallow: /reports/wwdr/2023/fr/search
1557Disallow: /reports/wwdr/2023/fr/search/*
1558Disallow: /reports/wwdr/2023/fr/explore/*
1559Disallow: /reports/wwdr/2023/fr/event/*/ical
1560Disallow: /reports/wwdr/2023/es/search
1561Disallow: /reports/wwdr/2023/es/search/*
1562Disallow: /reports/wwdr/2023/es/explore/*
1563Disallow: /reports/wwdr/2023/es/event/*/ical
1564Disallow: /reports/wwdr/2023/ru/search
1565Disallow: /reports/wwdr/2023/ru/search/*
1566Disallow: /reports/wwdr/2023/ru/explore/*
1567Disallow: /reports/wwdr/2023/ru/event/*/ical
1568Disallow: /reports/wwdr/2023/ar/search
1569Disallow: /reports/wwdr/2023/ar/search/*
1570Disallow: /reports/wwdr/2023/ar/explore/*
1571Disallow: /reports/wwdr/2023/ar/event/*/ical
1572Disallow: /reports/wwdr/2023/zh/search
1573Disallow: /reports/wwdr/2023/zh/search/*
1574Disallow: /reports/wwdr/2023/zh/explore/*
1575Disallow: /reports/wwdr/2023/zh/event/*/ical
1576Disallow: /reports/wwdr/2023/pt/search
1577Disallow: /reports/wwdr/2023/pt/search/*
1578Disallow: /reports/wwdr/2023/pt/explore/*
1579Disallow: /reports/wwdr/2023/pt/event/*/ical
1580Disallow: /reports/wwdr/2023/ja/search
1581Disallow: /reports/wwdr/2023/ja/search/*
1582Disallow: /reports/wwdr/2023/ja/explore/*
1583Disallow: /reports/wwdr/2023/ja/event/*/ical
1584Disallow: /reports/wwdr/2023/id/search
1585Disallow: /reports/wwdr/2023/id/search/*
1586Disallow: /reports/wwdr/2023/id/explore/*
1587Disallow: /reports/wwdr/2023/id/event/*/ical
1588Disallow: /reports/wwdr/2023/uk/search
1589Disallow: /reports/wwdr/2023/uk/search/*
1590Disallow: /reports/wwdr/2023/uk/explore/*
1591Disallow: /reports/wwdr/2023/uk/event/*/ical
1592Disallow: /reports/wwdr/2023/uz/search
1593Disallow: /reports/wwdr/2023/uz/search/*
1594Disallow: /reports/wwdr/2023/uz/explore/*
1595Disallow: /reports/wwdr/2023/uz/event/*/ical
1596Disallow: /reports/wwdr/2023/th/search
1597Disallow: /reports/wwdr/2023/th/search/*
1598Disallow: /reports/wwdr/2023/th/explore/*
1599Disallow: /reports/wwdr/2023/th/event/*/ical
1600Disallow: /reports/wwdr/2023/it/search
1601Disallow: /reports/wwdr/2023/it/search/*
1602Disallow: /reports/wwdr/2023/it/explore/*
1603Disallow: /reports/wwdr/2023/it/event/*/ical
1604Disallow: /reports/wwdr/2023/vi/search
1605Disallow: /reports/wwdr/2023/vi/search/*
1606Disallow: /reports/wwdr/2023/vi/explore/*
1607Disallow: /reports/wwdr/2023/vi/event/*/ical
1608Disallow: /TSPD/
1609Disallow: /*/TSPD/
1610Disallow: *?hub=*
1611Disallow: *?f[0]=*
1612Disallow: *?page*
1613Disallow: *TSPD*
1614Disallow: *?f[1]=*
1615Disallow: *?fq*
1616Disallow: */lightbox*
1617Disallow: *?country*
1618Disallow: */event/1*
1619Disallow: */event/2*
1620Disallow: */event/3*
1621Disallow: */event/4*
1622Disallow: */event/5*
1623Disallow: */event/6*
1624Disallow: */event/7*
1625Disallow: */event/8*
1626Disallow: */event/9*
1627Disallow: */event/10*
What do the bots actually get?
This page reads robots.txt, which states what unesco.org asks of crawlers. A firewall or CDN can still refuse a bot that robots.txt allows. The live check requests the page as each bot and shows the status it gets.
Run a live check on unesco.org Test a path in the robots.txt tester
History
- 19 October 2026: first checked.
Questions
Does unesco.org block GPTBot?
Yes. unesco.org’s robots.txt disallows GPTBot from the whole site (line 59: Disallow: /). GPTBot is OpenAI’s crawler for training data; blocking it doesn’t affect ChatGPT search, which uses OAI-SearchBot. Checked 9 November 2026.
Does unesco.org block ClaudeBot?
Yes. unesco.org’s robots.txt disallows ClaudeBot from the whole site (line 59: Disallow: /). ClaudeBot is Anthropic’s crawler for training data. Checked 9 November 2026.
Does unesco.org block Google-Extended?
No. unesco.org’s robots.txt lets Google-Extended crawl the site. Google-Extended is the token that controls whether Google may use pages for Gemini training and grounding; it doesn’t affect Google Search. Checked 9 November 2026.
Does unesco.org block PerplexityBot?
No. unesco.org’s robots.txt lets PerplexityBot crawl the site. PerplexityBot is the crawler that indexes pages for Perplexity’s answers. Checked 9 November 2026.
Can ChatGPT search find unesco.org?
Yes. unesco.org’s robots.txt lets OAI-SearchBot crawl the site. OAI-SearchBot is the crawler behind ChatGPT search; blocking it keeps the site out of ChatGPT’s search answers. Checked 9 November 2026.