Who blocks AI · #442 in the Majestic Million
robots.txt · checked 9 Nov 2026
Does dell.com block AI crawlers?
No. dell.com’s robots.txt lets all 19 AI crawlers we track crawl the site, though it keeps Bytespider out of some paths. It also blocks Baiduspider among search engines.
GPTBotAllowedOAI-SearchBotAllowedClaudeBotAllowedPerplexityBotAllowedGoogle-ExtendedAllowedCCBotAllowed- robots.txt
- www.dell.com/robots.txt · 1,254 lines
- llms.txt
- found
- Checked
- 9 November 2026, as canaicrawl. We check again every week.
Every bot, and the line that decides
Search engines
| Bot | Operator | robots.txt | Deciding line |
|---|---|---|---|
Googlebot | Allowed | no rule matches the home page | |
Bingbot | Microsoft | Allowed | no rule matches the home page |
DuckDuckBot | DuckDuckGo | Allowed | no rule matches the home page |
Applebot | Apple | Allowed | no rule matches the home page |
PetalBot | Huawei | Allowed | no rule matches the home page |
YandexBot | Yandex | Allowed | no rule matches the home page |
Baiduspider | Baidu | Blocked | line 989 · Disallow: / under User-agent: Baiduspider |
AI search
| Bot | Operator | robots.txt | Deciding line |
|---|---|---|---|
OAI-SearchBot | OpenAI | Allowed | no rule matches the home page |
PerplexityBot | Perplexity | Allowed | no rule matches the home page |
Claude-SearchBot | Anthropic | Allowed | no rule matches the home page |
DuckAssistBot | DuckDuckGo | Allowed | no rule matches the home page |
Amazonbot | Amazon | Allowed | no rule matches the home page |
YouBot | You.com | Allowed | no rule matches the home page |
AI training
| Bot | Operator | robots.txt | Deciding line |
|---|---|---|---|
GPTBot | OpenAI | Allowed | no rule matches the home page |
ClaudeBot | Anthropic | Allowed | no rule matches the home page |
Google-Extended | Allowed | no rule matches the home page | |
CCBot | Common Crawl | Allowed | no rule matches the home page |
Applebot-Extended | Apple | Allowed | no rule matches the home page |
GoogleOther | Allowed | no rule matches the home page | |
Meta-ExternalAgent | Meta | Allowed | no rule matches the home page |
Bytespider | ByteDance | Partly blocked | line 1190 · Disallow: /community/*=es line 1191 · Disallow: /community/*=de and 11 more |
User-triggered fetchers
| Bot | Operator | robots.txt | Deciding line |
|---|---|---|---|
ChatGPT-User | OpenAI | Allowed | no rule matches the home page |
Claude-User | Anthropic | Allowed | no rule matches the home page |
Perplexity-User | Perplexity | Allowed | no rule matches the home page |
MistralAI-User | Mistral | Allowed | no rule matches the home page |
Meta-ExternalFetcher | Meta | Allowed | no rule matches the home page |
SEO tools
| Bot | Operator | robots.txt | Deciding line |
|---|---|---|---|
AhrefsBot | Ahrefs | Allowed | no rule matches the home page |
SemrushBot | Semrush | Allowed | no rule matches the home page |
MJ12bot | Majestic | Allowed | no rule matches the home page |
DotBot | Moz | Allowed | no rule matches the home page |
View www.dell.com/robots.txt · 1,254 lines
1User-agent: *
2
3
4#Temporary launch entries
5Disallow: /csbapi/unified-communication/*
6Disallow: *.less$
7Disallow: /en-ca/shop/sfc/sf/alienware-desktops
8Disallow: /en-ca/shop/sfc/sf/alienware-desktops
9Disallow: /di/idp/dwa/
10Disallow: /dci/idp/dwa/
11Disallow: /en-in/shop/pdr/
12Disallow: /buyer/
13Disallow: /en-au/outlet/*?
14Disallow: /pt-br/outlet/*?
15Disallow: /zh-cn/outlet/*?
16Disallow: /post
17Disallow: */api/
18Disallow: *intake
19Disallow: *cacheKey
20
21
22#Shop
23Disallow: /*/cart/*
24Disallow: /*/buy/*
25Disallow: /*/ar/*?appliedRefinements=*,*
26Disallow: /*/ar/*?appliedrefinements=*,*
27Disallow: */gaming/performance/
28Disallow: /*.php
29Disallow: */reviewselections/
30Disallow: */pdr/
31Disallow: */iserviceselection/
32Disallow: */serviceselection/
33Disallow: */cty/pdp/ispd/
34Disallow: /quote/
35Disallow: /csbapi/
36Disallow: *boomerang
37Disallow: */apd/*/quickview$
38Disallow: *pr_rd_page=*
39Disallow: *bvstate=
40Allow: /en-gb/blog/
41
42#Support
43Disallow: /support/kbdoc/*/000287599
44Disallow: /support/ips/
45Disallow: /support/*/ru/ru/
46Disallow: /support/order-status/*/packingslip*
47Disallow: /support/home/cassette.axd/*
48Disallow: /support/home/*/*/*/Diagnostics/*
49Disallow: /support/home/*/*/*/Drivers/DownloadList/*
50Disallow: /support/home/*/*/*/Drivers/DriversList/*
51Disallow: /support/home/*/*/*/Drivers/EmailSubscription/*
52Disallow: /support/home/*/*/*/Drivers/OneClick/*
53Disallow: /support/manuals/*/*/*/flyout/
54Disallow: /support/csbapi/
55Disallow: /support/clientconnection/
56Disallow: /support/*/tab_mob/
57Disallow: /support/coveo-dell/*
58Disallow: /support/components/*
59Allow: */home/*/product-support/servicetag/*/configuration*
60Disallow: /support/Diagnostics/*
61Disallow: /support/diagnostics/*
62Disallow: /support/*/chat/*
63Disallow: /support/sitealerts/*-*/
64Disallow: /support/order-status/*/order-collection
65Disallow: */support/*/ru-ua
66Disallow: */support/*/en-ua
67Disallow: */support/*/ua/ru/
68Disallow: */support/*/ua/uk/
69Disallow: */support/*/ua/en/
70Disallow: /support/*/ips/
71
72#Sitelite
73Allow: /cs-cz/lp$
74Allow: /cs-cz/lp/contact-us
75Allow: /cs-cz/lp/homepage$
76Allow: /el-gr/lp$
77Allow: /el-gr/lp/contact-us
78Allow: /el-gr/lp/homepage$
79Allow: /en-ae/lp$
80Allow: /en-ae/lp/contact-us
81Allow: /en-ae/lp/homepage$
82Allow: /en-af/lp$
83Allow: /en-af/lp/contact-us
84Allow: /en-af/lp/homepage$
85Allow: /en-ag/lp$
86Allow: /en-ag/lp/contact-us
87Allow: /en-ag/lp/homepage$
88Allow: /en-ai/lp$
89Allow: /en-ai/lp/contact-us
90Allow: /en-ai/lp/homepage$
91Allow: /en-al/lp$
92Allow: /en-al/lp/contact-us
93Allow: /en-al/lp/homepage$
94Allow: /en-ao/lp$
95Allow: /en-ao/lp/contact-us
96Allow: /en-ao/lp/homepage$
97Allow: /en-aw/lp$
98Allow: /en-aw/lp/contact-us
99Allow: /en-aw/lp/homepage$
100Allow: /en-ba/lp$
101Allow: /en-ba/lp/contact-us
102Allow: /en-ba/lp/homepage$
103Allow: /en-bb/lp$
104Allow: /en-bb/lp/contact-us
105Allow: /en-bb/lp/homepage$
106Allow: /en-bd/lp$
107Allow: /en-bd/lp/contact-us
108Allow: /en-bd/lp/homepage$
109Allow: /en-bh/lp$
110Allow: /en-bh/lp/contact-us
111Allow: /en-bh/lp/homepage$
112Allow: /en-bm/lp$
113Allow: /en-bm/lp/contact-us
114Allow: /en-bm/lp/homepage$
115Allow: /en-bn/lp$
116Allow: /en-bn/lp/contact-us
117Allow: /en-bn/lp/homepage$
118Allow: /en-bs/lp$
119Allow: /en-bs/lp/contact-us
120Allow: /en-bs/lp/homepage$
121Allow: /en-bt/lp$
122Allow: /en-bt/lp/contact-us
123Allow: /en-bt/lp/homepage$
124Allow: /en-bw/lp$
125Allow: /en-bw/lp/contact-us
126Allow: /en-bw/lp/homepage$
127Allow: /en-bz/lp$
128Allow: /en-bz/lp/contact-us
129Allow: /en-bz/lp/homepage$
130Allow: /en-cv/lp$
131Allow: /en-cv/lp/contact-us
132Allow: /en-cv/lp/homepage$
133Allow: /en-cy/lp$
134Allow: /en-cy/lp/contact-us
135Allow: /en-cy/lp/homepage$
136Allow: /en-dm/lp$
137Allow: /en-dm/lp/contact-us
138Allow: /en-dm/lp/homepage$
139Allow: /en-ee/lp$
140Allow: /en-ee/lp/contact-us
141Allow: /en-ee/lp/homepage$
142Allow: /en-eg/lp$
143Allow: /en-eg/lp/contact-us
144Allow: /en-eg/lp/homepage$
145Allow: /en-et/lp$
146Allow: /en-et/lp/contact-us
147Allow: /en-et/lp/homepage$
148Allow: /en-fj/lp$
149Allow: /en-fj/lp/contact-us
150Allow: /en-fj/lp/homepage$
151Allow: /en-gd/lp$
152Allow: /en-gd/lp/contact-us
153Allow: /en-gd/lp/homepage$
154Allow: /en-gh/lp$
155Allow: /en-gh/lp/contact-us
156Allow: /en-gh/lp/homepage$
157Allow: /en-gm/lp$
158Allow: /en-gm/lp/contact-us
159Allow: /en-gm/lp/homepage$
160Allow: /en-gu/lp$
161Allow: /en-gu/lp/contact-us
162Allow: /en-gu/lp/homepage$
163Allow: /en-gy/lp$
164Allow: /en-gy/lp/contact-us
165Allow: /en-gy/lp/homepage$
166Allow: /en-ht/lp$
167Allow: /en-ht/lp/contact-us
168Allow: /en-ht/lp/homepage$
169Allow: /en-il/lp$
170Allow: /en-il/lp/contact-us
171Allow: /en-il/lp/homepage$
172Allow: /en-iq/lp$
173Allow: /en-iq/lp/contact-us
174Allow: /en-iq/lp/homepage$
175Allow: /en-is/lp$
176Allow: /en-is/lp/contact-us
177Allow: /en-is/lp/homepage$
178Allow: /en-jm/lp$
179Allow: /en-jm/lp/contact-us
180Allow: /en-jm/lp/homepage$
181Allow: /en-jo/lp$
182Allow: /en-jo/lp/contact-us
183Allow: /en-jo/lp/homepage$
184Allow: /en-ke/lp$
185Allow: /en-ke/lp/contact-us
186Allow: /en-ke/lp/homepage$
187Allow: /en-kh/lp$
188Allow: /en-kh/lp/contact-us
189Allow: /en-kh/lp/homepage$
190Allow: /en-ki/lp$
191Allow: /en-ki/lp/contact-us
192Allow: /en-ki/lp/homepage$
193Allow: /en-kn/lp$
194Allow: /en-kn/lp/contact-us
195Allow: /en-kn/lp/homepage$
196Allow: /en-kw/lp$
197Allow: /en-kw/lp/contact-us
198Allow: /en-kw/lp/homepage$
199Allow: /en-ky/lp$
200Allow: /en-ky/lp/contact-us
201Allow: /en-ky/lp/homepage$
202Allow: /en-la/lp$
203Allow: /en-la/lp/contact-us
204Allow: /en-la/lp/homepage$
205Allow: /en-lb/lp$
206Allow: /en-lb/lp/contact-us
207Allow: /en-lb/lp/homepage$
208Allow: /en-lc/lp$
209Allow: /en-lc/lp/contact-us
210Allow: /en-lc/lp/homepage$
211Allow: /en-lk/lp$
212Allow: /en-lk/lp/contact-us
213Allow: /en-lk/lp/homepage$
214Allow: /en-lr/lp$
215Allow: /en-lr/lp/contact-us
216Allow: /en-lr/lp/homepage$
217Allow: /en-ls/lp$
218Allow: /en-ls/lp/contact-us
219Allow: /en-ls/lp/homepage$
220Allow: /en-lt/lp$
221Allow: /en-lt/lp/contact-us
222Allow: /en-lt/lp/homepage$
223Allow: /en-lv/lp$
224Allow: /en-lv/lp/contact-us
225Allow: /en-lv/lp/homepage$
226Allow: /en-me/lp$
227Allow: /en-me/lp/contact-us
228Allow: /en-me/lp/homepage$
229Allow: /en-mk/lp$
230Allow: /en-mk/lp/contact-us
231Allow: /en-mk/lp/homepage$
232Allow: /en-mm/lp$
233Allow: /en-mm/lp/contact-us
234Allow: /en-mm/lp/homepage$
235Allow: /en-mn/lp$
236Allow: /en-mn/lp/contact-us
237Allow: /en-mn/lp/homepage$
238Allow: /en-ms/lp$
239Allow: /en-ms/lp/contact-us
240Allow: /en-ms/lp/homepage$
241Allow: /en-mt/lp$
242Allow: /en-mt/lp/contact-us
243Allow: /en-mt/lp/homepage$
244Allow: /en-mu/lp$
245Allow: /en-mu/lp/contact-us
246Allow: /en-mu/lp/homepage$
247Allow: /en-mv/lp$
248Allow: /en-mv/lp/contact-us
249Allow: /en-mv/lp/homepage$
250Allow: /en-mw/lp$
251Allow: /en-mw/lp/contact-us
252Allow: /en-mw/lp/homepage$
253Allow: /en-mz/lp$
254Allow: /en-mz/lp/contact-us
255Allow: /en-mz/lp/homepage$
256Allow: /en-na/lp$
257Allow: /en-na/lp/contact-us
258Allow: /en-na/lp/homepage$
259Allow: /en-np/lp$
260Allow: /en-np/lp/contact-us
261Allow: /en-np/lp/homepage$
262Allow: /en-om/lp$
263Allow: /en-om/lp/contact-us
264Allow: /en-om/lp/homepage$
265Allow: /en-pg/lp$
266Allow: /en-pg/lp/contact-us
267Allow: /en-pg/lp/homepage$
268Allow: /en-pr/lp$
269Allow: /en-pr/lp/contact-us
270Allow: /en-pr/lp/homepage$
271Allow: /en-pw/lp$
272Allow: /en-pw/lp/contact-us
273Allow: /en-pw/lp/homepage$
274Allow: /en-qa/lp$
275Allow: /en-qa/lp/contact-us
276Allow: /en-qa/lp/homepage$
277Allow: /en-rw/lp$
278Allow: /en-rw/lp/contact-us
279Allow: /en-rw/lp/homepage$
280Allow: /en-sa/lp$
281Allow: /en-sa/lp/contact-us
282Allow: /en-sa/lp/homepage$
283Allow: /en-si/lp$
284Allow: /en-si/lp/contact-us
285Allow: /en-si/lp/homepage$
286Allow: /en-sl/lp$
287Allow: /en-sl/lp/contact-us
288Allow: /en-sl/lp/homepage$
289Allow: /en-so/lp$
290Allow: /en-so/lp/contact-us
291Allow: /en-so/lp/homepage$
292Allow: /en-sr/lp$
293Allow: /en-sr/lp/contact-us
294Allow: /en-sr/lp/homepage$
295Allow: /en-sz/lp$
296Allow: /en-sz/lp/contact-us
297Allow: /en-sz/lp/homepage$
298Allow: /en-tc/lp$
299Allow: /en-tc/lp/contact-us
300Allow: /en-tc/lp/homepage$
301Allow: /en-tt/lp$
302Allow: /en-tt/lp/contact-us
303Allow: /en-tt/lp/homepage$
304Allow: /en-tz/lp$
305Allow: /en-tz/lp/contact-us
306Allow: /en-tz/lp/homepage$
307Allow: /en-ug/lp$
308Allow: /en-ug/lp/contact-us
309Allow: /en-ug/lp/homepage$
310Allow: /en-vc/lp$
311Allow: /en-vc/lp/contact-us
312Allow: /en-vc/lp/homepage$
313Allow: /en-vg/lp$
314Allow: /en-vg/lp/contact-us
315Allow: /en-vg/lp/homepage$
316Allow: /en-vi/lp$
317Allow: /en-vi/lp/contact-us
318Allow: /en-vi/lp/homepage$
319Allow: /en-vu/lp$
320Allow: /en-vu/lp/contact-us
321Allow: /en-vu/lp/homepage$
322Allow: /en-ws/lp$
323Allow: /en-ws/lp/contact-us
324Allow: /en-ws/lp/homepage$
325Allow: /en-ye/lp$
326Allow: /en-ye/lp/contact-us
327Allow: /en-ye/lp/homepage$
328Allow: /en-za/lp$
329Allow: /en-za/lp/contact-us
330Allow: /en-za/lp/homepage$
331Allow: /en-zm/lp$
332Allow: /en-zm/lp/contact-us
333Allow: /en-zm/lp/homepage$
334Allow: /en-zw/lp$
335Allow: /en-zw/lp/contact-us
336Allow: /en-zw/lp/homepage$
337Allow: /es-ar/lp$
338Allow: /es-ar/lp/contact-us
339Allow: /es-ar/lp/homepage$
340Allow: /es-bo/lp$
341Allow: /es-bo/lp/contact-us
342Allow: /es-bo/lp/homepage$
343Allow: /es-cl/lp$
344Allow: /es-cl/lp/contact-us
345Allow: /es-cl/lp/homepage$
346Allow: /es-co/lp$
347Allow: /es-co/lp/contact-us
348Allow: /es-co/lp/homepage$
349Allow: /es-cr/lp$
350Allow: /es-cr/lp/contact-us
351Allow: /es-cr/lp/homepage$
352Allow: /es-do/lp$
353Allow: /es-do/lp/contact-us
354Allow: /es-do/lp/homepage$
355Allow: /es-ec/lp$
356Allow: /es-ec/lp/contact-us
357Allow: /es-ec/lp/homepage$
358Allow: /es-gt/lp$
359Allow: /es-gt/lp/contact-us
360Allow: /es-gt/lp/homepage$
361Allow: /es-hn/lp$
362Allow: /es-hn/lp/contact-us
363Allow: /es-hn/lp/homepage$
364Allow: /es-mx/lp$
365Allow: /es-mx/lp/contact-us
366Allow: /es-mx/lp/homepage$
367Allow: /es-ni/lp$
368Allow: /es-ni/lp/contact-us
369Allow: /es-ni/lp/homepage$
370Allow: /es-pa/lp$
371Allow: /es-pa/lp/contact-us
372Allow: /es-pa/lp/homepage$
373Allow: /es-pe/lp$
374Allow: /es-pe/lp/contact-us
375Allow: /es-pe/lp/homepage$
376Allow: /es-pr/lp$
377Allow: /es-pr/lp/contact-us
378Allow: /es-pr/lp/homepage$
379Allow: /es-py/lp$
380Allow: /es-py/lp/contact-us
381Allow: /es-py/lp/homepage$
382Allow: /es-sv/lp$
383Allow: /es-sv/lp/contact-us
384Allow: /es-sv/lp/homepage$
385Allow: /es-uy/lp$
386Allow: /es-uy/lp/contact-us
387Allow: /es-uy/lp/homepage$
388Allow: /es-ve/lp$
389Allow: /es-ve/lp/contact-us
390Allow: /es-ve/lp/homepage$
391Allow: /fi-fi/lp$
392Allow: /fi-fi/lp/contact-us
393Allow: /fi-fi/lp/homepage$
394Allow: /fr-bf/lp$
395Allow: /fr-bf/lp/contact-us
396Allow: /fr-bf/lp/homepage$
397Allow: /fr-bi/lp$
398Allow: /fr-bi/lp/contact-us
399Allow: /fr-bi/lp/homepage$
400Allow: /fr-bj/lp$
401Allow: /fr-bj/lp/contact-us
402Allow: /fr-bj/lp/homepage$
403Allow: /fr-cd/lp$
404Allow: /fr-cd/lp/contact-us
405Allow: /fr-cd/lp/homepage$
406Allow: /fr-cf/lp$
407Allow: /fr-cf/lp/contact-us
408Allow: /fr-cf/lp/homepage$
409Allow: /fr-cg/lp$
410Allow: /fr-cg/lp/contact-us
411Allow: /fr-cg/lp/homepage$
412Allow: /fr-ci/lp$
413Allow: /fr-ci/lp/contact-us
414Allow: /fr-ci/lp/homepage$
415Allow: /fr-cm/lp$
416Allow: /fr-cm/lp/contact-us
417Allow: /fr-cm/lp/homepage$
418Allow: /fr-dj/lp$
419Allow: /fr-dj/lp/contact-us
420Allow: /fr-dj/lp/homepage$
421Allow: /fr-dz/lp$
422Allow: /fr-dz/lp/contact-us
423Allow: /fr-dz/lp/homepage$
424Allow: /fr-ga/lp$
425Allow: /fr-ga/lp/contact-us
426Allow: /fr-ga/lp/homepage$
427Allow: /fr-gf/lp$
428Allow: /fr-gf/lp/contact-us
429Allow: /fr-gf/lp/homepage$
430Allow: /fr-gn/lp$
431Allow: /fr-gn/lp/contact-us
432Allow: /fr-gn/lp/homepage$
433Allow: /fr-gq/lp$
434Allow: /fr-gq/lp/contact-us
435Allow: /fr-gq/lp/homepage$
436Allow: /fr-km/lp$
437Allow: /fr-km/lp/contact-us
438Allow: /fr-km/lp/homepage$
439Allow: /fr-lu/lp$
440Allow: /fr-lu/lp/contact-us
441Allow: /fr-lu/lp/homepage$
442Allow: /fr-ma/lp$
443Allow: /fr-ma/lp/contact-us
444Allow: /fr-ma/lp/homepage$
445Allow: /fr-mg/lp$
446Allow: /fr-mg/lp/contact-us
447Allow: /fr-mg/lp/homepage$
448Allow: /fr-ml/lp$
449Allow: /fr-ml/lp/contact-us
450Allow: /fr-ml/lp/homepage$
451Allow: /fr-mq/lp$
452Allow: /fr-mq/lp/contact-us
453Allow: /fr-mq/lp/homepage$
454Allow: /fr-mr/lp$
455Allow: /fr-mr/lp/contact-us
456Allow: /fr-mr/lp/homepage$
457Allow: /fr-nc/lp$
458Allow: /fr-nc/lp/contact-us
459Allow: /fr-nc/lp/homepage$
460Allow: /fr-ne/lp$
461Allow: /fr-ne/lp/contact-us
462Allow: /fr-ne/lp/homepage$
463Allow: /fr-pf/lp$
464Allow: /fr-pf/lp/contact-us
465Allow: /fr-pf/lp/homepage$
466Allow: /fr-re/lp$
467Allow: /fr-re/lp/contact-us
468Allow: /fr-re/lp/homepage$
469Allow: /fr-sc/lp$
470Allow: /fr-sc/lp/contact-us
471Allow: /fr-sc/lp/homepage$
472Allow: /fr-sn/lp$
473Allow: /fr-sn/lp/contact-us
474Allow: /fr-sn/lp/homepage$
475Allow: /fr-td/lp$
476Allow: /fr-td/lp/contact-us
477Allow: /fr-td/lp/homepage$
478Allow: /fr-tg/lp$
479Allow: /fr-tg/lp/contact-us
480Allow: /fr-tg/lp/homepage$
481Allow: /fr-tn/lp$
482Allow: /fr-tn/lp/contact-us
483Allow: /fr-tn/lp/homepage$
484Allow: /fr-yt/lp$
485Allow: /fr-yt/lp/contact-us
486Allow: /fr-yt/lp/homepage$
487Allow: /pt-pt/lp$
488Allow: /pt-pt/lp/contact-us
489Allow: /pt-pt/lp/homepage$
490Allow: /ro-md/lp$
491Allow: /ro-md/lp/contact-us
492Allow: /ro-md/lp/homepage$
493Allow: /ru-am/lp$
494Allow: /ru-am/lp/contact-us
495Allow: /ru-am/lp/homepage$
496Allow: /ru-az/lp$
497Allow: /ru-az/lp/contact-us
498Allow: /ru-az/lp/homepage$
499Allow: /ru-by/lp$
500Allow: /ru-by/lp/contact-us
501Allow: /ru-by/lp/homepage$
502Allow: /ru-ge/lp$
503Allow: /ru-ge/lp/contact-us
504Allow: /ru-ge/lp/homepage$
505Allow: /ru-kg/lp$
506Allow: /ru-kg/lp/contact-us
507Allow: /ru-kg/lp/homepage$
508Allow: /ru-kz/lp$
509Allow: /ru-kz/lp/contact-us
510Allow: /ru-kz/lp/homepage$
511Allow: /ru-ru/lp$
512Allow: /ru-ru/lp/contact-us
513Allow: /ru-ru/lp/homepage$
514Allow: /ru-tj/lp$
515Allow: /ru-tj/lp/contact-us
516Allow: /ru-tj/lp/homepage$
517Allow: /ru-tm/lp$
518Allow: /ru-tm/lp/contact-us
519Allow: /ru-tm/lp/homepage$
520Allow: /ru-uz/lp$
521Allow: /ru-uz/lp/contact-us
522Allow: /ru-uz/lp/homepage$
523Allow: /sk-sk/lp$
524Allow: /sk-sk/lp/contact-us
525Allow: /sk-sk/lp/homepage$
526Allow: /uk-ua/lp$
527Allow: /uk-ua/lp/contact-us
528Allow: /uk-ua/lp/homepage$
529Disallow: */ru-ru
530Disallow: /cs-cz/
531Disallow: /cs-cz/
532Disallow: /el-gr/
533Disallow: /el-gr/
534Disallow: /en-ae/
535Disallow: /en-af/
536Disallow: /en-ag/
537Disallow: /en-ai/
538Disallow: /en-al/
539Disallow: /en-an/
540Disallow: /en-ao/
541Disallow: /en-aw/
542Disallow: /en-ba/
543Disallow: /en-bb/
544Disallow: /en-bd/
545Disallow: /en-bg/
546Disallow: /en-bh/
547Disallow: /en-bm/
548Disallow: /en-bn/
549Disallow: /en-bo/
550Disallow: /en-bs/
551Disallow: /en-bt/
552Disallow: /en-bw/
553Disallow: /en-bz/
554Disallow: /en-cr/
555Disallow: /en-cv/
556Disallow: /en-cy/
557Disallow: /en-dm/
558Disallow: /en-do/
559Disallow: /en-ec/
560Disallow: /en-ed/
561Disallow: /en-ee/
562Disallow: /en-eg/
563Disallow: /en-en/
564Disallow: /en-et/
565Disallow: /en-fj/
566Disallow: /en-gd/
567Disallow: /en-gh/
568Disallow: /en-gm/
569Disallow: /en-gt/
570Disallow: /en-gu/
571Disallow: /en-gy/
572Disallow: /en-hn/
573Disallow: /en-hr/
574Disallow: /en-ht/
575Disallow: /en-id/
576Disallow: /en-il/
577Disallow: /en-iq/
578Disallow: /en-is/
579Disallow: /en-jm/
580Disallow: /en-jo/
581Disallow: /en-ke/
582Disallow: /en-kh/
583Disallow: /en-ki/
584Disallow: /en-kn/
585Disallow: /en-kw/
586Disallow: /en-ky/
587Disallow: /en-la/
588Disallow: /en-lb/
589Disallow: /en-lc/
590Disallow: /en-lk/
591Disallow: /en-lr/
592Disallow: /en-ls/
593Disallow: /en-lt/
594Disallow: /en-lv/
595Disallow: /en-ly/
596Disallow: /en-me/
597Disallow: /en-mk/
598Disallow: /en-mm/
599Disallow: /en-mn/
600Disallow: /en-ms/
601Disallow: /en-mt/
602Disallow: /en-mu/
603Disallow: /en-mv/
604Disallow: /en-mw/
605Disallow: /en-mz/
606Disallow: /en-na/
607Disallow: /en-ng/
608Disallow: /en-ni/
609Disallow: /en-np/
610Disallow: /en-om/
611Disallow: /en-pa/
612Disallow: /en-pg/
613Disallow: /en-ph/
614Disallow: /en-pk/
615Disallow: /en-pr/
616Disallow: /en-pw/
617Disallow: /en-py/
618Disallow: /en-qa/
619Disallow: /en-rw/
620Disallow: /en-sa/
621Disallow: /en-si/
622Disallow: /en-sl/
623Disallow: /en-so/
624Disallow: /en-sr/
625Disallow: /en-sv/
626Disallow: /en-sz/
627Disallow: /en-tc/
628Disallow: /en-th/
629Disallow: /en-tt/
630Disallow: /en-tz/
631Disallow: /en-ua
632Disallow: /en-ug/
633Disallow: /en-vc/
634Disallow: /en-vg/
635Disallow: /en-vi/
636Disallow: /en-vn/
637Disallow: /en-vu/
638Disallow: /en-ws/
639Disallow: /en-ye/
640Disallow: /en-yu/
641Disallow: /en-za/
642Disallow: /en-za/
643Disallow: /en-zm/
644Disallow: /en-zw/
645Disallow: /es-ag/
646Disallow: /es-ai/
647Disallow: /es-an/
648Disallow: /es-ar/
649Disallow: /es-aw/
650Disallow: /es-bb/
651Disallow: /es-bm/
652Disallow: /es-bo/
653Disallow: /es-bs/
654Disallow: /es-bz/
655Disallow: /es-cl/
656Disallow: /es-co/
657Disallow: /es-cr/
658Disallow: /es-dm/
659Disallow: /es-do/
660Disallow: /es-ec/
661Disallow: /es-gd/
662Disallow: /es-gt/
663Disallow: /es-gy/
664Disallow: /es-hn/
665Disallow: /es-ht/
666Disallow: /es-jm/
667Disallow: /es-kn/
668Disallow: /es-ky/
669Disallow: /es-lc/
670Disallow: /es-mx/
671Disallow: /es-ni/
672Disallow: /es-pa/
673Disallow: /es-pe/
674Disallow: /es-pr/
675Disallow: /es-py/
676Disallow: /es-sr/
677Disallow: /es-sv/
678Disallow: /es-tc/
679Disallow: /es-tt/
680Disallow: /es-uy/
681Disallow: /es-vc/
682Disallow: /es-ve/
683Disallow: /es-vg/
684Disallow: /es-vi/
685Disallow: /fi-fi/
686Disallow: /fi-fi/
687Disallow: /fr-bf/
688Disallow: /fr-bi/
689Disallow: /fr-bj/
690Disallow: /fr-cd/
691Disallow: /fr-cf/
692Disallow: /fr-cg/
693Disallow: /fr-ci/
694Disallow: /fr-cm/
695Disallow: /fr-dj/
696Disallow: /fr-dz/
697Disallow: /fr-ed/
698Disallow: /fr-ga/
699Disallow: /fr-gf/
700Disallow: /fr-gn/
701Disallow: /fr-gq/
702Disallow: /fr-km/
703Disallow: /fr-lu/
704Disallow: /fr-lu/
705Disallow: /fr-ma/
706Disallow: /fr-mg/
707Disallow: /fr-ml/
708Disallow: /fr-mq/
709Disallow: /fr-mr/
710Disallow: /fr-nc/
711Disallow: /fr-ne/
712Disallow: /fr-pf/
713Disallow: /fr-re/
714Disallow: /fr-sc/
715Disallow: /fr-sn/
716Disallow: /fr-td/
717Disallow: /fr-tg/
718Disallow: /fr-tn/
719Disallow: /fr-yt/
720Disallow: /hu-hu/
721Disallow: /pt-pt/
722Disallow: /pt-pt/
723Disallow: /ro-md/
724Disallow: /ro-ro/
725Disallow: /ru-am/
726Disallow: /ru-az/
727Disallow: /ru-by/
728Disallow: /ru-ge/
729Disallow: /ru-kg/
730Disallow: /ru-kz/
731Disallow: /ru-rs/
732Disallow: /ru-ru/
733Disallow: /ru-tj/
734Disallow: /ru-tm/
735Disallow: /ru-ua
736Disallow: /ru-uz/
737Disallow: /sk-sk/
738Disallow: /sk-sk/
739Disallow: /tr-tr/
740Disallow: /uk-ua
741Disallow: /uk-ua/
742
743#Incorrect ll-cc pattern
744Disallow: */en-ar$
745Disallow: */en-at$
746Disallow: */en-be$
747Disallow: */en-br$
748Disallow: */en-cd$
749Disallow: */en-cg$
750Disallow: */en-cg/
751Disallow: */en-ch$
752Disallow: */en-cl$
753Disallow: */en-cn$
754Disallow: */en-co$
755Disallow: */en-cz$
756Disallow: */en-de$
757Disallow: */en-dk$
758Disallow: */en-es$
759Disallow: */en-fi$
760Disallow: */en-fr$
761Disallow: */en-ga$
762Disallow: */en-gr$
763Disallow: */en-hu$
764Disallow: */en-it$
765Disallow: */en-jp$
766Disallow: */en-kr$
767Disallow: */en-lu$
768Disallow: */en-mx$
769Disallow: */en-nl$
770Disallow: */en-no$
771Disallow: */en-pe$
772Disallow: */en-pl$
773Disallow: */en-pt$
774Disallow: */en-ro$
775Disallow: /en-rs$
776Disallow: */en-se$
777Disallow: */en-tr$
778Disallow: */en-ua$
779Disallow: */en-ve$
780Disallow: */es-us$
781Disallow: /en-ck
782Disallow: /en-fm
783Disallow: /en-mp
784Disallow: /en-nr
785Disallow: /en-sb
786Disallow: /en-tl
787Disallow: /en-to
788Disallow: /en-tv
789Disallow: /fr-gp
790Disallow: */en-ar/
791Disallow: */en-at/
792Disallow: */en-be/
793Disallow: */en-br/
794Disallow: */en-cd/
795Disallow: */en-ch/
796Disallow: */en-cl/
797Disallow: */en-cn/
798Disallow: */en-co/
799Disallow: */en-cz/
800Disallow: */en-de/
801Disallow: */en-dk/
802Disallow: */en-es/
803Disallow: */en-fi/
804Disallow: */en-fr/
805Disallow: */en-ga/
806Disallow: */en-gr/
807Disallow: */en-hu/
808Disallow: */en-it/
809Disallow: */en-jp/
810Disallow: */en-kr/
811Disallow: */en-lu/
812Disallow: */en-mx/
813Disallow: */en-nl/
814Disallow: */en-no/
815Disallow: */en-pe/
816Disallow: */en-pl/
817Disallow: */en-pt/
818Disallow: */en-ro/
819Disallow: /en-rs/
820Disallow: */en-se/
821Disallow: */en-tr/
822Disallow: */en-ua/
823Disallow: */en-ve/
824Disallow: */es-us/
825
826#LP/DT
827Allow: /da-dk*/lp/
828Allow: /de-at*/lp/
829Allow: /de-ch*/lp/
830Allow: /de-de*/lp/
831Allow: /en-au*/lp/
832Allow: /en-ca*/lp/
833Allow: /en-hk*/lp/
834Allow: /en-ie*/lp/
835Allow: /en-my*/lp/
836Allow: /en-nz*/lp/
837Allow: /en-sg*/lp/
838Allow: /en-uk*/lp/
839Allow: /en-us*/lp/
840Allow: /es-es*/lp/
841Allow: /fr-be*/lp/
842Allow: /fr-ca*/lp/
843Allow: /fr-ch*/lp/
844Allow: /fr-fr*/lp/
845Allow: /it-it*/lp/
846Allow: /ja-jp*/lp/
847Allow: /ko-kr*/lp/
848Allow: /nl-be*/lp/
849Allow: /nl-nl*/lp/
850Allow: /no-no*/lp/
851Allow: /pt-br*/lp/
852Allow: /sv-se*/lp/
853Allow: /zh-cn*/lp/
854Allow: /zh-hk*/lp/
855Allow: /zh-tw*/lp/
856Allow: /en-in*/lp/
857Allow: /pl-pl*/lp/
858#Learn
859Disallow: /learn/in/
860Disallow: /learn/pl/
861Disallow: /learn/au/
862Disallow: /learn/us/
863Disallow: /learn/jp/
864Disallow: /learn/ca/
865Disallow: /learn/dk/
866Disallow: /learn/at/
867Disallow: /learn/ch/
868Disallow: /learn/de/
869Disallow: /learn/ie/
870Disallow: /learn/uk/
871Disallow: /learn/es/
872Disallow: /learn/be/
873Disallow: /learn/ch/
874Disallow: /learn/fr/
875Disallow: /learn/it/
876Disallow: /learn/be/
877Disallow: /learn/nl/
878Disallow: /learn/no/
879Disallow: /learn/se/
880Disallow: /learn/pt/
881Disallow: */learn/search*
882Disallow: /learn/nz/
883Disallow: /learn/sg/
884Disallow: /learn/my/
885Disallow: /learn/kr/
886Disallow: /learn/cn/
887Disallow: /learn/hk/
888Disallow: /learn/tw/
889#Internal Search
890Disallow: /koa/vocaas/
891Disallow: /koa/search
892Disallow: /Search/
893Disallow: /search/
894Disallow: */Search/*
895Disallow: */search/*
896
897#DT
898Disallow: */dt/microsites/cafe/
899Disallow: /en-us/dt/what-we-do/index-apex.htm
900Disallow: /en-us/dt/what-we-do/index-device.htm
901Disallow: /en-us/dt/what-we-do/index-server.htm
902Disallow: /en-us/dt/what-we-do/index-storage.htm
903Allow: /en-uk/dt/
904Allow: /en-au/dt/
905Allow: /en-ca/dt/
906Allow: /en-hk/dt/
907Allow: /en-in/dt/
908Allow: /en-ie/dt/
909Allow: /en-my/dt/
910Allow: /en-nz/dt/
911Allow: /en-sg/dt/
912Allow: /en-us/dt/
913Allow:/fr-ca/dt/
914Allow:/zh-cn/dt/
915Allow:/ja-jp/dt/
916Allow:/de-de/dt/
917Allow:/fr-fr/dt/
918Allow:/sv-se/dt/
919Allow:/es-es/dt/
920Allow:/de-ch/dt/
921Allow:/fr-ch/dt/
922Allow:/pt-br/dt/
923Allow:/nl-nl/dt/
924Allow:/de-at/dt/
925Allow:/nl-be/dt/
926Allow:/fr-be/dt/
927Allow:/da-dk/dt/
928Allow:/it-it/dt/
929Allow:/no-no/dt/
930Allow:/pl-pl/dt/
931Allow:/zh-hk/dt/
932Allow:/zh-tw/dt/
933Allow:/ko-kr/dt/
934Disallow: */dt/*.php$
935Disallow: */dt/*/configuration-page.htm
936Disallow: */dt/*/testing/
937Disallow: */dt/*/teammembers/
938Disallow: */dt/*/gobig
939Disallow: */dt/
940
941#Blog
942Disallow: */blog/
943Disallow: /*/blog*/page/
944Allow: /en-uk/blog/
945Allow: /en-au/blog/
946Allow: /en-ca/blog/
947Allow: /en-hk/blog/
948Allow: /en-in/blog/
949Allow: /en-ie/blog/
950Allow: /en-my/blog/
951Allow: /en-nz/blog/
952Allow: /en-sg/blog/
953Allow: /en-us/blog/
954Allow:/fr-ca/blog/
955Allow:/zh-cn/blog/
956Allow:/ja-jp/blog/
957Allow:/de-de/blog/
958Allow:/fr-fr/blog/
959Allow:/sv-se/blog/
960Allow:/es-es/blog/
961Allow:/de-ch/blog/
962Allow:/fr-ch/blog/
963Allow:/pt-br/blog/
964Allow:/nl-nl/blog/
965Allow:/de-at/blog/
966Allow:/nl-be/blog/
967Allow:/fr-be/blog/
968Allow:/da-dk/blog/
969Allow:/it-it/blog/
970Allow:/no-no/blog/
971Allow:/pl-pl/blog/
972Allow:/zh-hk/blog/
973Allow:/zh-tw/blog/
974Allow:/ko-kr/blog/
975Disallow: */blog/*.php$
976Disallow: */blog/*/configuration-page.htm
977Disallow: */blog/*/testing/
978Disallow: */blog/*/teammembers/
979Disallow: */blog/*/gobig
980
981#Community
982Disallow: /community/en/topics/community-events
983Disallow: /community/en/topics/leaderboard
984Disallow: /community/en/topics/superuser-central
985Disallow: /community/en/users/64c1061fe454291792ef4814
986
987
988User-agent: Baiduspider
989Disallow: /
990Disallow: /zh-cn/cart
991Disallow: /zh-cn/*/cart
992Disallow: /zh-cn/buy
993Disallow: /zh-cn/*/buy
994Disallow: /zh-cn/*/compare
995Disallow: /zh-cn/*/serviceselection/*
996Disallow: /zh-cn/*/rec/*
997Disallow: /zh-cn/outlet/*?
998Allow: /zh-cn
999Allow: /cn/
1000Allow: /tw/
1001Allow: /support/*/cn/zh/
1002Allow: /support/*/tw/zh/
1003Allow: /support/*/hk/zh/
1004Allow: /learn/hk/zh/
1005Allow: /learn/tw/zh/
1006Allow: /*&c=cn&*
1007Allow: /*?c=cn&*
1008Allow: /*&c=tw&*
1009Allow: /*?c=tw&*
1010Allow: /*&c=hk&*
1011Allow: /*?c=hk&*
1012Allow: /support/*/zh-cn/
1013Allow: /support/*/zh-tw/
1014Allow: /support/*/zh-hk/
1015Allow: /support/*/en-hk/
1016Allow: /zh-cn/dt/
1017Allow: /zh-cn/blog/
1018Disallow: /zh-cn/dt/*.php$
1019Disallow: /zh-cn/dt/*/configuration-page.htm
1020Disallow: /zh-cn/dt/*/testing/
1021Disallow: /zh-cn/dt/*/teammembers/
1022Disallow: /zh-cn/dt/*/gobig
1023Disallow: /zh-cn/blog/*.php$
1024Disallow: /zh-cn/blog/*/configuration-page.htm
1025Disallow: /zh-cn/blog/*/testing/
1026Disallow: /zh-cn/blog/*/teammembers/
1027Disallow: /zh-cn/blog/*/gobig
1028Disallow: /zh-cn/corporate/newsroom-test.htm
1029Disallow: /zh-cn/corporate/karen-quintos.htm
1030Disallow: /zh-cn/shop//sf/xps-2-in-1
1031Allow: /community/zh/
1032Disallow: /support/kbdoc/*/000287599
1033
1034User-agent: Sogou web spider
1035Disallow: /
1036Allow: /zh-cn
1037Allow: /cn/
1038Allow: /tw/
1039Allow: /support/*/cn/zh/
1040Allow: /support/*/tw/zh/
1041Allow: /support/*/hk/zh/
1042Allow: /learn/hk/zh/
1043Allow: /learn/tw/zh/
1044Allow: /*&c=cn&*
1045Allow: /*?c=cn&*
1046Allow: /*&c=tw&*
1047Allow: /*?c=tw&*
1048Allow: /*&c=hk&*
1049Allow: /*?c=hk&
1050Allow: /support/*/zh-cn/
1051Allow: /support/*/zh-tw/
1052Allow: /support/*/zh-hk/
1053Allow: /support/*/en-hk/
1054Disallow: /zh-cn/outlet/*?
1055Allow: /zh-cn/dt/
1056Allow: /zh-cn/blog/
1057Disallow: /zh-cn/dt/*.php$
1058Disallow: /zh-cn/dt/*/configuration-page.htm
1059Disallow: /zh-cn/dt/*/testing/
1060Disallow: /zh-cn/dt/*/teammembers/
1061Disallow: /zh-cn/dt/*/gobig
1062Disallow: /zh-cn/blog/*.php$
1063Disallow: /zh-cn/blog/*/configuration-page.htm
1064Disallow: /zh-cn/blog/*/testing/
1065Disallow: /zh-cn/blog/*/teammembers/
1066Disallow: /zh-cn/blog/*/gobig
1067Disallow: /zh-cn/corporate/newsroom-test.htm
1068Disallow: /zh-cn/corporate/karen-quintos.htm
1069Disallow: /zh-cn/shop//sf/xps-2-in-1
1070Allow: /community/zh/
1071Disallow: /support/kbdoc/*/000287599
1072
1073User-agent: Sogou inst spider
1074Disallow: /
1075Allow: /zh-cn
1076Allow: /cn/
1077Allow: /tw/
1078Allow: /support/*/cn/zh/
1079Allow: /support/*/tw/zh/
1080Allow: /support/*/hk/zh/
1081Allow: /learn/hk/zh/
1082Allow: /learn/tw/zh/
1083Allow: /*&c=cn&*
1084Allow: /*?c=cn&*
1085Allow: /*&c=tw&*
1086Allow: /*?c=tw&*
1087Allow: /*&c=hk&*
1088Allow: /*?c=hk&
1089Allow: /support/*/zh-cn/
1090Allow: /support/*/zh-tw/
1091Allow: /support/*/zh-hk/
1092Allow: /support/*/en-hk/
1093Disallow: /zh-cn/outlet/*?
1094Allow: /zh-cn/dt/
1095Allow: /zh-cn/blog/
1096Disallow: /zh-cn/dt/*.php$
1097Disallow: /zh-cn/dt/*/configuration-page.htm
1098Disallow: /zh-cn/dt/*/testing/
1099Disallow: /zh-cn/dt/*/teammembers/
1100Disallow: /zh-cn/dt/*/gobig
1101Disallow: /zh-cn/blog/*.php$
1102Disallow: /zh-cn/blog/*/configuration-page.htm
1103Disallow: /zh-cn/blog/*/testing/
1104Disallow: /zh-cn/blog/*/teammembers/
1105Disallow: /zh-cn/blog/*/gobig
1106Disallow: /zh-cn/corporate/newsroom-test.htm
1107Disallow: /zh-cn/corporate/karen-quintos.htm
1108Disallow: /zh-cn/shop//sf/xps-2-in-1
1109Allow: /community/zh/
1110Disallow: /support/kbdoc/*/000287599
1111
1112User-agent: Sogou spider2
1113Disallow: /
1114Allow: /zh-cn
1115Allow: /cn/
1116Allow: /tw/
1117Allow: /support/*/cn/zh/
1118Allow: /support/*/tw/zh/
1119Allow: /support/*/hk/zh/
1120Allow: /learn/hk/zh/
1121Allow: /learn/tw/zh/
1122Allow: /*&c=cn&*
1123Allow: /*?c=cn&*
1124Allow: /*&c=tw&*
1125Allow: /*?c=tw&*
1126Allow: /*&c=hk&*
1127Allow: /*?c=hk&
1128Allow: /support/*/zh-cn/
1129Allow: /support/*/zh-tw/
1130Allow: /support/*/zh-hk/
1131Allow: /support/*/en-hk/
1132Disallow: /zh-cn/outlet/*?
1133Allow: /zh-cn/dt/
1134Allow: /zh-cn/blog/
1135Disallow: /zh-cn/dt/*.php$
1136Disallow: /zh-cn/dt/*/configuration-page.htm
1137Disallow: /zh-cn/dt/*/testing/
1138Disallow: /zh-cn/dt/*/teammembers/
1139Disallow: /zh-cn/dt/*/gobig
1140Disallow: /zh-cn/blog/*.php$
1141Disallow: /zh-cn/blog/*/configuration-page.htm
1142Disallow: /zh-cn/blog/*/testing/
1143Disallow: /zh-cn/blog/*/teammembers/
1144Disallow: /zh-cn/blog/*/gobig
1145Disallow: /zh-cn/corporate/newsroom-test.htm
1146Disallow: /zh-cn/corporate/karen-quintos.htm
1147Disallow: /zh-cn/shop//sf/xps-2-in-1
1148Allow: /community/zh/
1149Disallow: /support/kbdoc/*/000287599
1150
1151User-agent: HaoSouSpider
1152Disallow: /
1153Allow: /zh-cn
1154Allow: /cn/
1155Allow: /tw/
1156Allow: /support/*/cn/zh/
1157Allow: /support/*/tw/zh/
1158Allow: /support/*/hk/zh/
1159Allow: /learn/hk/zh/
1160Allow: /learn/tw/zh/
1161Allow: /*&c=cn&*
1162Allow: /*?c=cn&*
1163Allow: /*&c=tw&*
1164Allow: /*?c=tw&*
1165Allow: /*&c=hk&*
1166Allow: /*?c=hk&
1167Allow: /support/*/zh-cn/
1168Allow: /support/*/zh-tw/
1169Allow: /support/*/zh-hk/
1170Allow: /support/*/en-hk/
1171Disallow: /zh-cn/outlet/*?
1172Allow: /zh-cn/dt/
1173Allow: /zh-cn/blog/
1174Disallow: /zh-cn/dt/*.php$
1175Disallow: /zh-cn/dt/*/configuration-page.htm
1176Disallow: /zh-cn/dt/*/testing/
1177Disallow: /zh-cn/dt/*/teammembers/
1178Disallow: /zh-cn/dt/*/gobig
1179Disallow: /zh-cn/blog/*.php$
1180Disallow: /zh-cn/blog/*/configuration-page.htm
1181Disallow: /zh-cn/blog/*/testing/
1182Disallow: /zh-cn/blog/*/teammembers/
1183Disallow: /zh-cn/blog/*/gobig
1184Disallow: /zh-cn/corporate/newsroom-test.htm
1185Disallow: /zh-cn/corporate/karen-quintos.htm
1186Disallow: /zh-cn/shop//sf/xps-2-in-1
1187Allow: /community/zh/
1188Disallow: /support/kbdoc/*/000287599
1189User-agent: Bytespider
1190Disallow: /community/*=es
1191Disallow: /community/*=de
1192Disallow: /community/*=pt
1193Disallow: /community/*=fr
1194Disallow: /community/*=ko
1195Disallow: /community/*=ja
1196Disallow: /community/*-es
1197Disallow: /community/*-de
1198Disallow: /community/*-pt
1199Disallow: /community/*-fr
1200Disallow: /community/*-ko
1201Disallow: /community/*-ja
1202Disallow: /support/kbdoc/*/000287599
1203User-agent: AmazonAdBot
1204Disallow: /community/*
1205Allow: /community/zh/
1206User-agent: 360Spider
1207Disallow: /
1208Allow: /zh-cn
1209Allow: /cn/
1210Allow: /tw/
1211Allow: /support/*/cn/zh/
1212Allow: /support/*/tw/zh/
1213Allow: /support/*/hk/zh/
1214Allow: /learn/hk/zh/
1215Allow: /learn/tw/zh/Allow: /*&c=cn&*
1216Allow: /*?c=cn&*
1217Allow: /*&c=tw&*
1218Allow: /*?c=tw&*
1219Allow: /*&c=hk&*
1220Allow: /*?c=hk&
1221Allow: /support/*/zh-cn/
1222Allow: /support/*/zh-tw/
1223Allow: /support/*/zh-hk/
1224Allow: /support/*/en-hk/
1225Disallow: /zh-cn/outlet/*?
1226Allow: /zh-cn/dt/
1227Allow: /zh-cn/blog/
1228Disallow: /zh-cn/dt/*.php$
1229Disallow: /zh-cn/dt/*/configuration-page.htm
1230Disallow: /zh-cn/dt/*/testing/
1231Disallow: /zh-cn/dt/*/teammembers/
1232Disallow: /zh-cn/dt/*/gobig
1233Disallow: /zh-cn/blog/*.php$
1234Disallow: /zh-cn/blog/*/configuration-page.htm
1235Disallow: /zh-cn/blog/*/testing/
1236Disallow: /zh-cn/blog/*/teammembers/
1237Disallow: /zh-cn/blog/*/gobig
1238Disallow: /zh-cn/corporate/newsroom-test.htm
1239Disallow: /zh-cn/corporate/karen-quintos.htm
1240Disallow: /zh-cn/shop//sf/xps-2-in-1
1241Allow: /community/zh/
1242Disallow: /support/kbdoc/*/000287599
1243
1244Sitemap: https://www.dell.com/index-sr-sitemap.xml.gz
1245Sitemap: https://www.dell.com/index-deals-href-sitemap.xml.gz
1246Sitemap: https://www.dell.com/index-product-pages-hreflang-sitemap.xml.gz
1247Sitemap: https://www.dell.com/index-sitemap.xml.gz
1248Sitemap: https://www.dell.com/index-snp-apj-sitemap.xml.gz
1249Sitemap: https://www.dell.com/index-support-href-sitemap.xml.gz
1250Sitemap: https://www.dell.com/index-video-xml-sitemap.xml.gz
1251Sitemap: https://www.dell.com/index-downloads-dell-pdfs-sitemap.xml.gz
1252Sitemap: https://www.dell.com/index-dt-sitemap.xml.gz
1253
1254# Last Modified on 2026 Oct 7th, 5:34AM CST
What do the bots actually get?
This page reads robots.txt, which states what dell.com asks of crawlers. A firewall or CDN can still refuse a bot that robots.txt allows. The live check requests the page as each bot and shows the status it gets.
Run a live check on dell.com Test a path in the robots.txt tester
History
- 19 October 2026: first checked.
Questions
Does dell.com block GPTBot?
No. dell.com’s robots.txt lets GPTBot crawl the site. GPTBot is OpenAI’s crawler for training data; blocking it doesn’t affect ChatGPT search, which uses OAI-SearchBot. Checked 9 November 2026.
Does dell.com block ClaudeBot?
No. dell.com’s robots.txt lets ClaudeBot crawl the site. ClaudeBot is Anthropic’s crawler for training data. Checked 9 November 2026.
Does dell.com block Google-Extended?
No. dell.com’s robots.txt lets Google-Extended crawl the site. Google-Extended is the token that controls whether Google may use pages for Gemini training and grounding; it doesn’t affect Google Search. Checked 9 November 2026.
Does dell.com block PerplexityBot?
No. dell.com’s robots.txt lets PerplexityBot crawl the site. PerplexityBot is the crawler that indexes pages for Perplexity’s answers. Checked 9 November 2026.
Can ChatGPT search find dell.com?
Yes. dell.com’s robots.txt lets OAI-SearchBot crawl the site. OAI-SearchBot is the crawler behind ChatGPT search; blocking it keeps the site out of ChatGPT’s search answers. Checked 9 November 2026.