Who blocks AI · #39 in the Majestic Million
robots.txt · checked 9 Nov 2026
Does adobe.com block AI crawlers?
No. adobe.com’s robots.txt lets all 19 AI crawlers we track crawl the site. Search engines such as Googlebot and Bingbot may crawl it.
GPTBotAllowedOAI-SearchBotAllowedClaudeBotAllowedPerplexityBotAllowedGoogle-ExtendedAllowedCCBotAllowed- robots.txt
- www.adobe.com/robots.txt · 228 lines
- llms.txt
- found
- Checked
- 9 November 2026, as canaicrawl. We check again every week.
Every bot, and the line that decides
Search engines
| Bot | Operator | robots.txt | Deciding line |
|---|---|---|---|
Googlebot | Partly blocked | line 195 · Disallow: /*promoid= line 196 · Disallow: /*trackingid= and 5 more | |
Bingbot | Microsoft | Allowed | no rule matches the home page |
DuckDuckBot | DuckDuckGo | Allowed | no rule matches the home page |
Applebot | Apple | Partly blocked | line 195 · Disallow: /*promoid= line 196 · Disallow: /*trackingid= and 5 more |
PetalBot | Huawei | Allowed | no rule matches the home page |
YandexBot | Yandex | Allowed | no rule matches the home page |
Baiduspider | Baidu | Allowed | no rule matches the home page |
AI search
| Bot | Operator | robots.txt | Deciding line |
|---|---|---|---|
OAI-SearchBot | OpenAI | Allowed | no rule matches the home page |
PerplexityBot | Perplexity | Allowed | no rule matches the home page |
Claude-SearchBot | Anthropic | Allowed | no rule matches the home page |
DuckAssistBot | DuckDuckGo | Allowed | no rule matches the home page |
Amazonbot | Amazon | Allowed | no rule matches the home page |
YouBot | You.com | Allowed | no rule matches the home page |
AI training
| Bot | Operator | robots.txt | Deciding line |
|---|---|---|---|
GPTBot | OpenAI | Allowed | no rule matches the home page |
ClaudeBot | Anthropic | Allowed | no rule matches the home page |
Google-Extended | Allowed | no rule matches the home page | |
CCBot | Common Crawl | Allowed | no rule matches the home page |
Applebot-Extended | Apple | Allowed | no rule matches the home page |
GoogleOther | Allowed | no rule matches the home page | |
Meta-ExternalAgent | Meta | Allowed | no rule matches the home page |
Bytespider | ByteDance | Allowed | no rule matches the home page |
User-triggered fetchers
| Bot | Operator | robots.txt | Deciding line |
|---|---|---|---|
ChatGPT-User | OpenAI | Allowed | no rule matches the home page |
Claude-User | Anthropic | Allowed | no rule matches the home page |
Perplexity-User | Perplexity | Allowed | no rule matches the home page |
MistralAI-User | Mistral | Allowed | no rule matches the home page |
Meta-ExternalFetcher | Meta | Allowed | no rule matches the home page |
SEO tools
| Bot | Operator | robots.txt | Deciding line |
|---|---|---|---|
AhrefsBot | Ahrefs | Allowed | no rule matches the home page |
SemrushBot | Semrush | Allowed | no rule matches the home page |
MJ12bot | Majestic | Allowed | no rule matches the home page |
DotBot | Moz | Allowed | no rule matches the home page |
View www.adobe.com/robots.txt · 228 lines
1# The use of robots or other automated means to access the Adobe site
2# without the express permission of Adobe is strictly prohibited.
3# Notwithstanding the foregoing, Adobe may permit automated access to
4# access certain Adobe pages but solely for the limited purpose of
5# including content in publicly available search engines. Any other
6# use of robots or failure to obey the robots exclusion standards set
7# forth at http://www.robotstxt.org/ is strictly prohibited.
8
9# Details about Googlebot available at: http://www.google.com/bot.html
10# The Google search engine can see everything
11User-agent: gsa-crawler-www
12Allow: /assets/globalnav/v1.1.4/css/globalnav.css
13Allow: /assets/globalnav/v1.1.4/*/globalnav.js
14Allow: /au/special/symposium
15Allow: /sea/special/symposium
16Allow: /*special/products/marketing-cloud/*.css$
17Allow: /*special/products/marketing-cloud/*.png$
18Allow: /*special/products/marketing-cloud/*.js$
19Allow: /www.adobe.com/special/ri/prod/assets/*.css$
20Allow: /www.adobe.com/special/ri/prod/assets/*.jpg$
21Allow: /www.adobe.com/special/ri/prod/assets/*.js$
22Allow: /www.adobe.com/special/ri/prod/assets/*.png$
23Allow: /www.adobe.com/special/retail-shopping-insights/
24Allow: /uk/special/landing/student/digitaledgeaward/
25Allow: /uk/special/landing/student/digitaledgeaward/
26Allow: /uk/special/landing/student/lovethejourney
27Allow: /fr/special/landing/student/lovethejourney
28Disallow: /assets/globalnav/
29Disallow: /*creativesuite/bridge
30Disallow: /cfusion/gpr/
31Disallow: /cfusion/search/
32Disallow: /cfusion/partnerportal/
33Disallow: /globalsearch
34Disallow: /cgi-local/
35Disallow: /de/products/index-notabs.html
36Disallow: /de/products/index-tabs.html
37Disallow: /devnet-archive/
38Disallow: /devnet/archive
39Disallow: /*devnet/tagsearch
40Disallow: /emea/eemeachannel/
41Disallow: /events/ciocouncil/
42Disallow: /events/executivecouncil/
43Disallow: /google_indexing/
44Disallow: /*.html_
45Disallow: /licensing/distribution/strategies/
46Disallow: /*limited/
47Disallow: /livedocs/
48Disallow: /*Ljava.lang.Object*
49Disallow: /products/acrobat/readstep2.html;JMICROSITE2SESSION*
50Disallow: /*products/flashplayer/distribution3.html
51Disallow: /*products/flashplayer/distribution4.html
52Disallow: /*special/
53Disallow: /*.sql$
54Disallow: /*supportservice/
55Disallow: /*.translate.html
56Disallow: /*products/adobeconnect/enterprise/trial.html
57Disallow: /*products/adobeconnect/enterprise/trial-check.html
58Disallow: /*products/adobeconnect/enterprise/trial-thankyou.html
59Disallow: /*products/adobeconnect/enterprise/trial-error.html
60Disallow: /*products/photoshop/selector
61Disallow: /*unsubscribe.html
62Disallow: /content/dam/acom/en/marketing-cloud/offers/pdf
63Disallow: /content/dam/www/us/en/adi/2020/pdfs/*
64Disallow: /content/dotcom/*.html$
65Disallow: /content/dam/www/us/en/offer/digital-trends-2020/digital-trends-2020-full-report.pdf
66Disallow: /plans-fragments/
67Disallow: /plans.modals*
68
69# The Omniture search engine can see everything
70User-agent: Atomz/1.0
71Disallow: /devnet-archive/
72Disallow: /devnet/archive
73Disallow: /events/ciocouncil/
74Disallow: /events/executivecouncil/
75Disallow: /licensing/distribution/strategies/
76Disallow: /*limited/
77Disallow: /livedocs/
78Disallow: /products/flashplayer/distribution3.html
79Disallow: /products/flashplayer/distribution4.html
80Disallow: /*special/
81Disallow: /*.sql$
82Disallow: /*supportservice/
83
84User-agent: *
85Allow: /assets/globalnav/v1.1.4/css/globalnav.css
86Allow: /assets/globalnav/v1.1.4/*globalnav.js
87Allow: /au/special/symposium
88Allow: /sea/special/symposium
89Allow: /*special/products/marketing-cloud/*.css$
90Allow: /*special/products/marketing-cloud/*.js$
91Allow: /*special/products/marketing-cloud/*.png$
92Allow: /www.adobe.com/special/ri/prod/assets/*.css$
93Allow: /www.adobe.com/special/ri/prod/assets/*.jpg$
94Allow: /www.adobe.com/special/ri/prod/assets/*.js$
95Allow: /www.adobe.com/special/ri/prod/assets/*.png$
96Allow: /www.adobe.com/special/retail-shopping-insights/
97Allow: /uk/special/landing/student/digitaledgeaward/
98Allow: /uk/special/landing/student/lovethejourney
99Allow: /fr/special/landing/student/lovethejourney
100Allow: /community/gallery$
101Allow: /community/gallery/assets$
102Allow: /community/gallery/challenges$
103Disallow: /assets/globalnav/
104Disallow: /*creativesuite/bridge
105Disallow: /cfusion/gpr/
106Disallow: /cfusion/search/
107Disallow: /cfusion/partnerportal/
108Disallow: /globalsearch
109Disallow: /cgi-local/
110Disallow: /de/products/index-notabs.html
111Disallow: /de/products/index-tabs.html
112Disallow: /devnet-archive/
113Disallow: /devnet/archive
114Disallow: /*devnet/tagsearch
115Disallow: /emea/eemeachannel/
116Disallow: /events/ciocouncil/
117Disallow: /events/executivecouncil/
118Disallow: /google_indexing/
119Disallow: /*.html_
120Disallow: /licensing/distribution/strategies/
121Disallow: /*limited/
122Disallow: /livedocs/
123Disallow: /*Ljava.lang.Object*
124Disallow: /products/acrobat/readstep2.html;JMICROSITE2SESSION*
125Disallow: /*products/flashplayer/distribution3.html
126Disallow: /*products/flashplayer/distribution4.html
127Disallow: /*special/
128Disallow: /*.sql$
129Disallow: /*supportservice/
130Disallow: /*.translate.html
131Disallow: /*products/adobeconnect/enterprise/trial.html
132Disallow: /*products/adobeconnect/enterprise/trial-check.html
133Disallow: /*products/adobeconnect/enterprise/trial-thankyou.html
134Disallow: /*products/adobeconnect/enterprise/trial-error.html
135Disallow: /*products/photoshop/selector
136Disallow: /*unsubscribe.html
137Disallow: /content/dam/acom/en/marketing-cloud/offers/pdf
138Disallow: /content/dotcom/*.html$
139Disallow: /plans-fragments/
140Disallow: /content/dam/www/us/en/offer/digital-trends-2020-in-media-and-entertainment-html/digital-trends-2020-in-media-and-entertainment.pdf
141Disallow: /content/dam/www/us/en/offer/digital-trends-2020-in-media-and-entertainment-html/digital-trends-2020-in-media-and-entertainment-DE.pdf
142Disallow: /content/dam/www/us/en/offer/digital-trends-2020-in-media-and-entertainment-html/digital-trends-2020-in-media-and-entertainment-EN-US.pdf
143Disallow: /content/dam/www/us/en/offer/digital-trends-2020-in-media-and-entertainment-html/digital-trends-2020-in-media-and-entertainment-EN.pdf
144Disallow: /content/dam/www/us/en/offer/digital-trends-2020-in-media-and-entertainment-html/digital-trends-2020-in-media-and-entertainment-ES.pdf
145Disallow: /content/dam/www/us/en/offer/digital-trends-2020-in-media-and-entertainment-html/digital-trends-2020-in-media-and-entertainment-FR.pdf
146Disallow: /content/dam/www/us/en/offer/digital-trends-2020-in-media-and-entertainment-html/digital-trends-2020-in-media-and-entertainment-IT.pdf
147Disallow: /content/dam/www/us/en/offer/digital-trends-2020-in-media-and-entertainment-html/digital-trends-2020-in-media-and-entertainment-JA.pdf
148Disallow: /content/dam/www/us/en/offer/digital-trends-2020-in-media-and-entertainment-html/digital-trends-2020-in-media-and-entertainment-KR.pdf
149Disallow: /content/dam/www/us/en/offer/digital-trends-2020-in-media-and-entertainment-html/digital-trends-2020-in-media-and-entertainment-zh-Hans.pdf
150Disallow: /content/dam/www/us/en/offer/digital-trends-2020-in-media-and-entertainment-html/digital-trends-2020-in-media-and-entertainment-zh-Hant.pdf
151Disallow: /content/dam/www/us/en/offer/digital-trends-2020-in-media-and-entertainment-html/digital_trends_21_core_report_TC_FA.pdf
152Disallow: /content/dam/dx/us/en/resources/reports/digital-trends-2021-in-high-tech/2021_Digital_Trends_High_Tech_in_Focus.pdf
153Disallow: /content/dam/dx/us/en/resources/digital-trends-2021-in-telecommunications-media-and-entertainment/Digital_Trends_2021_Media_Entertainment_and_Telecoms_Report.pdf
154Disallow: /content/dam/dx/us/en/resources/digital-trends-2021-in-retail/2021_Digital_Trends_Retail_in_Focus.pdf
155Disallow: /content/dam/dx/us/en/resources/digital-trends-2021-in-manufacturing/2021_Digital_Trends_Manufacturing_in_Focus.pdf
156Disallow: /content/dam/www/us/en/offer/technology-trends-2021/eConsultancy_Digital_Trends_2021_IT_Report.pdf
157Disallow: /content/dam/www/us/en/offer/pdf/digital-trends-2020-in-financial-services.pdf
158Disallow: /content/dam/www/us/en/offer/pdf/digital-trends-2020-in-retail.pdf
159Disallow: /content/dam/www/us/en/offer/pdf/digital-trends-2020-in-media-and-entertainment.pdf
160Disallow: /content/dam/www/us/en/offer/pdf/digital-trends-2020-in-travel-and-hospitality.pdf
161Disallow: /content/dam/www/us/en/offer/pdf/digital-trends-2020-in-telecommunications.pdf
162Disallow: /content/dam/www/us/en/offer/pdf/digital-trends-2020-in-advertising.pdf
163Disallow: /content/dam/www/us/en/offer/pdf/digital-trends-2020-in-high-tech.pdf
164Disallow: /content/dam/www/us/en/offer/pdf/digital-trends-2020-in-IT.pdf
165Disallow: /content/dam/www/us/en/adi/2020/pdfs/*
166Disallow: /content/dam/www/us/en/offer/digital-trends-2021-core/digital-trends-2021-full-report-*.pdf
167Disallow: /*langmaster/
168Disallow: /*express/*.mp4$
169Disallow: /en-AE/learn/
170Disallow: /en-BE/learn/
171Disallow: /en-CY/learn/
172Disallow: /en-GR/learn/
173Disallow: /en-HK/learn/
174Disallow: /en-ID/learn/
175Disallow: /en-IL/learn/
176Disallow: /en-LU/learn/
177Disallow: /en-MT/learn/
178Disallow: /en-MY/learn/
179Disallow: /en-PH/learn/
180Disallow: /en-SA/learn/
181Disallow: /en-SG/learn/
182Disallow: /en-TH/learn/
183Disallow: /en-VN/learn/
184Disallow: /en-x-MenA/learn/
185Disallow: /en-ZA/learn/
186Disallow: /*?searchId=
187Disallow: /*?tasks=
188Disallow: /*?campaignid=
189Disallow: /*?skwcid=
190Disallow: /products/photoshop/photoshop-brushes.h*
191Disallow: /community/gallery
192
193User-agent: Googlebot
194Allow: /?promoid=RTQCN3LX
195Disallow: /*promoid=
196Disallow: /*trackingid=
197Disallow: /*?searchId=
198Disallow: /*?tasks=
199Disallow: /*?campaignid=
200Disallow: /*?skwcid=
201Disallow: /products/photoshop/photoshop-brushes.h*
202
203User-agent: AdsBot-Google
204Allow: /
205
206User-agent: AdsBot-Google-Mobile
207Allow: /
208
209User-agent: SearchmetricsBot
210Disallow: /
211
212# XML sitemaps
213Sitemap: https://www.adobe.com/cc-product.index.xml
214Sitemap: https://www.adobe.com/creativecloud/sitemap-index.xml
215Sitemap: https://www.adobe.com/acrobat/sitemap-index.xml
216Sitemap: https://www.adobe.com/home-sitemap.xml
217Sitemap: https://www.adobe.com/plans-catalog-sitemap.xml
218Sitemap: https://www.adobe.com/learn/sitemap_index.xml
219
220# CC Express
221Sitemap: https://www.adobe.com/express/sitemap-index.xml
222
223#Naver
224Sitemap: https://www.adobe.com/naver.index.xml
225
226#Education
227Sitemap: https://www.adobe.com/edu/sitemap-index.xml
228
What do the bots actually get?
This page reads robots.txt, which states what adobe.com asks of crawlers. A firewall or CDN can still refuse a bot that robots.txt allows. The live check requests the page as each bot and shows the status it gets.
Run a live check on adobe.com Test a path in the robots.txt tester
History
- 19 October 2026: first checked.
Questions
Does adobe.com block GPTBot?
No. adobe.com’s robots.txt lets GPTBot crawl the site. GPTBot is OpenAI’s crawler for training data; blocking it doesn’t affect ChatGPT search, which uses OAI-SearchBot. Checked 9 November 2026.
Does adobe.com block ClaudeBot?
No. adobe.com’s robots.txt lets ClaudeBot crawl the site. ClaudeBot is Anthropic’s crawler for training data. Checked 9 November 2026.
Does adobe.com block Google-Extended?
No. adobe.com’s robots.txt lets Google-Extended crawl the site. Google-Extended is the token that controls whether Google may use pages for Gemini training and grounding; it doesn’t affect Google Search. Checked 9 November 2026.
Does adobe.com block PerplexityBot?
No. adobe.com’s robots.txt lets PerplexityBot crawl the site. PerplexityBot is the crawler that indexes pages for Perplexity’s answers. Checked 9 November 2026.
Can ChatGPT search find adobe.com?
Yes. adobe.com’s robots.txt lets OAI-SearchBot crawl the site. OAI-SearchBot is the crawler behind ChatGPT search; blocking it keeps the site out of ChatGPT’s search answers. Checked 9 November 2026.