Who blocks AI · #319 in the Majestic Million

robots.txt · checked 9 Nov 2026

Does cisco.com block AI crawlers?

No. cisco.com’s robots.txt lets all 19 AI crawlers we track crawl the site. Search engines such as Googlebot and Bingbot may crawl it.

GPTBotAllowed OAI-SearchBotAllowed ClaudeBotAllowed PerplexityBotAllowed Google-ExtendedAllowed CCBotAllowed

robots.txt: www.cisco.com/robots.txt · 310 lines
llms.txt: found
Checked: 9 November 2026, as canaicrawl. We check again every week.

Every bot, and the line that decides

Search engines

Bot	Operator	robots.txt	Deciding line
`Googlebot`	Google	Allowed	no rule matches the home page
`Bingbot`	Microsoft	Allowed	no rule matches the home page
`DuckDuckBot`	DuckDuckGo	Allowed	no rule matches the home page
`Applebot`	Apple	Allowed	no rule matches the home page
`PetalBot`	Huawei	Allowed	no rule matches the home page
`YandexBot`	Yandex	Allowed	no rule matches the home page
`Baiduspider`	Baidu	Allowed	no rule matches the home page

AI search

Bot	Operator	robots.txt	Deciding line
`OAI-SearchBot`	OpenAI	Allowed	no rule matches the home page
`PerplexityBot`	Perplexity	Allowed	no rule matches the home page
`Claude-SearchBot`	Anthropic	Allowed	no rule matches the home page
`DuckAssistBot`	DuckDuckGo	Allowed	no rule matches the home page
`Amazonbot`	Amazon	Allowed	no rule matches the home page
`YouBot`	You.com	Allowed	no rule matches the home page

AI training

Bot	Operator	robots.txt	Deciding line
`GPTBot`	OpenAI	Allowed	no rule matches the home page
`ClaudeBot`	Anthropic	Allowed	no rule matches the home page
`Google-Extended`	Google	Allowed	no rule matches the home page
`CCBot`	Common Crawl	Allowed	no rule matches the home page
`Applebot-Extended`	Apple	Allowed	no rule matches the home page
`GoogleOther`	Google	Allowed	no rule matches the home page
`Meta-ExternalAgent`	Meta	Allowed	no rule matches the home page
`Bytespider`	ByteDance	Allowed	no rule matches the home page

User-triggered fetchers

Bot	Operator	robots.txt	Deciding line
`ChatGPT-User`	OpenAI	Allowed	no rule matches the home page
`Claude-User`	Anthropic	Allowed	no rule matches the home page
`Perplexity-User`	Perplexity	Allowed	no rule matches the home page
`MistralAI-User`	Mistral	Allowed	no rule matches the home page
`Meta-ExternalFetcher`	Meta	Allowed	no rule matches the home page

SEO tools

Bot	Operator	robots.txt	Deciding line
`AhrefsBot`	Ahrefs	Allowed	no rule matches the home page
`SemrushBot`	Semrush	Allowed	no rule matches the home page
`MJ12bot`	Majestic	Allowed	no rule matches the home page
`DotBot`	Moz	Allowed	no rule matches the home page

View www.cisco.com/robots.txt · 310 lines

1#--------------------------------

2User-agent: cisco-googlebot-enterprise

3Disallow: /bug-navigator # Bug Data

4# Disallow: /cgi-bin # allow test crawls for TAC support content

5Disallow: /pcgi-bin # no programs

6Disallow: /univ-src/ccden # will get content through /univercd

7Disallow: /cpropub/univercd # obsolete

8Disallow: /jobs # temporary entry per performance team

9#--------------------------------

10User-agent: *

11Disallow: /*.html?cachemode=refresh$

12Disallow: /bug-navigator # Bug Data

13Disallow: /web/telepresence/pro/

14Disallow: /cgi-bin # no programs

15Disallow: /pcgi-bin # no programs

16Disallow: /univ-src/ccden # will get content through /univercd

17Disallow: /cpropub/univercd # obsolete

18Disallow: /jobs # temporary entry per performance team

19Disallow: /en/US/swassets/

20Disallow: /univercd/cc/td/doc

21Disallow: /univercd/

22Disallow: /tacpage

23Disallow: /survey/

24Disallow: /web/login

25Disallow: /ciscodotcom/

26Disallow: /c/contentReuse/

27Disallow: /c/contentreuse/

28Disallow: /c/dam/assets/offers/pdfs/

29Disallow: /c/dam/en/us/td/docs/Sanity/datasheet/

30Disallow: /c/dam/m/da_dk/offers/assets/images/

31Disallow: /c/dam/m/da_dk/offers/assets/pdfs/

32Disallow: /c/dam/m/de_de/offers/assets/images/

33Disallow: /c/dam/m/de_de/offers/assets/pdfs/

34Disallow: /c/dam/m/digital/

35Disallow: /c/dam/m/en_au/offers/assets/images/

36Disallow: /c/dam/m/en_au/offers/assets/pdfs/

37Disallow: /c/dam/m/en_au/offers/images/

38Disallow: /c/dam/m/en_au/offers/video/

39Disallow: /c/dam/m/en_emear/offers/assets/images/

40Disallow: /c/dam/m/en_emear/offers/assets/pdfs/

41Disallow: /c/dam/m/en_hk/offers/assets/images/

42Disallow: /c/dam/m/en_hk/offers/assets/pdf/

43Disallow: /c/dam/m/en_id/offers/assets/images/

44Disallow: /c/dam/m/en_id/offers/assets/pdfs/

45Disallow: /c/dam/m/en_id/offers/images/

46Disallow: /c/dam/m/en_in/offers/assets/images/

47Disallow: /c/dam/m/en_in/offers/assets/pdfs/

48Disallow: /c/dam/m/en_in/offers/images

49Disallow: /c/dam/m/en_in/offers/video/

50Disallow: /c/dam/m/en_my/offers/assets/images/

51Disallow: /c/dam/m/en_my/offers/assets/pdfs/

52Disallow: /c/dam/m/en_my/offers/images/

53Disallow: /c/dam/m/en_ph/offers/assets/images/

54Disallow: /c/dam/m/en_ph/offers/assets/pdfs/

55Disallow: /c/dam/m/en_ph/offers/images/

56Disallow: /c/dam/m/en_sg/offers/assets/images/

57Disallow: /c/dam/m/en_sg/offers/assets/pdfs/

58Disallow: /c/dam/m/en_sg/offers/images/

59Disallow: /c/dam/m/en_sg/offers/video/

60Disallow: /c/dam/m/en_th/offers/assets/images/

61Disallow: /c/dam/m/en_th/offers/assets/pdfs/

62Disallow: /c/dam/m/en_th/offers/images/

63Disallow: /c/dam/m/en_uk/offers/assets/images/

64Disallow: /c/dam/m/en_uk/offers/assets/pdfs/

65Disallow: /c/dam/m/en_vn/offers/assets/images/

66Disallow: /c/dam/m/en_vn/offers/assets/pdfs/

67Disallow: /c/dam/m/es_es/offers/assets/images/

68Disallow: /c/dam/m/es_es/offers/assets/pdfs/

69Disallow: /c/dam/m/es_la/offers/assets/images/

70Disallow: /c/dam/m/es_la/offers/assets/pdfs/

71Disallow: /c/dam/m/es_la/offers/video/

72Disallow: /c/dam/m/es_mx/offers/assets/images/

73Disallow: /c/dam/m/es_mx/offers/assets/pdfs/

74Disallow: /c/dam/m/fi_fi/offers/assets/images/

75Disallow: /c/dam/m/fi_fi/offers/assets/pdfs/

76Disallow: /c/dam/m/fr_ca/offers/assets/images/

77Disallow: /c/dam/m/fr_ca/offers/assets/pdfs/

78Disallow: /c/dam/m/fr_ca/offers/video/

79Disallow: /c/dam/m/fr_fr/offers/assets/images

80Disallow: /c/dam/m/fr_fr/offers/assets/pdfs/

81Disallow: /c/dam/m/it_it/offers/assets/images/

82Disallow: /c/dam/m/it_it/offers/assets/pdfs/

83Disallow: /c/dam/m/ja_jp/offers/161/security/images/

84Disallow: /c/dam/m/ja_jp/offers/161/security/pdf/

85Disallow: /c/dam/m/ja_jp/offers/162/network/images

86Disallow: /c/dam/m/ja_jp/offers/162/network/pdf/

87Disallow: /c/dam/m/ja_jp/offers/162/sc10/images/

88Disallow: /c/dam/m/ja_jp/offers/162/sc10/pdf/

89Disallow: /c/dam/m/ja_jp/offers/162/security/pdf/

90Disallow: /c/dam/m/ko_kr/offers/assets/images/

91Disallow: /c/dam/m/ko_kr/offers/assets/pdfs/

92Disallow: /c/dam/m/ko_kr/offers/images/

93Disallow: /c/dam/m/nl_be/offers/assets/images/

94Disallow: /c/dam/m/nl_be/offers/assets/pdfs/

95Disallow: /c/dam/m/nl_nl/offers/assets/images/

96Disallow: /c/dam/m/nl_nl/offers/assets/pdfs/

97Disallow: /c/dam/m/no_no/offers/assets/images/

98Disallow: /c/dam/m/no_no/offers/assets/pdfs/

99Disallow: /c/dam/m/pl_pl/offers/assets/images/

100Disallow: /c/dam/m/pl_pl/offers/assets/pdfs/

101Disallow: /c/dam/m/pt_br/offers/assets/images

102Disallow: /c/dam/m/pt_br/offers/assets/pdfs

103Disallow: /c/dam/m/pt_br/offers/img/

104Disallow: /c/dam/m/pt_br/offers/pdf/

105Disallow: /c/dam/m/pt_br/offers/video/

106Disallow: /c/dam/m/ru_ru/offers/assets/images/

107Disallow: /c/dam/m/ru_ru/offers/assets/pdfs/

108Disallow: /c/dam/m/sv_se/offers/assets/images/

109Disallow: /c/dam/m/sv_se/offers/assets/pdfs/

110Disallow: /c/dam/m/tr_tr/offers/assets/images/

111Disallow: /c/dam/m/tr_tr/offers/assets/pdfs/

112Disallow: /c/dam/m/zh_cn/offers/assets/images/

113Disallow: /c/dam/m/zh_cn/offers/assets/pdfs/

114Disallow: /c/dam/m/zh_tw/offers/assets/images/

115Disallow: /c/dam/m/zh_tw/offers/assets/pdfs/

116Disallow: /c/en/us/applicat/camp/

117Disallow: /c/en/us/applicat/non-camp/

118Disallow: /c/m/digital/

119Disallow: /cisco/web/UK/telepresence/pro/

120Disallow: /content/en/us/td/docs/Sanity

121Disallow: /c/en/us/products/products-sanity-test/

122Disallow: /c/en/us/solutions/solutions-sanity-test/

123Disallow: /en/US/swassets/sw274/sitewide_country_language_selector.html

124Disallow: /JP/support/public/mt/tac/100/1002347/techtip_conventions.shtml

125Disallow: /public/library/iosplanner/reldesignation.html

126Disallow: /web/AT/telepresence/pro/

127Disallow: /web/BE/fr/telepresence/pro/

128Disallow: /web/BE/nl/telepresence/pro/

129Disallow: /web/BE/telepresence/pro/

130Disallow: /web/CH/de/telepresence/pro/

131Disallow: /web/CH/fr/telepresence/pro/

132Disallow: /web/CZ/telepresence/pro/

133Disallow: /web/DE/telepresence/pro/

134Disallow: /web/DK/telepresence/pro/

135Disallow: /web/ES/telepresence/pro/

136Disallow: /web/FI/telepresence/pro/

137Disallow: /web/FR/telepresence/pro/

138Disallow: /web/go/

139Disallow: /web/help/index.html

140Disallow: /web/HU/telepresence/pro/

141Disallow: /web/IT/telepresence/pro/

142Disallow: /web/ME/ar/telepresence/pro/

143Disallow: /web/ME/telepresence/pro/

144Disallow: /web/NL/telepresence/pro/

145Disallow: /web/NO/telepresence/pro/

146Disallow: /web/offers/pdfs/

147Disallow: /web/PL/telepresence/pro/

148Disallow: /web/PO/telepresence/pro/

149Disallow: /web/PT/telepresence/pro/

150Disallow: /web/RO/telepresence/pro/

151Disallow: /web/RU/telepresence/pro/

152Disallow: /web/SE/telepresence/pro/

153Disallow: /web/TR/telepresence/pro/

154Disallow: /web/ZA/telepresence/pro/

155Disallow: /c/ru_ru/

156Disallow: /c/m/ru_ru/

157Disallow: /content/ru_ru/

158Disallow: /cisco/support/RU/

159Disallow: /cisco/web/support/RU/

160Disallow: /ru/

161Disallow: /c/dam/assets/global/pdfs/security/ru/

162Disallow: /c/dam/assets/global/RU/

163Disallow: /c/dam/global/ru_ru/

164Disallow: /c/dam/m/ru_ru/

165Disallow: /c/dam/marketing/ru_ru/

166Disallow: /web/offer/em/pdfs_innovators/RU/

167Disallow: /web/RU/

168Disallow: /global/RU/

169Disallow: /cisco/web/support/RU/

170Disallow: /c/en/us/support/web/tools/scm/help/stage/ru-RU/

171Disallow: /c/en/us/support/web/tools/scm/help/ru-RU/

172Disallow: /web/applicat

173Disallow: /c/en/us/applicat/

174Disallow: /cisco/web/

175Disallow: /site/us/en/test/

176Disallow: /assets/sol/sb/

177Disallow: /c/dam/r/newsroom/daily-news/

178Disallow: /en/US/docs/general/Test/

179Disallow: /c/login

180Disallow: /content/cdc/login

181Disallow: /web/solutions/

182Disallow: /c/en/us/support/index-beta.html

183Disallow: /resources

184Disallow: /abm/

185Disallow: /abx/

186#--------------------------------

187Sitemap: https://www.cisco.com/web/sitemap/www_cisco_com_ar_ae_index.xml

188Sitemap: https://www.cisco.com/web/sitemap/www_cisco_com_ar_eg_index.xml

189Sitemap: https://www.cisco.com/web/sitemap/www_cisco_com_ar_qa_index.xml

190Sitemap: https://www.cisco.com/web/sitemap/www_cisco_com_ar_sa_index.xml

191Sitemap: https://www.cisco.com/web/sitemap/www_cisco_com_az_az_index.xml

192Sitemap: https://www.cisco.com/web/sitemap/www_cisco_com_bg_bg_index.xml

193Sitemap: https://www.cisco.com/web/sitemap/www_cisco_com_bs_ba_index.xml

194Sitemap: https://www.cisco.com/web/sitemap/www_cisco_com_cs_cz_index.xml

195Sitemap: https://www.cisco.com/web/sitemap/www_cisco_com_da_dk_index.xml

196Sitemap: https://www.cisco.com/web/sitemap/www_cisco_com_de_at_index.xml

197Sitemap: https://www.cisco.com/web/sitemap/www_cisco_com_de_ch_index.xml

198Sitemap: https://www.cisco.com/web/sitemap/www_cisco_com_de_de_index.xml

199Sitemap: https://www.cisco.com/web/sitemap/www_cisco_com_el_gr_index.xml

200Sitemap: https://www.cisco.com/web/sitemap/www_cisco_com_en_ae_index.xml

201Sitemap: https://www.cisco.com/web/sitemap/www_cisco_com_en_au_index.xml

202Sitemap: https://www.cisco.com/web/sitemap/www_cisco_com_en_be_index.xml

203Sitemap: https://www.cisco.com/web/sitemap/www_cisco_com_en_ca_index.xml

204Sitemap: https://www.cisco.com/web/sitemap/www_cisco_com_en_cy_index.xml

205Sitemap: https://www.cisco.com/web/sitemap/www_cisco_com_en_dz_index.xml

206Sitemap: https://www.cisco.com/web/sitemap/www_cisco_com_en_eg_index.xml

207Sitemap: https://www.cisco.com/web/sitemap/www_cisco_com_en_hk_index.xml

208Sitemap: https://www.cisco.com/web/sitemap/www_cisco_com_en_id_index.xml

209Sitemap: https://www.cisco.com/web/sitemap/www_cisco_com_en_il_index.xml

210Sitemap: https://www.cisco.com/web/sitemap/www_cisco_com_en_in_index.xml

211Sitemap: https://www.cisco.com/web/sitemap/www_cisco_com_en_mt_index.xml

212Sitemap: https://www.cisco.com/web/sitemap/www_cisco_com_en_my_index.xml

213Sitemap: https://www.cisco.com/web/sitemap/www_cisco_com_en_ng_index.xml

214Sitemap: https://www.cisco.com/web/sitemap/www_cisco_com_en_ph_index.xml

215Sitemap: https://www.cisco.com/web/sitemap/www_cisco_com_en_qa_index.xml

216Sitemap: https://www.cisco.com/web/sitemap/www_cisco_com_en_sa_index.xml

217Sitemap: https://www.cisco.com/web/sitemap/www_cisco_com_en_sg_index.xml

218Sitemap: https://www.cisco.com/web/sitemap/www_cisco_com_en_th_index.xml

219Sitemap: https://www.cisco.com/web/sitemap/www_cisco_com_en_uk_index.xml

220Sitemap: https://www.cisco.com/web/sitemap/www_cisco_com_en_us_index.xml

221Sitemap: https://www.cisco.com/web/sitemap/www_cisco_com_en_za_index.xml

222Sitemap: https://www.cisco.com/web/sitemap/www_cisco_com_es_ar_index.xml

223Sitemap: https://www.cisco.com/web/sitemap/www_cisco_com_es_bo_index.xml

224Sitemap: https://www.cisco.com/web/sitemap/www_cisco_com_es_bz_index.xml

225Sitemap: https://www.cisco.com/web/sitemap/www_cisco_com_es_cl_index.xml

226Sitemap: https://www.cisco.com/web/sitemap/www_cisco_com_es_co_index.xml

227Sitemap: https://www.cisco.com/web/sitemap/www_cisco_com_es_cr_index.xml

228Sitemap: https://www.cisco.com/web/sitemap/www_cisco_com_es_ec_index.xml

229Sitemap: https://www.cisco.com/web/sitemap/www_cisco_com_es_es_index.xml

230Sitemap: https://www.cisco.com/web/sitemap/www_cisco_com_es_gt_index.xml

231Sitemap: https://www.cisco.com/web/sitemap/www_cisco_com_es_hn_index.xml

232Sitemap: https://www.cisco.com/web/sitemap/www_cisco_com_es_mx_index.xml

233Sitemap: https://www.cisco.com/web/sitemap/www_cisco_com_es_ni_index.xml

234Sitemap: https://www.cisco.com/web/sitemap/www_cisco_com_es_pa_index.xml

235Sitemap: https://www.cisco.com/web/sitemap/www_cisco_com_es_pe_index.xml

236Sitemap: https://www.cisco.com/web/sitemap/www_cisco_com_es_pr_index.xml

237Sitemap: https://www.cisco.com/web/sitemap/www_cisco_com_es_py_index.xml

238Sitemap: https://www.cisco.com/web/sitemap/www_cisco_com_es_sv_index.xml

239Sitemap: https://www.cisco.com/web/sitemap/www_cisco_com_es_uy_index.xml

240Sitemap: https://www.cisco.com/web/sitemap/www_cisco_com_es_ve_index.xml

241Sitemap: https://www.cisco.com/web/sitemap/www_cisco_com_et_ee_index.xml

242Sitemap: https://www.cisco.com/web/sitemap/www_cisco_com_fi_fi_index.xml

243Sitemap: https://www.cisco.com/web/sitemap/www_cisco_com_fr_be_index.xml

244Sitemap: https://www.cisco.com/web/sitemap/www_cisco_com_fr_ca_index.xml

245Sitemap: https://www.cisco.com/web/sitemap/www_cisco_com_fr_ch_index.xml

246Sitemap: https://www.cisco.com/web/sitemap/www_cisco_com_fr_dz_index.xml

247Sitemap: https://www.cisco.com/web/sitemap/www_cisco_com_fr_fr_index.xml

248Sitemap: https://www.cisco.com/web/sitemap/www_cisco_com_hr_hr_index.xml

249Sitemap: https://www.cisco.com/web/sitemap/www_cisco_com_hu_hu_index.xml

250Sitemap: https://www.cisco.com/web/sitemap/www_cisco_com_index.xml

251Sitemap: https://www.cisco.com/web/sitemap/www_cisco_com_it_it_index.xml

252Sitemap: https://www.cisco.com/web/sitemap/www_cisco_com_ja_jp_index.xml

253Sitemap: https://www.cisco.com/web/sitemap/www_cisco_com_ko_kr_index.xml

254Sitemap: https://www.cisco.com/web/sitemap/www_cisco_com_lt_lt_index.xml

255Sitemap: https://www.cisco.com/web/sitemap/www_cisco_com_lv_lv_index.xml

256Sitemap: https://www.cisco.com/web/sitemap/www_cisco_com_mk_mk_index.xml

257Sitemap: https://www.cisco.com/web/sitemap/www_cisco_com_nl_be_index.xml

258Sitemap: https://www.cisco.com/web/sitemap/www_cisco_com_nl_nl_index.xml

259Sitemap: https://www.cisco.com/web/sitemap/www_cisco_com_no_no_index.xml

260Sitemap: https://www.cisco.com/web/sitemap/www_cisco_com_pl_pl_index.xml

261Sitemap: https://www.cisco.com/web/sitemap/www_cisco_com_pt_br_index.xml

262Sitemap: https://www.cisco.com/web/sitemap/www_cisco_com_pt_pt_index.xml

263Sitemap: https://www.cisco.com/web/sitemap/www_cisco_com_ro_ro_index.xml

264Sitemap: https://www.cisco.com/web/sitemap/www_cisco_com_ru_kz_index.xml

265Sitemap: https://www.cisco.com/web/sitemap/www_cisco_com_ru_ua_index.xml

266Sitemap: https://www.cisco.com/web/sitemap/www_cisco_com_sk_sk_index.xml

267Sitemap: https://www.cisco.com/web/sitemap/www_cisco_com_sl_si_index.xml

268Sitemap: https://www.cisco.com/web/sitemap/www_cisco_com_sr_rs_index.xml

269Sitemap: https://www.cisco.com/web/sitemap/www_cisco_com_sv_se_index.xml

270Sitemap: https://www.cisco.com/web/sitemap/www_cisco_com_th_th_index.xml

271Sitemap: https://www.cisco.com/web/sitemap/www_cisco_com_tr_tr_index.xml

272Sitemap: https://www.cisco.com/web/sitemap/www_cisco_com_uk_ua_index.xml

273Sitemap: https://www.cisco.com/web/sitemap/www_cisco_com_vi_vn_index.xml

274Sitemap: https://www.cisco.com/web/sitemap/www_cisco_com_zh_cn_index.xml

275Sitemap: https://www.cisco.com/web/sitemap/www_cisco_com_zh_hk_index.xml

276Sitemap: https://www.cisco.com/web/sitemap/www_cisco_com_zh_tw_index.xml

277Sitemap: https://www.cisco.com/web/sitemap/rss/rss.xml

278Sitemap: https://www.cisco.com/web/sitemap/rss/rss1.xml

279Sitemap: https://www.cisco.com/web/sitemap/rss/rss2.xml

280Sitemap: https://www.cisco.com/web/sitemap/rss/rss3.xml

281Sitemap: https://www.cisco.com/web/sitemap/rss/rss4.xml

282Sitemap: https://www.cisco.com/web/sitemap/rss/rss5.xml

283Sitemap: https://www.cisco.com/web/sitemap/rss/rss6.xml

284Sitemap: https://www.cisco.com/web/sitemap/rss/rss7.xml

285Sitemap: https://www.cisco.com/web/sitemap/technical-documentation.xml

286#--------------------------------

287User-agent: Yandex

288Disallow: /c/ru_ru/

289Disallow: /c/m/ru_ru/

290Disallow: /content/ru_ru/

291Disallow: /cisco/support/RU/

292Disallow: /cisco/web/support/RU/

293Disallow: /ru/

294Disallow: /c/dam/assets/global/pdfs/security/ru/

295Disallow: /c/dam/assets/global/RU/

296Disallow: /c/dam/global/ru_ru/

297Disallow: /c/dam/m/ru_ru/

298Disallow: /c/dam/marketing/ru_ru/

299Disallow: /web/offer/em/pdfs_innovators/RU/

300Disallow: /web/RU/

301Disallow: /global/RU/

302Disallow: /cisco/web/support/RU/

303Disallow: /c/en/us/support/web/tools/scm/help/stage/ru-RU/

304Disallow: /c/en/us/support/web/tools/scm/help/ru-RU/

305

306

307#--------------------------------

308# All changes to robots.txt need to be approved by search-seo-and-site@cisco.com

309#

310

What do the bots actually get?

This page reads robots.txt, which states what cisco.com asks of crawlers. A firewall or CDN can still refuse a bot that robots.txt allows. The live check requests the page as each bot and shows the status it gets.

Run a live check on cisco.com Test a path in the robots.txt tester

History

19 October 2026: first checked.

Questions

Does cisco.com block GPTBot?

No. cisco.com’s robots.txt lets GPTBot crawl the site. GPTBot is OpenAI’s crawler for training data; blocking it doesn’t affect ChatGPT search, which uses OAI-SearchBot. Checked 9 November 2026.

Does cisco.com block ClaudeBot?

No. cisco.com’s robots.txt lets ClaudeBot crawl the site. ClaudeBot is Anthropic’s crawler for training data. Checked 9 November 2026.

Does cisco.com block Google-Extended?

No. cisco.com’s robots.txt lets Google-Extended crawl the site. Google-Extended is the token that controls whether Google may use pages for Gemini training and grounding; it doesn’t affect Google Search. Checked 9 November 2026.

Does cisco.com block PerplexityBot?

No. cisco.com’s robots.txt lets PerplexityBot crawl the site. PerplexityBot is the crawler that indexes pages for Perplexity’s answers. Checked 9 November 2026.

Can ChatGPT search find cisco.com?

Yes. cisco.com’s robots.txt lets OAI-SearchBot crawl the site. OAI-SearchBot is the crawler behind ChatGPT search; blocking it keeps the site out of ChatGPT’s search answers. Checked 9 November 2026.