Bot directory · Search engines
Search engines · Microsoft
Bingbot
Microsoft’s crawler for Bing. Its index also powers Copilot answers, DuckDuckGo results and Yahoo Search.
- Operator
- Microsoft
- Purpose
- Search index. The crawlers behind Google, Bing and the other search engines. Blocking one drops you out of its results.
- robots.txt token
Bingbot- User agent
Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko; compatible; bingbot/2.0; +http://www.bing.com/bingbot.htm) Chrome/131.0.0.0 Safari/537.36
Block Bingbot
Add a group naming it to /robots.txt. Compliant crawlers stop before requesting any page. A firewall rule on the user agent is the only way to enforce it against crawlers that ignore robots.txt.
User-agent: Bingbot
Disallow: /Allow Bingbot when everything else is blocked
A group naming the bot wins over User-agent: *, so an explicit allow lets it in while your catch-all rules stay in place.
User-agent: Bingbot
Allow: /
User-agent: *
Disallow: /Does Bingbot reach your site?
The checker applies your robots.txt the way Bingbot does, then requests your page with the user agent above, alongside 29 other bots.
Other search engines: Googlebot, DuckDuckBot, Applebot, PetalBot, YandexBot, Baiduspider.
Build a complete file with the robots.txt generator.