30 bots · copy and paste
robots.txt generator
Choose which AI crawlers, search engines and SEO bots may read your site and get a robots.txt to paste at https://your-site.com/robots.txt. Then check the result.
Start from your current robots.txt
We read your site’s /robots.txt once and set every choice below to match it. Nothing is stored.