Loading...
Loading...
Loading tool...
Create and validate robots.txt files to control search engine crawling.
User-agent: * Allow: / Disallow: /admin/ Disallow: /private/ Sitemap: https://example.com/sitemap.xml
Pro includes sitemap validation, crawl analytics, and automatic robots.txt testing against Google's rules.
robots.txt is a text file that tells search engine crawlers which pages on your site they can and cannot access. It lives at the root of your domain (example.com/robots.txt).
robots.txt can prevent crawling, but it doesn't guarantee a page won't appear in search results. Use a noindex meta tag instead to prevent indexing.
Robots.txt Generator is part of BriskTool's collection of free online tools. All processing runs entirely in your browser for maximum privacy and speed.