Robots.txt Rule Builder
Draft robots.txt Allow and Disallow rules and a sitemap line.
Runs in your browser. Nothing you enter is sent to our servers.
Result
Enter your details and select Run tool.
How to use this tool
Enter path prefixes that crawlers should not fetch and optional exceptions. The output targets all bots (User-agent: *). Add a full sitemap URL if available. Review your server’s existing rules before replacing them, and remember: robots.txt cannot keep sensitive data private or reliably prevent indexing.
Questions and answers
Does Disallow remove pages from Google?
No. Disallow stops crawling by cooperative bots but a URL may still be indexed from external links. Use appropriate noindex or access control instead.
Can different search engines behave differently?
Yes. Robots directives depend on bot behavior and supported syntax. Test important rules with search engine tools.
More tools
All tools-
Meta Title & Description Preview
Draft a page title and meta description and see an approximate search-result snippet.
-
SEO-Friendly URL Slug Generator
Turn a title into a concise lowercase URL slug, including Unicode letters.
-
UTM Campaign URL Builder
Add consistent utm_source, utm_medium and utm_campaign tags to a URL.