indexing

Robots.txt Disallow Examples and Common Mistakes

Practical robots.txt Disallow examples for folders, pages, and file types — plus the mistakes that can hide your whole site from search results.

7 Nov 2026 9 min read