Robots.txt Generator

Standard site Open public pages and close only typical private areas. WordPress Close WordPress admin and search duplicates without blocking public assets. Online store Keep products and categories open, close cart, checkout, account and filter duplicates. Multilingual site Keep language folders indexable and close language-switcher parameter duplicates. Blog / media Close internal search, author archives and comment reply duplicates. Corporate site Close service pages that should not appear in search, while keeping sales pages open. SaaS / web app Close app, dashboard, API and billing areas, keep marketing pages open. Close from indexing Block crawling for staging, test projects or sites that should not be public yet.