Home » SEO Tools » Robots Txt Generator
Build a simple robots.txt file for your site, control which paths crawlers should skip, and add your sitemap address.
Note: a robots.txt file is a request to crawlers, not a security wall. Also note that Google ignores crawl delay, so that line is only a hint for crawlers that respect it.
A robots.txt file tells search engine crawlers which parts of a site they may visit and which parts they should skip. It lives at the root of a site, for example at the address that ends with robots.txt, and crawlers read it before they crawl.
This tool builds a simple robots.txt file from your choices. Tick the paths you want to block, add your sitemap address, and copy or download the result to upload at your site root.
No. Robots.txt asks crawlers not to visit a path, but it is a request, not a guarantee, and it is not a way to hide private data.
Upload it at the root of your site, so it opens at the address that ends with robots.txt, not inside a subfolder.
In most cases yes, a login page gives searchers nothing useful, so blocking it keeps crawlers focused on your real content.
No. Everything runs in your browser on your own device, so your text never leaves your computer.