How to use Robots.txt Generator
Generate crawler rules and an optional sitemap directive. The operation runs locally on this device with a 200 KB input limit. Copy or download the output to keep it; no input text is sent to the tool server.
- Set User-agent.
- Set Disallow path.
- Set Sitemap URL (optional).
- Run the tool, review the result and its stated limitations, then copy or download if needed.
Example & practical use
Generate crawler rules and an optional sitemap directive. The example below demonstrates User-agent; Disallow path; Sitemap URL (optional).
Example input:
{
"agent": "*",
"path": "/admin",
"sitemap": "https://example.com/sitemap.xml"
}
Expected result (relevant fields):
"User-agent: *\nDisallow: /admin\nSitemap: https://example.com/sitemap.xml\n"Features & result limitations
The operation runs locally on this device with a 200 KB input limit. Copy or download the output to keep it; no input text is sent to the tool server.
Frequently asked questions
What inputs does Robots.txt Generator accept?
User-agent; Disallow path; Sitemap URL (optional)
How should I interpret the Robots.txt Generator result?
The operation runs locally on this device with a 200 KB input limit. Copy or download the output to keep it; no input text is sent to the tool server.
Where does Robots.txt Generator process my data?
Inputs are processed locally in your browser. Submitted text is not sent to external services. A successful-run event increments an aggregate tool counter without your input.