⚡FreeToolJetFTJ

Robots.txt Generator

Generate robots.txt file for your website. Create allow/disallow rules, set crawl delays, and block AI bots with presets.

Sitemap URL

User-agent 1

Rules

Crawl-delay

robots.txt Preview

# robots.txt generated by FreeToolJet
User-agent: *

# Sitemap: https://example.com/sitemap.xml

Related Articles

SEOMay 28, 2026

Robots.txt Generator: Block AI Bots and Control Crawler Access

Generate a robots.txt file for your website. Includes presets to block AI training bots like GPTBot, Claude, and more.

Related Tools

Meta Tag Generator

Generate HTML meta tags for SEO and social media. Includes Open Graph, Twitter Card, and search engine meta tags with live preview.

Convert titles and phrases into clean URL slugs. Create lowercase hyphenated slugs for blogs, docs, and product pages.

JSON Formatter & Validator

Format, validate, and beautify JSON data instantly. Paste messy JSON and get clean, indented output.

Base64 Encoder / Decoder

Encode text to Base64 or decode Base64 back to text. Fast, free, and runs entirely in your browser.