AI .txt Toolkit

Generate your custom robots.txt + llms.txt kit free.
Control crawlers and guide AI models to your content — simple, fast.

Learn how it works →

Website URL*

Email*

I consent to the analysis of my website and processing of submitted data.

Please enter a valid email.

By default, all bots are blocked from crawling your website, except those you explicitly allow below. Certain repositories listed below will be restricted even for approved bots.

Baseline SEO - All robots are*:

AI Training Bots - All robots are*:

AI Browsing Bots:

OpenAI

OpenAI User

ChatGPT-Plugins

DuckAssist

Browse.AI

PhantomJS

Puppeteer

Playwright

SeleniumBot

Diffbot

SurferBot

Claude-User

Restricted Directories (comma-separated, optional)

Sitemap (optional)

Crunching the data…
Big sites need a bit more time.

Please review the results and change if needed before saving:

robots.txt llms.txt

Need schema and a full fix report too? The AI-Ready Kit adds JSON-LD schema snippets, llms-full.txt and a step-by-step install guide written for your site.

Get the AI-Ready Kit – €19

AI .txt Generator by Web2Agents.com
© 2026 All rights reserved | www.web2agents.com