[ Free tool · AI search ]
llms.txt generator &
AI crawler check.
Give AI assistants a clean summary of your site with an llms.txt file — and make sure your robots.txt isn't quietly blocking the crawlers behind ChatGPT, Claude, Perplexity and Gemini.
- Free, no sign-up
- llms.txt in seconds
- 14 AI crawlers checked
One page per line. Put your most important pages first.
Secondary content AI can skip when it's short on space — blog, tools, policies.
Your llms.txt
Where it goes
Upload the file to the root of your site so it loads at https://yoursite.com/llms.txt
AI crawler access
| Crawler | Status |
|---|
The basics
What llms.txt and AI crawlers do.
- 01
llms.txt
A plain-text Markdown file at the root of your site that summarizes who you are and links to your most important pages — a cheat sheet for AI assistants.
- 02
robots.txt
Tells crawlers what they may read. Many sites block AI crawlers by accident through security plugins, CDN settings or copied rules.
- 03
Why it matters
If an assistant can't read your site, it can't cite you. Blocking search and user-request crawlers can keep you out of AI answers entirely.
FAQ
llms.txt & AI crawler questions
Is llms.txt an official standard?
It's a proposed convention, not a formal standard, and AI companies don't all say they use it. It takes minutes to add, does no harm, and gives assistants a clean summary of your business — so we recommend it as a low-cost step alongside the fundamentals.
Should I block AI crawlers?
It depends on your goals. Training crawlers (like GPTBot or Google-Extended) and search or user-request crawlers (like OAI-SearchBot, ChatGPT-User or PerplexityBot) do different jobs. If you want to appear in AI answers, keep the search and user-request crawlers allowed at a minimum.
Does blocking Google-Extended affect my Google rankings?
No. Google-Extended controls whether your content is used for Gemini models; it doesn't affect Googlebot or your search rankings. AI Overviews are driven by Googlebot.
Where do I put the llms.txt file?
At the root of your domain, so it loads at yoursite.com/llms.txt — the same place as robots.txt. On WordPress, upload it via your host's file manager or SFTP, or use an SEO plugin that supports it.
From the team behind this tool
Getting crawled is step one. Getting cited is the goal.
Our AEO and AIO work makes your brand the source AI assistants understand, trust and cite.