# llms.txt Explained and How to Create One

https://citaeotion.ai/llms-txt-explained/

llms.txt is a proposed plain-text file that lists your most important pages in Markdown so AI models can find clean content fast, but adoption stays limited and no major AI company has confirmed its crawlers read it.
llms.txt is a plain-text file you place at the root of your website that points AI language models to your key pages in Markdown format. It sits at yourdomain.com/llms.txt and works as a hand-picked map of your best content for tools like ChatGPT, Claude, and Perplexity.
Quick answer: llms.txt was proposed in September 2024 by Jeremy Howard of Answer.AI. It lives at /llms.txt and uses Markdown to list your highest-value URLs with short descriptions. It is a proposal, not an official standard, and as of 2026 OpenAI, Anthropic, and Google have not publicly confirmed that their crawlers consume it. It does not allow or block any bot; that job belongs to robots.txt. Treat llms.txt as an optional content map, then check your server logs to see which AI crawlers actually hit you.
What llms.txt is
llms.txt is a single text file that lives at the root of your domain, the same place robots.txt sits. The idea is simple. Large language models work from a limited context window, and most web pages bury the useful text under navigation, ads, and scripts. An llms.txt file gives models a short, clean list of the pages that matter most, written in Markdown so it is easy to parse.
The proposal also describes an optional second file, llms-full.txt, which holds the full Markdown text of your documentation in one place. A docs site might ship both: llms.txt as the index, llms-full.txt as the complete body.
What goes in an llms.txt file
The format is plain Markdown with a fixed shape. You start with an H1 title, add an optional blockquote summary, then group links under H2 headings. Here is the structure:
# Your Site Name

> One sentence that says what this site is and who it serves.

Optional paragraph of plain context in Markdown.

## Docs
- [Getting started](https://example.com/start): how to install and configure
- [Pricing](https://example.com/pricing): plans and current rates

## Optional
- [Changelog](https://example.com/changelog): release history

Links under an H2 called "Optional" signal lower priority, so a model short on context can skip them. Every link should be an absolute URL with a short note after the colon.
How to create one in five steps
1. List the 10 to 30 URLs you most want an AI model to read: product pages, docs, pricing, key guides.
2. Write the file in Markdown using the H1, blockquote, and H2 structure above. Keep descriptions to one line each.
3. Save it as a UTF-8 text file named exactly llms.txt.
4. Upload it to your web root so it resolves at https://yourdomain.com/llms.txt. On WordPress you can drop it in the root directory next to robots.txt, or serve it with a small plugin or rule.
5. Load the URL in a browser to confirm it returns plain text with a 200 status. If you also want a full-text version, publish llms-full.txt the same way.
llms.txt vs robots.txt vs sitemap.xml
These three files do different jobs, and people mix them up constantly.
robots.txt tells crawlers what they may and may not fetch. It is the file that actually allows or blocks GPTBot, ClaudeBot, and Google-Extended.
sitemap.xml lists every indexable URL for search engines so nothing gets missed.
llms.txt does neither of those. It does not grant or deny access and it does not guarantee indexing. It only suggests which pages an AI model should prefer if it chooses to read the file at all.
Does llms.txt actually work in 2026?
Honest answer: the evidence is thin. llms.txt remains a community proposal with no central standards body behind it. Adoption among publishers is uneven, and the major AI companies have not published documentation saying their crawlers request or parse llms.txt. Some site owners report no change in how their content is used after adding one.
That does not make the file worthless. It costs almost nothing to publish, it can help internal tools and smaller agents that do read it, and it forces you to decide which pages represent you best. Just do not expect it to change what ChatGPT or Claude cites, and do not treat it as a control mechanism.
Measure which AI crawlers actually reach your pages
llms.txt is a guess about what AI bots want. Your server logs are proof of what they take. Every time GPTBot, ClaudeBot, PerplexityBot, Google-Extended, CCBot, Amazonbot, or Bytespider requests a page, it leaves a real record: user agent, URL, timestamp, and HTTP status. That is data you can act on.
citAEOtion reads that real server traffic on WordPress and classifies every AI and search crawler by actual hits. You see which bots hit which pages, how often, in which category, and with which status code. Prompt-based visibility tools guess by sampling model answers; citAEOtion measures the requests themselves. Read how citAEOtion reads your server traffic to see the method, or compare plans on the citAEOtion pricing page, starting at $34.99/mo.
The community proposal and its format live at the official source: llmstxt.org.
Frequently Asked Questions
Where do I put the llms.txt file?
At the root of your domain, so it resolves at https://yourdomain.com/llms.txt. It does not work in a subfolder.
Does llms.txt replace robots.txt?
No. robots.txt controls crawler access. llms.txt only suggests preferred content. You still need robots.txt to allow or block AI bots.
Do OpenAI, Anthropic, or Google read llms.txt?
As of 2026 none of them has confirmed that their crawlers request or parse llms.txt. Treat any claim that they do as unverified.
Is llms.txt an official standard?
No. It is a proposal introduced in September 2024. There is no standards organization maintaining it and no enforcement behind it.
Will llms.txt improve my Google rankings?
No. It has no known effect on search rankings. Use robots.txt, sitemaps, and content quality for search, and use server-log tracking to see AI crawler behavior.
See which AI bots hit your site in a live demo

---
Site: citAEOtion - https://citaeotion.ai/
LLM index: https://citaeotion.ai/llms.txt | Full text: https://citaeotion.ai/llms-full.txt