This site published an llms.txt months ago and then shipped sixty-two new tools. The file kept confidently telling every AI assistant that came to read it that Quill Tools had seventy-six tools across eight categories. It was wrong by sixty-two tools and one entire category, and nothing anywhere surfaced a warning — a static file has no way to know the thing it describes has changed.
That is the most useful thing we can tell you about llms.txt, so it goes first. The format is easy; keeping it true is the actual work.
What is llms.txt?
llms.txt is a Markdown file served at the root of a domain that gives language models a short, curated description of what a site is and which pages matter. It is the AI-era counterpart to robots.txt and sitemap.xml: robots.txt controls access, sitemap.xml lists URLs, and llms.txt supplies meaning.
The reasoning behind it is practical. A model arriving at a modern site has to infer what the site is from navigation chrome, marketing copy and whatever markup survived the conversion to text. llms.txt lets you simply say it.
The format
Deliberately minimal — it is Markdown, and only the first element is strictly required:
- One H1 with the project or brand name.
- A blockquote immediately after it, summarising the site in a few sentences.
- H2 sections containing link lists, each entry formatted as
- [Title](https://absolute.url): one-line description. - An optional H2 named "Optional" for links a model can safely skip.
A minimal but complete example:
# Example Docs
> Example is a TypeScript library for parsing invoices.
> It runs in Node 18+ and in browsers, and has no runtime dependencies.
## Docs
- [Quickstart](https://example.com/start): install and parse a first invoice.
- [API reference](https://example.com/api): every exported function.
## Optional
- [Changelog](https://example.com/changelog)The blockquote is the whole file
If you take one structural point from the spec, take this one: the blockquote is what gets lifted verbatim as the answer to "what is this?". Skipping it, or filling it with positioning language rather than facts, wastes the file.
Write it as claims. "A free suite of 138 browser-based utilities; every tool runs client-side, so files are never uploaded" is quotable. "The web's most-loved productivity destination" is not — there is nothing in it a model can assert on your behalf.
llms.txt vs llms-full.txt
llms.txt is the concise index — usually under 5 KB — meant to be read in full and understood quickly. llms-full.txt is the exhaustive companion containing every page or item with a description. Keep the short file curated and let the long file carry the completeness.
The split exists because the two have incompatible goals. Comprehensiveness makes a file long; comprehension wants it short. Trying to satisfy both in one file produces something that is neither.
llms.txt vs robots.txt vs sitemap.xml
- robots.txt — permission. Which agents may fetch what. Enforced by convention and widely respected. Build and test one with the robots.txt Generator.
- sitemap.xml — inventory. Every URL worth crawling, with metadata. Generate one with the XML Sitemap Generator.
- llms.txt — meaning. What the site is, and which parts matter most.
They do not overlap and none substitutes for another. llms.txt in particular grants no permissions: if robots.txt blocks a crawler, listing pages in llms.txt changes nothing.
The stale-file problem
Now the part that cost us something.
Every ordinary staleness signal is missing here. A stale page gets flagged by an audit, gets fewer clicks, or shows a visibly old date. A stale llms.txt produces none of that. It is never rendered, so nobody reads it. It returns HTTP 200, so no monitor complains. And it is trusted — its whole purpose is to be believed, which means its errors propagate straight into answers stated as fact.
Ours claimed 76 tools while the site had 138, described eight categories when there were nine, and cited 31 blog posts against an actual 33. Every assistant that read it received precise, confident, wrong numbers.
The fix: generate it, do not write it
A hand-maintained llms.txt is a second copy of facts that already live somewhere authoritative. Second copies drift — not because anyone is careless, but because nothing connects the copy to the change.
On this site the file is now produced at build time from the same tool registry that generates the sitemap, the category pages and the navigation. The count in the blockquote is Object.keys(TOOLS).length. Adding a tool updates llms.txt, necessarily, because there is no longer a separate number to forget.
If your stack cannot generate it, the fallback is a checklist item on the same release step that updates your sitemap. Weaker, but far better than trusting memory.
Does llms.txt actually do anything?
Honest answer: it is a convention, not a ratified standard, and no engine guarantees it is read. Adoption is real but uneven, and anyone promising you a measurable ranking effect from it is guessing.
The case for publishing one is the asymmetry rather than the certainty. The cost is a file of a few kilobytes. The benefit, when a model does consult it, is that your description of your site becomes the one it uses — instead of whatever it inferred from your homepage. That trade is worth making. It only turns negative if the file is wrong, which is why the generation point above matters more than the format details.
A checklist
- Serve it at
/llms.txtastext/plain, returning 200. - One H1, then a blockquote written as factual claims.
- Absolute
https://URLs only — relative paths are meaningless to a fetcher that has your file and no page context. - Keep it under about 5 KB; push exhaustive detail to
llms-full.txt. - Verify every link returns 200. Dead links in a trusted file are worse than absent ones.
- Generate it from your source of truth. If you cannot, put it on a release checklist.
Frequently Asked Questions
Where exactly does llms.txt go?
The domain root — https://example.com/llms.txt. Not a subdirectory, and not behind a redirect chain. Serve it as text/plain; some fetchers will not parse it if it arrives as HTML.
Do I need llms.txt if I already have a good sitemap?
They answer different questions. A sitemap tells a crawler which URLs exist; it says nothing about what any of them mean or which matter most. llms.txt is where that judgement goes, and judgement is exactly what a model cannot derive from a URL list.
Can llms.txt stop AI companies training on my content?
No. It confers no permissions whatsoever. Access control is robots.txt — and, if you need it to be enforceable rather than advisory, server-side blocking. Treat llms.txt purely as description.
Ours is at /llms.txt, with the exhaustive version at /llms-full.txt. Both are generated from the tool registry, for the reason described above.