What is llms.txt, and does your site need one?

llms.txt is a markdown file at the root of a website — yoursite.com/llms.txt — that describes the site for a language model: what it is, and which pages are worth reading. It was proposed in September 2024 by Jeremy Howard of Answer.AI, on the reasoning that a model reading a web page spends most of its attention on navigation, scripts and layout, when what it wanted was a short list of the pages that matter.

We publish one. This is what we learned building it, including the part where we found ours was pointing at a page it should not have been.

What goes in it

The format is deliberately small. A heading with the site's name, a one-paragraph summary in a blockquote, then sections of links, each followed by a sentence saying what is behind it. The shape of ours:

# Populr

> Populr is your AI CMO. It reads your site, runs SEO, AI-search
> visibility, Reddit and content daily, and sends you only what
> is worth approving.

## Guides
- [What is GEO? Getting named by AI assistants](https://www.trypopulr.in/guides/what-is-geo-generative-engine-optimization): ...

## Site
- [Home](https://www.trypopulr.in)
- [Early Access](https://www.trypopulr.in/early-access)

There is an optional companion, llms-full.txt, which inlines the full text of those pages so a model can take in everything in one request. It earns its place when your important content is long and stable, like documentation. For a marketing site it mostly repeats the sitemap.

What it cannot do

It is not a ranking factor. Google has said publicly that its search systems do not use the file, and no major AI assistant has committed to reading it. A model that already crawls your site will go on crawling your site. Anyone promising that llms.txt will get you cited by ChatGPT is selling you a text file.

Treat it the way you would a good README: cheap to write, occasionally read, and embarrassing only when it is wrong.

The one way it goes wrong

A hand-written llms.txt is accurate on the day it is written and drifts from then on. Pages get renamed, removed, or turn out weaker than you thought, and the file keeps describing the site as it used to be. Any model that reads it is then told something false by the one document whose only job is to be true.

So ours is not written by hand. It is generated from the same list of public routes that produces our sitemap and robots.txt, and from the same data that produces each guide's title and description. Add a guide and it appears in all three. Remove a route and it leaves all three.

That is how we caught our own mistake. We noticed our results page — the screen that shows a signed-in customer which of our recommendations moved their numbers — was listed in both the sitemap and llms.txt. To a logged-out visitor, which is what every crawler is, it rendered a back link, a loading message and an empty table. We had been pointing machines at our thinnest page and labelling it as proof. Taking it out of the route list took it out of both files in one change.

Should you have one?

• Yes, if it can be generated. If your sitemap is built from code, building llms.txt from the same source is an hour's work and cannot go stale.
• Probably not, if someone would have to remember to update it. A stale llms.txt is worse than none.
• Either way, it is not where AI visibility comes from. Models name products they have read about in places they trust: documentation, comparisons, forums, other people's articles. A file you publish about yourself is the weakest evidence there is.

If you want to know whether AI assistants mention you at all, ask them the question your customer would ask and see whose name comes back. Our guide to GEO covers how to do that systematically.

Common questions