What is llms.txt?

llms.txt is a plain-text Markdown file a website publishes at https://example.com/llms.txt. It tells large language models and AI agents, in a form they can read directly, what the site is, who it is for, and which pages matter. Think of it as robots.txt for understanding rather than access: robots.txt says where a crawler may go; llms.txt says what is worth reading and why.

The convention was proposed in September 2024 by Jeremy Howard (llmstxt.org) and has since been adopted by documentation sites, developer tools, SaaS products and publishers. Agent Web Map tracks who publishes one and ranks them.

What goes in an llms.txt file

The format is deliberately small:

  1. An H1 title with the site or project name. This is the only required line.
  2. A blockquote summary: one short paragraph with the key facts an agent needs to decide whether the site is relevant.
  3. Any free-text sections with more detail: what the site does, conventions, how to use the docs.
  4. H2 sections of links, each link as - [name](url): short description. A section named Optional marks links that can be skipped when context is short.
# Acme Payments

> Acme is a payments API for marketplaces. The docs cover onboarding sellers, splitting payouts, and handling disputes; the REST reference is the source of truth.

## Docs

- [Quickstart](https://acme.example/docs/quickstart): First charge in ten minutes
- [API reference](https://acme.example/docs/api): Every endpoint, with examples

## Optional

- [Changelog](https://acme.example/changelog): What changed, by version

llms.txt vs llms-full.txt

llms.txt is the index: short, curated, links out. llms-full.txt is the whole thing: the full text of the documentation in one Markdown file, for agents that want everything in a single fetch. Publishing both is best; llms.txt alone is enough to be listed here, and llms-full.txt adds to a site's quality score.

llms.txt vs robots.txt vs sitemap.xml

robots.txt controls which URLs crawlers may fetch. sitemap.xml lists URLs for search engines, without saying what they are. llms.txt is written for a reader: it explains the site in prose and points at the pages that matter, in Markdown a model can consume as-is. They complement each other; none replaces another.

Does it help with SEO or AI search?

Honest answer: search engines have not announced ranking signals for llms.txt. What it does today is make a site easier for agents and AI tools to use correctly, which is the point. Directories like this one also index and link to published files, so an llms.txt is a way of being found by the systems that read them.

How to publish one

  1. Write the file (or draft it from your site in one click).
  2. Serve it at /llms.txt with Content-Type: text/plain or text/markdown, UTF-8.
  3. Check it for the common mistakes: an HTML page returned instead of the file, a redirect to another domain, no H1, no links.
  4. Submit it here, then verify ownership to rank first.

More: llms.txt examples · llms.txt best practices · checker · generator