llms.txt
llms.txt is a plain-markdown index file served at a website's root path (/llms.txt) that tells AI systems where the pages that matter are, so an agent can route straight to the right documentation instead of crawling HTML. Proposed by Jeremy Howard in September 2024, the format specifies a required H1 title, a summary blockquote, and H2 sections containing lists of described links to markdown versions of key pages.
The proposal exists because context windows are too small to hold most websites, and converting HTML pages full of navigation, ads, and JavaScript into model-friendly text is lossy and imprecise. llms.txt gives a language model a curated entry point: brief background, guidance on how to interpret the site, and links to clean markdown sources. The same proposal recommends serving a markdown mirror of each important page at the original URL with .md appended, so the linked destinations are as legible as the index.
For an API producer, llms.txt is the declared agent-facing front door. When an agent lands on your domain, this file is the cheapest, most deterministic way to route it to your API reference, authentication docs, and error documentation. Quality matters as much as presence: an llms.txt that points agents at marketing pages, or at pages that render empty without JavaScript, actively misroutes the reader it was written for. Because the file is your own declaration of what agents should read, it gets treated as exactly that — the surface you chose to present.
Discry's llms.txt discovery check fetches /llms.txt on the API's docs domain and, when the file exists, lints it mechanically: headings present, at least one link, descriptions attached to links, and overall size within a sane budget. A present, well-formed file passes; a stub or oversized file earns partial credit; absence fails. The fetched URL ships in the profile as a receipt.
Checks: llms.txt · llms.txt quality · how each is scored
What does an AI agent make of your API?
Find out in about a minute — no signup.
Discry your API — free