What is llms.txt?

llms.txt is a proposed Markdown file at a website's root that gives large language models a short guide to the site's most useful pages.

Some people describe llms.txt as "robots.txt for AI". It is an easy way to remember the name, but it is not accurate. llms.txt does not block anything or allow anything. It is more like a menu card that tells an AI system which dishes your kitchen is known for.

llms.txt is a proposed plain-text file, written in Markdown, that sits at the root of a website (for example, yoursite.com/llms.txt). It gives large language models a short summary of the site and a curated list of its most useful pages. Jeremy Howard of Answer.AI proposed it in September 2024, mainly to help AI tools read websites and documentation more easily.

How it differs from other files

Robots.txt tells crawlers which parts of a site they may or may not crawl.
An XML sitemap lists the URLs you want search engines to discover.
llms.txt offers a short, human-readable guide to your best content, with a line of context for each link.

What the file contains

The proposal suggests a simple structure. It starts with a heading for the site or project name, then a short summary. After that come sections with lists of links, each with a brief note on what the page covers. An optional section can list pages that are less important. Some sites also publish a larger companion file with the full text of their key pages.

Where it stands in 2026

This is the part that needs honesty. llms.txt is still a community proposal, not an official web standard. Google has said its Search does not use llms.txt, and there is no clear public evidence that the file improves how often a site is cited in AI answers. Adoption has been strongest among software companies and developer documentation sites, where AI coding tools and assistants may read the file when a user points them to it.

That said, the file is small and cheap to create. For some businesses, it is a reasonable low-effort experiment. For others, it is a distraction from more useful work.

An example

Suppose a software company sells an invoicing API. Developers often ask AI assistants for help integrating it. The company creates an llms.txt that links to its getting-started guide, API reference, authentication page and pricing page, each with a one-line description. When a developer gives this file to an AI tool, the tool has a clean map of the right pages instead of guessing.

A neighborhood bakery, on the other hand, gains far more from a complete Google Business Profile than from an llms.txt file.

If you decide to add one

Finish the basics first: crawlable pages, a sitemap and solid technical SEO.
List only your most useful, current pages.
Write short, factual descriptions for each link.
Update the file when important pages change.
Do not expect ranking or citation gains, and do not pay anyone who promises them.

Simple takeaway

llms.txt is an interesting idea, not a magic file. It can help AI tools read some sites more easily, especially documentation. Treat it as an optional extra within generative engine optimization, never as your main plan.