What llms.txt is, and who actually reads it

A file written for language models

Most sites already carry two files that exist only for machines. robots.txt says which paths a crawler may fetch. sitemap.xml lists the pages you want found.

llms.txt is a newer third one. Jeremy Howard of Answer.AI proposed it in September 2024 as a short Markdown file at the root of a site, written for large language models rather than for search crawlers. It tells a model what the site is and which pages matter.

It is a proposal, not a standard. No body governs it, nobody is obliged to read it, and one of the biggest engines has said it does not.

What robots.txt and sitemaps leave out

A robots.txt file answers whether a crawler may fetch a path. A sitemap answers which pages exist. Neither says what the site is about, or which of its hundred pages answers a real question well.

A model reading your site has to work that out from the HTML, around menus, cookie banners and footers. llms.txt hands it the summary in plain text, if it looks.

The format

The proposal keeps the format small. A level 1 heading with the site's name comes first, then a blockquote with a one or two line summary, then optional paragraphs, then level 2 sections that list links with a short note after each:

# Acme Hotel Belgrade

> A hotel in Belgrade, Serbia, with rooms, a restaurant and conference facilities.

## Key pages

- [Home](https://acme-hotel.com/): Hotel overview and booking
- [Rooms](https://acme-hotel.com/rooms): Room types and rates
- [Restaurant](https://acme-hotel.com/restaurant): Dining options
- [Contact](https://acme-hotel.com/contact): Address and directions

## Optional

- [History](https://acme-hotel.com/history): The building and its past

A section headed "Optional" marks links a tool can skip when it is short of space. The proposal also suggests a Markdown copy of each page at the same address with .md added, so a tool can read the text without the HTML around it.

Who reads it

Most articles on llms.txt overstate this part.

Google does not use it. Google has said so in public, and the file plays no part in Google Search or in AI Overviews. A guide that tells you Googlebot reads your llms.txt first is wrong.

For OpenAI, Anthropic, Perplexity and the rest, I have found no public commitment either way. Treat any claim that a named assistant reads the file with suspicion until it points to that company's own documentation.

Any AI tool that can fetch a URL can read the file when a person asks it about your site, and many developer documentation sites publish one. That is the honest size of the audience today. Publish it because it costs close to nothing, and do not expect a channel with guaranteed readers.

What it helps with

An agent that arrives without it starts at the home page, follows some links and forms its own picture. It can miss a good page buried three levels down in a category archive, or decide the site is about whatever it read first. A summary and a short list of your best pages leave less to guesswork, for the agents that open the file.

The weak point of a hand-written llms.txt is upkeep. You add a page, change a phone number or drop a service, and the file keeps describing the old site until someone remembers it.

llms.txt in AI Boost for Joomla

AI Boost for Joomla builds llms.txt on each request once you switch it on, so it describes the site as it is at that moment. The file opens with your organisation's name, or the site name when that is empty, and a description taken from Site Description for AI or, failing that, from your business description. After those come the base URL, your menu and home pages, and your most recent articles, 10 unless you choose another number. It also lists your business type, country, phone, email and social profiles, unless you switch that off. By default it lists the same pages your XML sitemap does.

It can also publish /llms-full.txt, which groups every page by kind, up to 500 articles by default, with a Markdown copy of each article, category and tag page once AI Boost has fetched it. A separate switch, Serve pages as Markdown for AI agents, answers the page address plus .md for articles, categories and tags, with a noindex header so search engines keep the copies out of their index.

What it cannot do is make anyone read the file. Google will ignore it, and whether another assistant opens it is that assistant's choice.