llms.txt: what it is, how to write one, and an honest view of whether it helps

Few topics in AI visibility are as confused as llms.txt. Vendors sell it as the new robots.txt. Sceptics call it pointless. Both positions skip the evidence, so this guide reads the proposal itself, compares it with what the platforms say, and tells you what to do. It sits in the sequence described in how to get cited by ChatGPT, after crawler access and good pages.

What is llms.txt?

The llms.txt proposal suggests adding a /llms.txt markdown file to websites to provide content that is friendly to large language models. Its reasoning is practical. Agents now use websites constantly: a coding agent fetches a library’s documentation to get an API call right, and a chat assistant with search reads pages to answer questions about a product. But web pages are built for people, wrapped in navigation, advertising and scripts, so converting them back into clean text is difficult and imprecise. Context windows are larger than they were but still too small for most websites in their entirety. Agents are best served by concise, expert-level information gathered in a single, accessible location.

The file is meant to offer brief background information, guidance and links to detailed markdown files. It can be placed at the site root or at any path within it, in which case it covers the pages under that path, and where more than one file applies, agents should use the most specific one.

Who actually uses it?

The current version of the proposal reports that thousands of sites publish an llms.txt file, that documentation platforms generate one automatically, and that the AI labs publish llms.txt files for their own developer documentation. You can see this yourself: Perplexity’s crawler documentation opens with a note that you can fetch the complete documentation index at /llms.txt, so that agents can discover all available pages.

That is adoption by publishers and documentation sites, which is real and useful for developer docs. It is a different thing from evidence that an assistant consults a business’s llms.txt when deciding who to recommend. I am not aware of any major assistant that has published such a commitment, so I will not claim that one exists.

What does Google say?

Google’s documentation on AI features and your website is explicit. It says you do not need to create new machine-readable files, AI text files or markup to appear in AI Overviews and AI Mode, and that there is also no special schema.org structured data you need to add. What it asks for instead is the SEO fundamentals: crawling allowed in robots.txt and by any CDN or hosting infrastructure, content findable through internal links, a great page experience, important content in text form and structured data that matches the visible page. So if your goal is to appear in Google’s AI features, llms.txt is not the thing standing between you and success.

What goes in an llms.txt file?

The proposal specifies a markdown file with these parts, in this order:

1. An optional byte-order mark.
2. An H1 with the name of the project or site. This is the only required section.
3. A blockquote with a short summary of the project, containing the key information necessary for understanding the rest of the file.
4. Zero or more markdown sections of any type except headings, with more detailed information about the project and how to interpret the files.
5. Zero or more sections delimited by H2 headings, each containing a file list of links to further detail, with an optional short description after each link.

The format deliberately uses markdown rather than XML or JSON because the files are expected to be read by language models and agents, while still following a precise structure that classic programmatic tools can parse. The proposal also suggests that pages with information agents might need provide a clean markdown version at the same URL with .md appended, and recommends standard link relations to help clients find them: rel=”alternate” type=”text/markdown” pointing to the markdown version of a page, and rel=”describedby” pointing to the llms.txt file that covers it, provided either as HTML link elements or as an HTTP Link header.

What does a good file look like?

A personal brand or consultancy

This is a shortened version of the file on this site, which is generated from the same content as the pages themselves.

# Victor Olanrewaju

> Victor Olanrewaju is a copywriter, SEO and AEO/GEO strategist
> and crypto and financial markets analyst based in Kigali,
> Rwanda. He founded Vatam, a consultancy that helps businesses get
> cited and recommended by AI assistants and ranked on Google.

## Key facts
- Based in: Kigali, Rwanda (remote)
- Founder of: Vatam
- Bylines: CCN, BeInCrypto, AMBCrypto, Benzinga, Coin Edition

## Pages
- [Results](/work/): Rankings, AI citations and reach with sources
- [Services](/services/): Copywriting, SEO, AEO and GEO
- [Pricing](/pricing/): Published prices in US dollars

## Articles
- [How to get cited by ChatGPT](/how-to-get-cited-by-chatgpt/): A sourced guide

An e-commerce store (illustrative)

# Example Eyewear

> Example Eyewear sells blue light, photochromic and photochromic
> blue-cut glasses online to buyers across Nigeria. Ready lenses
> are delivered in 24 to 48 hours in Lagos and 3 to 7 working days
> elsewhere. Frames start from 10,000 naira.

## Buying guides
- [Where to buy blue light glasses in Lagos](https://example.com/guides/where-to-buy/): prices, delivery, store address
- [Photochromic vs blue cut lenses](https://example.com/guides/photochromic-vs-blue-cut/): which to choose

## Shop
- [Anti blue light glasses](https://example.com/product-category/anti-blue-light-glasses/)
- [Photochromic glasses](https://example.com/product-category/photochromic-glasses/)

## Policies
- [Delivery and returns](https://example.com/delivery/): windows by state
- [Upload a prescription](https://example.com/upload-prescription/)

A local business (illustrative)

# Example Hotel Kigali

> Example Hotel is a boutique hotel in Kigali, Rwanda, three
> minutes from the airport, with airport pickup, breakfast and
> free Wi-Fi included in every room rate.

## Stay
- [Rooms and rates](https://example.com/rooms/): room types and nightly prices
- [Airport pickup](https://example.com/airport-pickup/): how to book and what it costs
- [Getting around Kigali](https://example.com/getting-around/)

## Contact
- [Contact and directions](https://example.com/contact/): address, phone, WhatsApp

Notice what all three contain: facts, not slogans. An assistant reading the file should be able to answer “who is this, what do they do, where, and where do I read more” without opening anything else.

How do you write one well?

1. Write the summary first. If you cannot say who you are and what you do in a few plain sentences, no file will fix that.
2. Put the facts people ask about in the file: location, delivery, prices or price floors, opening times, credentials, contact methods.
3. Link to your most important pages only, each with a one-line description. Curation is the point.
4. Use plain, specific language. Avoid marketing superlatives that cannot be verified.
5. Match your site exactly. A fact in llms.txt that contradicts your pages undermines trust in both.
6. Keep it short. It is a guide, not a copy of your site. Some sites publish a longer companion file with fuller content, but that is a convention beyond the core proposal.

How do you publish it?

1. Create the file as plain text with the .txt extension, in UTF-8.
2. Serve it at the root of your domain so it loads at yourdomain.com/llms.txt, as text, with a 200 response. Avoid redirects to a trailing-slash URL.
3. Do not block it in robots.txt.
4. Test it with a simple request:

   curl -i https://example.com/llms.txt

5. Reference it in your documentation or footer if you wish, and mention it in your robots.txt as a comment for humans.

On WordPress you can upload a file to the site root, use a plugin, or generate it from your content, which is what my theme does. Some hosts also offer a toggle that creates one, so check yours for a conflict before adding a second source: a physical file and a generated route can fight, and the physical file will win.

How is llms.txt different from robots.txt, a sitemap and schema?

Will llms.txt improve my AI visibility?

Nobody outside the AI companies can say, and I would not sell it as a result. What it reliably does is two useful things. First, it makes you write down your facts in one place, which is the same discipline that makes answer-first pages and consistent structured data work. Second, it costs almost nothing. For documentation sites and developer products, where agents fetch pages programmatically, the value is more direct and well established. For a typical business, treat it as a tidy summary that may help some tools and harms none.

How can you tell if anyone reads it?

You cannot attribute citations to the file, but you can see requests for it. Look in your server or CDN logs for requests to /llms.txt and note the user agents. Fetches by AI-related agents tell you the file is being discovered, nothing more. Keep expectations modest and your own outcome measures where they belong: the monthly panel in how to check what ChatGPT says about your business, AI referral traffic and enquiries.

What mistakes should you avoid?

• Treating it as a magic switch. It is a summary.
• Letting it go stale. A file that contradicts your site is worse than none.
• Stuffing it with keywords or promotional language.
• Listing every URL. That is what a sitemap is for.
• Two competing sources (a plugin and a physical file).
• Skipping the foundations. No file replaces crawlable, indexed, specific pages.

A checklist

1. Is the H1 your exact business name?
2. Does the blockquote state who you are, what you do and where in a few plain sentences?
3. Are your key facts present and identical to your site?
4. Are the linked pages your most important, with one-line descriptions?
5. Does the file load as plain text at /llms.txt with a 200 response?
6. Is there a process to update it when your business changes?

For the bigger picture, read SEO, AEO and GEO explained, and for a specialist to implement the whole sequence, see my GEO strategist page.

Should you publish markdown versions of your pages?

The proposal goes beyond the single file. It suggests that pages with information agents might need provide a clean markdown version at the same URL with .md appended, so a tool can fetch the content without navigation, scripts and layout. It also recommends standard link relations to help clients find those versions. Whether this is worth doing depends on your site:

If you do publish markdown versions, generate them automatically from your content so they cannot drift from the HTML, and make sure each links back to the canonical page.

How does llms.txt differ by site type?

Documentation and SaaS

The proposal started with this use case. The file lists the key documentation pages by section with one-line descriptions, and the optional markdown versions give agents clean text. The benefit is direct, because developers’ tools and coding assistants fetch documentation constantly.

Publishers

List your most important evergreen guides and your author or about page, not every article. Include editorial standards and corrections policy, since trust signals matter. Consider describing your topical focus in the blockquote.

Service businesses and consultants

State what you do, where, for whom and how to contact you. List services, case studies and pricing pages if you publish prices, and your key articles. Include credentials that can be verified.

Local businesses

Address, hours, services, prices or price floors, how to book, and a short description of what makes the business what it is. Keep these identical to your Google Business Profile.

Who should own the file?

An llms.txt file is a small piece of documentation that goes stale fast. Give it an owner, put it on your update checklist for any change to services, prices, locations or key pages, and review it quarterly alongside your robots.txt and structured data. If you generate it from your content, as my theme does, most of the maintenance disappears, because the file changes when the content changes. If you maintain it by hand, treat a stale file as a bug.

A worked example: building the file for a consultancy, step by step

Here is how I built the llms.txt for this site, as a template you can follow.

1. Name: the H1 is the exact name used everywhere, “Victor Olanrewaju”.
2. Summary blockquote: who he is, where he is based, what he does and what he founded, in four sentences, using the same wording as the “in short” block on the home page.
3. Key facts section: location, roles, the consultancy, bylines, certifications and contact details, each on one line.
4. Pages section: the results, services, pricing, specialist and about pages, each with a one-line description taken from its meta description.
5. Services section: each service with its one-line promise.
6. Selected results section: the main case studies with the headline figure and the link.
7. Articles section: the latest articles with their excerpts.
8. Pricing section: the published prices, in the currency published.
9. Generation: the file is built from the same content that renders the pages, so it cannot drift.

Does caching or a CDN affect it?

Yes, in two ways. A cache can serve an old copy after you update the file, so purge it when you change the content. And some CDN or security configurations treat a .txt file differently from a page, for example by adding challenges. Test the live URL with a plain request from outside your network and confirm you get the current text with a 200 status.

Questions clients ask about llms.txt

• “Does it expose anything private?” It is a public file. Only include what you are happy for anyone to read.
• “Will it make us appear in ChatGPT?” No promise can be made. Crawler access and specific pages come first.
• “Can we copy a competitor’s?” Use it for structure, but write your own facts. A copied summary describes someone else’s business.
• “How long should it be?” Short enough that a person would read it. If it is long, you are probably duplicating your sitemap.
• “Should we list prices?” If you publish them on your site, yes, and keep them identical.
• “What if our plugin and our file disagree?” Remove one source. A physical file at the site root normally takes precedence over a generated route.

What is the bottom line?

Publish an accurate, concise llms.txt if you can do it in an hour or generate it automatically. Do not expect it to move rankings or citations on its own, do not pay a premium for it and do not skip the work that matters more: letting the right crawlers in, writing specific pages and keeping your facts consistent. If your site is documentation or a developer product, the case for doing it, and for publishing markdown versions, is stronger.

Sources

• llmstxt.org: the /llms.txt file
• Google Search Central: AI features and your website
• Google Search Central: learn about sitemaps
• Perplexity: bots and user agents