llms.txt is a proposed plain-text file placed at a site’s root to give large language models a curated guide to its most important content; it is a community proposal, not a standard adopted by the major engines. It earns an Index entry for exactly that reason: it is the most asked-about artefact in AI SEO relative to its documented effect, and a reference page owes readers the gap between the two.
In one sentence
llms.txt is a menu a site offers to language models, and the honest current status is that the major engines have not documented ordering from it.
How llms.txt is proposed to work
The proposal, introduced in 2024, places a markdown file at the site root, /llms.txt, containing a short description of the site and a curated, annotated list of its most important pages, so a model with limited context can find the substance without crawling everything. It is deliberately the inverse of robots.txt: an invitation rather than a restriction. Some sites pair it with expanded variants offering fuller flattened content for model consumption.
Adoption status
What can be stated firmly: Google’s guidance on generative AI features says no special AI files or markup are required and that standard crawling, indexing and quality systems govern eligibility, and neither OpenAI’s nor the other major engines’ publisher documentation instructs sites to provide llms.txt or documents consuming it. Adoption has run ahead of consumption: many sites publish the file, and tooling generates it, without platform documentation confirming it influences retrieval or citation on the surfaces that matter commercially. That may change, which is why this page carries a verification date; as of it, llms.txt is a low-cost signal of intent, not a documented lever.
Why llms.txt matters
Practically: it is cheap, harmless and occasionally convenient, and there is no documented case for making it a priority, a launch blocker, or a promise to a client. Instructively: it is the cleanest available case study in AI SEO folklore, a plausible mechanism spreading through repetition while the platforms’ own documentation points elsewhere, to crawler access, extractable content and evidence. The habit it teaches, checking claimed levers against documented behaviour, is worth more than the file.
What would change this assessment
Platform documentation from a major engine stating that it fetches and uses llms.txt for retrieval, grounding or citation would move the file from signal to lever, and this page would be updated accordingly. Server logs showing declared AI crawlers fetching the file are a weaker but observable interim signal worth logging where seen.
Related concepts
References
- Google Search Central, Optimising your website for generative AI features on Google Search: developers.google.com
- OpenAI Help Center, Publishers and Developers FAQ: help.openai.com
Author: Harpal Singh · Last verified: 7 August 2026