What is llms.txt?
The file lives at a predictable address, such as example.com/llms.txt, so any tool that looks for it knows where to find it. The idea is that a model can read one clean file instead of parsing navigation, scripts, and layout.
It was proposed in 2024 as a community convention. It isn't an adopted web standard, and adoption by the companies that build AI models is limited and unclear.
What does an llms.txt file contain?
The proposal describes a simple Markdown structure. A typical file includes:
- A title. The name of the site or project as a top-level heading.
- A short summary. A sentence or two describing what the site is and who it's for.
- Key details. Optional notes that help a model interpret the site correctly.
- Sections of links. Lists of the most useful pages, each with a one-line description, often pointing to clean Markdown versions.
What is an example of llms.txt?
A documentation site
A software company publishes developer documentation across hundreds of pages. It adds an llms.txt file listing its getting-started guide, API reference, and common tasks, each with a short description.
An AI coding tool that chooses to read llms.txt can find the right pages quickly. A tool that doesn't read it, which includes Google Search, behaves exactly as before.
llms.txt vs. robots.txt vs. sitemap.xml
| robots.txt | sitemap.xml | llms.txt | |
|---|---|---|---|
| Purpose | Tells crawlers what they may access | Lists URLs for search engines to discover | Summarizes a site for AI models |
| Status | Long-established standard | Long-established standard | Proposed convention |
| Format | Plain-text directives | XML | Markdown |
| Controls crawling or training | Yes, for crawlers that respect it | No | No |
| Used by Google Search | Yes | Yes | Google has said no |
Should you add an llms.txt file?
Adding one is low effort and low risk, and some AI developer tools read it. But it isn't a shortcut to AI visibility, and it doesn't replace the work that actually shapes how AI models describe a brand.
- Don't expect ranking gains. Treat it as a convenience for the tools that read it, not as an SEO tactic.
- Don't use it for access control. Use robots.txt and other documented controls to manage crawling.
- Focus on fundamentals. Clear, accurate pages and consistent mentions across trusted sources matter more.
- Measure the outcome. Track how AI models actually describe your brand, rather than assuming a file changed it.
How does Alli do this?
Creating and maintaining llms.txt files is one of the AI visibility services PMG offers. PMG's teams, and its Forward Deployed Engineering team, build them as part of broader GEO and AEO work, alongside the content and structure changes that matter most.
Alli measures the outcome: Share of LLM Presence, a signal in Alli Brand Equity, tracks how a range of AI models describe, rank, and recommend your brand compared with your competitors, and how that changes over time, so teams can see whether the work is changing how AI talks about them.
Explore Alli Brand Equityllms.txt FAQ
Does Google use llms.txt?
Google has said its search systems don't use llms.txt. Adding the file won't affect Google rankings or AI Overviews.
Is llms.txt an official standard?
No. It's a proposed community convention from 2024. It hasn't been adopted as a web standard, and support among AI companies is limited.
Is llms.txt the same as robots.txt?
No. robots.txt tells crawlers what they may access. llms.txt only summarizes a site, and it doesn't allow or block anything.
Can llms.txt stop AI models from training on my content?
No. To manage AI crawlers, use robots.txt rules and the controls each AI company documents.
Last updated ‹ Back to the glossary