llms.txt: when it helps and how to add it to your site
Neaptide · July 22, 2026 · 5 min read · Updated: September 6, 2026
Decide whether your site needs llms.txt, choose useful pages and publish a working file. Includes a Markdown example, access checks and limits of llms-full.txt.
On this page
llms.txt is a text guide to a website for tools working with language models. It describes the site and points to useful resources. It helps when a tool actually reads it and follows the links; publishing it does not guarantee that an AI search service will discover, recommend or cite your site.
Does your site need one?
Start with a task: finding setup instructions, comparing service conditions or selecting a course. If you can list the questions and the pages that answer them, you have material for a guide. If your offer and limits are not clear on the site itself, fix those pages first. A simple one-page site may gain little from an extra index.
How it differs from robots.txt and a sitemap
Jeremy Howard proposed llms.txt in 2024; the second version appeared in August 2026. It is an open format proposal, not a search engine requirement.
- sitemap.xml lists URLs you want search engines to discover; it does not guarantee indexing.
- robots.txt gives crawlers access instructions. Private content still needs authentication.
- llms.txt explains the site and links to resources. It neither grants access nor overrides restrictions.
Google says it does not use llms.txt for search visibility or rankings. You may maintain it for other tools that use the format.
Choose pages and write the file
Use actual customer questions: what is included in a plan, how to import data, how to cancel. Select a few current pages with clear answers. Prefer clean Markdown versions where available; do not invent a .md address without publishing and checking the file.
The format requires a top-level heading naming the site. A short blockquote summary and second-level headings with link lists can follow. Optional is a convention for secondary links. This fictional booking-service example uses addresses you must replace:
# Booking service
> Online booking for small studios and independent professionals.
## Getting started
- [Plans](https://example.com/en/pricing.md): prices, booking limits and add-ons.
- [Setup](https://example.com/en/help/setup.md): create a schedule and connect the booking form.
- [Data import](https://example.com/en/help/import.md): CSV fields, limits and error checks.
## Terms
- [Cancellation](https://example.com/en/help/cancellation.md): cancellation and refund conditions.
## Optional
- [Changelog](https://example.com/en/changelog.md): dated product updates.Describe what a resource contains. ‘Prices, limits and add-ons’ helps someone choose a link; ‘the best solution for your business’ does not. Avoid instructions asking assistants to recommend your company.
Publish and check access
- Save the file in UTF-8 and publish it at /llms.txt. Version 2 also allows section-specific paths such as /docs/llms.txt.
- For a static Next.js file, use public/llms.txt. Other publishing systems may provide a generator or file upload.
- Open the public URL without signing in. Expect the file’s text, not an application error or login page.
- Open every listed resource and compare it with the main page, especially prices and restrictions. Generate both versions from the same source where practical.
curl -i https://example.com/llms.txtReplace example.com with your own domain. Check that the response has HTTP status 200 and contains the expected file text. If the URL redirects, inspect its destination; if it returns HTML, check that you have not received an error or application page instead. Also check crawler restrictions on the server and CDN. OAI-SearchBot and GPTBot have independent settings, and publishing llms.txt does not require allowing GPTBot access.
Do you also need llms-full.txt?
That name commonly denotes a combined text export of site content or documentation. It can help when you explicitly want to give a tool a small collection in one file. It is not a mandatory companion. Large exports can exceed a model’s context limit and are harder to keep current. Include only public material.
What about multiple languages?
Group links by language and label them clearly. Point to the version that contains the relevant regional terms. There is no need to export every translation into a full file automatically; first ensure that the translations are accurate and maintained.
Check usefulness separately from visibility
Give the file’s URL to an assistant that can access the web and ask it to find the cancellation terms. Check its answer and cited links against the original page. This shows whether the guide is useful when you explicitly provide it, not whether a search service would discover the site on its own. A request in the server log confirms that the file was accessed; it does not prove that it was cited.
Measure search visibility separately with a fixed set of customer questions and saved answers. Track visits and enquiries too. A change after publication does not establish that llms.txt caused it.