Skip to content
Neaptidestudio
blog

llms.txt: helping AI find and cite your website

July 22, 2026 · 5 min read

llms.txt is a site map for AI models. What the standard is, how it differs from sitemap.xml, how the format works, and a 5-step rollout together with llms-full.txt.

llms.txt is a plain-text file at the root of your site that helps AI models quickly understand what the site contains and what is worth citing. If robots.txt tells crawlers where they may go, and sitemap.xml lists where the pages are, llms.txt answers the question “what is this site about, and where is the essence”.

The standard was proposed in 2024 (llmstxt.org) and spread quickly: documentation platforms support it, and AI crawlers increasingly request the file alongside robots.txt. It is a young standard — it doesn't guarantee citations — but it costs almost nothing and removes the main barrier: models no longer have to fight through HTML, navigation and scripts.

How the file is structured

llms.txt is regular Markdown with a strict structure:

  • an H1 heading — the project name;
  • a blockquote — one paragraph of essence: who you are and what you do;
  • H2 sections with link lists: page — short description;
  • an optional Optional section — things that can be skipped under a tight context budget.

Next to it goes llms-full.txt — the full text version of the site in a single file: no markup, ready to be loaded into a model's context whole. Multilingual sites should publish full versions per language: engines answer in the user's language.

Rollout in 5 steps

  1. Map out the essentials: 5–15 key pages with one-line descriptions — no marketing, just substance.
  2. Generate llms.txt following the structure above and place it at the root: your-domain/llms.txt.
  3. Create llms-full.txt — a full markdown export of your core content; keep it in sync with the site.
  4. Allow AI crawlers in robots.txt: GPTBot, ClaudeBot, PerplexityBot and the rest.
  5. Update the files with every meaningful content change: a stale llms.txt is worse than none.

Typical mistakes

  • A business-card file with no links: the model won't know where to go for details.
  • Divergence from the site: prices, services and facts in llms.txt must match the pages.
  • One language on a multilingual site: part of your audience is lost.
  • The file lives in the repo but never ships — check production by direct URL.

llms.txt is one of the cheapest GEO steps there is: half an hour of work, and AI engines get a ready-made map of your content. You can see ours live: neaptide.ai/llms.txt.

faq

The short version

Is llms.txt an official standard?

No — it is a community-proposed standard (llmstxt.org, 2024). Engines aren't required to support it, but documentation platforms already use the format and AI crawlers request the file more and more often. The risk is zero: it's a plain text file.

How does llms.txt differ from sitemap.xml?

A sitemap lists every URL for indexing. llms.txt is a curated map of meaning: what matters, what each page is about, what is worth citing. It is written in Markdown and meant to be read by a model, not a crawler.

Do I need llms-full.txt?

It's recommended. llms.txt is an index with links, while llms-full.txt gives the model all your content in one request, without crawling pages. For smaller sites it is the most reliable way to get into the context window whole.

More on the topic

← All articles