What is an llms.txt file?

llms.txt is a Markdown file placed at a domain root (e.g. example.com/llms.txt) that summarises the site's content and structure for LLMs. It was proposed by Jeremy Howard (Answer.AI / fast.ai) on 3 September 2024. Adoption by major LLMs is still partial in 2026, it is a hygiene signal, not yet a confirmed retrieval anchor.

What is an llms.txt file, in detail?

llms.txt was proposed by Jeremy Howard, founder of Answer.AI and co-founder of fast.ai, on 3 September 2024 (source: llmstxt.org and answer.ai/posts/2024-09-03-llmstxt.html). The file lives at a domain root, example.com/llms.txt, and contains a Markdown document with an H1 title, a short description, and a series of sections (Documentation, FAQ, Articles, Case studies) listing absolute URLs. The spec also recommends a second file, llms-full.txt, which concatenates the entire site content as Markdown for full-context LLM ingestion.

Why it exists

Robots.txt tells crawlers what to fetch. Sitemap.xml lists URLs. Neither tells an LLM how a site is organised or what is most important to read. llms.txt fills that gap: it is a human-curated map of the site optimised for LLM consumption. The format is intentionally minimal, Markdown rather than XML, so any technical writer can maintain it.

Adoption status in 2026

No major answer engine (OpenAI, Anthropic, Google, Perplexity) has publicly confirmed that it systematically uses llms.txt in its crawl or retrieval pipeline in 2026. Independent server-log analyses by SEO cohorts suggest ChatGPT and Perplexity read it occasionally, usually after the file is referenced from an article or a known authority page, but there is no guarantee. The format has gained traction in the technical-documentation community (FastAPI, Stripe, Anthropic docs all publish a llms.txt), less in marketing sites. "We install llms.txt on every GEO-Optim deployment as a low-cost hygiene signal with potential upside and no downside," adds Lorenzo Eeman, founder of PROEMA. Sources: llmstxt.org, answer.ai/posts/2024-09-03-llmstxt.html, FastAPI documentation, Anthropic docs.

Counterintuitive finding. The llms.txt technical proposal by Jeremy Howard (answer.ai, September 2024) is neither an IETF standard nor a scheme referenced in OpenAI's or Anthropic's public 2026 documentation. Yet of the 1,000 sites most cited by Perplexity (Cloudflare Radar sample, April 2026), fewer than 9% publish a llms.txt. That gap is exactly the first-mover window: an easy signal on near-empty ground.