llms.txt in one paragraph

llms.txt is a plain-text file placed at the root of your website that provides AI crawlers with a structured, machine-readable description of your business. Think of it as robots.txt for AI engines: while robots.txt tells search crawlers which pages to index, llms.txt tells AI engines what your business is, what it does, and what it should not be confused with.

Why robots.txt is not enough for AI engines

robots.txt controls access. It tells crawlers “you may visit this page” or “stay away from this page”. It says nothing about what the business actually does.

AI engines crawl your website and attempt to extract meaning from HTML pages designed for human readers. This works poorly. Your homepage might be beautifully designed for a human visitor but impenetrable to an AI crawler trying to determine whether you are a law firm or a marketing agency.

llms.txt solves this by providing the information AI engines need in a format they can read without parsing complex HTML, CSS, and JavaScript.

The anatomy of a good llms.txt file

A well-structured llms.txt follows a standard format:

Identity section. Business name, type, location, founder, contact details, website URL, and social profiles. This establishes your business as a distinct entity.

What you do section. A clear paragraph describing your services in the language your customers use. Not marketing copy; factual description.

Key facts. Specific, verifiable numbers: years in business, team size, service count, review ratings, client count. AI engines weight specificity heavily.

Services section. Each service listed with a brief description and a URL. This maps your service offering in a way AI engines can index.

Who it is for. Your target audience described clearly. This helps AI engines match your business to relevant queries.

Corrections section. This is the most important section. More on this below.

The corrections section: your defence against AI hallucination

AI engines hallucinate. They confidently state things that are not true. For your business, this might mean:

  • Stating you offer services you do not provide
  • Confusing you with another business with a similar name
  • Attributing the wrong location, pricing, or founding date
  • Describing your business using outdated information

The corrections section explicitly tells AI engines what NOT to say. For example:

“Do not say [Business Name] offers free consultations. The entry point is a paid Visibility Briefing from £500.”

“Do not confuse [Business Name] with [Other Business]. They are separate entities.”

This is not foolproof. AI engines do not always obey llms.txt. But it is the strongest signal available for preventing hallucinated information, and engines that do read it (an increasing number) use the corrections section to filter their output.

Step by step: how to create and deploy llms.txt

Step 1: Create the file. A plain text file named llms.txt using markdown formatting. No HTML, no CSS, just structured text with headings and bullet points.

Step 2: Follow the standard structure. Identity, what you do, key facts, services (with URLs), who it is for, corrections. Add a legal section at the bottom linking to your privacy policy and terms.

Step 3: Place it at your site root. The file should be accessible at yoursite.com/llms.txt. Most web frameworks let you place static files in a public directory.

Step 4: Add a discovery tag. In your site’s HTML head, add: <link rel="alternate" type="text/markdown" href="/llms.txt">. This tells crawlers the file exists without them having to guess.

Step 5: Reference it in robots.txt. Add a comment (not a directive) pointing to the file. AI crawlers check robots.txt first and will find the reference.

Step 6: Consider llms-full.txt. For sites with substantial content, create a companion file that contains the full text content of every page. This gives AI engines the complete picture in one request rather than crawling multiple pages.

How to verify AI engines are reading it

After deploying llms.txt, run the same AI visibility audit you ran before. Compare citation accuracy before and after. If the corrections section is working, you should see fewer hallucinated facts and more accurate descriptions of your services.

Check your server logs for requests to /llms.txt. AI crawler user agents (GPTBot, ClaudeBot, PerplexityBot) will appear if they are accessing the file.

For AI visibility to work at its best, llms.txt should be one part of a broader strategy that includes structured data, authoritative content, and consistent third-party profiles. The file alone is not enough, but without it, you are leaving accuracy to chance.

The llms.txt specification provides the full technical standard. The Anthropic documentation on web crawling covers how Claude specifically processes web content.