Enter any website URL and get a ready-to-publish llms.txt file built from your real sitemap, homepage title, and meta description. No login required.
Generate llms.txt
Live sitemap and homepage scan
What You Get
llms.txt File
Result
No result yet
Enter a website URL on the left and hit "Generate llms.txt."
llms.txt is a plain markdown file at yoursite.com/llms.txt that gives AI models and coding agents a short, curated map of a website: a title, a one or two sentence summary, and grouped lists of links to the pages that actually matter. It was proposed in September 2024 by Jeremy Howard of Answer.AI as a lightweight alternative to making an LLM parse an entire HTML page just to understand what a site is and where its documentation lives. It sits alongside robots.txt and sitemap.xml rather than replacing either one.
The spec defines a strict order. Skipping a part or putting sections in the wrong order will still parse, but tools built specifically for llms.txt expect this structure.
The only mandatory line. A single # heading with your project or site name. Nothing else belongs on this line.
A short > blockquote directly under the title. One to two sentences that explain what the site is and who it serves. This is the single most load-bearing line in the file.
Plain paragraphs or lists, no headings allowed here, covering anything an agent needs before recommending you: scope, constraints, or what you explicitly are not.
Group related pages under ## headings like Docs, Blog, or Products. Each entry is a markdown link, optionally followed by a colon and a short description.
A special ## Optional heading has a specific meaning in the spec: links here can be skipped by an agent that needs a shorter context window. Use it for secondary material.
Use this as a checklist when you write or edit your file by hand after generating it above.
| Part | What to Include | Example |
|---|---|---|
| H1 title | Your site or project name, one line, nothing else | # Acme Analytics |
| Blockquote summary | One to two sentences on what the site does and who it is for | > Acme Analytics is a lightweight event analytics platform... |
| Context paragraph | Anything an agent needs before recommending you, including what you are not | Acme Analytics is not a full BI suite. |
| H2 section | A named group of related pages, like Docs, Blog, or Products | ## Docs |
| Link list item | A markdown link plus an optional colon and short description | - [Quick Start](/docs/quick-start): Install in five minutes |
| Optional section | Secondary links an agent can skip for a shorter context window | ## Optional |
This example follows the spec exactly: title, summary, a context note, two named sections, and an Optional section. Use it as a starting shape.
# Acme Analytics > Acme Analytics is a lightweight event analytics platform for indie SaaS teams. It tracks product usage without cookies or a JavaScript-heavy tag manager. Acme Analytics is not a full business intelligence suite. It focuses on product usage events, funnels, and retention, not marketing attribution or ad spend reporting. ## Docs - [Quick Start](https://acme-analytics.com/docs/quick-start): Install the snippet and send your first event in under five minutes - [Event Reference](https://acme-analytics.com/docs/events): Every built-in event type and its required fields ## Product - [Pricing](https://acme-analytics.com/pricing): Plans, limits, and self-serve billing details - [Changelog](https://acme-analytics.com/changelog): Recent feature releases and fixes ## Optional - [Blog](https://acme-analytics.com/blog): Product updates and analytics tutorials, safe to skip for a shorter context
From generating the file to it being live on your domain, here is the full path.
Generate a draft above
Paste your domain into the tool at the top of this page. It pulls your sitemap and homepage metadata automatically so you are not starting from a blank file.
Rewrite the summary in your own voice
The auto-pulled meta description is a starting point, not a finished product. Rewrite the blockquote so it reads like something a founder would actually say about the site, not SEO boilerplate.
Trim to what actually matters
Do not list every URL your sitemap returns. Keep the sections tight: docs, pricing, key product pages, and maybe a blog. A shorter, curated file beats a bloated one.
Add a context paragraph if you have edge cases
If your product is easily confused with a competitor category, say so directly. This is the one place in the file where you can pre-empt a wrong assumption from an agent.
Save the file as llms.txt
Plain text, UTF-8, no file extension tricks. Most code editors handle this correctly by default.
Upload it to your domain root
It needs to resolve at yoursite.com/llms.txt exactly, the same pattern as robots.txt. On Next.js, drop it in the public folder. On other stacks, check your static asset routing.
Verify it resolves
Visit the URL directly in a browser and confirm it renders as plain text, not a 404 page or your site's default HTML shell.
Re-generate every few months
Come back to this tool whenever you ship a new major section or restructure URLs, and refresh the file so it does not go stale.
Short Answer
It depends entirely on who is reading it. Coding agents and documentation tools benefit today. Mainstream AI search does not confirm using it for ranking or citation yet.
Long Answer
The clearest, most verifiable use case for llms.txt today is developer tooling. IDE assistants, coding agents, and documentation frameworks such as Mintlify auto-generate the file for hosted docs, and CLI tools like llms_txt2ctx exist specifically to expand an llms.txt file into a full context bundle for a model. If your product has documentation that developers paste into an agent, publishing llms.txt is a genuinely useful, low-cost move.
What is not yet confirmed is whether general-purpose AI search products such as ChatGPT web browsing, Perplexity, or Google AI Overviews treat llms.txt as an input to what they cite. None of them have published documentation stating that they crawl or prioritize it. In practice, what gets cited in AI answers is overwhelmingly pulled from normally indexed web content, discussion threads, and structured pages, which is a big part of why MediaFast focuses on getting founders visible where AI models actually source answers today, on Reddit, rather than betting solely on a file with unproven pull on ranking.
None of that makes llms.txt a bad idea. It costs almost nothing to publish, it future-proofs you if adoption grows, and it is a genuinely helpful map for any agent that does fetch it. Treat it as cheap insurance and a documentation nicety, not a replacement for the harder work of getting your product discussed in places AI models already trust.
No independent, regularly updated census of llms.txt adoption exists, so treat any precise percentage you see elsewhere with skepticism. Here is what is verifiable.
llms.txt was proposed in September 2024 by Jeremy Howard of Answer.AI. It is still an informal community standard hosted on GitHub, not a ratified web standard, and it continues to evolve based on public feedback.
Adoption is concentrated among documentation platforms, API providers, and AI-forward SaaS tools. Frameworks like VitePress and Docusaurus have plugins that auto-generate the file, and hosted docs platforms such as Mintlify create one automatically for sites they host.
Ecommerce, media, local business, and most mainstream consumer sites have not meaningfully adopted llms.txt. Consumer AI search products are not known to require or reward it, so the incentive to publish one is weaker outside developer-facing products.
Most llms.txt files that exist today were generated once and never revisited. These are the recurring problems.
Dumping the entire sitemap into one section. A 400-link wall under a single heading defeats the purpose. The whole point of llms.txt is curation. Keep each section to the pages that genuinely matter.
Leaving the auto-generated meta description as the summary. SEO meta descriptions are written for search snippets, not for an agent deciding whether your product fits a use case. Rewrite the blockquote in plain, direct language.
Forgetting the file exists after publishing it once. URLs move, products get renamed, and sections go stale. A dead link in llms.txt is arguably worse than no file at all, since it actively sends an agent to a 404.
Treating it as an SEO or ranking hack. There is no confirmed ranking benefit from publishing llms.txt. Do it for the coding agent and documentation use case it is actually built for, not as a shortcut around real content and link building work.
Skipping the H2 section structure entirely. A wall of unstructured links with no ## headings still technically parses in some implementations, but it breaks tools built strictly to the spec, which expect named sections.
Putting marketing copy in the blockquote. "Award-winning, industry-leading platform" tells an agent nothing useful. State plainly what the product does and who it is for, the same way you would explain it to a colleague.
There is no public confirmation from Google that its crawlers read or use llms.txt for AI Overviews or standard search ranking. Google's crawling and ranking systems are built around the full page content and existing structured data formats. Publishing llms.txt will not hurt you, but do not expect it to move an AI Overview citation on its own.
Not by default, and not documented as standard behavior. ChatGPT's web browsing tool fetches and reads normal page content when a user asks it to look something up. If a user or a connected agent explicitly points it at your llms.txt URL, it can read it like any other text file, but it is not something the model proactively seeks out on every domain it visits.
Yes, this is the clearest working use case. Developers frequently paste an llms.txt URL directly into an IDE agent or chat interface to give it fast, accurate context on a library or API before writing code against it. This is exactly the scenario the spec was designed for, and it is why documentation-heavy products see the most practical benefit from publishing one.
Three root-level files, three different jobs. None of them replace the others.
| File | Purpose | Read By | Format |
|---|---|---|---|
| robots.txt | Tells crawlers what they are allowed or disallowed to fetch | Search engine crawlers and most well-behaved bots | Plain text, Allow/Disallow directives |
| sitemap.xml | Lists every indexable URL so search engines can find and prioritize pages | Search engine crawlers, submitted via Search Console and similar tools | XML, exhaustive, machine generated |
| llms.txt | Gives a curated, human-readable overview and link map for AI agents on demand | Coding agents, IDE assistants, and tools built specifically for the spec | Markdown, curated, meant to be read by a person too |
Publishing llms.txt takes ten minutes and costs nothing. Getting cited by ChatGPT and Perplexity today mostly comes from being discussed on Reddit. MediaFast finds the right subreddits and helps you show up there.
Everything you need to know about llms.txt, the format, and whether it actually matters yet.
An llms.txt file is a plain markdown file placed at the root of a website (yoursite.com/llms.txt) that gives AI models and coding agents a short, structured summary of a site. It follows a proposed format from Answer.AI: an H1 title, a blockquote summary, optional context paragraphs, and then markdown sections of links pointing to the site's most useful pages. It exists because full HTML pages are noisy and expensive for a language model to parse, while llms.txt gives a clean, curated entry point instead.
You do not need one, and no search engine currently requires it. That said, it is a low-effort file to publish, and it costs you nothing to have one ready in case AI agents and AI-powered browsers lean on it more heavily going forward. Documentation-heavy sites, developer tools, and SaaS products get the most practical value today because coding agents and IDE assistants are the most consistent readers of llms.txt right now.
Not in any confirmed way. No major AI search product has publicly stated that publishing an llms.txt file improves your odds of being cited. What actually gets cited in AI answers today is usually pulled from indexed web content, including forums like Reddit, review sites, and well-structured pages, not a special file most consumer AI search crawlers do not fetch by default. llms.txt is closer to a convenience file for agentic tools than a ranking lever.
Upload it to the root of your domain so it is reachable at yoursite.com/llms.txt, the same pattern as robots.txt and sitemap.xml. On most static site generators and frameworks this means dropping the file into your public or static assets folder. On a CMS, you may need a redirect rule or a small route handler, since many CMS platforms do not let you serve arbitrary files from the domain root by default.
robots.txt tells automated crawlers what they are allowed or not allowed to access, mainly for search indexing. llms.txt is not a permissions file at all. It is a curated summary and link list meant to be read on demand, typically when a user or an agent explicitly wants a compact overview of a site. The two files can coexist without conflict, and robots.txt still governs what crawlers may fetch regardless of what llms.txt lists.
Treat it like a lightweight sitemap. Regenerate it whenever you add a new major section, launch a product, or significantly restructure your site's URLs. There is no crawl schedule to worry about since nothing currently re-fetches llms.txt automatically the way search engines re-crawl sitemap.xml, so stale links are a bigger risk than stale metadata. Re-running this generator every few months is enough for most sites.
