3 spots left for AugustGrab Yours →

What Is llms.txt and Does Your Website Need One in 2026?

AD

Abhishek Dwivedi

Team Lead, SEO

|
Jul 17, 20269 min read
Share:
What Is llms.txt and Does Your Website Need One in 2026?

In September 2024, Jeremy Howard, the co-founder of fast.ai and Answer.AI, proposed a new file format called llms.txt. The idea was simple. Large language models struggle to process full HTML pages the way a browser does, so give them a clean, structured summary instead. Two years on, adoption has grown across SaaS companies, documentation sites, and content-heavy publishers, even though no major AI lab has confirmed it officially crawls the file yet. If you're building a GEO or AEO strategy for your brand, it's worth understanding exactly what this file does before you add one.

What Is llms.txt, in Plain Terms?

llms.txt is a markdown file you place at the root of your domain, at yourdomain.com/llms.txt. It lists your most important pages in plain language, grouped under simple headers, with a one-line description of what each page covers. Some sites also publish an llms-full.txt, which includes the entire text content of key pages in one file so a model can ingest it without crawling the live site page by page.

Where Did This Idea Come From, and Is It an Official Standard?

It isn't an official web standard the way robots.txt is, and it hasn't been ratified by the W3C or adopted as documented policy by OpenAI, Anthropic, or Google. It started as a proposal on llmstxt.org and spread the way a lot of useful web conventions do: through voluntary adoption. Companies like Anthropic, Mintlify, and Cursor have published their own llms.txt files. That's a meaningful adoption signal even without formal backing, but it also means you shouldn't treat it as a guaranteed ranking or citation lever.

Close-up of website code on a screen representing technical SEO and llms.txt setup
Setting up llms.txt is a lightweight technical addition to your site's root directory

How Is llms.txt Different From robots.txt and sitemap.xml?

robots.txt is a permissions file. It tells crawlers what they're allowed and not allowed to fetch, and it says nothing about content quality or importance. sitemap.xml is a complete index, often thousands of URLs long, meant for exhaustive discovery. llms.txt does a third, different job. It's a short, curated, human-readable brief pointing to maybe 10 to 30 pages that matter most, written in a format a language model can parse without rendering JavaScript or navigating a menu. None of the three files replace each other.

Does Adding llms.txt Actually Improve AI Citations?

Here's the honest answer: the evidence right now is mixed and mostly anecdotal. Some documentation-heavy sites have reported showing up more often in AI-generated answers after publishing one, but there's no independent, large-scale study confirming causation as of this writing. What we do know is that clean, well-structured content with clear headings and direct answers tends to get cited more often in tools like ChatGPT, Perplexity, and Google's AI Overviews, and llms.txt is one more way to present that structure. This mirrors a broader shift we've covered before, where content graphs are replacing social graphs as the backbone of discovery. Treat it as a small, low-cost addition to a broader content strategy, not a replacement for one.

Which Websites Get the Most Value From It?

Sites with deep, evergreen content libraries benefit the most: SaaS products with documentation, EdTech platforms with course catalogs (our complete guide to EdTech SEO covers this in more depth), agencies with detailed service pages and guides, and publishers with large archives. If your site is a five-page brochure site with no blog or documentation, an llms.txt file won't do much for you. Fix your on-page content and technical SEO first. Structure follows substance, not the other way around.

Laptop displaying a website analytics dashboard with traffic and engagement graphs
Track AI-referral traffic in GA4 after publishing your llms.txt file

How Do You Create Your First llms.txt File?

Start by listing the 10 to 20 pages you'd want an AI assistant to read if it could only read a handful. Group them under markdown H2 headers such as "Docs," "Guides," "Product," or "Case Studies." Write one plain sentence under each link describing what it covers and who it's for; skip marketing language and get straight to the point. Save the file as llms.txt and upload it to your site's root directory so it's reachable at yourdomain.com/llms.txt. Once it's live, keep an eye on referral traffic from ChatGPT, Perplexity, and Gemini inside GA4 over the following month to see if anything shifts.

Handwritten checklist on paper, representing the step-by-step process of building your first llms.txt file
Creating your first llms.txt file is a simple, step-by-step process

What Should You Do Next?

Don't treat llms.txt as a checkbox you tick once and forget. Update it whenever you publish major new guides or retire outdated pages, the same way you'd maintain a sitemap. Pair it with solid technical SEO, clear heading structure, and content that answers real questions directly near the top of the page. That combination, not any single file, is what actually improves your odds of being read and cited accurately by AI systems in 2026.

Frequently Asked Questions

Do I need to remove or change my robots.txt file if I add llms.txt?

No. llms.txt and robots.txt serve different purposes and work together. robots.txt controls what crawlers can access, while llms.txt highlights your most useful pages for AI models to read. Keep both files active and update them independently as your site changes.

Will Google use my llms.txt file for search rankings?

Google has not confirmed that it reads or ranks pages based on llms.txt. It is aimed at AI assistants and chat tools rather than traditional search crawlers. Adding one is unlikely to hurt your SEO, but treat it as a supplement to strong technical SEO, not a ranking factor on its own.

How long should an llms.txt file be?

Most effective llms.txt files list 10 to 30 links, each with a one-line description. Keep it focused on your most valuable pages, such as core service pages, documentation, or flagship guides, rather than trying to include your entire site.

Can I automate the creation of an llms.txt file?

Yes. Several open-source scripts and CMS plugins can generate an llms.txt file automatically from your sitemap or content management system. For small sites, writing it by hand often produces a more useful, curated result since you can choose which pages truly matter.

Does WordPress or Shopify support llms.txt out of the box?

Not natively as of 2026. You will need to manually upload an llms.txt file to your site's root directory, or use a plugin or app built for this purpose. Check with your hosting provider or developer to confirm root-level file access.

What happens if I don't create an llms.txt file?

Nothing breaks. Your site will continue to be crawled and indexed normally by search engines and AI tools. You may simply miss a small, low-cost opportunity to guide AI assistants toward your best content, so it's worth doing once your core SEO and content are solid.

Like what you're reading?

Get a free marketing plan tailored to your brand

Get Your Free Plan

yes, actually free. we're not kidding.

Key Takeaways

  • llms.txt is a curated, plain-language map of your best pages for AI models, not a crawler permissions file like robots.txt.
  • It was proposed by Jeremy Howard of Answer.AI in September 2024 and has voluntary adoption, but it isn't an official web standard yet.
  • No major AI lab has publicly confirmed it crawls the file, so results are a signal, not a guarantee.
  • It works best for content-rich sites like SaaS, EdTech, agencies, and publishers with real documentation or guides.
  • Fix technical SEO and content quality first; llms.txt supplements a strong content strategy, it doesn't replace one.
  • Update the file whenever your key pages change, and track AI-referral traffic in GA4 to see if it's making a difference.

the TL;DR your boss will love

This Isn't Just Theory. We Do This Daily.

Let us show you exactly what we'd do for your brand - strategy, channels, budget, timeline. Free. No pitch deck.

Get Smarter Every Week

Ideas, data, and the occasional hot take. Biweekly, no spam.

we promise not to sell your email to robots

You Might Also Dig These

^ we picked these just for you (okay, the algorithm did)

How Indian Brands Can Beat Rising Ad Costs with AI-Driven Marketing

Garage Collective TeamJul 24, 2026

Hey Indian homeowners and farmers! Did you know that Google and Meta ad costs have jumped by over 30% in the last year? 📈 If you’re tired of throwing money at ads that don’t deliver, it’s time to rethink your marketing strategy from scratch. This blog breaks down how using AI and first principles thinking can help you get the most bang for your marketing buck - without wasting a rupee.

Read

Ultimate Guide to AI-First Agency Month: Optimize Performance Marketing ROI Amid Rising Ad Costs in India

Garage Collective TeamJul 25, 2026

Struggling to keep up with rising ad costs and ever-changing Google and Meta algorithms? Indian marketers, startups, and D2C brands are feeling the pressure to scale smarter - not harder. Ready to cut through the noise and supercharge your performance marketing ROI? This guide reveals AI-first strategies, proprietary tools, and proven tactics to help you stay ahead and win big in India’s digital market.

Read

Proof of Work in Indian Marketing: Beyond Case Studies to Real Metrics and Client Trust

Garage Collective TeamJul 27, 2026

Indian founders and CMOs managing Rs.1Cr+ marketing budgets often struggle to verify agency claims beyond polished case studies. In a digital landscape where Google and Meta algorithms constantly change, authentic proof of work is essential. This article examines what proof of work truly entails - actionable metrics, transparent reporting, and honest client conversations that establish trust and demonstrate real value for serious Indian brands.

Read