- The llms.txt standard was proposed in September 2024 and more than 7.3 million live sites now serve the file — yet 97% of those domains received zero requests for it in May 2026, according to (Ahrefs, 2026).
- Over a 90-day experiment, only 84 out of 62,100+ total AI bot visits landed on /llms.txt — that is 0.1% of all AI crawler traffic.
- AI traffic to US retail sites rose 393% year over year in Q1 2026, and that traffic converts 42% better than non-AI traffic — the opportunity is real even if llms.txt is not the lever.
- Shopify turned on Agentic Storefronts by default for eligible US merchants on March 24, 2026, and began serving llms.txt files automatically in May 2026.
- Google's own May 2026 guidance states machine-readable files like llms.txt are not needed to optimize for generative AI features.
- Your highest-return AI visibility moves are content depth, structured data, and review volume — not the file itself.
Introduction
Your store's product page ranks on page one of Google, but a shopper asks ChatGPT "what's the best waterproof hiking boot under $120" and your brand is not in the answer. That gap — between traditional search presence and AI-generated recommendations — is what the llms.txt conversation is really about.
The llms.txt file is a plain-text, machine-readable document placed at the root of your domain (yourstore.com/llms.txt) that lists URLs and structured information intended to help AI crawlers and agents understand what your site contains. Think of it as a curated table of contents written for AI readers rather than human visitors. The concept emerged in 2024 as AI search engines began pulling answers directly from web content, and operators started asking whether there was a robots.txt equivalent for the new generation of crawlers.
The honest answer in mid-2026 is: the file exists, adoption is growing fast, Shopify has baked it in, and Google has simultaneously said it does not matter much. Both things are true at once. This article walks through what the data actually shows about AI bot behavior, where the real AI visibility levers are, and how to prioritize your time.
What is the llms.txt file and why should e-commerce operators care?
The llms.txt file is a plain-text document at your domain root that lists pages, products, and structured content for AI agents to discover — proposed as a standard in September 2024 and now served automatically by Shopify for eligible merchants. Understanding what it does (and does not do) matters because the infrastructure around AI-driven product discovery is being built right now, and the decisions platforms make in 2026 will shape crawl behavior for years.
According to (Ahrefs, 2026), 28% of domains studied publish an llms.txt file, and Google included llms.txt in their Agent2Agent (A2A) protocol launched in April 2025. That same source reports that 97% of domains with an llms.txt file received zero requests for it in May 2026 — a number that should recalibrate how much engineering time you spend on the file versus the content it points to.
Shopify's move is the most concrete signal for store operators. Shopify turned on its Agentic Storefronts feature by default for eligible US merchants on March 24, 2026, and began serving llms.txt, agents.md, and agent discovery files automatically in May 2026, according to (Shero Commerce, 2026). If you run a Shopify store, you likely already have a version of this file live. The question shifts from "should I create it?" to "what does it point to, and is that content good enough to be cited?"
Google's position adds a layer of complexity. In late May 2026, Google's guide on optimizing for generative AI features stated that machine-readable files like llms.txt are not needed — a direct reversal of the signal their A2A protocol inclusion sent 13 months earlier. For operators, this means the file is low-risk to have but should not be the centerpiece of your AI SEO strategy.
How is AI Search Changing E-commerce Discovery?
AI search is changing e-commerce discovery by routing purchase-intent queries through answer engines that synthesize content rather than list links — which means your product pages need to be citable, not just rankable. The scale of this shift is not marginal.
AI Overviews now reaches 2.5 billion monthly users, and AI Mode tops 1 billion monthly users. Perplexity surpassed 100 million monthly active users in 2025, and ChatGPT Search launched to all users that same year. ChatGPT reached 100 million users in just two months — smartphones took 16 months to hit the same threshold, according to (BigCommerce, 2026).
The traffic numbers for retail are striking. AI traffic to US retail sites rose 393% year over year in Q1 2026, and that traffic converts 42% better than non-AI traffic. That conversion premium makes sense: a shopper who arrives via an AI recommendation has already received a synthesized answer and is closer to a decision than someone who clicked a generic organic result.
Zero-click behavior is the other side of this coin. 60% of Google searches in 2026 produce no click at all. Your product needs to be the answer inside the AI Overview, not just a link below it. That requires content that an AI can quote — specific, factual, well-structured prose on your category and product pages.
Can llms.txt Really Improve Your AI Visibility?
The honest answer is: probably not on its own, and the crawl data makes that case clearly. Only 0.1% of AI crawler requests touched /llms.txt over a 90-day period — just 84 requests out of over 62,100 total AI bot visits. The site's average content page received approximately 265 AI bot visits over the same 90 days, meaning AI crawlers were more than three times as likely to visit a standard content page than the llms.txt file itself.
A SERanking study in late 2025 found no measurable improvement in AI citations from publishing llms.txt. Google's own May 2026 guidance reinforces this: machine-readable files like llms.txt are not needed to optimize for generative AI features.
None of this means the file is harmful. Publishing it costs almost nothing — the file format is plain text, and automatically generated versions cannot exceed 100,000 characters. The risk is opportunity cost: an operator who spends two weeks perfecting their llms.txt while leaving category pages at 80 words of useful content has made the wrong trade.
The more productive frame is to treat llms.txt as infrastructure hygiene — publish it, point it at your best content, and then spend your actual optimization budget on the pages it references. AI crawlers are already visiting those pages at a rate 3x higher than the discovery file itself.
What are the best practices for creating an effective llms.txt file?
Effective llms.txt practice starts with what the file points to, not the file itself — your category and product pages need to be worth citing before any discovery mechanism matters. That said, the structural decisions around the file do have practical implications.
Automatically generated llms.txt files cannot exceed 100,000 characters, which means you need to be selective about what you include. A sample llms.txt for an outdoor retailer includes approximately 25 links. For a large catalog, that selectivity is a feature: prioritize your highest-margin categories, your most review-rich product pages, and any editorial content that answers common purchase-intent questions.
The programmatic opportunity is real for stores with location or variant depth. 100 products multiplied by 50 locations can create 5,000 unique, crawlable, AI-optimized entry points. Each of those pages needs substance, though — any category page with under 200 words of useful content above the product grid is at risk of being ignored or downweighted by AI synthesis engines.
Page speed is a non-negotiable input. AI crawlers may abandon a site if product APIs take 5 seconds to respond, and the practical target is a sub-2 to 3 second time-to-first-byte. Approximately 11.5% of ChatGPT's requests were JavaScript files, and 23.8% for Claude — files that likely went unused. Heavy JS-rendered pages that require client-side execution to surface product data are a structural liability for AI crawlability, regardless of what your llms.txt contains.
The format itself is straightforward: plain text, one URL per line, grouped by section with a brief markdown-style header. Keep it updated as your catalog evolves. If Shopify is generating it automatically for your store, audit the output quarterly to confirm it is pointing at your strongest pages rather than auto-including thin variant URLs.
How do AI referrals impact sales and conversion?
AI referral traffic is a measurably different audience — higher intent, better conversion, and growing fast enough to warrant dedicated tracking in your analytics stack. The 42% conversion premium over non-AI traffic is the number to anchor your business case around.
Engaged shoppers who interacted with reviews account for 56% of all sales, according to (Yotpo, 2026). That figure matters for AI visibility because AI synthesis engines pull from the same signals human shoppers use to evaluate products — review volume, specificity, and recency. Displaying just 10 reviews can lead to a 53% uplift in conversion for human shoppers, and verified customer photos increase purchase likelihood by 137%, according to (Yotpo, 2026). These are not just CRO tactics; they are the content layer that makes your product pages citable by AI.
In late 2025, 10% of all new signups for some leading tech platforms were coming directly from AI referrals, according to (Yotpo, 2026). For e-commerce, the equivalent metric is first-order attribution — tracking which orders came from sessions that originated at an AI search engine. Most stores are not doing this yet, which means the operators who instrument it now will have a data advantage when the channel matures.
The conversion data also reframes the llms.txt debate. If AI-referred traffic converts 42% better, the question is not "does llms.txt drive more AI traffic?" but "are my product pages good enough to be recommended by an AI to a high-intent shopper?" The answer to that second question depends on content quality, review density, and structured data — not on a text file at your domain root.
Beyond llms.txt: Broader AI Optimization Strategies for E-commerce
AI visibility for e-commerce comes down to three things: content that AI can quote, structured data that AI can parse, and a post-purchase loop that generates the review signals AI synthesis engines weight heavily. The llms.txt file is a pointer to these assets, not a substitute for them.
Content depth is the most direct lever. A site with 5,000 well-crafted pages will outperform a site with 50,000 thin ones in the AI Overview landscape. For a mid-market store, this means writing genuine buying guides, comparison pages, and use-case content — not spinning product descriptions into 50 near-duplicate variants. Each page should answer a specific question a shopper might ask an AI assistant.
Loyalty and repeat-purchase programs have a structural connection to AI visibility that is easy to miss. Loyalty programs generated an 8x return on investment and a 44% increase in average order value from loyalty members, according to (Yotpo, 2026). Repeat customers are also the cohort most likely to leave detailed, specific reviews — and those reviews feed directly into the content layer that AI synthesis engines draw from. The flywheel is: better retention → more reviews → richer product pages → higher AI citation rate → better-qualified new traffic.
Trust remains the constraint. Only 24% of consumers are comfortable sharing data with an AI shopping tool, according to (BigCommerce, 2026). Stores that make their data practices transparent — clear privacy policies, explicit opt-ins for AI-personalized recommendations — will convert AI-referred visitors at a higher rate than stores that treat data collection as a background process. Nearly 9 in 10 senior executives expect to increase AI-related budgets in the next year, which means the competitive pressure on AI visibility is about to intensify. Getting your content, structured data, and review infrastructure right now is the practical preparation.
Editor's Take — Michal Baloun, Co-founder
The llms.txt debate reminds me of the early days of schema markup — a lot of operators asked "do I need this?" when the more useful question was "what does my content look like to a machine?"
The crawl data from OtterlyAI is the most honest signal we have right now: 84 visits to /llms.txt versus 62,100+ total AI bot visits over 90 days. That is not an argument against publishing the file — it takes an afternoon to set up, and Shopify does it for you anyway. It is an argument against treating the file as a strategy.
What I keep coming back to is the conversion premium. AI-referred traffic converting 42% better than non-AI traffic is a number that should change how you think about your content budget. A shopper who arrives from an AI recommendation has already been pre-sold to some degree — they asked a question, got an answer that named your product, and clicked through. Your job at that point is to not lose them with a thin product page, missing reviews, or a slow load time.
The stores I see winning the AI visibility game are not the ones who spent the most time on their llms.txt. They are the ones who wrote a genuine 800-word buying guide for their core category, collected 50+ verified reviews on their top SKUs, and made sure their product pages load in under two seconds. Those are the pages AI crawlers visit 265 times in 90 days. Make them worth citing.
The file is infrastructure. The content is the strategy.
Here's what advice from Margly looks like
Most analytics dashboards stop at "your number is X". Margly stops at the next sentence — what to do, where, how much it's worth. Recommendations Margly would surface for the patterns described in this article:
-
High priority "Audit category pages with under 200 words of useful content above the product grid and expand them to 500+ words of buying-guide copy." Your AI-referred traffic converts 42% better than non-AI traffic — thin pages are where that premium leaks. Estimated impact: +$1,800 to +$3,200 / month
-
High priority "Instrument AI referral attribution in your analytics — tag sessions originating from ChatGPT, Perplexity, and Google AI Mode as a separate channel." Without this, you cannot measure the 42% conversion premium or know which product pages are being cited. Estimated impact: +$900 to +$1,500 / month in recovered attribution clarity
-
Medium priority "Run a post-purchase review request sequence targeting your top 20 SKUs to reach at least 10 verified reviews per product." Displaying just 10 reviews produces a 53% uplift in conversion, and review density is a primary signal AI synthesis engines use when selecting which products to recommend. Estimated impact: +$600 to +$1,200 / month
-
Medium priority "Audit your llms.txt output (or generate one if absent) and confirm it points to your highest-margin category pages rather than thin variant URLs." The file costs nothing to publish and Shopify generates it automatically — the audit is about making sure the pointer is aimed at your best content. Estimated impact: +$200 to +$500 / month
Notice none of those needed a CSV export. That's the difference between raw analytics and concrete advice.
What is the llms.txt file?
The llms.txt file is a machine-readable plain-text document placed at the root of your domain, proposed as a standard in September 2024. Its purpose is to help AI crawlers and agents discover and understand what your website contains — essentially a curated index written for AI readers rather than human visitors.
Adoption by major AI models is still evolving. Google included llms.txt in their Agent2Agent (A2A) protocol in April 2025, but their own guidance in May 2026 stated that such files are not strictly needed to optimize for generative AI features. More than 7.3 million live sites now serve the file, but 97% of those domains received zero requests for it in May 2026.
How do I create an llms.txt generator?
There is no single canonical generator tool. Creating an llms.txt file means listing relevant URLs, product links, and discoverable content in plain text, grouped by section with brief markdown-style headers. The format is intentionally simple.
Shopify began serving these files automatically for eligible US merchants from May 2026 onward — if you run a Shopify store, check whether yours is already live at yourstore.com/llms.txt. Automatically generated files cannot exceed 100,000 characters, so for large catalogs you will need to be selective about which pages to include. Prioritize high-margin categories, review-rich product pages, and editorial buying guides.
What is an llms.txt example for e-commerce?
A sample llms.txt file for an outdoor retailer includes approximately 25 links to products and categories. The structure is plain text: a brief markdown header for each section (e.g., "## Products", "## Buying Guides") followed by one URL per line with an optional short description.
The file cannot exceed 100,000 characters in auto-generated form. The goal is to give AI crawlers direct access to your most valuable, content-rich pages — not to list every URL in your sitemap. Think of it as a highlight reel rather than a complete index.
What is the llms.txt standard?
The llms.txt standard was proposed in September 2024 to give AI agents a structured way to understand what a website contains. It is not an official W3C or IETF standard — it emerged from the developer community as a practical convention, similar to how robots.txt started.
Google included llms.txt in their Agent2Agent (A2A) protocol in April 2025, lending it institutional weight. Their May 2026 guidance then stated that machine-readable files like llms.txt are not needed to optimize for generative AI features — a signal that the file is useful infrastructure but not a ranking factor in any direct sense.
LLMs.txt vs robots.txt — what is the difference?
Robots.txt tells search engine crawlers which pages not to crawl — it is a restriction mechanism, primarily used to manage crawl budget and keep private or duplicate content out of indexes. LLMs.txt is the opposite: it is an invitation, designed to help AI agents discover and understand the content that should be surfaced by generative AI search engines.
The two files serve complementary roles. Robots.txt says "don't go here." LLMs.txt says "here is what matters most." A well-configured store should have both: a robots.txt that protects checkout flows, account pages, and thin variant URLs, and an llms.txt that highlights your best category pages, buying guides, and review-rich product pages.
Sources:
- Ahrefs — What Is llms.txt?
- Yotpo — What Is llms.txt?
- BigCommerce — E-commerce LLMs.txt
- Mintlify — llms.txt Docs
- Firecrawl — How to Create an llms.txt File
- OtterlyAI — The llms.txt Experiment
- Shero Commerce — LLMs.txt and agents.md for E-commerce
- TNG Shopper — LLMs.txt for E-commerce: A Practical Guide
- 1Digital Agency — llms.txt Implementation
- Similar.ai — Google AI Overviews