The llms.txt file is a plain text file that websites use to give AI crawlers context about what the site is and which pages matter most. It sits at the root of a domain, the same place you would find robots.txt or sitemap.xml. If you want to know whether a site has one, and what it says, checking takes about five seconds.
This guide covers how to find and read the llms.txt file for any website, including your own. It also covers what a well-structured file looks like, based on files we have pulled from hundreds of audited sites, and what it means when a site does not have one at all.
The quick answer: just open the URL
The file lives at `yourdomain.com/llms.txt`. That is the entire convention. There is no registry, no central index, and no tool required to view it. Type the domain into your browser, add `/llms.txt` to the end, and hit enter.
If the site has an llms.txt file, you see plain text in your browser window. No formatting, no HTML rendering, just text. That is intentional. The format is designed to be readable by both humans and language models.
If the site does not have one, you get a 404 page or a blank response. That tells you the site has not adopted the file. It is not an error on your end.
Try it right now. Open a new tab and go to `parceit.com/llms.txt`. You will see our file. Then try `openai.com/llms.txt` or `anthropic.com/llms.txt` to compare. Some major AI companies publish one. Many do not.
Why people check llms.txt files
There are three common reasons to look at a site's llms.txt file.
**You are auditing your own site.** You want to confirm the file exists, is accessible, and contains accurate information. This is the most basic check in AI search visibility work. If your site is supposed to have one and returns a 404, that is a problem to fix before anything else.
**You are checking a competitor.** You want to see whether competitors in your space have adopted the file, what they chose to include, and how they describe themselves to AI crawlers. This is useful intelligence. The file is public by design, so reading a competitor's file is fair game.
**You are evaluating whether llms.txt matters for you.** The standard is new enough that adoption is still uneven. Looking at what comparable sites do helps you decide whether to invest time in creating or maintaining your own file.
How to check with curl (for developers)
If you work in a terminal, use curl to fetch the file and check the HTTP status at the same time:
curl -I https://yourdomain.com/llms.txtThe `-I` flag returns just the headers. A `200 OK` response means the file exists and is accessible. A `404 Not Found` means the site does not have one. A `403 Forbidden` or `503` usually means a bot protection layer is blocking your request, not that the file is missing.
To see the actual content:
curl https://yourdomain.com/llms.txtThis is the fastest way to check multiple sites in bulk. We use this pattern when auditing dozens of domains at once.
How to check with browser DevTools
If you want to confirm the file is being served correctly without caching interference, open your browser's developer tools (F12 in Chrome or Firefox), go to the Network tab, then load the `yourdomain.com/llms.txt` URL.
Look for three things in the response headers:
- **Status code 200.** Anything else means the file is missing or blocked.
- **Content-Type: text/plain.** Some misconfigured servers serve text files as `text/html`, which can cause browsers to render them oddly. The correct type is plain text.
- **No redirect.** If the URL redirects to the homepage or a 404 page, the file is not actually there. A real llms.txt file is served directly at the root path.
What a good llms.txt file looks like
The format is simple but most files we see get it wrong. Here is what a clean, useful file contains, based on the best examples from our crawl data.
# Parceit
> Parceit is an AI search visibility audit service. We crawl websites and score them on the signals that determine how they appear in Google AI Overviews, ChatGPT, Perplexity, and other AI search engines.
## Core Pages
- [Parceit Audit Tool](https://parceit.com/audit): Free audit. Scores your site on crawlability, entity data, content structure, and technical signals.
- [What Is AEO?](https://parceit.com/blog/what-is-aeo-answer-engine-optimization): Complete guide to Answer Engine Optimization, with technical evidence.
- [llms.txt Guide](https://parceit.com/blog/llms-txt-guide): How to create an llms.txt file, with working examples.
## Optional
- [Blog](https://parceit.com/blog): Research and analysis on AI search visibility.That is the whole thing. A title, a one-sentence description in a blockquote, and a few links to the most important pages with short descriptions.
The description line matters more than people realize. It should match the description in your Organization JSON-LD and your meta description. When an AI crawler reads your site, it cross-references these signals. If they say the same thing, trust goes up. If they conflict, the crawler has to guess which one to believe.
What a bad llms.txt file looks like
From our audit data, here are the most common mistakes, in order of frequency.
**Too many URLs.** The file is not a sitemap. Listing 40 pages defeats the purpose. The point is to surface the three to five pages that matter most. Anything beyond that dilutes the signal. If you want crawlers to find every page, that is what sitemap.xml is for.
**Wrong format.** Some sites serve an HTML page at `/llms.txt` instead of plain text. Others put JSON or XML in the file. The spec is plain text with markdown-style links. Anything else is noise that crawlers may ignore or misread.
**Stale descriptions.** The file says the company does one thing, but the homepage says another. This happens when teams update their homepage copy and forget to update llms.txt. The mismatch is worse than having no file at all, because it forces the crawler to reconcile conflicting information.
**No description at all.** A file that is just a list of URLs with no context gives the crawler nothing to work with. The description line is what tells an AI model what your business actually is. Without it, the file is barely useful.
What it means when a site has no llms.txt file
Most websites do not have one. That is normal and not necessarily a problem.
In our crawl data, roughly 70 percent of audited sites return a 404 at `/llms.txt`. The file is an emerging standard, not a requirement. Google has not said it uses the file. Neither has OpenAI, Anthropic, or Perplexity, at least not officially.
So why bother? Because the cost is low and the upside is real. Creating an llms.txt file takes about five minutes. Maintaining one takes almost nothing. If even one major AI search engine starts reading the file for context, the sites that have a good one are ahead. If none of them ever do, you lost five minutes.
The sites that benefit most are those whose business is hard to describe accurately from page content alone. If you run a niche B2B service, a company with a non-obvious name, or a brand that does something AI models tend to misunderstand, the file is worth your time.
How to check whether your llms.txt is actually working
Having the file is step one. Confirming AI crawlers can reach it is step two. Here is how to verify.
First, check that the file returns a 200 status when fetched by a non-browser client. Some bot protection services (Cloudflare, Akamai, Sucuri) block automated requests to unusual paths. Your browser might load the file fine while a crawler gets a 403. Run the curl check from the section above to test.
Second, check your server logs for requests to `/llms.txt`. If you see hits from user agents like `GPTBot`, `ClaudeBot`, `PerplexityBot`, or `Google-Extended`, crawlers are finding and requesting the file. If you see zero hits over a few weeks, either crawlers are not interested or they cannot reach the file.
Third, make sure your robots.txt is not blocking access. The llms.txt file should be accessible to all crawlers, including the AI-specific ones. If your robots.txt blocks `GPTBot` or `ClaudeBot` globally, those crawlers cannot read your llms.txt file even if it exists. Check that AI crawlers are explicitly allowed, as we cover in our [guide to robots.txt for AI crawlers](https://parceit.com/blog/how-to-get-cited-by-chatgpt).
The relationship between llms.txt and your other files
People ask whether llms.txt replaces structured data or sitemaps. It does not. The three files do different jobs.
**robots.txt** controls who can crawl your site. Without AI crawler permissions here, nothing else matters. This is the gate.
**sitemap.xml** lists every URL on your site. It is essential for indexing. Every major crawler reads it.
**llms.txt** gives AI crawlers context about what your site is and which pages matter most. It is optional but helpful.
Priority order: robots.txt first, sitemap second, llms.txt third. If your robots.txt blocks AI crawlers or your sitemap is broken, fixing your llms.txt file will not help. Get the foundation right first.
In our audits, sites with strong JSON-LD structured data but no llms.txt still score well on AI visibility. Sites with an llms.txt file but missing structured data still score poorly. Fix your structured data first, then add llms.txt on top. We break down exactly what structured data does for AI search in our [complete guide to llms.txt](https://parceit.com/blog/llms-txt-guide) and our [AEO fundamentals article](https://parceit.com/blog/what-is-aeo-answer-engine-optimization).
Quick reference: checking any site in under 30 seconds
- Open your browser.
- Go to `yourdomain.com/llms.txt`.
- If you see plain text, the file exists. Read it.
- If you see a 404 or error page, the site has no llms.txt file.
- To confirm with headers, run `curl -I https://yourdomain.com/llms.txt` in a terminal.
That is it. No tool required, no account needed, no cost. The file is public by design.
Want to know how your whole site scores?
Checking your llms.txt file is one signal among many. The Parceit audit engine crawls your site and scores it across four categories: Crawlability, Entity data, Content structure, and Structural fundamentals. Each maps directly to the signals AI search engines use.
You can [run a free audit at parceit.com](https://parceit.com/audit) to see your llms.txt status alongside every other signal, with a prioritized list of what to fix first. It takes under five seconds.