llms.txt at the edge

What it does

A request for /llms.txt from a known AI crawler gets the content you configured, as text/plain. Every other request goes to your origin.

Untouched is exact: the origin response is returned as-is, with no header added, no HTML rewritten and no cache directive changed. A human visitor cannot tell it is installed. Leave it unconfigured and it does nothing at all.

Zero outbound requests

The only network call in the source is the pass-through to your own origin. It never contacts TriRank, never reports that a crawler visited, and never phones home.

That is a property you can check rather than a promise you have to take: the source is short enough to read in a couple of minutes, and a check in our repository asserts it contains no absolute URL at all, so the property cannot be lost by accident.

Installing it

1. Copy the template directory and put your llms.txt into the LLMS_TXT variable in wrangler.toml.
2. Set the route to your own zone.
3. Run wrangler deploy, then check it twice: once as a crawler, once as a browser.

The KV path for an llms.txt over Cloudflare's variable size cap, the two verification commands and what the template deliberately leaves out are in integrations/cloudflare-worker/README.md.

Which crawlers count

A generated list of published crawler User-Agent tokens ships with the template. It comes from the same roster our own crawler statistics are built on, where every token carries the vendor page it was read from and the day it was read — along with the reasoning for the ones deliberately left out.

See which AI crawlers actually visit