Insight

The Web Is Getting a Second Reader

Agents now read your site on behalf of people who never will. What that changes about publishing, docs, and the open web.

The web is getting a second reader. For thirty years we built pages for one audience: a human with eyes and a mouse. Now there is a second audience. AI agents visit sites to answer a question, compare options, or finish a task — for a person who will never see the page.

Most of the web serves that second reader badly

Fetch a normal page the way an agent does and you get 50–100 KB of markup — menus, banners, scripts — with the actual content buried inside. We spent twenty years making sites readable for one specific robot, the search crawler, and we called it SEO. Nobody has done the same work for agents yet. I think this is the most interesting gap on the web right now.

Two views of the same content, chosen at request time

My bet — and I built a product on it — is that the web splits into two views of the same content. The human gets the designed page. The agent gets clean Markdown, a discovery index, and structured data. The same truth, in the shape each reader can use. Not a separate “API version” that someone maintains by hand. The same content, served twice.

llms.txt, content negotiation, and JSON-LD already exist

The pieces already have names. /llms.txt is becoming for agents what robots.txt was for crawlers. Content negotiation — an HTTP feature that has existed since the 90s — turns out to be exactly the right tool. JSON-LD carries the facts. None of this is new technology. What is missing is the habit of doing it on purpose, site by site.

You cannot improve what you do not measure

And the part almost everyone skips: “Agent-friendly” means nothing until it is a number. When I built Mokhai, the most useful design decision was turning agent-readiness into a 0–100 score across seven parts — discoverability, readability, schema, safety, health, conversion quality, discovery channels. The same decision made pages measurable at the byte level: a 50–100 KB HTML page becomes a 4–8 KB Markdown twin. The moment readiness became a number, site owners stopped debating and started fixing.

The winning sites will be the ones agents can read

What this changes for builders: If agents complete tasks for users, the winning sites are the ones agents can read, trust, and operate. Some old habits turn against you. Tricks that pressure human visitors stop working — an agent does not get tired or confused. It leaves, and it reports the competitor’s price. Content buried five clicks deep is invisible. And “we will add structured data later” becomes the new “we will make it mobile-friendly later” — fine, until the traffic goes somewhere else.

Docs become the interface

Documentation changes the most. Docs used to be what humans read when the product confused them. For an agent, the docs are the interface. A precise sentence in your docs is now executable: an agent reads it and acts on it. A vague sentence used to cost you a support ticket. Now it costs you a wrong action, taken with confidence, on a user’s behalf.

I am building for the open web

Why I care: I came to this from years of support work, teaching people to run their own websites. The agent-readable web can go two ways. In one, a few platforms sit between users and everything else, and the open web becomes a backend nobody visits. In the other, any small site — a clinic, a shop, an independent writer — can be found and used by agents, because publishing a little structure is enough. I am building for the second one.

The web stayed open last time because the tools to be found were free and public. That needs to happen again, and this time there is less time.

← All insights