Agentic Web Census

August 2026by Omitsis

English

Trends · May 2026 – August 2026

The agentic web is getting better, and we can prove it

The index went from 14.3% in May 2026 to 19.9% in August 2026 across 4 monthly measurements. Every single step of that clears its own margin of ±0.44, which is what makes it a trend and not a wobble.

20.4%13.7%AWCS · May: 14.3%AWCS · June: 16.5%AWCS · July: 19.1%AWCS · August: 19.9%AWCSMayJuneJulyAugust
The AWCS, May 2026 to August 2026. The last step was up 0.8%.

The dimensions underneath

37.7%0.0%Sites with an AI crawler policy · May: 8.9%Sites with an AI crawler policy · June: 9.8%Sites with an AI crawler policy · July: 9.9%Sites with an AI crawler policy · August: 9.6%Pages an agent can parse · May: 26.2%Pages an agent can parse · June: 27.7%Pages an agent can parse · July: 29.1%Pages an agent can parse · August: 30.4%Sites serving an llms.txt · May: 29.9%Sites serving an llms.txt · June: 34.6%Sites serving an llms.txt · July: 34.3%Sites serving an llms.txt · August: 34.0%Sites with a valid llms.txt · May: 3.1%Sites with a valid llms.txt · June: 9.1%Sites with a valid llms.txt · July: 13.0%Sites with a valid llms.txt · August: 13.1%Sites with an AI crawler policyPages an agent can parseSites serving an llms.txtSites with a valid llms.txtMayJuneJulyAugust
The dimensions that make up the index, in percent of sites. Same months, same sample. Sites serving an llms.txt is not part of the index — it is here because having a file and having a valid one are different problems.

Every figure, and how much of the web moved with it

How to read these figures

A difference smaller than the wider of the two margins is not a small movement, it is a tie: with a different sample of pages it could have gone the other way. We label those = and we do not describe them as up or down. It is why the last column matters — a figure can move while almost none of the segments underneath it do.Every figure here comes from a sample, not from counting every site — so it carries a margin: how much it could move if we had happened to sample different pages. A figure measured on a thousand pages has a wide margin; one measured on a hundred thousand has a narrow one. It matters for one thing: two figures that differ by less than the wider of their margins are not one above the other, they are tied. That is why nothing here has a position number.
FigureAugust 2026Last monthSegments that moved
AWCS19.9%▲ +0.8366 of 2,490
Pages an agent can parse30.4%▲ +1.3 pp851 of 2,490
Sites with an AI crawler policy9.6%= −0.3 pp322 of 1,891
Sites with a valid llms.txt13.1%= +0.1 pp400 of 2,490
Sites serving an llms.txt34.0%= −0.3 pp963 of 2,490
Sites exposing tools735▲ +111181 of 2,490
Google's category score57.3%▲ +0.9207 of 1,497
Layout stability88.6%▲ +0.91 of 1
Agents that can reach the site96.0%= −0.2 pp58 of 2,529
Forms an agent can fill98.4%= +0.8 pp0 of 1
Valid tool schemas98.8%= +1.8 pp0 of 1