Writing · Data
We measured 16 sites. 77% of the HTML was not the page.
Everyone knows web pages carry a lot of markup. Almost nobody has a number for it, which makes it easy to shrug at. So here is one, measured rather than estimated.
Sixteen public pages, fetched once each with the cache disabled, extracted to markdown. Across the fourteen that returned content: 3,305,533 characters of HTML carrying 758,093 characters of page. 77% of what came down the wire was not the page.
The table
Sorted by how much was markup. Ratio is HTML characters per character of extracted content.
| Page | HTML | Content | Ratio |
|---|---|---|---|
| developer.mozilla.org/en-US/docs/Web/HTTP/Methods | 211,616 | 3,215 | 65.8× |
| blog.cloudflare.com | 189,743 | 5,124 | 37× |
| react.dev/learn | 265,023 | 16,055 | 16.5× |
| pypi.org/project/httpx | 121,092 | 7,562 | 16× |
| arxiv.org/abs/1706.03762 | 44,193 | 4,312 | 10.2× |
| fastapi.tiangolo.com/tutorial/first-steps | 143,606 | 14,982 | 9.6× |
| news.ycombinator.com | 34,755 | 4,108 | 8.5× |
| docs.python.org/3/library/asyncio.html | 25,455 | 3,739 | 6.8× |
| go.dev/doc/tutorial/getting-started | 35,393 | 5,602 | 6.3× |
| en.wikipedia.org/wiki/Large_language_model | 1,067,563 | 171,101 | 6.2× |
| example.com | 559 | 113 | 4.9× |
| peps.python.org/pep-0008 | 121,843 | 48,087 | 2.5× |
| gnu.org/software/bash/manual/bash.html | 1,040,953 | 470,527 | 2.2× |
| httpbin.org/html | 3,739 | 3,566 | 1× |
Median 7.7×. Five of fourteen were 10× or worse. The spread matters more than the average, and the two ends of it are the interesting part.
The worst case is a documentation site
MDN's HTTP methods page is 211,616 characters of HTML around 3,215 characters of reference content. Sixty-six to one.
This is not a criticism of MDN, which is an excellent site to read. It is a large sidebar of every HTTP topic, a language switcher, a version banner, inline SVG icons, and an interactive example widget, all of which serve a human reader and none of which serve a model asking what PATCH does.
The pattern is general: the better a documentation site is at navigation, the worse its content ratio. And documentation is exactly what people feed to retrieval systems.
The best case is a page with nothing on it
httpbin.org/html came in at 1.0×. It is a deliberately minimal test page containing an excerpt of Moby-Dick and essentially no markup. There was nothing to strip.
The GNU Bash manual is 2.2× for a related reason: it is a single enormous document of nearly pure prose, generated by Texinfo, with no framework and no navigation chrome to speak of. Its 470,527 characters of content are genuinely content.
So the ratio is not a law of nature. It is a measure of how much application a page has been wrapped in, and it varies by more than an order of magnitude across ordinary sites.
Two pages we could not measure
Reporting these because leaving them out would flatter the result.
w3.org/TR/html52/ returned 403 with the title “Just a moment...”. That is a bot challenge, not a page. It would have scored 147×, which would have been a lie: we measured an interstitial. One in sixteen public URLs handed back a challenge instead of content, which is its own useful number.
curl.se/docs/manpage.html failed on a malformed URL in the page's own markup, where a port could not be parsed. Real HTML is full of things like this, and a scraper that falls over on one is a scraper that fails on real inputs.
What it costs
At the usual four-characters-per-token estimate, sending the raw HTML of these fourteen pages is roughly 826,000 tokens. Sending the extracted content is roughly 190,000. The difference is about 637,000 tokens of navigation, attributes and scripts.
That is fourteen pages. A modest documentation crawl is a few thousand, and the boilerplate is largely the same on every one of them, so you pay for the same sidebar a few thousand times.
The bill is the least of it. The retrieval consequences are worse.
None of them needed a browser
Worth noting, because it cuts against the assumption that modern sites all require rendering: not one of the fourteen was a JavaScript shell. React's own documentation site is server-rendered and extracted fine at 16.5×.
Rendering everything by default would have made this survey slower by two orders of magnitude and changed none of the results. That is why detecting shells beats rendering blindly.
Method
One fetch per URL on 31 July 2026, cache disabled, no rendering forced, robots.txt respected. HTML length is the raw response body; content length is the extracted markdown. Both in characters, not bytes, so multi-byte characters do not inflate one side. Token figures use the four-characters-per-token approximation for English and are indicative rather than exact for any particular tokenizer.
Every number above is reproducible against the live sites, though the sites themselves change. Ratios drift as pages get redesigned; the shape of the result has been stable every time we have run it.
snakereach turns any URL into the shape described here: markdown first, structure preserved, with per-stage timings. Try it on a page of your own, or read what comes back.