Why LLMs read markdown better than HTML

If you are feeding web pages to a language model, the first decision you make is what to send it. Most pipelines send the HTML, because that is what the fetch returned. It is an expensive default.

Here is the same content measured both ways, on three real pages:

The Wikipedia article is the one worth sitting with. Roughly 245,000 characters of HTML carry about 25,000 characters of article. At the usual four-characters-per-token rule of thumb, that is around 61,000 tokens against 6,400. Nine tenths of what you paid to send was not the page.

What the other nine tenths are

Not padding, exactly. Every byte does a job in a browser. None of those jobs is answering your question.

• Attributes. Class lists, data attributes, ARIA roles, inline styles. On a modern site a single <div> routinely carries more characters of attribute than of text.
• Navigation. Menus, sidebars, breadcrumbs, footers, language switchers. Repeated identically on every page of the site, so if you crawl a thousand pages you send it a thousand times.
• Scripts and styles. Inline JSON blobs, analytics snippets, consent-manager configuration. Occasionally kilobytes of base64.
• Chrome. Cookie banners, newsletter prompts, related-article widgets, social buttons.

Three costs, not one

The obvious cost is money, and it is the least interesting of the three.

Context. Whatever your window is, boilerplate competes with content for it. Send raw HTML and one Wikipedia article can fill a context that would have held ten of them extracted.

Attention. A model attends over everything you give it. Repeated navigation, cookie text and script fragments are not neutral filler; they are tokens that look like content and are not. This is the failure people describe as the model “quoting the menu”.

Retrieval. If you are chunking for RAG, boilerplate gets embedded too. A chunk that is half footer produces an embedding that is half footer, and it will match queries it has nothing to say about. More on that in RAG quality starts at extraction.

Why markdown specifically

Plain text would also be small. Markdown is the better target because it is small and keeps the structure, and the structure is information the model can use.

HTML      <h2 class="section-title" id="s-3" data-track="hdr">Pricing</h2>
plain     Pricing
markdown  ## Pricing

The plain-text version has thrown away the fact that this is a heading, and therefore that what follows belongs under it. The markdown version costs three characters to keep that. Models have seen enormous quantities of markdown in training, so the convention is one they already read fluently.

That argument goes further than headings. Why models prefer structured content takes it apart properly.

What good extraction has to get right

“Strip the tags” is not extraction. A regex over HTML gets you the navigation text too, and turns tables into a run-on sentence. The things that separate a usable result from a mess:

• Finding the content. Which subtree is the article and which is furniture, without a hand-written rule per site.
• Keeping code intact. Whitespace in a <pre> is meaningful. Collapse it and you have destroyed the sample.
• Tables as tables. The single most common way extracted content becomes useless.
• Resolving links. Relative hrefs are worthless once the page is out of its origin.

Keep the metadata separately

Title, author, publication date and canonical URL are usually in the HTML as JSON-LD or OpenGraph tags, and they are worth pulling out before you discard the markup. Not in the prose, where the model has to infer them, but as fields alongside it.

They are what lets you cite a source, filter retrieval by recency, or deduplicate a crawl. Throwing them away with the markup is the quietest mistake in this whole pipeline.