snakereach

Feature

You can see where the time went.

“The scrape took 4.2 seconds” is not a fact you can act on. Was the site slow? Did it need a browser? Is the page enormous? Are you being rate limited? Every answer implies a different fix, and a single number tells you none of them.

Every snakereach response carries a per-stage breakdown instead.

What comes back

part of every response
"trace": {
  "stages": [
    { "name": "fetch",    "elapsed_ms": 22.2 },
    { "name": "parse",    "elapsed_ms": 1.3 },
    { "name": "content",  "elapsed_ms": 185.8 },
    { "name": "metadata", "elapsed_ms": 10.0 },
    { "name": "links",    "elapsed_ms": 0.1 },
    { "name": "media",    "elapsed_ms": 0.0 },
    { "name": "records",  "elapsed_ms": 0.1 },
    { "name": "combine",  "elapsed_ms": 0.0 }
  ],
  "cache_hit": false,
  "rendered": false,
  "redirects": [],
  "warnings": []
}

That is a real trace, not an illustration, captured from this project's own test suite. Nine fields, no scores, no grades. A record of what happened.

Read it as a shape

The same total can mean completely different things. Here is where the time actually goes on a static article, 219.5 ms, and almost all of it in one place:

  1. 01fetch22.2 ms
  2. 02parse1.3 ms
  3. 03content185.8 ms
  4. 04metadata10.0 ms
  5. 05links0.1 ms
  6. 06media0.0 ms
  7. 07records0.1 ms
  8. 08combine0.0 ms

Extraction dominates, and the network barely registers. Buying a faster connection would change nothing here. That is a decision the trace makes for you in one glance and a stopwatch never could.

Four shapes worth knowing

When this dominatesIt meansSo you
fetchThe target site is slow, or politeness delays are throttling you.Cannot fix it with faster hardware. Cache, or accept it.
renderThe page was a JavaScript shell and needed a browser.Know which of your targets cost 10× more than the rest.
contentA large or deeply nested page. This is CPU.Scale by core count, not by bandwidth.
Nothing, but the total is largeYou are queued or rate limited.Check rate limits, not the pipeline.

rendered is the expensive bit

Rendering happens automatically, and only when a page turns out to be a JavaScript shell. You do not ask for it, but the trace tells you when it happened, via rendered: true and a render stage that is usually the largest number present.

That is how you find out that a site you assumed was static is actually assembling its markup in the browser. Nothing else in a scraping pipeline surfaces that.

Why nobody else ships this

Because it is only possible when the pipeline is deterministic. Fixed stages, in a fixed order, doing the same work every time. You can instrument that. If “main content” is decided by a model, there is no stage to time and no honest number to report.

The trace is a side effect of the extraction being inspectable rather than a feature bolted on. That is also why the same page always produces the same document.

Try it on your own slow page

The playground renders the trace as a bar chart on its Trace tab. Run something static, then something JavaScript-heavy, and compare the shapes. It is the clearest explanation of the pipeline there is, and it takes about twenty seconds.

Full field reference in the docs.