Everything a page
contains, normalized.

The pipeline

One path. Every page takes all four phases.

1. 01

   fetch

   Browser-realistic headers, robots.txt, per-domain politeness, retries with backoff, and a revalidating HTTP cache.

2. 02

   render

   Only when the page turns out to be a JavaScript shell. Always headless, no browser window ever opens.

3. 03

   extract

   Every harvester runs on every page: precision and density content passes, JSON-LD, OpenGraph, meta, links, images, tables, repeated structures.

4. 04

   combine

   One deterministic merge with fixed precedence. The result is a Document, markdown first, metadata complete.

There are no engines to pick, no modes to learn, and no scores to interpret. The trace on every document breaks these phases into the individual stages that ran, fetch, parse, content, metadata, links, media, records, combine. Each with the milliseconds it took.

Writing

What models actually need from a page.

Notes on the part most pipelines skip: how much of a page is content, why structure survives better than styling, and what it costs when it does not.

Or stay in the terminal

It is a CLI first.

The HTTP API wraps the same Python client the command line uses, so whichever surface you reach for, the document is identical. Script it from a shell, or call it from your application.

snakereach scrape example.com          # one URL to one document
snakereach crawl docs.example.com -d 2  # whole site to a local store
snakereach export -o corpus.jsonl       # RAG-ready output