Run it yourself

Python.

The HTTP API is a thin shell over a Python client. If you are already in Python, skip the network entirely, no key, no rate limit, no serialization round trip.

Install and scrape

shell

pip install snakereach

python

import asyncio
from snakereach import SnakereachClient

async def main():
    async with SnakereachClient() as client:
        doc = await client.scrape("https://example.com")
        print(doc.title)
        print(doc.markdown)

asyncio.run(main())

doc is a pydantic model with the fields on the Document page, so attribute access rather than dictionary keys, and doc.to_json() when you want the wire format.

Several at once

One client owns one connection pool, cache and politeness state, so reuse it rather than creating one per URL.

python

async with SnakereachClient() as client:
    docs = await asyncio.gather(
        *(client.scrape(url) for url in urls),
        return_exceptions=True,
    )

for url, doc in zip(urls, docs):
    if isinstance(doc, Exception):
        print(f"{url}: {doc}")
        continue
    print(f"{url}: {doc.title}")

Configuration

python

from snakereach.config import load_config

config, origins = load_config()
config.render.enabled = False        # skip rendering entirely
config.crawl.max_depth = 1

async with SnakereachClient(config) as client:
    ...

load_config() applies the same layering the CLI uses, and returns where each value came from alongside the config itself.

Crawling

python

from snakereach.crawl.crawler import Crawler
from snakereach.crawl.scope import ScopeRules
from snakereach.plugins.builtin.queue_memory import MemoryQueue

async with SnakereachClient() as client:
    crawler = Crawler(
        client,
        queue=MemoryQueue(),
        storage=None,                     # collect in memory
        scope=ScopeRules.from_config(seed, config.crawl),
        max_pages=50,
        concurrency=4,
    )
    stats = await crawler.run([seed])

print(stats.crawled, "pages")
for doc in crawler.documents:
    print(doc.url, doc.title)

Pass a storage backend instead of None to stream results into sqlite, JSONL or a filesystem tree as the crawl runs.

Errors

Everything derives from SnakereachError. The ones worth catching individually are FetchError, RobotsDisallowed, BlockedAddress and ParserUnavailable, the same conditions the API maps onto HTTP statuses.

Is there a Node SDK?

No. The engine is Python, and there is no JavaScript client, calling the HTTP API with fetch is the supported path from Node. The quickstart has a JavaScript example.