snakereachReference
Rate limits.
Defaults for the hosted API. If your workload does not fit inside them, say so and we will look at it.
| Limit | Default | Variable |
|---|---|---|
| Requests | 10 per minute, per client IP | SNAKEREACH_API_RATE_LIMIT |
| Window | 60 seconds | SNAKEREACH_API_RATE_WINDOW |
| Concurrent requests | 8 across the deployment | SNAKEREACH_API_MAX_CONCURRENCY |
| Concurrent crawls | 4 jobs in flight | SNAKEREACH_API_MAX_JOBS |
| Crawl pages | 25 per job | SNAKEREACH_API_CRAWL_MAX_PAGES |
| Crawl depth | 2 from the seed | SNAKEREACH_API_CRAWL_MAX_DEPTH |
| Response size | 10 MB, on decoded bytes | SNAKEREACH_API_MAX_RESPONSE_BYTES |
| Request timeout | 30 seconds | SNAKEREACH_API_REQUEST_TIMEOUT |
| Job retention | 15 minutes after finishing | SNAKEREACH_API_JOB_TTL |
| Searches | 30 per hour, per client IP | SNAKEREACH_API_SEARCH_RATE_LIMIT |
| Search budget | 800 upstream calls per month | SNAKEREACH_API_SEARCH_MONTHLY_BUDGET |
| Search results | 10 per search | SNAKEREACH_API_SEARCH_MAX_COUNT |
Crawl budgets are clamped, not rejected
Asking for 10,000 pages does not fail. It gives you the deployment's ceiling. The client asks; the server decides. Check stats.crawled on the finished job for what you actually got.
Search is guarded harder
Every uncached search spends upstream quota, so /v1/search carries its own hourly per-IP limit and a monthly budget ceiling on top of the general limit. Identical queries are cached for an hour and never touch the budget. A budget at or below zero disables search rather than unlimiting it.
Counted per process
Limits are held in memory, so a horizontally scaled deployment gives a client the allowance once per instance. A shared store is the fix when that starts to matter.
Staying under
Leave use_cache on, repeated URLs are served from cache and cost you almost nothing. Batch work with a delay between requests rather than firing everything at once. And read the detail on a 429: it says exactly how long to wait.
The playground on this site is capped harder than the API, five requests a minute, because it runs on our key. That limit is not the API's.