snakereach

Endpoint

Discover links.

Every link on a page, canonicalized and classified. Useful for building a crawl frontier yourself, auditing internal linking, or mapping a site before committing to a full crawl.

POST/v1/discoverAPI key

Every link on a page, canonicalized and classified as internal or external.

Body

FieldTypeDetail
url requiredstringThe page to read links from.
internal_onlybooleanReturn same-site links only. `counts` still describes the whole page, so you can see what was filtered.
request
curl https://api.snakereach.com/v1/discover \
  -H "X-API-Key: $SNAKEREACH_API_KEY" \
  -H "content-type: application/json" \
  -d '{"url": "https://example.com", "internal_only": true}'
200 response
{
  "url": "https://example.com/",
  "final_url": "https://example.com/",
  "status": 200,
  "links": [
    { "url": "https://example.com/c", "text": "Chapter C", "internal": true, "nofollow": false },
    { "url": "https://example.com/d", "text": "Chapter D", "internal": true, "nofollow": false }
  ],
  "counts": { "total": 3, "internal": 2, "external": 1, "nofollow": 0, "returned": 3 }
}

Errors

  • 403 disallowed by robots.txt or blocked address
  • 502 the page could not be fetched

Canonicalization

Links come back canonicalized: relative URLs resolved, fragments dropped, default ports removed, query parameters sorted, and known tracking parameters (utm_*, fbclid, and friends) stripped. Two links that point at the same page therefore look the same, which is what makes them safe to deduplicate.

Related

To follow those links automatically, use crawl. To enumerate a site without fetching a page at all, use sitemap.