Scraping crates.io
crates.io requires Discovered API (Tier -1) because the site uses internal json api replay. Cheaper engine tiers fail; this is the cheapest one that actually works.
Why Discovered API wins on crates.io
DreamScrape captured this site's internal JSON endpoint during a previous browser scrape. Future requests replay the API directly — same data, HTTP speed, 1 credit instead of 10.
Cost math: at 1 credit per request, scraping crates.io costs $0.10–$0.18 per 1,000 requests on the Starter tier. Compare to ScrapingBee ($1.40/1K) or Firecrawl (~$5.33/1K flat). For high-volume workloads on this domain, the credit-based model lands cheaper.
API Discovery: we've captured 1 internal JSON endpoint on this domain during browser scrapes. When you scrape crates.io through DreamScrape, the router checks for these endpoints first and replays them at HTTP speed (1 credit) instead of firing a full browser session. On domains with discovered APIs, this is typically 10× cheaper and 15× faster than the primary tier.
Try crates.io in the playground
10 free requests per day, no signup. The router picks the engine — you get clean markdown back.
Other domains we route through Discovered API
Sites with similar protection profiles. Each link goes to its own intel page with real production routing data.