← Back to home

Docs

Everything you need to start scraping with DreamScrape.

Quickstart

Get your first scrape in 30 seconds.

1. Get an API key

Contact us or use the demo key on the homepage to test.

2. Make your first request

curl

curl -X POST https://api.dreamscrape.app/scrape \
  -H "Authorization: Bearer ds_pk_your_key" \
  -H "Content-Type: application/json" \
  -d '{"url": "https://news.ycombinator.com", "renderMode": "auto"}'

Node.js

const res = await fetch('https://api.dreamscrape.app/scrape', {
  method: 'POST',
  headers: {
    'Authorization': 'Bearer ds_pk_your_key',
    'Content-Type': 'application/json',
  },
  body: JSON.stringify({
    url: 'https://news.ycombinator.com',
    renderMode: 'auto',
  }),
});

const data = await res.json();
console.log(data.markdown);       // Clean markdown
console.log(data.engineTier);     // "http", "stealth-playwright", etc.
console.log(data.timing.fetchMs); // Latency in ms

Python

import requests

res = requests.post('https://api.dreamscrape.app/scrape',
    headers={'Authorization': 'Bearer ds_pk_your_key'},
    json={'url': 'https://news.ycombinator.com', 'renderMode': 'auto'})

data = res.json()
print(data['markdown'])       # Clean markdown
print(data['engineTier'])     # Which engine tier was used
print(data['timing']['fetchMs'])  # Latency

3. Read the response

Every response includes:

  • markdown — Clean extracted content
  • engineTier — Which engine served the request ("http", "stealth-playwright", "camoufox")
  • timing.fetchMs — Request latency in milliseconds
  • metadata — Page title, description, language
  • links — All extracted links from the page

API Reference

POST /scrape

Scrape a URL and get clean markdown + metadata.

ParameterTypeDefaultDescription
urlstring(required)URL to scrape
renderMode"auto" | "http" | "browser""auto"How to fetch the page
engine"auto" | "stealth-playwright" | "camoufox"Force a specific engine
timeoutnumber30000Request timeout in ms
asyncbooleanfalseReturn 202 + job ID for polling
impersonateBrowser"chrome" | "firefox"TLS fingerprint impersonation
tlsFingerprint"ja3" | "ja4"JA3 (legacy) or JA4 (recommended)
acceptContentType"html" | "json" | "any""html"Accept JSON API responses
solveCaptchabooleanfalseAuto-solve CAPTCHAs
useProxybooleanfalseUse residential proxy
headersobjectCustom request headers
extractobjectStructured extraction schema

GET /jobs/:id

Poll for async scrape results (when async: true).

GET /usage

Get your current month's usage by engine tier.

GET /health

Check API status, engine availability, and dependency health.

Error Taxonomy

Every error includes a blockReason that tells you exactly why it failed and a suggestion with what to try next. No more guessing at 403s.

ipDatacenter IP blocked — needs residential proxy

Fix: Use useProxy: true with a residential proxy configured

stealthFingerprint detected — needs browser engine

Fix: Try engine: "stealth-playwright" or "camoufox"

captchaCAPTCHA challenge detected

Fix: Use engine: "camoufox" with solveCaptcha: true

authSite requires authentication

Fix: Pass auth cookies/headers via the headers option

rate_limitTarget site is throttling requests

Fix: Reduce request frequency or use proxy rotation

timeoutRequest timed out

Fix: Increase timeout or try a browser engine for JS-heavy sites

robots_txtDisallowed by robots.txt

Fix: Check the target's robots.txt and respect their rules

geoContent is geo-restricted

Fix: Use a proxy in the target region

paywallContent behind a paywall

Fix: Pass subscription auth headers

not_found404 — page does not exist

Fix: Verify the URL is correct

unknownUnclassified failure

Fix: Check the error message for details and contact support

Engine Tiers

DreamScrape automatically picks the cheapest engine that works for each domain. Every response includes engineTier so you always know which one was used.

HTTPhttp
~200ms$0.0001/req

Plain fetch with JA4 TLS fingerprint. Handles 70%+ of sites.

Stealth Browserstealth-playwright
~3s$0.002/req

Headless Chromium with fingerprint injection. For JS-heavy SPAs.

Anti-detect Firefoxcamoufox
~10s$0.006/req

Patched Firefox with C++ fingerprinting. For the hardest targets.

Changelog

2026-04-10

Launch

  • • API keys with per-key rate limits, quotas, and engine ACLs
  • • 5 engine tiers: HTTP, JA4, StealthPlaywright, Camoufox, Browserless
  • • Async scrape via BullMQ (POST with async: true)
  • • 11-value error taxonomy with actionable suggestions
  • • CAPTCHA detection as hard failure (no silent garbage)
  • • Live scorecard at dreamscrape.app
  • • Sports + crypto data packages

Questions? Email us — we reply fast.