Skip to main content
POST /v1/hosted/crawl, costs 1 standard credit per page actually crawled (capped by your requested max_pages and your plan’s max_pages_per_run limit).

​
Request

Fields on CrawlRequest:
FieldTypeDefaultNotes
urlstring,required, crawl start point
max_depthint2link-following depth
max_pagesint10hard cap on pages fetched
url_patternstring""restrict crawl to matching URLs
include_externalboolfalsefollow links off the start domain
use_jsboolfalserender with a browser
timeout_msint60000
curl -X POST "$SCOUT_HOSTED_BASE_URL/v1/hosted/crawl" \
  -H "Authorization: Bearer $SCOUT_HOSTED_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"url":"https://example.com","max_pages":5}'

​
Response

CrawlResponse, wrapped in hosted usage metadata: 
{
  "success": true,
  "hosted": {
    "tenant_id": "tn_..",
    "key_id": "key_..",
    "credits_charged": 5,
    "credit_type": "standard"
  },
  "crawl": {
    "success": true,
    "start_url": "https://example.com",
    "pages": [
      {
        "url": "https://example.com/about",
        "markdown": "..",
        "metadata": {"url": "https://example.com/about", "crawled_at": "2026-07-06T18:00:00Z"},
        "success": true
      }
    ],
    "total_pages": 5,
    "duration_ms": 4210
  }
}
Credits are charged for pages actually fetched, not the requested max_pages, and never exceed your plan’s per-run page limit.