Crawl - Scout

POST /v1/hosted/crawl, costs 1 standard credit per page actually crawled (capped by your requested max_pages and your plan’s max_pages_per_run limit).

Request

Fields on CrawlRequest:

Field	Type	Default	Notes
`url`	string	,	required, crawl start point
`max_depth`	int	`2`	link-following depth
`max_pages`	int	`10`	hard cap on pages fetched
`url_pattern`	string	`""`	restrict crawl to matching URLs
`include_external`	bool	`false`	follow links off the start domain
`use_js`	bool	`false`	render with a browser
`timeout_ms`	int	`60000`

curl -X POST "$SCOUT_HOSTED_BASE_URL/v1/hosted/crawl" \
  -H "Authorization: Bearer $SCOUT_HOSTED_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"url":"https://example.com","max_pages":5}'

Response

CrawlResponse, wrapped in hosted usage metadata:

{
  "success": true,
  "hosted": {
    "tenant_id": "tn_..",
    "key_id": "key_..",
    "credits_charged": 5,
    "credit_type": "standard"
  },
  "crawl": {
    "success": true,
    "start_url": "https://example.com",
    "pages": [
      {
        "url": "https://example.com/about",
        "markdown": "..",
        "metadata": {"url": "https://example.com/about", "crawled_at": "2026-07-06T18:00:00Z"},
        "success": true
      }
    ],
    "total_pages": 5,
    "duration_ms": 4210
  }
}

Credits are charged for pages actually fetched, not the requested max_pages, and never exceed your plan’s per-run page limit.