> ## Documentation Index
> Fetch the complete documentation index at: https://docs.scout.chowmes.com/llms.txt
> Use this file to discover all available pages before exploring further.

# Crawl

> Follow links from a start URL across a small site section.

`POST /v1/hosted/crawl`, costs **1 standard credit per page actually crawled** (capped by your requested `max_pages` and your plan's `max_pages_per_run` limit).

## Request

Fields on `CrawlRequest`:

| Field              | Type   | Default | Notes                             |
| ------------------ | ------ | ------- | --------------------------------- |
| `url`              | string | ,       | required, crawl start point       |
| `max_depth`        | int    | `2`     | link-following depth              |
| `max_pages`        | int    | `10`    | hard cap on pages fetched         |
| `url_pattern`      | string | `""`    | restrict crawl to matching URLs   |
| `include_external` | bool   | `false` | follow links off the start domain |
| `use_js`           | bool   | `false` | render with a browser             |
| `timeout_ms`       | int    | `60000` |                                   |

```bash theme={null}
curl -X POST "$SCOUT_HOSTED_BASE_URL/v1/hosted/crawl" \
  -H "Authorization: Bearer $SCOUT_HOSTED_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"url":"https://example.com","max_pages":5}'
```

## Response

`CrawlResponse`, wrapped in hosted usage metadata:

```json theme={null}
{
  "success": true,
  "hosted": {
    "tenant_id": "tn_..",
    "key_id": "key_..",
    "credits_charged": 5,
    "credit_type": "standard"
  },
  "crawl": {
    "success": true,
    "start_url": "https://example.com",
    "pages": [
      {
        "url": "https://example.com/about",
        "markdown": "..",
        "metadata": {"url": "https://example.com/about", "crawled_at": "2026-07-06T18:00:00Z"},
        "success": true
      }
    ],
    "total_pages": 5,
    "duration_ms": 4210
  }
}
```

Credits are charged for pages actually fetched, not the requested `max_pages`, and never exceed your plan's per-run page limit.