> ## Documentation Index
> Fetch the complete documentation index at: https://docs.scrape.st/llms.txt
> Use this file to discover all available pages before exploring further.

# Start Crawl

> Crawl a website and scrape every discovered page

Start a crawl over a website. The crawler discovers pages from the entry URL (and sitemaps) and scrapes each one with the same output options as [Scrape URL](/rest-api-reference/web-scrape).

Crawls are always asynchronous: the response returns a `crawl_id` immediately — poll [Crawl Status](/rest-api-reference/web-crawl-status) for progress and results, or register a `webhook` for delivery.

## Request Body

<ParamField body="url" type="string" required>
  The entry URL to start crawling from.
</ParamField>

<ParamField body="crawler" type="object">
  Crawler options: - `limit` (number, default `100`) — max pages to crawl -
  `include` / `exclude` (string\[]) — URL patterns to include or skip -
  `max_depth` (number, default `3`) — link depth from the entry URL -
  `include_entire_domain` (boolean, default `false`) — allow crawling any path
  on the entry domain - `include_subdomains` (boolean, default `false`) — allow
  crawling subdomains of the entry domain - `include_external_links` (boolean,
  default `false`) — allow crawling external links - `sitemaps` (boolean,
  default `true`) — seed from sitemap.xml
</ParamField>

<ParamField body="output" type="object">
  Output options per page — same as [Scrape
  URL](/rest-api-reference/web-scrape).
</ParamField>

<ParamField body="proxy" type="object">
  Proxy options: `location` (default `US`) and `sticky_session`.
</ParamField>

<ParamField body="request" type="object">
  Request options — same as [Scrape URL](/rest-api-reference/web-scrape).
</ParamField>

<ParamField body="js_render" type="object">
  JavaScript rendering options — same as [Scrape
  URL](/rest-api-reference/web-scrape).
</ParamField>

<ParamField body="webhook" type="object">
  Delivery webhook: `url`, optional `headers`, `events` (default `["started",
      "completed", "failed"]`).
</ParamField>

## Response

<ResponseField name="crawl_id" type="string">
  Unique ID for this crawl job. Use it with [Crawl
  Status](/rest-api-reference/web-crawl-status).
</ResponseField>

<ResponseField name="status" type="string">
  `crawling` on accept.
</ResponseField>

<RequestExample>
  ```bash theme={null}
  curl -X POST https://scrape.st/web/crawl \
    -H "x-api-key: YOUR_API_KEY" \
    -H "Content-Type: application/json" \
    -d '{
      "url": "https://example.com",
      "crawler": { "limit": 50, "max_depth": 2 },
      "output": { "formats": ["markdown"] }
    }'
  ```
</RequestExample>

<ResponseExample>
  ```json 202 theme={null}
  {
    "crawl_id": "crl_01HZX...",
    "version": "1.0.0",
    "status": "crawling"
  }
  ```
</ResponseExample>