curl -X GET https://scrape.st/web/crawl/crl_01HZX... \
-H "x-api-key: YOUR_API_KEY"
{
"crawl_id": "crl_01HZX...",
"version": "1.0.0",
"status": "crawling",
"completed": 12,
"total": 50
}
{
"crawl_id": "crl_01HZX...",
"version": "1.0.0",
"status": "completed",
"completed": 50,
"total": 50,
"url": "https://example.com",
"data": [
{
"markdown": "# Example Domain...",
"metadata": { "title": "Example Domain", "status_code": 200 }
}
]
}
Web Queries
Crawl Status
Poll the progress and results of a crawl job
GET
/
web
/
crawl
/
{id}
curl -X GET https://scrape.st/web/crawl/crl_01HZX... \
-H "x-api-key: YOUR_API_KEY"
{
"crawl_id": "crl_01HZX...",
"version": "1.0.0",
"status": "crawling",
"completed": 12,
"total": 50
}
{
"crawl_id": "crl_01HZX...",
"version": "1.0.0",
"status": "completed",
"completed": 50,
"total": 50,
"url": "https://example.com",
"data": [
{
"markdown": "# Example Domain...",
"metadata": { "title": "Example Domain", "status_code": 200 }
}
]
}
Poll a crawl started via Start Crawl. While running, the response reports progress (
completed / total); once finished, it includes the scraped data for every page.
Path Parameters
string
required
The
crawl_id returned when the crawl was started.Response
string
crawling, completed, or failed.number
Pages scraped so far.
number
Total pages discovered.
array
Present once completed: one scrape result per page (same shape as Scrape
URL
data).curl -X GET https://scrape.st/web/crawl/crl_01HZX... \
-H "x-api-key: YOUR_API_KEY"
{
"crawl_id": "crl_01HZX...",
"version": "1.0.0",
"status": "crawling",
"completed": 12,
"total": 50
}
{
"crawl_id": "crl_01HZX...",
"version": "1.0.0",
"status": "completed",
"completed": 50,
"total": 50,
"url": "https://example.com",
"data": [
{
"markdown": "# Example Domain...",
"metadata": { "title": "Example Domain", "status_code": 200 }
}
]
}