curl -X POST https://scrape.st/web/map \
-H "x-api-key: YOUR_API_KEY" \
-H "Content-Type: application/json" \
-d '{
"url": "https://example.com",
"limit": 100
}'
{
"map_id": "map_01HZX...",
"version": "1.0.0",
"status": "completed",
"url": "https://example.com",
"data": {
"links": ["https://example.com/", "https://example.com/about"],
"total_links": 2
}
}
Web Queries
Map Site
Discover all links on a website
POST
/
web
/
map
curl -X POST https://scrape.st/web/map \
-H "x-api-key: YOUR_API_KEY" \
-H "Content-Type: application/json" \
-d '{
"url": "https://example.com",
"limit": 100
}'
{
"map_id": "map_01HZX...",
"version": "1.0.0",
"status": "completed",
"url": "https://example.com",
"data": {
"links": ["https://example.com/", "https://example.com/about"],
"total_links": 2
}
}
Map a website and return the links discovered on it. Useful for building a crawl plan or auditing site structure. Runs synchronously.
Request Body
string
required
The site URL to map.
string
Only return links containing this substring.
number
default:"5000"
Maximum number of links to return. Between
1 and 100000.boolean
default:"true"
Include links on subdomains of the target site.
boolean
default:"true"
Treat URLs that differ only by query string as the same link.
number
Max time in milliseconds to wait. Between
1000 and 600000.Response
object
links (string[]) — discovered URLs; total_links (number) — count.curl -X POST https://scrape.st/web/map \
-H "x-api-key: YOUR_API_KEY" \
-H "Content-Type: application/json" \
-d '{
"url": "https://example.com",
"limit": 100
}'
{
"map_id": "map_01HZX...",
"version": "1.0.0",
"status": "completed",
"url": "https://example.com",
"data": {
"links": ["https://example.com/", "https://example.com/about"],
"total_links": 2
}
}