Private SSE - Scrapest

The private SSE stream requires a token and delivers full SourceEvent payloads — no truncation. Generate a token via /stream/token, then connect.

Setup

Generate a token

curl -X POST https://scrape.st/stream/token \
  -H "x-api-key: YOUR_API_KEY"

Response:

{
  "token": "a1b2c3d4...64_char_hex",
  "expiresIn": 86400
}

Tokens are valid for 24 hours.

Connect to the stream

GET https://scrape.st/stream?token=YOUR_TOKEN

Connection

const token = "YOUR_TOKEN";
const es = new EventSource(`https://scrape.st/stream?token=${token}`);

es.onmessage = (event) => {
  const data = JSON.parse(event.data);
  console.log(`[${data.source}] ${data.mid}`, data.payload);
};

es.onerror = () => {
  console.log("Connection lost — auto-reconnecting...");
};

Query Parameters

Parameter	Type	Description
`token`	string	Required. Token from `/stream/token`
`useFastX`	boolean	Include `fast-x` push events
`ignoreFullPayload`	boolean	Skip enriched `x` payloads, keep fast-x + others

Response Format

Full SourceEvent payloads (same as webhooks and WebSocket):

data: {"mid":"1886493083207827456","sid":"44196397","source":"x","timestamp":1712072400000,"payload":{"id":"1886493083207827456","text":"Bitcoin hits new ATH!","author":{"id":"44196397","name":"Elon Musk","screen_name":"elonmusk"},...}}

Auto-Reconnect & Buffer

EventSource automatically reconnects on connection loss
Server sends keepalive comments (:) every 15 seconds
Last 5 events are buffered and replayed on reconnection

vs Public SSE

Feature	Public SSE	Private SSE
Auth	None	Token required
Payload	Truncated (100 chars)	Full `SourceEvent`
X	fast-x only	fast-x + enriched
Use case	Demos, monitoring	Production integrations

​Setup

​Connection

​Query Parameters

​Response Format

​Auto-Reconnect & Buffer

​vs Public SSE