Advanced Features

Once you’re comfortable with the basics, Scrapest offers powerful advanced features for sophisticated integrations and high-performance applications.

Real-Time Streaming

WebSocket Connections

For applications requiring instant data delivery and bidirectional communication: WebSocket Documentation

const ws = new WebSocket("wss://scrape.st/ws", [], {
  headers: { "x-api-key": "YOUR_API_KEY" },
});

ws.onmessage = (event) => {
  const data = JSON.parse(event.data);
  // Process real-time data
};

Use Cases:

Live dashboards and monitoring
Real-time analytics
Interactive applications
High-frequency trading

Server-Sent Events (SSE)

For unidirectional real-time data with automatic reconnection: Public SSE - Free, public data with privacy protection Private SSE - Full data access with token authentication

const eventSource = new EventSource(
  "https://scrape.st/stream/private?token=TOKEN",
);
eventSource.onmessage = (event) => {
  const data = JSON.parse(event.data);
  // Process streaming data
};

Keyword Filtering

Attach keywords to an API key (Keywords API) and every delivery channel (webhook, Telegram, WebSocket, SSE) only receives events whose text matches at least one keyword — case-insensitive substring match. Matching events gain a keyword_matches array:

{
  "source": "x",
  "keyword_matches": ["pepe", "airdrop"],
  "payload": { "...": "..." }
}

An API key with no keywords receives everything, unfiltered.
Filtering also applies to ocr_result events — a keyword that only appears inside an attached image still triggers a delivery.
profile_change events bypass keyword filtering.

Contract Address Detection (buy_ca)

Every delivered event carries a top-level buy_ca array with contract addresses detected in the event text, across all sources:

{
  "buy_ca": [
    {
      "network": "evm",
      "address": "0x7a250d5630B4cF539739dF2C5dAcb4c659F2488D"
    },
    {
      "network": "svm",
      "address": "EPjFWdd5AufqSSqeM2qN1xzybapC8G4wEGGkZwyTDt1v"
    }
  ]
}

Supported networks: evm, svm (Solana), tron, and the cosmos family (cosmos, juno, osmo). Empty array when nothing is detected.

Derived Events

Beyond plain source events, two derived event kinds are delivered through the same channels. They are flat objects — source is the discriminator:

`source`	What it is
`profile_change`	An X handle or profile picture change for a watched account
`ocr_result`	OCR text extracted from an image attached to a tracked post

Next Steps

Ready to dive deeper into specific features?

Streaming Documentation: Complete streaming guide
REST API Reference: Full API documentation
Health Monitoring: System monitoring and metrics
Dashboard Guide: Web interface usage

Need help with a specific advanced feature? Check our detailed documentation or contact support.

Overview

Guides

Advanced Features

Real-Time Streaming

WebSocket Connections

Server-Sent Events (SSE)

Keyword Filtering

Contract Address Detection (buy_ca)

Derived Events

Next Steps

​Real-Time Streaming

​WebSocket Connections

​Server-Sent Events (SSE)

​Keyword Filtering

​Contract Address Detection (buy_ca)

​Derived Events

​Next Steps

Real-Time Streaming

WebSocket Connections

Server-Sent Events (SSE)

Keyword Filtering

Contract Address Detection (buy_ca)

Derived Events

Next Steps