Skip to main content

​
Private Server Sent Events

Private SSE provides access to complete, untruncated Twitter/X data through token-based authentication for enhanced privacy and control.

​
Overview

Private SSE offers:
  • Full data payloads without truncation
  • Token-based authentication for secure access
  • Real-time streaming with automatic reconnection
  • Enhanced filtering capabilities

​
Authentication

Private SSE requires an access token generated via the token endpoint. See Generate SSE Token for details.

​
Connection Setup

​
Endpoint

https://scrape.st/sse/private?token=YOUR_TOKEN

​
Connection Example

const eventSource = new EventSource("https://scrape.st/sse/private?token=YOUR_ACCESS_TOKEN");

eventSource.onmessage = function (event) {
  const data = JSON.parse(event.data);
  console.log("Received:", data);
};

eventSource.onerror = function (error) {
  console.error("SSE Error:", error);
  // Implement reconnection logic
};

​
Data Format

Private SSE provides complete, untruncated data: 
{
  "type": "tweet",
  "data": {
    "id": "1234567890123456789",
    "text": "This is the complete tweet text without any truncation or limits on character count",
    "user": {
      "id": "987654321",
      "username": "fullusername",
      "name": "Complete Display Name",
      "verified": true,
      "followers_count": 15000,
      "following_count": 500
    },
    "created_at": "2024-01-01T12:00:00Z",
    "metrics": {
      "retweets": 150,
      "likes": 500,
      "replies": 25,
      "quotes": 10
    },
    "entities": {
      "hashtags": ["#technology", "#programming"],
      "mentions": ["@user1", "@user2"],
      "urls": ["https://example.com"]
    },
    "geo": {
      "place": "San Francisco, CA"
    }
  }
}

​
Advanced Filtering

Private SSE supports enhanced filtering options: 
// Connect with filters
const eventSource = new EventSource(
  "https://scrape.st/sse/private?token=TOKEN&keywords=javascript,programming&min_followers=1000&verified=true",
);

​
Available Filters

  • keywords: Comma-separated keywords
  • users: Comma-separated usernames
  • min_followers: Minimum follower count
  • verified: Only verified users (true/false)
  • language: Tweet language code
  • min_retweets: Minimum retweet count

​
Event Types

​
Tweet Events

eventSource.addEventListener("tweet", function (event) {
  const data = JSON.parse(event.data);
  // Handle complete tweet data
});

​
User Events

eventSource.addEventListener("user", function (event) {
  const userData = JSON.parse(event.data);
  // Handle user profile updates
});

​
Status Events

eventSource.addEventListener("status", function (event) {
  const status = JSON.parse(event.data);
  console.log("Stream status:", status.status);
});

​
Token Security

  • Token Expiration: Tokens expire after 24 hours
  • Single Use: Each token can be used for one connection
  • Rate Limiting: 10 tokens per hour per API key
  • Secure Transmission: Always use HTTPS

​
Reconnection Strategy

function connectPrivateSSE() {
  const token = getValidToken(); // Get or refresh token
  const eventSource = new EventSource(`https://scrape.st/sse/private?token=${token}`);

  eventSource.onerror = function (error) {
    if (eventSource.readyState === EventSource.CLOSED) {
      // Token expired or connection lost
      setTimeout(() => {
        connectPrivateSSE(); // Reconnect with new token
      }, 5000);
    }
  };

  return eventSource;
}

​
Rate Limits

  • 5 events per second per connection
  • Maximum 3 concurrent connections per API key
  • Automatic disconnection after 30 minutes of inactivity
  • 10 token generations per hour per API key

​
Use Cases

Private SSE is ideal for:
  • Analytics platforms requiring complete data
  • Monitoring applications needing full context
  • Enterprise integrations with data processing pipelines
  • Research applications requiring comprehensive datasets
Next: Generate SSE Token