Skip to main content

​
Public Server Sent Events

Public SSE provides a simple way to receive real-time Twitter/X data without authentication. Data is truncated to maintain privacy.

​
Connection Setup

​
Endpoint

https://api.scrape.st/sse/public

​
Connection Example

const eventSource = new EventSource("https://api.scrape.st/sse/public");

eventSource.onmessage = function (event) {
  const data = JSON.parse(event.data);
  console.log("Received:", data);
};

eventSource.onerror = function (error) {
  console.error("SSE Error:", error);
};

​
Data Format

Public SSE provides truncated data to protect user privacy: 
{
  "type": "tweet",
  "data": {
    "id": "1234567890",
    "text": "This is a truncated tweet text...",
    "user": {
      "username": "user...",
      "name": "Display..."
    },
    "created_at": "2024-01-01T00:00:00Z",
    "metrics": {
      "retweets": 10,
      "likes": 25
    }
  }
}

​
Data Limitations

Public SSE includes the following restrictions:
  • Text: Limited to 100 characters
  • Usernames: Limited to 10 characters
  • Display Names: Limited to 15 characters
  • No sensitive information: Email, phone numbers removed
  • Rate limited: 1 event per second maximum

​
Event Types

​
Tweet Events

eventSource.addEventListener("tweet", function (event) {
  const data = JSON.parse(event.data);
  // Handle tweet data
});

​
Status Events

eventSource.addEventListener("status", function (event) {
  const status = JSON.parse(event.data);
  console.log("Stream status:", status);
});

​
Automatic Reconnection

SSE includes built-in reconnection:
  • Automatic retry on connection loss
  • Exponential backoff (1s, 2s, 4s, 8s, 15s, 30s)
  • Maximum 30 second retry interval

​
Usage Example

<!DOCTYPE html>
<html>
  <head>
    <title>Public SSE Example</title>
  </head>
  <body>
    <div id="tweets"></div>

    <script>
      const eventSource = new EventSource("https://api.scrape.st/sse/public");
      const tweetsDiv = document.getElementById("tweets");

      eventSource.onmessage = function (event) {
        const data = JSON.parse(event.data);

        if (data.type === "tweet") {
          const tweetElement = document.createElement("div");
          tweetElement.innerHTML = `
                    <h4>${data.data.user.name} (@${data.data.user.username})</h4>
                    <p>${data.data.text}</p>
                    <small>${new Date(data.data.created_at).toLocaleString()}</small>
                    <hr>
                `;
          tweetsDiv.appendChild(tweetElement);
        }
      };
    </script>
  </body>
</html>

​
Rate Limits

  • 1 event per second per connection
  • Maximum 5 concurrent connections from same IP
  • Automatic disconnection after 1 hour of inactivity

​
Use Cases

Public SSE is ideal for:
  • Public dashboards and displays
  • Demo applications
  • Testing and development
  • Real-time public monitoring
Next: Private SSE