Skip to main content
The Health section provides comprehensive monitoring and metrics for the Scrapest platform, ensuring transparency about system performance and availability.

What’s Included

System Monitoring

  • System Status: Overall platform health and availability
  • Service Health: Individual component status checks
  • Performance Metrics: Latency and throughput measurements

Data Metrics

  • Source Metrics: Performance metrics by data source
  • Internal Metrics: Platform internal processing metrics
  • Historical Data: 24-hour rolling performance windows

Health Endpoints

  • Health Checks: Automated system health verification
  • Metrics API: Programmatic access to performance data
  • Status Monitoring: Real-time system status information

Key Metrics Tracked

Latency Metrics

  • Source Latency: Time to fetch data from external sources
  • Internal Latency: Time to process and deliver data
  • Percentiles: P50, P95, P99 latency measurements

Performance Metrics

  • Request Volume: Number of requests processed
  • Success Rates: Percentage of successful operations
  • Error Rates: Frequency and types of errors

System Health

  • Service Availability: Uptime and downtime tracking
  • Resource Usage: CPU, memory, and network utilization
  • Dependency Health: Status of external services and APIs

Why Health Monitoring Matters

For Developers

  • Performance Optimization: Identify bottlenecks and optimization opportunities
  • Debugging Support: Access detailed metrics for troubleshooting
  • Capacity Planning: Understand system limits and scaling needs

For Operations

  • SLA Monitoring: Track service level agreement compliance
  • Incident Response: Quick identification of system issues
  • Preventive Maintenance: Early warning for potential problems

For Business

  • Service Reliability: Ensure consistent service delivery
  • Cost Management: Optimize resource utilization
  • Customer Satisfaction: Maintain high service quality

Accessing Health Information

Web Dashboard

  • Visit the System Status page for overview
  • Check Performance Metrics for detailed performance data
  • Review Health Endpoints for technical documentation

API Access

  • Use the Metrics API for programmatic access
  • Implement Health Checks for automated monitoring
  • Integrate Status Monitoring into your systems

Getting Started

  1. System Status: Check current platform health
  2. Metrics Overview: Understand available metrics
  3. Performance Metrics: Review current performance data
  4. Source-Specific Metrics: Analyze performance by data source
  5. Health Endpoints: Learn about API access

Data Freshness

  • Real-time Status: Updated every 30 seconds
  • Performance Metrics: 24-hour rolling window
  • Historical Data: Available for the past 24 hours
  • Latency Measurements: Continuously updated

Privacy and Security

  • Aggregate Data: All metrics are aggregated and anonymized
  • No User Data: Health metrics contain no user-specific information
  • Secure Access: Health endpoints require appropriate authentication
  • Rate Limited: Health API access is rate limited to prevent abuse
Next: System Status