API Documentation

The Real-Time Data Processing System provides a comprehensive REST API for event ingestion, metrics monitoring, and system management.

https://your-deployment-url.com

Currently, the API does not require authentication. In production, implement proper authentication and authorization.

Ingest a single event into the processing system.

Request Body:

{
  "type": "user_action",
  "source": "web_app",
  "data": {
    "user_id": "12345",
    "action": "click",
    "page": "/dashboard",
    "metadata": {}
  },
  "timestamp": 1640995200000,
  "id": "optional-custom-id"
}

Response:

{
  "success": true,
  "eventId": "evt_1640995200000_abc123",
  "processingTime": 15.5
}

Ingest multiple events in a single request (max 1000 events).

Request Body:

[
  {
    "type": "user_action",
    "source": "web_app",
    "data": { "user_id": "12345", "action": "click" }
  },
  {
    "type": "page_view",
    "source": "web_app", 
    "data": { "user_id": "12345", "page": "/home" }
  }
]

Response:

{
  "success": true,
  "processed": 2,
  "failed": 0,
  "totalProcessingTime": 45.2,
  "errors": []
}

Get current system metrics.

Response:

{
  "timestamp": 1640995200000,
  "throughput": {
    "eventsPerSecond": 1250.5,
    "batchesPerSecond": 12.3,
    "totalEvents": 1000000
  },
  "latency": {
    "avgProcessingTime": 15.2,
    "p50ProcessingTime": 12.0,
    "p95ProcessingTime": 45.0,
    "p99ProcessingTime": 89.0
  },
  "errors": {
    "errorRate": 0.001,
    "totalErrors": 1000,
    "errorsByType": {
      "validation_error": 500,
      "storage_error": 300,
      "processing_error": 200
    }
  },
  "system": {
    "processingInstances": 5,
    "queueDepth": 1500,
    "memoryUsage": 512,
    "cpuUsage": 65.5
  },
  "health": {
    "status": "healthy",
    "uptime": 86400000
  }
}

Get historical metrics data.

Query Parameters:

• hours (optional): Number of hours of history to retrieve (default: 1)

Get detailed system health status.

Response:

{
  "status": "healthy",
  "timestamp": 1640995200000,
  "uptime": 86400000,
  "checks": [
    {
      "name": "event_ingestion",
      "status": "pass",
      "message": "Event ingestion healthy: 1250.5 events/sec",
      "duration": 2.5,
      "timestamp": 1640995200000,
      "metadata": {
        "eventsPerSecond": 1250.5,
        "totalEvents": 1000000
      }
    }
  ],
  "summary": {
    "total": 7,
    "passing": 7,
    "warning": 0,
    "critical": 0
  }
}

Start a load test to generate synthetic events.

Request Body:

{
  "eventsPerSecond": 5000,
  "durationSeconds": 30
}

Response:

{
  "success": true,
  "message": "Load test started: 5000 events/sec for 30 seconds"
}

Scale the number of processing instances.

Request Body:

{
  "instances": 10
}

Response:

{
  "success": true,
  "message": "Scaled to 10 processing instances"
}

Reset system metrics and state.

Response:

{
  "success": true,
  "message": "System metrics and state reset"
}

Connect to receive real-time metrics updates.

Message Format:

{
  "type": "metrics",
  "data": {
    // Same structure as GET /api/metrics
  },
  "timestamp": 1640995200000
}

All endpoints return errors in the following format:

{
  "success": false,
  "error": "Error message describing what went wrong"
}

• Event ingestion: 10,000 requests per second per instance
• API endpoints: 1,000 requests per minute per IP
• WebSocket connections: 10 concurrent connections per IP

• type: String - Event type (e.g., "user_action", "page_view")
• source: String - Event source (e.g., "web_app", "mobile_app")

• id: String - Custom event ID (auto-generated if not provided)
• timestamp: Number - Unix timestamp in milliseconds (current time if not provided)
• data: Object - Event payload data

Common event types supported:

• user_action - User interactions (clicks, scrolls, etc.)
• page_view - Page navigation events
• purchase - E-commerce transactions
• signup - User registration events
• login - Authentication events
• api_call - API usage events
• system_event - System-generated events
• error_event - Error occurrences

Common event sources:

• web_app - Web application
• mobile_app - Mobile application
• api - Direct API calls
• load_test - Load testing
• system - System-generated
• external - External integrations