Speech API Wrapper with Document Text Extraction

This is a wrapper API for the LemonFox AI speech generation service with usage tracking, rate limiting, and document text extraction capabilities. It simplifies the interface by only exposing the essential parameters while handling authentication, usage tracking, and configuration internally.

• Authentication: RevenueCat subscription verification + admin bypass
• Multiple Subscription Tiers: Plus (900k/month) and Pro (2.7M/month) plans
• Non-Expiring Credit Packs: Purchase one-off credit packs (25k, 100k, 500k characters)
• Dual-Bucket Credit System: Subscription credits expire monthly, purchased credits never expire
• Usage Tracking: Comprehensive ledger-based tracking with SQLite storage
• Document Text Extraction: Extract clean text from .txt, .rtf, .docx, .md, .pdf, and .epub files for TTS
• URL Content Extraction: Extract readable content from web pages using reader view algorithms
• Modular Architecture: Separated middleware and database modules

├── backend/
│   ├── index.ts              # Main Hono app with webhook endpoints
│   ├── middleware/
│   │   ├── auth.ts          # Authentication & plan detection
│   │   └── usage.ts         # Dual-bucket usage tracking
│   ├── database/
│   │   ├── usage.ts         # Monthly usage tracking
│   │   └── credits.ts       # Non-expiring credits ledger
│   ├── plans/
│   │   └── config.ts        # Plan & credit pack configuration
│   └── README.md
├── main.tsx                  # Frontend (if applicable)
└── README.md

1. LEMONFOX_API_KEY - Your LemonFox API key
2. REVENUECAT_API_KEY - Your RevenueCat API key for subscription verification
3. ADMIN_ACCESS_KEY - Admin bypass key for development/testing
4. PDF_PARSER_API_KEY - Admin key for the primary OCR PDF parser
5. PDFVECTOR_API_KEY - Your PDFVector API key (fallback)
6. REVENUECAT_WEBHOOK_AUTH - Authentication key for RevenueCat webhook endpoint

Update the REVENUECAT_PROJECT_ID constant in /backend/middleware/auth.ts with your actual RevenueCat project ID.

All API endpoints (except /health) require authentication via the Authorization header:

Authorization: Bearer YOUR_ADMIN_ACCESS_KEY

Authorization: Bearer CUSTOMER_ID

or

Authorization: Customer CUSTOMER_ID

The API will verify the customer has active entitlements via RevenueCat before allowing access.

• Plus Plan: 900,000 characters per month
• Pro Plan: 2,700,000 characters per month
• Reset Period: Calendar month (1st to last day of month)

• Initial Free Grant: 45,000 characters for all users
• Credit Packs: 25k, 100k, or 500k character packs available for purchase
• Never Expire: Purchased credits last forever

1. Subscription credits are used first (if available)
2. Non-expiring credits are used when subscription is exhausted or unavailable
3. Requests are rejected when both buckets are empty

• Admin Users: Unlimited usage
• Tracking: Automatic dual-bucket allocation and recording

Generates speech from text using the LemonFox AI service.

Request Body:

{
  "voice": "sarah",
  "input": "Text to convert to speech"
}

Response:

{
  "audio": "_base_64_encoded_audio_here", 
  "word_timestamps": [
    {
      "word": "Hello!", 
      "start": 0.275, 
      "end": 0.7
    }
  ]
}

Error Response (Rate Limited):

{
  "error": "Character limit exceeded",
  "details": {
    "monthly_limit": 900000,
    "current_monthly_usage": 900000,
    "requested_characters": 10000,
    "remaining_characters": 5000,
    "user_tier": "plus"
  }
}

Extracts clean text content from uploaded documents (.txt, .rtf, .docx, .md, .pdf, .epub) or from web URLs for TTS processing.

Request:

• Method: POST
• Content-Type: multipart/form-data
• Body: Form data with 'file' field containing the document

Supported File Types:

• .txt - Plain text files
• .rtf - Rich Text Format files
• .docx - Microsoft Word documents
• .md - Markdown files
• .pdf - PDF documents
• .epub - EPUB eBook files

File Size Limit: 10MB

Processing Details:

• Markdown files: Processed locally using the remove-markdown package
  • Strips all Markdown formatting (headers, links, lists, etc.)
  • Preserves alt text from images
  • Supports GitHub-Flavored Markdown
• PDF files: Processed via a primary OCR endpoint with cleanup; falls back to PDFVector if OCR fails
  • Primary: PDF_PARSER_API_KEY authorizes the OCR endpoint with cleanup rules optimized for TTS
  • Fallback: PDFVector API (PDFVECTOR_API_KEY) is used if OCR fails
  • Returns clean text suitable for TTS
• EPUB files: Parsed with epubix, converted to readable text via html-to-text, and backed by a linkedom DOMParser polyfill so extraction works in the Deno runtime

Request:

• Method: POST
• Content-Type: application/json
• Body: JSON object with URL

{
  "url": "https://example.com/article"
}

URL Requirements:

• Must be HTTP or HTTPS protocol
• Must be publicly accessible
• Should contain article-like content for best results

Success Response:

{
  "text": "Extracted clean text content suitable for TTS processing...",
  "filename": "document.pdf",
  "fileType": "pdf",
  "wordCount": 1250,
  "characterCount": 6890
}

For URL extraction, fileType will be "url" and filename will be the page title or domain name.

Error Responses:

{
  "error": "PDF processing error: <details>"
}

{
  "error": "PDF processing error: Document processing timed out",
  "code": "timeout-error"
}

{
  "error": "Failed to fetch URL: 404 Not Found"
}

{
  "error": "Invalid URL format"
}

Text Processing:

• Removes excessive whitespace and formatting artifacts
• Strips common header/footer patterns and page numbers
• Optimizes text flow for natural TTS reading
• Handles complex document structures (tables, lists, etc.)
• For URLs: Extracts main article content using reader view algorithms
• For Markdown: Converts to plain text while preserving readability
• For PDFs: Uses AI when needed to handle complex layouts and maintain reading order

Get usage statistics for the authenticated customer. Response includes legacy top-level fields for compatibility plus new credits field.

Plus/Pro User Response:

{
  "user_tier": "premium",
  "monthly_limit": 900000,
  "current_usage": 150000,
  "remaining_characters": 750000,
  "usage_percentage": 17,
  "reset_date": "2024-02-01T00:00:00.000Z",
  "plan_key": "plus",
  "credits": {
    "subscription": {
      "plan_key": "plus",
      "monthly_limit": 900000,
      "current_usage": 150000,
      "remaining_characters": 750000,
      "usage_percentage": 17,
      "reset_date": "2024-02-01T00:00:00.000Z"
    },
    "non_expiring_tokens": {
      "balance": 70000,
      "total_granted": 95000,
      "total_consumed": 25000,
      "purchases": [
        {
          "source": "free_grant",
          "characters": 45000,
          "price_usd": 0
        },
        {
          "source": "iap",
          "product_lookup_key": "credit_pack_1hr",
          "characters": 25000,
          "price_usd": 2.99
        }
      ]
    }
  }
}

Free User Response:

{
  "user_tier": "free",
  "lifetime_limit": 45000,
  "current_usage": 5000,
  "remaining_characters": 40000,
  "usage_percentage": 11,
  "message": "Subscribe for monthly credits or purchase additional credits.",
  "credits": {
    "subscription": null,
    "non_expiring_tokens": {
      "balance": 40000,
      "total_granted": 45000,
      "total_consumed": 5000,
      "purchases": [
        {
          "source": "free_grant",
          "characters": 45000,
          "price_usd": 0
        }
      ]
    }
  }
}

Admin User Response (no query parameter):

{
  "message": "Admin users have unlimited usage",
  "is_admin": true
}

Admin Query Parameter:

GET /api/usage?customer_id=specific_customer_id

Allows admins to check usage for any specific customer ID.

Health check endpoint that returns the service status.

Response:

{
  "status": "ok",
  "timestamp": "2024-01-01T00:00:00.000Z"
}

Webhook endpoint for RevenueCat to notify about purchases and refunds.

Headers:

• Authorization: {REVENUECAT_WEBHOOK_AUTH} (must match environment variable)

Handled Events:

• NON_RENEWING_PURCHASE - Grants credits for credit pack purchases
• REFUND - Deducts credits for refunded purchases

Response:

{
  "success": true
}

Root endpoint that provides basic API information.

Response:

{
  "message": "Text Extraction API",
  "version": "1.0.0",
  "endpoints": {
    "health": "/health",
    "speech": "/api/speech",
    "extractText": "/api/extract-text",
    "usage": "/api/usage"
  },
  "supportedFileTypes": ["txt", "rtf", "docx", "md", "pdf"],
  "timestamp": "2024-01-01T00:00:00.000Z"
}

The system uses SQLite with multiple tables for comprehensive tracking:

Tracks all usage (both subscription and non-expiring):

• id - Auto-incrementing primary key
• customer_id - Customer identifier from RevenueCat
• character_count - Number of characters in the request
• request_timestamp - ISO timestamp of the request
• created_at - Database insertion timestamp

Ledger of all credit additions:

• id - Auto-incrementing primary key
• customer_id - Customer identifier
• source - Type of grant (free_grant, iap, admin, refund)
• product_lookup_key - RevenueCat product ID for purchases
• revenuecat_tx_id - Transaction ID for idempotency
• characters_granted - Number of characters granted (negative for refunds)
• price_usd - Price paid for the credits
• granted_at - Timestamp of grant

Ledger of non-expiring credit consumption:

• id - Auto-incrementing primary key
• customer_id - Customer identifier
• usage_id - Foreign key to customer_usage_v1
• characters_consumed - Number of non-expiring characters used
• consumed_at - Timestamp of consumption

The API automatically configures:

• response_format: Always set to "mp3"
• word_timestamps: Always set to true

Only voice and input parameters need to be provided by the client.

• Used for PDF text extraction
• Pricing: 1-2 credits per page depending on AI usage
• AI Enhancement: Automatically enabled for complex layouts
• Timeout: 3 minutes maximum per document
• Supported formats: PDF and Word documents

• 400 - Bad Request (missing required fields, invalid file type)
• 401 - Unauthorized (missing/invalid auth header)
• 403 - Forbidden (no active subscription)
• 429 - Too Many Requests (monthly limit exceeded)
• 500 - Internal Server Error

• hono - Web framework
• mammoth - DOCX text extraction
• rtf-parser - RTF text extraction
• remove-markdown - Markdown to plain text conversion
• pdfvector - PDF text extraction via API
• @extractus/article-extractor - Web page content extraction