• Blog
  • Docs
  • Pricing
  • We’re hiring!
Log inSign up
dl4senses

dl4senses

x_thread_archiver

Public
Like
x_thread_archiver
Home
Code
9
src
5
.vtignore
AGENTS.md
DATABASE_ERROR_HANDLING.md
README.md
UI_IMPROVEMENTS.md
deno.json
H
main.ts
test_db.ts
Branches
1
Pull requests
Remixes
History
Environment variables
2
Val Town is a collaborative website to build and scale JavaScript apps.
Deploy APIs, crons, & store data – all from the browser, and deployed in milliseconds.
Sign up now
Code
/
README.md
Code
/
README.md
Search
…
README.md

X Thread Archiver

Archive Twitter/X threads with Grok AI and optional OCR for images.

Features

  • Thread Fetching: Uses xAI's Grok API to fetch complete X/Twitter threads
  • OCR Processing: Extract text from tweet images using Amazon Nova via OpenRouter
  • Markdown Export: Generate markdown files with YAML frontmatter and save to blob storage
  • Owner-Only Filtering: Option to exclude replies and show only thread author's tweets
  • Interactive Web UI: Dashboard with comprehensive visual feedback
    • Real-time loading states with animated spinners
    • Clear success/error messages with emojis
    • Auto-scrolling to messages for visibility
    • Button disabling during operations (prevents double-submission)
    • Smooth fade-in animations for all dynamic content
    • Auto-dismiss success messages (5 seconds)
  • RESTful API: Programmatic access for archiving and retrieving threads

Project Structure

individual_vals/x_thread_archiver/
β”œβ”€β”€ main.ts              # HTTP handler + Web UI
β”œβ”€β”€ src/
β”‚   β”œβ”€β”€ types.ts         # TypeScript interfaces
β”‚   β”œβ”€β”€ database.ts      # SQLite operations
β”‚   β”œβ”€β”€ grok_client.ts   # Grok API for thread fetching
β”‚   β”œβ”€β”€ ocr_client.ts    # Amazon Nova OCR via OpenRouter
β”‚   └── markdown.ts      # Markdown generation and frontmatter
β”œβ”€β”€ deno.json           # Deno configuration
└── README.md           # This file

Environment Variables

Set these in Val Town Dashboard β†’ Settings β†’ Environment Variables:

  1. XAI_API_KEY - xAI API key for Grok (required)
  2. OPENROUTER_API_KEY - OpenRouter API key for Amazon Nova OCR (required for OCR)
  3. X_THREAD_ARCHIVER_PASSWORD - Password for dual authentication (optional)

Deployment

Using Val Town CLI

# Navigate to project directory cd individual_vals/x_thread_archiver # Initialize val (if not already done) vt init # Watch for changes and auto-deploy vt watch # Or manually push vt push

Usage

Web Interface

  1. Open the val URL in your browser
  2. Enter a Twitter/X thread URL
  3. Optionally enable owner-only filtering
  4. Optionally enable OCR for images
  5. Click "Archive Thread"

API Endpoints

Archive a thread (POST /)

curl -X POST https://your-val.web.val.run \ -H "Content-Type: application/json" \ -H "X-API-Key: your-password" \ -d '{ "url": "https://x.com/username/status/123456789", "ownerOnly": true, "enableOCR": true, "ocrModel": "amazon/nova-2-lite-v1:free" }'

List archived threads (GET /?list)

curl https://your-val.web.val.run?list=true \ -H "X-API-Key: your-password"

Get specific thread (GET /?thread=ID)

curl https://your-val.web.val.run?thread=1 \ -H "X-API-Key: your-password"

Download markdown (GET /?download=BLOB_KEY)

curl https://your-val.web.val.run?download=x-threads/user_123_2025.md \ -H "X-API-Key: your-password"

Delete thread (GET /?delete=ID)

curl https://your-val.web.val.run?delete=1 \ -H "X-API-Key: your-password"

Authentication

Supports dual authentication:

  • X-API-Key header: For API/script access
  • Basic Auth: For browser access (username: admin, password: env var)

Database Schema

x_threads

  • Stores thread metadata (author, tweet count, settings, blob key)

x_tweets

  • Stores individual tweets with full data including images and engagement metrics

Markdown Format

Generated markdown includes:

  • YAML frontmatter with metadata
  • Formatted tweets with engagement metrics
  • Embedded images with optional OCR text
  • Links back to original tweets

Technical Details

Grok API Integration

  • Model: grok-4-1-fast-reasoning for thread fetching
  • Endpoint: /responses (supports tools like x_search)
  • Tool: x_search for accessing X/Twitter data
  • Test Function: Uses simple /chat/completions endpoint for fast API key validation

OCR Processing

  • Provider: OpenRouter API
  • Models: amazon/nova-2-lite-v1:free (default) or amazon/nova-2-pro-v1:free
  • Format: Images converted to JPEG base64 for compatibility
  • Rate Limiting: 1 second delay between images, 3 retry attempts

Development

# Run locally deno run --allow-all main.ts # Watch mode deno task dev

Based On

This Val is a TypeScript/Deno port of the Python script try_grok_nova.py, adapted for the Val Town platform.

License

MIT

FeaturesVersion controlCode intelligenceCLIMCP
Use cases
TeamsAI agentsSlackGTM
DocsShowcaseTemplatesNewestTrendingAPI examplesNPM packages
PricingNewsletterBlogAboutCareers
We’re hiring!
Brandhi@val.townStatus
X (Twitter)
Discord community
GitHub discussions
YouTube channel
Bluesky
Open Source Pledge
Terms of usePrivacy policyAbuse contact
Β© 2025 Val Town, Inc.