Fast X (Twitter) thread extraction using Grok's x_search API with full media support.

This Val extracts complete X/Twitter threads from any tweet URL, including all media attachments (images, videos, and GIFs). It uses xAI's Grok API with the x_search tool to intelligently fetch the entire thread authored by a specific user.

• Fast Thread Extraction - Uses grok-4-1-fast-reasoning model for quick responses
• Media Support - Captures images, videos, and GIFs from all tweets in the thread
• Smart Querying - Automatically constructs optimal search queries to fetch complete threads
• Schema Validation - Type-safe responses using Zod schemas
• URL Normalization - Accepts both x.com and twitter.com URLs
• Structured Output - Returns clean JSON with all thread data

The val includes a beautiful dark-themed web UI for easy thread extraction with comprehensive debugging tools.

Access the UI: Simply visit the val URL in your browser:

https://yourusername-xthreadwithmedia.web.val.run/

Features:

• 🎨 Dark theme - X/Twitter-inspired color scheme optimized for readability
• ⚡ Real-time loading indicator - Animated spinner with status updates during extraction
• 🐛 Debug panel - Shows API status, response codes, and raw JSON for troubleshooting
• 📱 Mobile-friendly - Responsive design that works on all devices
• 🖼️ Media preview - Inline image and video display in a responsive grid
• 🔗 Direct links - Click to view original tweets on X
• 📊 Tweet metadata - Shows username, timestamp, and tweet count
• 🎯 Auto-scroll - Automatically scrolls to results when ready
• ✨ Smart formatting - Tweet cards with numbered badges and hover effects

Debug Panel: The yellow debug panel shows:

• Status: Real-time processing status (Idle → Loading → Success/Error)
• URL: The tweet URL being extracted
• Response Status: HTTP status code from the API
• Raw Response: Expandable JSON view for troubleshooting

The debug panel automatically collapses when results are ready and can be toggled with the "Show/Hide" button.

Serves the HTML web interface when accessed without parameters.

Extract a full thread from any tweet URL (API endpoint).

Parameters:

• url (required) - X/Twitter post URL (supports both x.com and twitter.com)

Example Request:

curl "https://yourusername-xthreadwithmedia.web.val.run/?url=https://x.com/username/status/1234567890"

Response:

[
  {
    "index": 1,
    "tweet_id": "1234567890",
    "username": "example_user",
    "timestamp": "2024-12-07T12:00:00Z",
    "text": "First tweet in the thread",
    "images": ["https://pbs.twimg.com/media/..."],
    "videos": [],
    "gifs": []
  },
  {
    "index": 2,
    "tweet_id": "1234567891",
    "username": "example_user",
    "timestamp": "2024-12-07T12:01:00Z",
    "text": "Second tweet in the thread",
    "images": [],
    "videos": ["https://video.twimg.com/..."],
    "gifs": []
  }
]

Health check endpoint.

Response:

{
  "ok": true,
  "time": "2024-12-07T12:00:00.000Z"
}

1. URL Validation - Validates and normalizes the provided X/Twitter URL
2. Query Construction - Extracts username and post ID, then builds an optimized search query:

   {postId} from:{username} (conversation_id:{postId} OR filter:self_thread)
   

3. Grok API Call - Sends the query to Grok's x_search tool with structured output schema
4. Response Parsing - Parses the JSON response and validates against the Zod schema
5. Returns Thread - Returns the complete thread with all media URLs

• grok-4-1-fast-reasoning - Fast reasoning model optimized for quick responses
• max_tokens: 4096 - Ensures complete JSON responses without truncation

• zod@3 - Schema validation
• zod-to-json-schema@3 - Convert Zod schemas to JSON Schema for Grok API

• GROK_API_KEY - xAI API key (set in Val Town environment variables)

• JSON Parsing - Robust error handling with automatic retry for malformed JSON
• Response Validation - Zod schema validation ensures type-safe responses
• Timeout Protection - 120-second timeout prevents hanging requests
• Error Reporting - Detailed server-side logging with [SERVER] and [PARSE] prefixes
• Smart Querying - Optimized X search query: {postId} from:{username} (conversation_id:{postId} OR filter:self_thread)

const TweetSchema = z.object({
  index: z.number(),           // Position in thread (1-based)
  tweet_id: z.string(),        // Tweet ID
  username: z.string(),        // Author username
  timestamp: z.string(),       // ISO 8601 timestamp
  text: z.string(),            // Tweet text content
  images: z.array(z.string()), // Array of image URLs
  videos: z.array(z.string()), // Array of video URLs
  gifs: z.array(z.string()),   // Array of GIF URLs
});

• 400 Bad Request - Missing or invalid URL parameter
• 500 Internal Server Error - API errors or parsing failures
• Timeout - Requests abort after 120 seconds

grok_thread/
├── main.ts              # Main production version
├── main_golden.ts       # Golden version backup
├── main_golden_2.ts     # Golden version backup 2
├── deno.json            # Deno configuration
├── AGENTS.md            # Val Town agent guidelines
└── README.md            # This file

# Using x.com URL
curl "https://yourusername-xthreadwithmedia.web.val.run/?url=https://x.com/elonmusk/status/1234567890"

# Using twitter.com URL (also works)
curl "https://yourusername-xthreadwithmedia.web.val.run/?url=https://twitter.com/elonmusk/status/1234567890"

const response = await fetch(
  `https://yourusername-xthreadwithmedia.web.val.run/?url=${encodeURIComponent(tweetUrl)}`
);
const thread = await response.json();

console.log(`Found ${thread.length} tweets in thread`);
thread.forEach(tweet => {
  console.log(`${tweet.index}. ${tweet.text}`);
  if (tweet.images.length) console.log(`  Images: ${tweet.images.length}`);
  if (tweet.videos.length) console.log(`  Videos: ${tweet.videos.length}`);
});

• Author-Only Threads - Only extracts tweets from the original thread author (not replies from others)
• API Timeout - Requests timeout after 120 seconds
• Rate Limits - Subject to Grok API rate limits
• Media URLs - Returns media URLs as provided by X; URLs may expire

If you don't see results after extraction:

1. Check the Debug Panel - Look for error messages or response status
2. Scroll Down - Results appear below the debug panel
3. Check Browser Console - Press F12 and look for errors
4. Hard Refresh - Press Ctrl+Shift+R (Cmd+Shift+R on Mac) to clear cache

If you see "Unterminated string in JSON" errors:

• The val automatically retries with JSON cleanup
• Check vt watch terminal logs for [PARSE] messages
• Ensure max_tokens: 4096 is set in the API payload

Monitor vt watch terminal output for detailed logs:

cd individual_vals/grok_thread
vt watch

Look for these log prefixes:

• [SERVER] - Request handling and validation
• [PARSE] - JSON parsing and error recovery

500 Internal Server Error

• Check if GROK_API_KEY environment variable is set
• Verify the tweet URL is valid (x.com or twitter.com)
• Check Val Town logs for detailed error messages

Empty Thread Results

• Verify the tweet exists and is public
• Some threads may be too old or deleted
• Check if the user has protected tweets

Slow Response Times

• Grok reasoning model can take 30-60 seconds for complex threads
• Check the loading indicator is visible
• Wait for up to 120 seconds before timing out

• Grok Search Tools - Complete Grok search documentation
• xAI API Docs - Official xAI API documentation
• Val Town Docs - Val Town platform documentation

Use the /test endpoint to verify the val is running:

curl "https://yourusername-xthreadwithmedia.web.val.run/test"

This val uses vt watch for live development:

cd individual_vals/grok_thread
vt watch

Changes are automatically deployed to Val Town.

• Uses grok-4-1-fast-reasoning model which has lower costs than standard Grok models
• Each thread extraction costs tokens based on:
  • Search query complexity
  • Number of tweets in thread
  • Amount of media content
  • Reasoning tokens used

• ✨ New: Beautiful dark-themed web UI with real-time updates
• 🐛 New: Comprehensive debug panel for troubleshooting
• 🔧 Fixed: JSON truncation issue with max_tokens: 4096
• 🔧 Fixed: Robust JSON parsing with automatic error recovery
• 🎨 Enhanced: Auto-scroll to results, responsive design
• 📊 Enhanced: Detailed server-side logging with [SERVER] and [PARSE] prefixes
• ⚡ Enhanced: Smart querying for complete thread coverage
• 🖼️ Enhanced: Media support for images, videos, and GIFs

• Core thread extraction functionality
• Grok API integration with x_search tool
• Schema validation with Zod
• Basic error handling