Readwise Mastra - Implementation Recap

Project: AI-powered Readwise Reader inbox triage and organization
Completed: 2026-02-02
Status: ✅ Fully Functional

This is a remixable Val Town app template that lets users:

1. Sync their Readwise Reader library to a local SQLite index
2. Triage documents in their inbox (move, tag, annotate)
3. Search through 10k+ docs with filters (location, category, tags, date, progress)
4. AI-Powered Plans - Generate batch organization plans using Pollinations AI
5. Projects - Create collections with automated AI rules
6. Image Generation - Generate cover images for projects (optional)

Users remix this val, paste their API tokens, and instantly get the full experience.

• Runtime: Deno (Val Town)
• Router: Hono (HTTP framework)
• Database: SQLite (Val Town std/sqlite@14-main)
• AI Provider: Pollinations (OpenAI-compatible endpoint)
• Frontend: Vanilla JS SPA (dark theme)

readwise-mastra/
├── app/
│   └── http.ts              # Main HTTP router (Hono)
├── lib/
│   ├── env.ts               # Environment config + validation
│   ├── db.ts                # SQLite schema + helpers
│   ├── readwise.ts          # Readwise Reader API client
│   ├── sync.ts              # Incremental sync runner
│   ├── llm_pollinations.ts  # Pollinations wrapper + model discovery
│   └── agent.ts             # AI plan generation + application
├── jobs/
│   └── sync/
│       └── interval.ts      # 15-min sync + daily model refresh
├── web/
│   ├── index.html           # SPA entry point
│   ├── app.js               # Frontend JS
│   └── styles.css           # Dark theme styles
├── plan.md                  # Original build spec
├── grounding.md             # Pollinations API docs
├── keyupdates.md            # Update plan details
├── tracker.md               # Implementation progress tracker
└── IMPLEMENTATION_RECAP.md  # This file

• docs - Readwise documents (id, title, author, site_name, url, category, location, reading_progress, notes, summary, tags, etc.)
• tags - Tag definitions (key, name, type)
• doc_tags - Junction table for document-tag relationships
• sync_state - Sync watermark + status (single row)

• views - Saved search views
• projects - Collections with AI rules
• ai_plans - Generated organization plans
• ai_jobs - Plan execution logs

• model_cache - Discovered Pollinations models (name, type, tools, reasoning, context_window, pricing)
• model_config - Selected primary/fallback models + last discovery timestamp

• docs(location, updated_at) - Inbox sorting
• docs(category), docs(site_name) - Filters
• doc_tags(tag_key) - Tag views
• docs(saved_at) - Date sorting

• GET /health - Health check (no auth)
• POST /api/sync/run - Trigger sync now
• POST /api/sync/reset - Reset sync state (full resync)
• GET /api/stats - Document counts + sync status
• GET /api/auth/check - Validate Readwise token

• GET /api/inbox/queue?location=new&limit=50 - List inbox items
• POST /api/docs/:id/triage - Move/tag/annotate document
• GET /api/docs/:id - Get single document

• GET /api/search?q=&tag=&category=&location=&from=&to=&progressMin=&progressMax= - Full search
• POST /api/views - Create saved view
• GET /api/views - List saved views
• DELETE /api/views/:id - Delete view

• POST /api/projects - Create project
• GET /api/projects - List projects
• POST /api/projects/:id/apply - Run AI rule, return plan

• POST /api/ai/plan - Generate organization plan
• POST /api/ai/apply - Execute plan
• GET /api/ai/plans - List recent plans
• GET /api/ai/plans/:id - Get plan details
• GET /api/ai/models (NEW) - List cached models + config
• POST /api/ai/discover (NEW) - Trigger model rediscovery

• POST /api/image/generate - Generate image from prompt (stored in blob)
• GET /api/image/models - List available image models

• POST /webhooks/readwise - Receive Readwise webhooks (triggers sync)

• GET /web/:file - Serve SPA assets (index.html, app.js, styles.css)
• GET /* - SPA fallback (serves index.html)

READWISE_TOKEN=your_readwise_token
APP_ADMIN_SECRET=your_secret_for_api_auth
POLLINATIONS_TOKEN=your_pollinations_sk_token

POLLINATIONS_BASE_URL=https://gen.pollinations.ai
POLLINATIONS_CHAT_PATH=/v1/chat/completions
POLLINATIONS_PRIMARY_MODEL=  # leave empty for auto-discovery
READWISE_WEBHOOK_SECRET=  # for webhook verification

Old (wrong): Used legacy endpoint https://text.pollinations.ai/openai with hardcoded "openai" model

New (correct):

1. Canonical endpoint: https://gen.pollinations.ai/v1/chat/completions
2. Model discovery: Fetches available models from /v1/models and /text/models
3. Smart fallbacks: Auto-retries with fallback models on error
4. Cache: Stores discovered models in SQLite for 24 hours
5. Task-based selection: Uses cheap models for simple tasks, stronger models for complex planning

• Auto-discovery: Runs daily via interval job (or manually via /api/ai/discover)
• Fallback on errors: If gemini fails, tries openai, then gemini-large
• Response metadata: Returns { text, modelUsed, fallbacksAttempted }
• Image generation: generateImage() stores images in Val Town blob storage

• Gemini code_execution: Can generate images/plots inline
• Gemini google_search: Can search the web (if using gemini-search)
• Content blocks: Handles content_blocks in responses (text, image_url, thinking)

• Uses updatedAfter watermark to fetch only changed documents
• Stores watermark in sync_state table
• Runs automatically every 15 minutes via jobs/sync/interval.ts
• Manual trigger: POST /api/sync/run

• Documents (title, author, url, category, location, progress, notes, summary, etc.)
• Tags (as key-value pairs)
• Document-tag relationships

1. Get watermark from sync_state.updated_after
2. Call Readwise /api/v3/list/?updatedAfter=<watermark>
3. Paginate through all results (handles rate limits)
4. Upsert each document + tags to SQLite
5. Update watermark to newest updated_at

1. User provides a goal (e.g., "Archive all articles older than 30 days with 0% progress")
2. Scope (optional): location, category, tag, limit
3. Fetches matching docs from SQLite
4. Sends to Pollinations with complexTask: true (uses gemini-large)
5. Returns structured JSON plan:

   {
     "items": [
       {
         "docId": "abc123",
         "patch": { "location": "archive", "tagsAdd": ["read"] },
         "rationale": "Not started, over 30 days old"
       }
     ]
   }

6. User reviews plan in UI
7. User clicks "Apply" → executes patches via Readwise API

{
  items: Array<{
    docId: string;
    patch: {
      location?: "new" | "later" | "shortlist" | "archive" | "feed";
      tagsAdd?: string[];
      tagsRemove?: string[];
      notes?: string;
      summary?: string;
      title?: string;
    };
    rationale: string;
  }>
}

• Respects Readwise Retry-After header on 429 errors
• Auto-retries with exponential backoff
• 200ms delay between patch applications

• Pure Vanilla JS (no framework)
• Dark theme (custom CSS)
• State management: Simple in-memory state object
• Auth: Bearer token stored in localStorage

1. Inbox - List by location (new, later, shortlist, archive, feed)
2. Search - Full-text + filters
3. AI Triage - Generate and apply plans
4. Projects - Collections with AI rules
5. Settings - Sync status, reset, logout

• Batch triage: Select multiple docs, move all at once
• Real-time stats: Document counts by location/category
• Toast notifications: Success/error feedback
• Modal dialogs: Create projects, review plans
• Responsive: Works on desktop and mobile

Runs every 15 minutes

1. Readwise sync: Fetch new/updated documents
2. Model discovery: Refresh model cache (once per 24 hours)

Next run: Check Val Town UI for interval status

1. No authentication UI - Users must know their APP_ADMIN_SECRET
2. No user management - Single-user per deployment
3. Basic search - No full-text search index (SQLite LIKE only)
4. No document content - Only metadata (Readwise API limitation)
5. Blob storage URLs - Generated image URLs are Val Town-specific

1. Better auth - OAuth or magic links
2. Full-text search - Use SQLite FTS5
3. Webhook processing - Handle specific events (tags_updated, etc.)
4. Batch operations - Bulk move/tag in UI
5. Saved filters - Reusable search queries
6. Project dashboards - Stats per project
7. Export - Download filtered docs as CSV/JSON
8. Multi-provider AI - Support OpenAI, Anthropic, etc.

https://kamenxrider--2325eac8ffd511f0b0a842dde27851f2.web.val.run

1. Set environment variables in Val Town:

   • READWISE_TOKEN
   • APP_ADMIN_SECRET
   • POLLINATIONS_TOKEN

2. Trigger initial sync:

   curl -X POST https://your-val.web.val.run/api/sync/run \
     -H "Authorization: Bearer YOUR_ADMIN_SECRET"

3. Trigger model discovery:

   curl -X POST https://your-val.web.val.run/api/ai/discover \
     -H "Authorization: Bearer YOUR_ADMIN_SECRET"

4. Open the UI and sign in with APP_ADMIN_SECRET

• Sync status: GET /api/stats shows last sync time + error
• Model status: GET /api/ai/models shows cached models + last discovery
• Logs: Check Val Town execution logs for errors

• Sync fails: Check READWISE_TOKEN is valid
• AI fails: Check POLLINATIONS_TOKEN is valid (sk_...)
• Model errors: Run POST /api/ai/discover to refresh cache
• UI won't load: Check APP_ADMIN_SECRET in localStorage

• Health check: GET /health returns {"ok":true}
• Sync: POST /api/sync/run starts sync
• Stats: GET /api/stats returns doc counts
• Inbox: GET /api/inbox/queue?location=new lists docs
• UI: Open in browser, sign in with admin secret

• Model discovery: POST /api/ai/discover caches models
• Model list: GET /api/ai/models shows cached models
• Plan generation: POST /api/ai/plan with goal returns plan
• Plan application: POST /api/ai/apply with planId executes patches
• Image generation: POST /api/image/generate with prompt returns blob URL

• Check interval settings: 15 minutes (900000ms)
• Check logs: Should run sync + model discovery
• Verify model discovery only runs once per day

1. app/http.ts - All HTTP routes (start here)
2. jobs/sync/interval.ts - Background jobs

3. lib/sync.ts - Readwise sync algorithm
4. lib/agent.ts - AI plan generation + application
5. lib/llm_pollinations.ts - Pollinations API wrapper + discovery
6. lib/readwise.ts - Readwise API client

7. lib/db.ts - SQLite schema + helpers
8. lib/env.ts - Environment config

9. web/app.js - SPA state + API calls
10. web/index.html - HTML entry point
11. web/styles.css - Dark theme

• plan.md - Original build specification
• grounding.md - Pollinations API documentation
• keyupdates.md - Pollinations integration plan
• tracker.md - Implementation progress (all phases complete)
• IMPLEMENTATION_RECAP.md - This file

✅ All planned features implemented

• Schema + sync ✓
• Inbox triage ✓
• AI plan generation ✓
• Image generation ✓
• Model discovery + fallbacks ✓
• SPA UI ✓

✅ Pollinations integration complete

• Correct canonical endpoint ✓
• Model discovery + caching ✓
• Smart fallback logic ✓
• Task-based model selection ✓
• Image generation with blob storage ✓

✅ Production ready

• Error handling ✓
• Rate limit handling ✓
• Authentication ✓
• Interval jobs ✓
• Logging ✓

1. All code is commented - Read inline comments for context
2. tracker.md shows progress - All 8 phases complete
3. Environment variables are documented - See "Environment Variables" section above
4. API is RESTful - Standard JSON responses
5. Frontend is vanilla JS - Easy to understand/modify

1. Check Val Town execution logs for errors
2. Use GET /api/stats to verify sync is working
3. Use GET /api/ai/models to verify model discovery
4. Use POST /api/ai/discover to force model refresh
5. Check SQLite tables directly via Val Town's sqlite tool

1. Add new endpoints: Edit app/http.ts
2. Add new tables: Edit lib/db.ts runMigrations()
3. Add new AI features: Edit lib/agent.ts
4. Change model logic: Edit lib/llm_pollinations.ts
5. Update UI: Edit web/app.js and web/styles.css

• Val Town Docs: https://docs.val.town
• Readwise Reader API: https://readwise.io/reader_api
• Pollinations API: https://github.com/pollinations/pollinations/blob/main/APIDOCS.md
• Val: https://www.val.town/x/kamenxrider/readwise-mastra

End of Implementation Recap

Last updated: 2026-02-02