readwise-ingester
Val Town is a collaborative website to build and scale JavaScript apps.
Deploy APIs, crons, & store data – all from the browser, and deployed in milliseconds.
readwise-http.ts
https://thesolarmonk--30563c2848d511f0b5b176b3cceeab13.web.val.run
This project syncs your Readwise shortlist articles to a SQLite database for easy access and management.
readwise-bot.ts- Standalone function version (can be used as cron)readwise-http.ts- HTTP endpoint version with web dashboard ⭐ RecommendedREADME.md- This documentation
-
Get your Readwise API token:
- Go to https://readwise.io/access_token
- Copy your access token
-
Set environment variable:
- In Val Town, add an environment variable named
READWISE_TOKEN - Set its value to your Readwise access token
- In Val Town, add an environment variable named
The readwise-http.ts file provides a web interface and API endpoints:
- Dashboard: Visit the root URL to see a simple web interface
- Sync API:
POST /sync- Triggers a sync of your shortlist - Links API:
GET /links?limit=50- Retrieves saved links from database
The readwise-bot.ts can be:
- Run manually by calling the function
- Set up as a cron job for automatic syncing
The bot creates a readwise_links table with the following fields:
id- Unique document ID from Readwiseurl- Article URLtitle- Article titleauthor- Article authorsource- Source publicationcategory- Document category (article, email, etc.)location- Current location (shortlist, archive, etc.)tags- JSON string of tagssite_name- Website nameword_count- Article word countcreated_at- When document was created in Readwiseupdated_at- When document was last updated in Readwisepublished_date- Original publication datesummary- Article summaryimage_url- Featured image URLreading_progress- Reading progress percentagefirst_opened_at- When first openedlast_opened_at- When last openedsaved_at- When saved to Readwiselast_moved_at- When last moved between locationssynced_at- When synced to our database
- ✅ Incremental sync: Only updates documents that have changed
- ✅ Pagination handling: Automatically handles large shortlists
- ✅ Error handling: Robust error handling and logging
- ✅ Duplicate prevention: Prevents duplicate entries
- ✅ Web dashboard: Simple interface to trigger syncs and view links
- ✅ SQLite compatibility: Properly handles Val Town's SQLite format
curl -X POST https://your-val-url/sync
curl https://your-val-url/links?limit=20
To run automatically, you can:
- Use the standalone
readwise-bot.tsversion and set it as a cron job - Set up an external service to call the
/syncendpoint periodically - Manually trigger syncs through the web dashboard
READWISE_TOKEN- Your Readwise API access token
- ✅ Fixed SQLite result format handling
- ✅ Improved error handling for individual document failures
- ✅ Added proper incremental sync logic
- ✅ Web dashboard now displays saved articles correctly