Public
RSS feed of Paul Graham's essays
paul-grahamrssscraper
Val Town is a collaborative website to build and scale JavaScript apps.
Deploy APIs, crons, & store data – all from the browser, and deployed in milliseconds.

pg-essays-rss

An automatically-updated RSS feed of Paul Graham's essays.

Feed URL: https://pg-essays.val.run/

How it works

  • scraper.ts fetches articles.html, parses the ordered list of essays, and resolves each essay's publication date from the earliest paulgraham.com submission on the Hacker News Algolia API. The feed data is cached as a single JSON document in the val's blob storage (std/blob), which keeps it independent of the SQLite client.
  • main.ts (HTTP) serves an RSS 2.0 feed from that cache. When a request finds the cache older than an hour it re-checks the index (cheap) and only resolves dates for newly-seen essays, so responses stay fast.
  • refresh.ts (interval, cron 0 */3 * * * UTC) keeps the index warm between requests.
  • seed.ts is a one-time backfill script — run it once to populate the database with all ~230 essays and their dates (run_file ivryb/pg-essays-rss seed.ts).

New essays are picked up the next time the index is re-checked (hourly on request, or every 3 hours via the interval), so the feed stays current without hitting Paul Graham's server on every read.

Notes

  • Publication dates are approximate (inferred from Hacker News submissions).
  • The archive contains ~230 items; some very old essays predating Hacker News may have no date.

Files

FileTypePurpose
main.tshttpServes the RSS feed at /
scraper.tsscriptIndex scraping, date resolution, caching
refresh.tsintervalWarm-up job (every 3 hours)
seed.tsscriptOne-time backfill