Public
Like
rss-generator
Val Town is a collaborative website to build and scale JavaScript apps.
Deploy APIs, crons, & store data β all from the browser, and deployed in milliseconds.
Viewing readonly version of main branch: v85View latest version
A RSS Feed Generator that creates RSS feeds for blogs that don't provide their own.
- JSON Configuration: Define blogs and scraping rules in
/blogs.json - Automated Scraping: Cron job runs hourly to scrape configured blogs
- RSS Generation: Converts scraped posts to valid RSS XML
- Feed Serving: Access feeds at
/rss/<slug>.xml
flowchart TD %% Configuration A[blogs.json<br/>π Blog configurations] --> B[cron_generateFeeds.ts<br/>β° Hourly cron job] %% Scraping Flow B --> C[scraper.ts<br/>π·οΈ CSS selector scraping] C --> D[rss.ts<br/>π RSS XML generation] D --> E[(Blob Storage<br/>πΎ Val Town storage)] %% HTTP API Flow F[π€ User Request] --> G[index.ts<br/>π HTTP API Server] G --> H{Request Type?} %% Different endpoints H -->|GET /| I[π Status Dashboard<br/>HTML page] H -->|GET /status| J[π JSON Status API] H -->|GET /rss/:slug.xml| K[π‘ RSS Feed XML] %% Data retrieval I --> E J --> E K --> E %% Storage details E --> L[RSS Files<br/>rss_slug.xml] E --> M[Metadata<br/>meta_slug.json] %% External sources N[π Blog Websites] --> C %% Styling classDef service fill:#e1f5fe,stroke:#01579b,stroke-width:2px classDef storage fill:#f3e5f5,stroke:#4a148c,stroke-width:2px classDef config fill:#e8f5e8,stroke:#1b5e20,stroke-width:2px classDef endpoint fill:#fff3e0,stroke:#e65100,stroke-width:2px class B,C,D,G service class E,L,M storage class A config class I,J,K endpoint
- HTTP API (
/services/index.ts): Serves RSS feeds via Hono routing - Cron Scraper (
/services/cron_generateFeeds.ts): Hourly blog scraping and RSS generation - Generic Scraper (
/services/scraper.ts): CSS selector-based blog parsing - RSS Generator (
/services/rss.ts): Converts posts to RSS XML format
- Neynar Blog:
/rss/neynar.xml- https://artivilla--442b5b3e6f9511f08f640224a6c84d84.web.val.run/rss/neynar.xml
- Add a new entry to
/blogs.jsonwith CSS selectors - The cron job will automatically pick it up on the next run
- Feed will be available at
/rss/<your-slug>.xml
-
GET /- Interactive status dashboard showing all feeds and their last successful generation times -
GET /rss/<slug>.xml- RSS feed for a specific blog -
GET /status- JSON status of all feeds with metadata
- Uses Val Town Blob Storage with single key per feed:
rss_{slug}.xml - Metadata stored as
meta_{slug}with generation timestamps and post counts - Feeds are updated only on successful scrapes
- Stale feeds are served indefinitely until next successful update