Public
Up Bank mirror: bronze/silver/gold transaction pipeline
pipelinesqliteup-bank
Val Town is a collaborative website to build and scale JavaScript apps.
Deploy APIs, crons, & store data – all from the browser, and deployed in milliseconds.
A personal mirror of an Up Bank account: transactions arrive by webhook within seconds (daily sweep as backup), land raw in bronze, become typed rows in silver, and roll up into gold marts for spend and cashflow.
- Remix this val. Migrations bootstrap the schema on first run.
- Set
UP_API_TOKENin env vars (token from https://api.up.com.au). - Trigger
jobs/bronze.cron.tsa few times until the logs report backfill done. Each run resumes from its cursor. - Silver and gold fill on the next quarter-hour, or trigger their crons in order.
- In
tools/webhookSetup.tssetMODE = "register", run, paste the printed secret into env vars asUP_WEBHOOK_SECRET, thenMODE = "ping"and expect DELIVERED / HTTP 200. LeaveMODEonstatus. - Verify in the SQLite panel: bronze count matches your history, silver matches bronze minus deleted minus rejects, gold tables have rows.
- After changing
models/silverTransform.ts, runtools/silverReplay.tsto re-transform anything insilver_rejects.
| Table | Holds |
|---|---|
bronze_transactions | Latest state per id: raw JSON as last received, upserted by id, deleted_at tombstones |
bronze_transactions_log | Every observation, append-only: one row per cron page row, webhook fetch or delete event, with source and run id |
silver_transactions | Typed columns, integer cents, UTC timestamps, live rows only |
silver_rejects | Bronze ids toSilver refused, with reason and attempt count; clears itself when the id transforms cleanly |
gold_category_spend | Settled spend per day and category, rebuilt whole; durations roll up at query time |
gold_cashflow | Settled money in and out per day and account, rebuilt whole |
gold_balance | Reconstructed balance per day and account, dense day spine |
gold_income | Income per day and type (salary, interest, tax refunds), rebuilt whole |
meta | Cursors, watermarks, locks, per-job run state |
Why it is built this way: docs/adr/.