up

A personal mirror of an Up Bank account: transactions arrive by webhook within seconds (daily sweep as backup), land raw in bronze, become typed rows in silver, and roll up into gold marts for spend and cashflow.

Architecture: Up API feeds a webhook and a daily sweep into bronze, silver and gold, which serve analysis

Remix quickstart

  1. Remix this val. Migrations bootstrap the schema on first run.
  2. Set UP_API_TOKEN in env vars (token from https://api.up.com.au).
  3. Trigger jobs/bronze.cron.ts a few times until the logs report backfill done. Each run resumes from its cursor.
  4. Silver and gold fill on the next quarter-hour, or trigger their crons in order.
  5. In tools/webhookSetup.ts set MODE = "register", run, paste the printed secret into env vars as UP_WEBHOOK_SECRET, then MODE = "ping" and expect DELIVERED / HTTP 200. Leave MODE on status.
  6. Verify in the SQLite panel: bronze count matches your history, silver matches bronze minus deleted minus rejects, gold tables have rows.
  7. After changing models/silverTransform.ts, run tools/silverReplay.ts to re-transform anything in silver_rejects.

Tables

TableHolds
bronze_transactionsLatest state per id: raw JSON as last received, upserted by id, deleted_at tombstones
bronze_transactions_logEvery observation, append-only: one row per cron page row, webhook fetch or delete event, with source and run id
silver_transactionsTyped columns, integer cents, UTC timestamps, live rows only
silver_rejectsBronze ids toSilver refused, with reason and attempt count; clears itself when the id transforms cleanly
gold_category_spendSettled spend per day and category, rebuilt whole; durations roll up at query time
gold_cashflowSettled money in and out per day and account, rebuilt whole
gold_balanceReconstructed balance per day and account, dense day spine
gold_incomeIncome per day and type (salary, interest, tax refunds), rebuilt whole
metaCursors, watermarks, locks, per-job run state

Why it is built this way: docs/adr/.