ADR 0012. Bronze keeps an append-only observation log

Status: accepted. Implemented 2026-08-25 in kevinvu184/up. Extends ADR 0002 (bronze table) and ADR 0007 (webhook tombstones).

Context

bronze_transactions was built as a latest-state mirror: the page writer upserts by id and replaces raw, so a transaction moving HELD → SETTLED overwrites the earlier payload, and a webhook fetch overwrites whatever the daily sweep last saw. That fails two things bronze is supposed to guarantee under the medallion contract: history is not preserved (there is no way to answer "what did this row look like on day X"), and there is no provenance (nothing records whether a row arrived by cron backfill, cron incremental sweep, or webhook, nor which run wrote it). first_seen_at and last_seen_at record when, not what or by whom. Two alternatives were rejected: making the mirror itself append-only (every reader — silver's watermark scan, the tombstone path — is keyed by id and would need rewriting), and storing diffs (a diff format is a second interpretation layer bronze is not supposed to have).

Decisions

A second table, insert-only, alongside the mirror

Migration 13 adds bronze_transactions_log (seq, id, seen_at, source, run_id, event_type, raw). Every observation appends one row: each row of a cron page, each webhook fetch, and each DELETED event (whose raw is the event body, labelled by event_type). The mirror is unchanged and remains what silver reads. The log's seen_at equals the last_seen_at written to the mirror in the same statement.

Log and mirror land in one transaction

The log insert is pushed into the same sqlite.batch as the mirror upsert (and, for backfill, the cursor advance), so the log can never lack an observation the mirror has, and a killed run leaves both or neither. bronzeLogInsert in core/sqliteStore.ts is the only producer of statements against the log; there is no UPDATE or DELETE against it anywhere in the codebase.

Provenance is the entry point plus its run

source is one of cron_backfill, cron_incremental, webhook. run_id is the bronze lock token for cron rows (unique per run, already in hand) and a fresh UUID per delivery for the webhook. event_type carries the webhook eventType and is NULL for cron rows.

Every observation, no dedupe

The daily 30-day sweep re-observes rows whose raw has not changed; those append too. seen_at is the fact recorded, and deduping would require the writer to read before writing. Volume at personal-account scale is tens to low hundreds of rows a day.

Seeded from the mirror

The migration copies every existing mirror row into the log with source = 'migration_13_seed', so the log starts no emptier than the mirror. Pre-migration history is still lost — this record starts the archive, it cannot recover what the upserts already discarded.

Consequences

Bronze now meets the append-only and lineage halves of its contract: the full observation history of any id is SELECT * FROM bronze_transactions_log WHERE id = ? ORDER BY seq, and every row says how it got there. A full silver rebuild still reads the mirror, unchanged. Costs: the log grows without bound (a retention rule is a future record if it matters), and the mirror is still overwritten — the immutable property lives in the log, not the primary table. Change data capture as diffs is not provided; consumers compare consecutive log rows themselves.