gissue-rolodex
Val Town is a collaborative website to build and scale JavaScript apps.
Deploy APIs, crons, & store data β all from the browser, and deployed in milliseconds.
| name: | Layer refactor | |||||||||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| overview: | Refactor cron layers into shared, configurable primitives so each layer composes selection, gating (updated-since / stalest), and Beeminder dispatch without duplicating logic. | |||||||||||||||||||||
| todos: |
|
- Make layer behavior composable: selector (which issues), gate (when to count), and sink (Beeminder send + daily de-dupe) should be reusable.
- Keep separate cron entrypoints while sharing logic and storage helpers.
- Introduce a small βlayer pipelineβ module that exposes:
collectUpdatedIssues(...)(shared fetch + last-run handling)filterByLabels(...)(subset selection for tagged rules)gateByStalestPrefix(...)(oldest/stalest gating using snapshot table + temporal ordering)sendBeeminderDatapoints(...)(daily de-dupe + comment + dispatch)
- Keep each cron thin: define layer config (selector + gate + goal routing), then call the pipeline.
- New module:
backend/layers/pipeline.tsfor shared flow pieces. - Extend state helpers in
backend/database/lastRunState.tsto support multiple jobs and optional βlast snapshot timestampβ for stalest gating. - Reuse stalest snapshot logic in
backend/database/stalestSnapshot.tsas a generic βprefix snapshotβ keyed byjobIdto support stalest gating for multiple layers. - Refactor crons to use the pipeline:
main.cron.tsxusescollectUpdatedIssues+sendBeeminderDatapoints.only-oldest-issues.cron.tsxusescollectUpdatedIssues+gateByStalestPrefix+sendBeeminderDatapoints.third-layer.cron.tsxusescollectUpdatedIssues+filterByLabels+ optionalgateByStalestPrefix(for βonly-oldestβ on subset) +sendBeeminderDatapoints.
Rendering mermaid diagram...
- Gate composition allows βonly-oldestβ to apply to any selector (e.g., tagged subset) without duplicating query logic.
- Snapshot storage becomes keyed by
jobIdso multiple layers can persist their own βstalest prefixβ. - Stalest gating enforces temporal order using
issue.updated_at: process the stalest prefix in order and only emit a contiguous run of updated issues (if the first stale item is unchanged, stop).
- Table schema updates require new table names per Val Town rules; plan includes new tables (e.g.,
_1suffix) where needed.