Flexible Layer Design

• Make layer behavior composable: selector (which issues), gate (when to count), and sink (Beeminder send + daily de-dupe) should be reusable.
• Keep separate cron entrypoints while sharing logic and storage helpers.

• Introduce a small “layer pipeline” module that exposes:
  • collectUpdatedIssues(...) (shared fetch + last-run handling)
  • filterByLabels(...) (subset selection for tagged rules)
  • gateByStalestPrefix(...) (oldest/stalest gating using snapshot table + temporal ordering)
  • sendBeeminderDatapoints(...) (daily de-dupe + comment + dispatch)
• Keep each cron thin: define layer config (selector + gate + goal routing), then call the pipeline.

• New module: backend/layers/pipeline.ts for shared flow pieces.
• Extend state helpers in backend/database/lastRunState.ts to support multiple jobs and optional “last snapshot timestamp” for stalest gating.
• Reuse stalest snapshot logic in backend/database/stalestSnapshot.ts as a generic “prefix snapshot” keyed by jobId to support stalest gating for multiple layers.
• Refactor crons to use the pipeline:
  • main.cron.tsx uses collectUpdatedIssues + sendBeeminderDatapoints.
  • only-oldest-issues.cron.tsx uses collectUpdatedIssues + gateByStalestPrefix + sendBeeminderDatapoints.
  • third-layer.cron.tsx uses collectUpdatedIssues + filterByLabels + optional gateByStalestPrefix (for “only-oldest” on subset) + sendBeeminderDatapoints.

Rendering mermaid diagram...

• Gate composition allows “only-oldest” to apply to any selector (e.g., tagged subset) without duplicating query logic.
• Snapshot storage becomes keyed by jobId so multiple layers can persist their own “stalest prefix”.
• Stalest gating enforces temporal order using issue.updated_at: process the stalest prefix in order and only emit a contiguous run of updated issues (if the first stale item is unchanged, stop).

• Table schema updates require new table names per Val Town rules; plan includes new tables (e.g., _1 suffix) where needed.