claude/zealous-lovelace-cbc97f
needs attentionviewing older commit4c3d40d · fullPR #215reviewed 2026-07-03 06:05 UTC6H · 15M · 12L · 8I- Purpose
- Backstop for ADR-020's drift risk: the utility_contract_overview read-model is trigger-maintained but drift can occur via trigger-bypass writes (replica role, DISABLE TRIGGER, pg_restore, TRUNCATE). Without this, drift persists silently until the next feeder write.
- Goal
- Nightly reconcile-and-repair (pg_cron) that recomputes every overview row from source, repairs each divergent pair, logs an audit row, and makes drift observable and alertable.
- Sub-goals
- SQL reconcile function (SECURITY DEFINER + pg_cron schedule at 09:17 UTC)
- Audit log table utility_contract_overview_reconcile_runs (service_role-only RLS)
- TS wrapper runOverviewReconcile() for EventBridge/Lambda fallback path
- Integration tests covering all 3 drift kinds (missing, orphaned, mismatch)
- Independent CI drift watchdog GHA workflow checking staging + prod daily
- ADR-020 follow-up marker + migration slot re-slotted to 0053 after merge collision
- What
- Added 14 files: pg_cron-scheduled SQL reconcile function, Drizzle schema for the audit table, TS wrappers in queries.ts, integration test suite, GHA watchdog workflow, squawk lint config, and watchdog SQL query.
- Why
- ADR-020 committed to scheduling a nightly reconcile-and-repair as a follow-up item. This PR delivers that commitment, making drift observable (drift_count should always be 0) and auto-repairable.
- Areas
- packages/database+12097−2domains/utility/src/utility-contract-overview+318−0.github/workflows+121−0scripts/db+67−0docs/ADRs+6−2.squawk.toml+30−0
- Blast
- 14 files, +838/−4 (excluding 11812-line Drizzle snapshot). New pg_cron job touches utility_contract_overview and reconcile_runs tables at 09:17 UTC daily. No app deployment required — pg_cron calls the SQL function directly. Requires out-of-band pg_cron enablement on staging + prod Supabase.
Findings · 18
correctness4
runId silently returns empty string when execute() yields no rows
domains/utility/src/utility-contract-overview/utility-contract-overview.queries.ts:605
The TS wrapper uses `row?.run_id ?? ''` as the fallback. A caller receiving runId='' cannot distinguish a missed audit INSERT from normal operation. Add a guard: `if (!row) throw new Error('utility_contract_overview_reconcile() returned no rows')`.
Watchdog 40-min window risks false NO_RECENT_RUN alarm when reconcile is slow
.github/workflows/uco-drift-watchdog.yml:17
Watchdog fires 43 minutes after the pg_cron schedule. A >43-minute reconcile (plausible with 2000 rows + advisory locks during a real bypass incident) fires a false alarm at the worst possible moment. Widen to 60-90 minutes or use cron.job_run_details start_time as a secondary signal.
Reconcile holds up to 2000 advisory locks simultaneously, blocking concurrent feeder writes
packages/database/drizzle/0053_organic_changeling.sql:127
All advisory locks are transaction-level and held until the repair loop and audit INSERT complete. Every concurrent trigger write for affected contracts stalls. Low risk in normal operation (drift expected 0), but a real bypass incident + reconcile = double write stall.
TEMP TABLE snapshot is READ COMMITTED — new drift unaccounted between snapshot and repair
packages/database/drizzle/0053_organic_changeling.sql:83
Drift counts in the audit row reflect only pairs visible at snapshot time. A concurrent trigger write can create a new divergent pair after snapshot that the reconcile never sees, leaving drift_count understated.
security2
PGURL_RAW never masked — only rewritten PGURL is masked
.github/workflows/uco-drift-watchdog.yml:59
Both PGURL_RAW (port 6543) and PGURL (port 5432) differ. Only PGURL is masked. Add `echo '::add-mask::$PGURL_RAW'` before the port rewrite to prevent credential exposure under ACTIONS_STEP_DEBUG.
err.log contents could include connection strings in GitHub issue body
.github/workflows/uco-drift-watchdog.yml:89
QUERY_ERROR branch embeds err.log into DETAIL step output which ends up in the issue body. If psql emits 'could not connect to server at host:port' in err.log, partial credential information could leak before masking.
conventions5
runOverviewReconcile belongs in .shells.ts, not .queries.ts
domains/utility/src/utility-contract-overview/utility-contract-overview.queries.ts:1
runOverviewReconcile() writes to reconcile_runs, repairs overview rows, and emits pg_notify. Per FCIS ADR-016, shells.ts owns transaction boundaries and write side effects. queries.ts files are thin read wrappers. Move this function to a new utility-contract-overview.shells.ts.
runOverviewReconcile throws on error instead of returning Result<T,E>
domains/utility/src/utility-contract-overview/utility-contract-overview.queries.ts:1
FCIS guardrails: all fallible operations return Result<T,E> from @batu/result, never thrown exceptions. As a shell-level write with side effects, this should return Result<OverviewReconcileSummary, ReconcileError> so callers can exhaustively handle error cases.
No errors.ts — reconcile error cases are unmodelled
domains/utility/src/utility-contract-overview/:1
The canonical entity structure requires {entity}.errors.ts. With runOverviewReconcile added, callers cannot distinguish pg_cron scheduling failure from repair cap exceeded or DB connectivity errors without a typed discriminated union.
Double cast (as unknown as Array<...>) bypasses type safety
domains/utility/src/utility-contract-overview/utility-contract-overview.queries.ts:1
A Zod schema for the raw row shape would surface column name mismatches at runtime. A SQL function column rename would silently produce NaN via Number(undefined) with the current approach.
pgTable used instead of project's createTable wrapper
packages/database/src/schema/utility-contract-overview-reconcile.ts:1
If the project uses a custom createTable wrapper to enforce RLS or naming conventions, using raw pgTable bypasses those guardrails. .enableRLS() is called explicitly, but other wrapper invariants may be missed.
tests7
latestRun() is DB-global — can be contaminated by concurrent test suites
domains/utility/src/utility-contract-overview/__tests__/utility-contract-overview.reconcile.integration.test.ts:66
latestRun() queries reconcile_runs with ORDER BY ranAt DESC LIMIT 1 with no test-specific filter. Concurrent suites calling runOverviewReconcile() contaminate this. The missing and orphaned tests do NOT cross-reference run!.id == summary.runId.
Audit log rows not cleaned up in afterAll — accumulate on persistent DBs
domains/utility/src/utility-contract-overview/__tests__/utility-contract-overview.reconcile.integration.test.ts:139
afterAll deletes all test entities but never deletes rows written to utilityContractOverviewReconcileRuns. Each test run appends 4-5 audit rows indefinitely on preview/staging DBs, affecting latestRun() assertions in future runs.
Missing and orphaned tests do not verify run!.id == summary.runId
domains/utility/src/utility-contract-overview/__tests__/utility-contract-overview.reconcile.integration.test.ts:213
The mismatch test correctly cross-references expect(run!.id).toBe(summary.runId). The missing (lines 194-216) and orphaned (lines 218-247) tests assert on whichever run happens to be latest without verifying it's the expected one.
repairedCount never asserted to equal driftCount
domains/utility/src/utility-contract-overview/__tests__/utility-contract-overview.reconcile.integration.test.ts:177
Tests assert toBeGreaterThanOrEqual(1) separately. A regression where refresh() silently fails (repairs 0 out of 1) would not be caught. Assert summary.repairedCount === summary.driftCount explicitly.
No test for zero-drift baseline (clean-state)
domains/utility/src/utility-contract-overview/__tests__/utility-contract-overview.reconcile.integration.test.ts:249
The convergence test does not assert summary.driftCount === 0 or summary.repairedCount === 0. The zero-drift code path — correct audit row with all counts at zero, no RAISE WARNING, no pg_notify — is never directly verified.
RPU collision risk with Date.now() in parallel CI shards
domains/utility/src/utility-contract-overview/__tests__/utility-contract-overview.reconcile.integration.test.ts:43
Two parallel shards starting within the same millisecond produce identical RPU strings causing UNIQUE constraint violations in beforeAll. Combine with crypto.randomUUID() or Math.random() for a robust unique suffix.
watchdog SQL has no automated test — verdict logic untested
scripts/db/uco-drift-watchdog.sql:1
The verdict CTE logic (OK, DRIFT, RUN_FAILED, NO_RECENT_RUN, NOT_SCHEDULED, PENDING_FIRST_RUN) is only testable via 'Run workflow' manually. Edge cases have no automated coverage.
History · 8 commits
- df474d4needs attentionincremental2H · 3M · 6L2026-07-22 23:12
- 8b749afneeds attentionincremental0H · 1M · 3L2026-07-14 17:58
- 8f01e3bsafeincremental0H · 0M · 0L2026-07-07 16:58
- 6fb3b0aneeds attentionincremental0H · 1M · 3L2026-07-06 16:26
- b7dc960needs attentionincremental0H · 1M · 4L2026-07-04 02:25
- c0927dfneeds attentionincremental0H · 2M · 3L2026-07-03 06:34
- 1328ef5needs attentionincremental1H · 4M · 5L2026-07-03 06:26
- 4c3d40dneeds attentionfull6H · 15M · 12L2026-07-03 06:05current