fix/tariff-cov
blockedviewing older commit92353bd · incrementalpre-PRreviewed 2026-08-10 19:32 UTC1H · 9M · 7L · 5I- Purpose
- Close a series of security, correctness, and test-coverage findings from 4 prior review panels on the CFE tariff scrape/persist/transition pipeline
- Goal
- Land a production-safe tariff pipeline: no silent data loss, loud failures, proper auth gates, and CI infrastructure that enforces these properties going forward
- Sub-goals
- SG-1: Fix silent data loss in CFE rate scraper (no-price guard, shrink guard)
- SG-2: Make the pipeline fail loudly on bad input (non-zero exit, error reporting)
- SG-3: Close security findings (auth gate pins, error sanitization, admin-only tariff-jobs endpoint)
- SG-4: Add CI required-check gates (pr-checks.yml, tariff-coverage-watchdog, uco-drift-watchdog)
- SG-5: Add test infrastructure to make gates testable (scripts/ci package, SQL watchdog tests)
- What
- Added test infrastructure for CI gates (scripts/ci package with classify-changed-files.sh + vitest), watchdog SQL with textual tests, updated pr-checks.yml to make unit tests required and add tariff coverage gate, e2e.yml refactored to use external classifier script, .claude/rules/ontology.md updated to track new scope, turbo.json updated for CI scripts
- Why
- The prior 4 review panels found the pipeline had critical holes: silent data loss on empty scrapes, wrong baseline comparison, unknown divisions killing sibling zones. This diff closes the CI infrastructure gap (gates were advisory-only before) and adds testability for the watchdog SQL
- Areas
- scripts/ci+330−0.github/workflows+308−22scripts/db+104−0.claude/rules+157−0turbo.json+25−1
- Blast
- 40 files total across branch (+2230/−544); incremental window: 5 areas, CI/workflow infrastructure only — no runtime domain or service code changed in these 2 commits
Findings · 22
correctness2
e2e.yml calls classify-changed-files.sh before actions/checkout — script never exists on runner
.github/workflows/e2e.yml:96
The 'Gate — does this PR touch live code?' step calls `./scripts/ci/classify-changed-files.sh` but `actions/checkout@v4` doesn't run until AFTER this step (line 132) and is itself conditional on `steps.gate.outputs.has_preview == 'true'`. The script file does not exist on the runner at call time. With `set -euo pipefail`, the subshell returns exit 127 (command not found), the step fails, and the job named 'e2e testing' — a required check — reports FAILURE for every PR with ≤300 changed files and a working GitHub API. PRs with >300 files or a GitHub API failure escape via the earlier fail-safe. Fix: add an unconditional `actions/checkout@v4` step before the scope step.
Vacuous assertion: q.findByTariffZoneAndPeriod is always undefined on the mock object
domains/utility/src/tariff-rate/__tests__/persist-scraped-rates.test.ts:182
`q` is constructed with `{ findEffectiveByTariffZoneAndPeriod, ... }` — never with `findByTariffZoneAndPeriod`. The assertion `expect(q.findByTariffZoneAndPeriod).toBeUndefined()` is a tautology. A meaningful pin is the already-present `expect(q.findEffectiveByTariffZoneAndPeriod).toHaveBeenCalled()` at line 176.
security3
PROD_POSTGRES_URL is repo-level secret — ref guard is the sole enforcement (BAT-316)
.github/workflows/tariff-coverage-watchdog.yml:55
PROD_POSTGRES_URL and STG_POSTGRES_URL are repository-level secrets, not environment-scoped. The `if: matrix.env == 'staging' || github.ref == format('refs/heads/{0}', ...)` condition is the only thing preventing a workflow_dispatch on a non-default ref from reaching the production database. A collaborator with write access can dispatch the workflow against any ref. The `environment: Production` binding is acknowledged as non-load-bearing in the comment. uco-drift-watchdog.yml has the same pattern. Fix: move PROD_POSTGRES_URL to a GitHub `Production` environment with required reviewers + main-only deployment-branch policy (tracked BAT-316).
sanitizePipelineError enforced at call site, not query layer — future callers can bypass
services/utility/tariffs/cfe/__tests__/transition.test.ts:2694
tariff_jobs has `FOR SELECT TO authenticated USING (true)` — every tenant can read `error` and `result` columns. `sanitizePipelineError` is called in the Lambda's finalize/fail path, but a future caller writing to `error` directly would silently expose infra detail to all tenants. Consider enforcing at the query layer inside `tariffJobQueries.update`.
43 RLS/auth integration tests run in zero CI gates (BAT-319)
.github/workflows/pr-checks.yml:90
The pr-checks.yml comment added in this diff correctly documents that `rls-cross-org`, `savings-config-idor`, and `site-energy-streams-rls` integration tests run in no gate. A tenant-isolation regression currently merges green. This is acknowledged debt (BAT-319), not introduced by this diff — the comment makes the gap more visible.
conventions5
ScrapedRateSkip error type lives in .decisions.ts instead of .errors.ts
domains/utility/src/tariff-rate/tariff-rate.decisions.ts:850
Canonical form places discriminated-union error types in `{entity}.errors.ts`. `ScrapedRateSkip` is the `E` side of `Result<RateComponents, ScrapedRateSkip>` — the error type for `decideScrapedRateWrite` — so it belongs in `tariff-rate.errors.ts` alongside `TariffRateNotFoundError`, `TariffRateDuplicateError`, etc. This also makes `.decisions.ts` import from `.scrape.ts` for pure helpers, which is the first such import in the codebase and sets a precedent that could erode the pure-decisions boundary.
.decisions.ts imports from sibling .scrape.ts — unprecedented in codebase
domains/utility/src/tariff-rate/tariff-rate.decisions.ts:9
This is the only `.decisions.ts` that imports from a sibling `.scrape.ts`. The `.scrape.ts` module is pure (no Drizzle, no Zod) so it doesn't violate the infrastructure-import ban, but it introduces a dependency from the business-logic layer into the transform layer that has historically lived only in shells. Alternative: move `droppedComponents` and `hasEnergyComponent` out of `.scrape.ts` into `.decisions.ts` directly to eliminate the novel cross-sibling import.
scripts/ci package missing lint script present in peer packages
scripts/ci/package.json:7
Peer package `scripts/metrics/package.json` also has no lint script, so this is consistent within the `scripts/` tier. Informational — the gap is pre-existing.
findEffectiveByTariffZoneAndPeriod naming slightly deviates from findBy{Field} convention
domains/utility/src/tariff-rate/tariff-rate.queries.ts:119
Canonical naming is `findBy{Field}`. The new function uses `findEffectiveBy{Fields}` with a semantic qualifier. Readable and self-documenting — the distinction from `findByTariffZoneAndPeriod` is meaningful. Minor deviation noted.
turbo.json test inputs changed to $TURBO_DEFAULT$ — documented tradeoff accepted
turbo.json:35
Changing inputs from explicit glob list to `$TURBO_DEFAULT$` (all git-tracked files) means README.md or `.claude/` rule changes now invalidate the turbo test cache. The comment documents this as the intended trade ('re-running tests when a README changes is the correct trade'). No convention violation.
tests6
SQL watchdog tests are textual-only — no execution proof
scripts/ci/__tests__/tariff-coverage-watchdog-sql.test.ts:1
All 10 assertions are regex/string matches on raw SQL text, not executions of the query. The file itself warns: 'These are TEXTUAL assertions on the query, not an execution of it. They cannot prove the SQL returns the right verdict.' The verdict-ordering test checks string index position — a comment containing 'NO_BASELINE' before the real CASE branch satisfies it. The provider-scoping test counts two regex matches but cannot verify the clauses connect to the right CTEs. Stated plan (real round-trip once BAT-319 lands) is honest, but this is the only guard on the query that detects coverage holes the product hides.
Verdict-ordering test uses string index — fragile against inline string references in wrong position
scripts/ci/__tests__/tariff-coverage-watchdog-sql.test.ts:87
The ordering test uses `CODE.indexOf("'NO_BASELINE'")` which is satisfied by any occurrence of that literal — including a string_agg label, format() format string, or detail message — not just the CASE branch. A refactor moving the literal into a detail format string while leaving the CASE branch wrong would still pass.
No test asserting .sh files in the live arm — inert misclassification would go undetected
scripts/ci/__tests__/classify-changed-files.test.ts:46
The inert test matrix covers `.test.ts`, `.spec.tsx`, `.md`, `.snap`, `.vscode/*`, `LICENSE`, and `.claude/*` — but no `.sh` file. Adding `*.sh` to the inert arm would silently exempt deployment script changes from triggering E2E; no test would catch it.
pnpm-workspace.yaml live classification not tested
scripts/ci/__tests__/classify-changed-files.test.ts:30
The live arm includes `pnpm-workspace.yaml` and `*/pnpm-workspace.yaml`, but tests only cover `pnpm-lock.yaml`. A change to workspace membership (e.g. adding `scripts/ci` to the workspace) would be live but is not explicitly tested.
Deferred SQL integration tests have a complete spec — plan is good
scripts/ci/__tests__/tariff-coverage-watchdog-sql.test.ts:123
The end-of-file comment documents exactly the 5 integration test cases needed once BAT-319 provides a DB gate, including parameterizing `now()` to test the day-9/day-10 boundary. The rationale for deferring is sound.
tariff-job.shells.test.ts correctly includes negative controls
domains/utility/src/tariff-job/__tests__/tariff-job.shells.test.ts:520
Shell-level idempotency pins include 'fresh path' negative controls that would fail if the shell unconditionally short-circuited. Correct pattern per testing.md.
improvement6
scripts/ci package is classified as inert by the classifier it houses
scripts/ci/__tests__/classify-changed-files.test.ts:29
Changes to `scripts/ci/classify-changed-files.sh` and its tests match `*__tests__/*` and are classified as `inert`. A PR that only changes the classifier would report 'no runtime code changed', skip E2E, and merge green even if the classifier accidentally makes everything inert. Consider adding `scripts/ci/classify-changed-files.sh` to the live-first arm explicitly.
classifyTerminated helper is redundant with classify for all but one test
scripts/ci/__tests__/classify-changed-files.test.ts:23
`classifyTerminated` is only used in one test (`classifyTerminated(['README.md'])`) and the assertion it produces is already covered by `classify(['README.md'])` on the same line. Having two helpers with subtly different behaviors risks future confusion about which to use. Either collapse into one helper, or rename `classifyTerminated` to `classifyNewlineSafe` with a comment explaining why both exist.
Test patterns mirror the script case statement verbatim — structural bugs could pass both
scripts/ci/__tests__/classify-changed-files.test.ts:47
Inert test cases are drawn from the same glob patterns used in the case statement. A structural bug (e.g. glob matching differently in fnmatch vs globstar for `*__tests__/*`) would be masked because test inputs are canonical examples of the same patterns. A few boundary cases NOT drawn from the existing pattern arms (e.g. `__tests__` appearing mid-path in a non-test directory) would add meaningful coverage.
Deferred SQL tests in block comment instead of it.todo blocks
scripts/ci/__tests__/tariff-coverage-watchdog-sql.test.ts:1908
The 5 integration test cases deferred to BAT-319 are described in a block comment invisible to vitest output. Using `it.todo(...)` blocks instead would make them visible in test run output as pending, surface in coverage analysis, and prevent accidental deletion.
scripts/ci uses vitest ^3.2.4 while monorepo resolves vitest@4.0.15
scripts/ci/package.json:1
Version divergence: `scripts/ci` declares `vitest: ^3.2.4` but the lockfile shows other packages resolve to `vitest@4.0.15`. These tests run on a different major than the rest of the suite and will cause upgrade confusion. The specifier should be `^4.0.0` to align with what is already resolved.
Two nearly-identical watchdog YAML files diverge under copy-paste pressure
.github/workflows/tariff-coverage-watchdog.yml:157
The NOTE comment correctly flags that tariff- and uco-drift-watchdog.yml share ~90 lines and a shared `scripts/ci/redact-psql-error.sh` would have prevented the original divergence. BAT-322 tracks extraction. The `sed` redaction block now appears 3× in uco-drift-watchdog.yml alone.
History · 14 commits
- 1ed035eneeds attentionincremental0H · 2M · 5L2026-08-11 02:03
- fcbe80dblockedfull6H · 12M · 14L2026-08-10 22:02
- 92353bdblockedincremental1H · 9M · 7L2026-08-10 19:32current
- 5f2213eneeds attentionincremental1H · 6M · 10L2026-08-09 05:35
- c30da44needs attentionincremental1H · 4M · 2L2026-08-09 04:54
- 5940f56needs attentionincremental0H · 3M · 7L2026-08-07 19:15
- f222512needs attentionfull2H · 9M · 15L2026-08-07 18:58
- eec3b04needs attentionincremental2H · 1M · 4L2026-08-07 18:27
- 0b43396needs attentionincremental0H · 2M · 6L2026-08-07 01:39
- 3eb9789needs attentionincremental0H · 3M · 5L2026-08-06 18:30
- 87fc06aneeds attentionincremental0H · 4M · 9L2026-08-06 01:21
- 76bcc76needs attentionincremental0H · 1M · 5L2026-08-06 00:51
- 8948608needs attentionincremental0H · 3M · 4L2026-08-05 23:49
- e44f6bbneeds attentionfull3H · 6M · 9L2026-08-05 19:29