← all branches

fix/tariff-cov

needs attentionviewing older commit
5940f56 · incrementalpre-PRreviewed 2026-08-07 19:15 UTC0H · 3M · 7L · 5I
The branch
Purpose
Fix silent correctness holes in the CFE tariff scrape pipeline that caused jobs to report completion while leaving coverage gaps undetected
Goal
Three fixes: (1) unknown requested divisions surface as explicit failures instead of being silently filtered; (2) finalize denominator computed from job request config rather than Map branch survivors; (3) SFN Finalize state gets a Catch to FailJob so pipeline failures don't leave jobs stuck in 'running'
Sub-goals
  • SG-1: Extract division logic to shared lib so collector and finalize compute the same denominator
  • SG-2: resolveTargets returns { targets, unmatched } with explicit unknown names
  • SG-3: UNKNOWN_DIVISION failures emitted per unknown name × rate type in runCollection
  • SG-4: finalizeTariffJobShell receives expectedPairs from request config, not aggregated survivor reports
  • SG-5: Catch on Finalize SFN state routes failures to FailJob
The changes (whole branch)
What
New lib module divisions.ts extracted from tariff-collector.lambda.ts; resolveTargets return shape changed to { targets, unmatched }; runCollection emits UNKNOWN_DIVISION failures; transition handler computes denominator from job config; SFN Finalize gets Catch handler; tests added for all three behaviors
Why
Silent partial-match bug: repairing a coverage gap with a misspelled division name matched some divisions, persisted them, then reported 'completed' with the gap still open. Denominator bug: a Map branch that died reported 0 discovered, shrinking the denominator so a job missing 17 rows read as '136/136 persisted'. Finalize had no Catch so DB/state failures left jobs in running status indefinitely.
Areas
services/utility/tariffs/cfe+1195177domains/utility/src/tariff-job+6910domains/utility/src/tariff-rate+1771infra/cdk/src/stacks/services/utility/tariffs/cfe+254101.github/workflows+25311packages/api/src/schemas+182scripts/db+1040.claude/rules+950
Blast
Tariff-only pipeline changes — no org/user data, no billing reads, no public API surface changes. Lambda service + SFN definition + domain shells. Tests cover all changed behaviors. CDK change requires deploy to take effect.
cdk-change-needs-deploy
ci· GitHub auth unavailable — CI signals not checkedcoderabbit· No .coderabbit.yaml in repo

Findings · 15

correctness4

low

Catch on Finalize + oversized $.perRateType could defeat the very protection it adds

infra/cdk/src/stacks/services/utility/tariffs/cfe/stepfunctions.stack.ts:403

When Finalize catches an error, `ResultPath: '$.error'` merges the error into the existing state, which still carries `$.perRateType` (the full Map output). The Finalize comment explicitly names 'an aggregated payload over the state-size limit' as a scenario being protected against. But if the perRateType payload was what pushed state past SFN's 256 KB limit, the Catch execution could itself exceed the limit before reaching FailJob — leaving the job stuck in `running`. Common failure modes (DB down) are well under 256 KB, so this is an edge-within-an-edge, but worth a note in the runbook.

low

Empty `divisions: []` silently falls back to survivor denominator

services/utility/tariffs/cfe/src/handlers/tariff-transition.lambda.ts:244

`divisions: []` (empty array, distinct from null) yields `resolveTargets([])` → targets=[], so `expectedPairs = 0 * rateTypes.length = 0`. The fallback is `totals.divisionsDiscovered ?? null` — the survivor-denominated value the commit fixes. This is likely unreachable if the job schema rejects empty arrays at creation (null = all, non-empty array = subset), but that guard should be confirmed. If reachable, the fallback silently reintroduces the bug.

info

UNKNOWN_DIVISION 'division' field carries the requested name, not a zone name

services/utility/tariffs/cfe/src/handlers/tariff-collector.lambda.ts

For matched-but-failed divisions, failures carry the individual zone name. For UNKNOWN_DIVISION, the failure carries the raw requested name (e.g., 'ATLANTIS'). Intentional and tested, but noteworthy: any downstream that joins failures on zone name will not find a match for UNKNOWN_DIVISION entries.

info

Unmatched names carry the un-trimmed raw string in failure messages

services/utility/tariffs/cfe/src/lib/divisions.ts:73

`wanted = requested.map(r => ({ raw: r, key: r.trim().toUpperCase() }))`. A name with trailing whitespace ('BAJIO ') will match via `key` but appear in `unmatched` and failure messages as the un-trimmed 'BAJIO ' (with space). Low risk given operator-controlled inputs, but trimming `raw` too would be more consistent.

security2

low

Operator-supplied division name reflected verbatim in failure messages stored in SFN state

services/utility/tariffs/cfe/src/handlers/tariff-collector.lambda.ts

The raw operator-supplied division name is embedded in `UNKNOWN_DIVISION` failure messages, which flow into SFN execution state and CloudWatch Logs. This is an internal operator input path with no injection risk, but an oversized string could bloat SFN state and logs. Capping the name at a reasonable length (e.g., `name.slice(0, 200)`) before embedding it is a low-cost guard.

info

Catch on States.ALL writes Lambda error objects (including stack traces) into SFN execution history

infra/cdk/src/stacks/services/utility/tariffs/cfe/stepfunctions.stack.ts

`ErrorEquals: ["States.ALL"]` captures Lambda service errors, timeouts, and throttles, putting raw error objects (potentially including stack traces or internal ARNs) into `$.error` in SFN execution history. Ensure `states:GetExecutionHistory` on this state machine is restricted to ops/admin IAM roles. This is an IAM review item, not a code change.

conventions1

medium

Re-export of lib symbols from handler file is a leaky abstraction

services/utility/tariffs/cfe/src/handlers/tariff-collector.lambda.ts:64

`export { resolveTargets, type DivisionConfig } from '../lib/divisions'` is a transitional shim so collector tests (which import from the handler) keep working. But a Lambda handler is an entry-point boundary, not a library surface. The comment already admits this is a crutch. Tests in collector.test.ts should import from `../src/lib/divisions` directly, and the re-export should be removed. Leaving it creates an implicit API contract on the handler file that makes future removal a breaking change.

tests5

medium

`expectedPairs > 0` fallback path is untested

services/utility/tariffs/cfe/__tests__/transition.test.ts

The production handler uses `expectedPairs > 0 ? expectedPairs : (totals.divisionsDiscovered ?? null)`. No test exercises the `expectedPairs = 0` branch (e.g., a job created with `rateTypes: []` or all divisions unrecognized). In that branch, the handler silently falls back to the survivor-denominated value — exactly the bug this commit fixes. If this branch is legitimately unreachable, a guard or assertion at the call site would make that explicit. If reachable, a test should cover it.

medium

No test exercises the finalize handler wiring end-to-end

services/utility/tariffs/cfe/__tests__/transition.test.ts

The denominator arithmetic is tested as a pure calculation against the library functions (`zoneCount(resolveTargets(...))`) but the full handler path — receiving `results`, calling `aggregatePersistResults`, computing `expectedPairs` from `job.config`, and passing it to `finalizeTariffJobShell` — is never exercised with a stubbed job object. A wiring error (e.g., reading `job.config.rateTypes` from the wrong field) would pass the arithmetic tests. An integration test of the `finalize` branch with a real job config would close this gap.

low

CRON_RATE_TYPES.length pinned implicitly via `17 * 9` magic literal

services/utility/tariffs/cfe/__tests__/transition.test.ts

`expect(allZones * CRON_RATE_TYPES.length).toBe(17 * 9)` — the `9` is a magic literal. Adding a tenth rate type to rate-pages.json produces a confusing arithmetic failure rather than a clear `expect(CRON_RATE_TYPES.length).toBe(9)`. Splitting into two assertions (explicit length + the product) would make failures self-diagnosing.

low

UNKNOWN_DIVISION test doesn't assert distinct rateType per failure

services/utility/tariffs/cfe/__tests__/collector.test.ts

The test asserts `unknown.toHaveLength(2)` and `unknown.every(f => f.division === 'ATLANTIS')`, confirming one failure per rate type. But it doesn't assert that the two failures carry distinct `rateType` values ('GDMTH' and 'GDMTO'). An implementation that emits two failures for the same rate type would pass. Adding `expect(unknown.map(f => f.rateType).sort()).toEqual(['GDMTH', 'GDMTO'])` (or checking `f.message` prefix) would close this.

info

Three behavioral changes each covered by a focused, well-named test

services/utility/tariffs/cfe/__tests__/collector.test.ts

All three stated behavioral changes (unmatched surfacing, UNKNOWN_DIVISION failures, honest denominator) have direct test coverage with specific assertions and clear descriptions. The accented 'VALLE DE MÉXICO' test case is particularly good at pinning the exact bug scenario.

improvement3

low

Double iteration over DIVISIONS in resolveTargets — minor but asymmetric

services/utility/tariffs/cfe/src/lib/divisions.ts:65

`targets` is built with `DIVISIONS.filter(...)` and `unmatched` is built with `wanted.filter(w => !DIVISIONS.some(...))` — two passes duplicating the same predicate in opposite polarity. A single pass building a Set of matched wanted keys while collecting targets, then deriving unmatched from the difference, would be both cleaner and marginally more efficient. At 17 divisions this is trivial, but the current structure makes the logic harder to follow at a glance.

low

`matches` inner function closes over nothing — could be module-level

services/utility/tariffs/cfe/src/lib/divisions.ts:68

The `matches` predicate is defined inside `resolveTargets` on every invocation but captures no variables from the enclosing scope. Hoisting it to module level would make it directly testable, clarify that it's a pure predicate (not a closure), and avoid a per-call function allocation.

info

Zero-pairs fallback silently proceeds where a loud error would be clearer

services/utility/tariffs/cfe/src/handlers/tariff-transition.lambda.ts:249

`expectedPairs > 0 ? expectedPairs : (totals.divisionsDiscovered ?? null)` — if `expectedPairs` is 0 (unreachable if job creation validates), the handler silently falls through with a potentially-wrong denominator rather than throwing. A `throw new Error('finalize: expectedPairs resolved to 0 — job config has empty rateTypes or all divisions unrecognized')` would be both safer and easier to debug.

History · 14 commits

  1. 1ed035eneeds attentionincremental0H · 2M · 5L2026-08-11 02:03
  2. fcbe80dblockedfull6H · 12M · 14L2026-08-10 22:02
  3. 92353bdblockedincremental1H · 9M · 7L2026-08-10 19:32
  4. 5f2213eneeds attentionincremental1H · 6M · 10L2026-08-09 05:35
  5. c30da44needs attentionincremental1H · 4M · 2L2026-08-09 04:54
  6. 5940f56needs attentionincremental0H · 3M · 7L2026-08-07 19:15current
  7. f222512needs attentionfull2H · 9M · 15L2026-08-07 18:58
  8. eec3b04needs attentionincremental2H · 1M · 4L2026-08-07 18:27
  9. 0b43396needs attentionincremental0H · 2M · 6L2026-08-07 01:39
  10. 3eb9789needs attentionincremental0H · 3M · 5L2026-08-06 18:30
  11. 87fc06aneeds attentionincremental0H · 4M · 9L2026-08-06 01:21
  12. 76bcc76needs attentionincremental0H · 1M · 5L2026-08-06 00:51
  13. 8948608needs attentionincremental0H · 3M · 4L2026-08-05 23:49
  14. e44f6bbneeds attentionfull3H · 6M · 9L2026-08-05 19:29