← all branches

fix/tariff-cov

needs attentionviewing older commit
8948608 · incrementalpre-PRreviewed 2026-08-05 23:49 UTC0H · 3M · 4L · 4I
The branch
Purpose
Fix silent data-loss in the CFE tariff rate scraper: Valle de México (one dropdown selection covering 3 zones) was producing a single flat cell list, so cfeRateRowToComponents kept the last table's values and wrote Sur's prices to Norte and Centro. This branch adds per-table scraping, a table-count guard, and comprehensive test coverage.
Goal
Eliminate the Valle de México per-zone price data-loss bug and harden the collector against future shape mismatches with loud failures instead of silent wrong data.
Sub-goals
  • SG-1: Rename extractRateCells→extractRateCellTables returning ScrapedRateCell[][] (one array per table)
  • SG-2: Add table-count === zones-count guard with a loud error
  • SG-3: Wire componentsByZone[i] to zones[i] (correct pairing)
  • SG-4: Add tariff-coverage-watchdog CI workflow to detect future gaps
  • SG-5: Add rate-type isolation tests and config-integrity tests
  • SG-6: Fix month-reach bug in the rate engine (reach any month, fail loudly)
  • SG-7: Plug silent data-loss in the pipeline (stop overwriting with nulls)
The changes (whole branch)
What
tariff-collector.lambda.ts: extractRateCells refactored to extractRateCellTables (returns per-table arrays); table-count mismatch guard added; zones.forEach paired with componentsByZone[i]. collector.test.ts: StubOptions.tables added; rateRows made a function of t for distinct values per table; two Valle tests upgraded from touPages to vallePages (3-table stub); two new tests added pinning per-zone prices and mismatch failures.
Why
The last commit (89486083) closes the pairing gap: prior commits fixed the month-reach and pipeline data-loss bugs; this one ensures each of Valle de México's 3 zones receives the rates from its own table, not a shared last-table override.
Areas
services/utility/tariffs/cfe+784127infra/cdk/src/stacks/services/utility/tariffs/cfe+246101domains/utility/src/tariff-job+10814.github/workflows+1445scripts/db+1040apps/platform/src/api+130.claude/rules+950
Blast
19 files, +1711/-277 lines across 7 areas; all changes are within the CFE tariff scraping service, utility domain job logic, CDK Lambda/SFN stacks, and CI. No shared domain types or API contracts changed.
no-gh-auth pre-pr-branch
gh-checks· GitHub CLI not authenticated on this runner — CI status not retrievedcoderabbit· No .coderabbit.yaml present

Findings · 11

security4

low

Unbounded table count from CFE HTML response

services/utility/tariffs/cfe/src/handlers/tariff-collector.lambda.ts

extractRateCellTables iterates all matching tables with no upper bound before the zones.length guard fires. A malformed page could accumulate many tables in memory before the error is thrown. A simple early-exit (break-equivalent once tables.length > max_expected) would bound memory usage.

low

Unbounded cell count per scraped table row

services/utility/tariffs/cfe/src/handlers/tariff-collector.lambda.ts

No per-table row count cap on scraped cells. Unexpected page growth could produce unexpectedly long strings passed to cfeRateRowToComponents. Low risk since CFE controls the source, but a max-row guard is cheap defense-in-depth.

info

Scraped values stored in JSONB without explicit length cap

services/utility/tariffs/cfe/src/handlers/tariff-collector.lambda.ts

parseRateValue strips non-numeric characters but does not assert a length bound before persisting. In practice a valid CFE rate value is ≤10 chars after cleaning. Not exploitable, noted for defense-in-depth.

info

Error message zone names sourced from static config — confirmed safe

services/utility/tariffs/cfe/src/handlers/tariff-collector.lambda.ts

cfg.divisionGroup and zones.join() in the error template string originate from bundled divisions.json (not user input). No injection risk in current form; note this if zones ever sourced from external input.

tests5

medium

Missing NORTE vs CENTRO price-distinctness assertion

services/utility/tariffs/cfe/__tests__/collector.test.ts:217

The 'gives each contained zone ITS OWN table's prices' test checks NORTE≠SUR and CENTRO≠SUR but never checks NORTE≠CENTRO. A bug that assigns table[0]'s prices to both NORTE and CENTRO (while giving SUR table[2]) would produce 3 distinct Set values and pass all current assertions. Add: expect(byZone.get('VALLE DE MEXICO NORTE')).not.toEqual(byZone.get('VALLE DE MEXICO CENTRO')).

medium

Non-null assertion on data.rates[0]! may hide regression

services/utility/tariffs/cfe/__tests__/collector.test.ts:221

In 'gives each zone its own prices', data.rates[0]! throws a TypeError (not a test failure) if runCollection emits no rate entries. A guard expect(data.rates).toHaveLength(1) before the .map() would produce a clear, actionable failure instead.

medium

Mismatch test division assertion is vacuously correct on undefined

services/utility/tariffs/cfe/__tests__/collector.test.ts:242

expect(data.rates[0]?.divisions ?? []).toHaveLength(0) passes both when rates is empty AND when rates[0] exists with an empty divisions array. Verify the production path — if a page-level error fires, rates may be completely empty, and adding expect(data.rates).toHaveLength(1) would pin the expected shape.

low

Table-to-zone index mapping correct but not explicitly pinned

services/utility/tariffs/cfe/__tests__/collector.test.ts

The test proves three zones have distinct prices but not which zone received which table's prices. A pairing inversion (NORTE ↔ CENTRO swap) passes all current assertions. Low risk given the index comes directly from divisions.json order, but a direct component equality check would fully close the regression surface.

info

Mismatch failure test does not assert errorCode

services/utility/tariffs/cfe/__tests__/collector.test.ts

The test validates stage and message but not errorCode (expected DIVISION_SCRAPE_FAILED). Adding this assertion would catch a future refactor that changes the code while leaving the message intact.

improvement2

low

zones.forEach with index is less readable than a for-loop zip

services/utility/tariffs/cfe/src/handlers/tariff-collector.lambda.ts

zones.forEach((zone, i) => { rates.push({ ..., components: componentsByZone[i]! }) }) pairs two arrays by index with a non-null assertion. A classic for-loop or a pre-zip makes the pairing explicit and drops the !, since the length guard above already guarantees safety.

info

isTou derivation from .some() assumes all tables share TOU character

services/utility/tariffs/cfe/src/handlers/tariff-collector.lambda.ts

isTou = componentsByZone.some(c => ...) marks the whole page as TOU if any zone has TOU fields. This is correct for current CFE structure, but a short comment stating the assumption (all tables on one page are uniformly TOU or non-TOU) would save future readers from reconstructing the intent.

History · 14 commits

  1. 1ed035eneeds attentionincremental0H · 2M · 5L2026-08-11 02:03
  2. fcbe80dblockedfull6H · 12M · 14L2026-08-10 22:02
  3. 92353bdblockedincremental1H · 9M · 7L2026-08-10 19:32
  4. 5f2213eneeds attentionincremental1H · 6M · 10L2026-08-09 05:35
  5. c30da44needs attentionincremental1H · 4M · 2L2026-08-09 04:54
  6. 5940f56needs attentionincremental0H · 3M · 7L2026-08-07 19:15
  7. f222512needs attentionfull2H · 9M · 15L2026-08-07 18:58
  8. eec3b04needs attentionincremental2H · 1M · 4L2026-08-07 18:27
  9. 0b43396needs attentionincremental0H · 2M · 6L2026-08-07 01:39
  10. 3eb9789needs attentionincremental0H · 3M · 5L2026-08-06 18:30
  11. 87fc06aneeds attentionincremental0H · 4M · 9L2026-08-06 01:21
  12. 76bcc76needs attentionincremental0H · 1M · 5L2026-08-06 00:51
  13. 8948608needs attentionincremental0H · 3M · 4L2026-08-05 23:49current
  14. e44f6bbneeds attentionfull3H · 6M · 9L2026-08-05 19:29