← all branches

feat/powerradar

needs attentionviewing older commit
b876b54 · incrementalpre-PRreviewed 2026-07-08 04:44 UTC2H · 2M · 5L · 2I
The branch
Purpose
Port the PowerRadar (Centrica/Smarter) integration into the v2 metrics engine: 31 legacy sites using operator-uploaded latin-1 hourly CSVs with DST gotchas and site-name-keyed registry rows.
Goal
CSV-first intake worker processing uploaded PowerRadar CSVs (S3 reference in params) + D3 legacy registry migration support with powerradarChannel passthrough.
Sub-goals
  • SG-1: Study csv-upload path + legacy intake; worker + parser + unit tests (DST cases)
  • SG-2: CDK + main.ts + coordinator grant; synth clean
  • SG-3: Preview data plane + powerradarChannel D3 unblock + real sample CSV validation
  • SG-4: Live end-to-end intake validation + Tinybird + legacy parity
  • SG-N: Framework learnings fold-back
The changes (whole branch)
What
SG-3 + SG-N: (1) powerradarChannel passthrough in ProvisionAssetStreamCreate → ensureMetricSource → source_config JSONB — unblocks D3 apply for powerradar; (2) case/diacritic-insensitive channel matching via channelMatchKey() to handle legacy UPPERCASE/accent-stripped registry names vs CSV headers; (3) seed-migration-sites.ts dev script for creating test sites on fresh preview DBs; (4) integrations CLAUDE.md Tier-3 section + 8 framework learnings in scope.md.
Why
The legacy registry stores channel names UPPERCASE and sometimes accent-stripped relative to actual CSV column headers (e.g. MENJURJERIA MILAGROS vs MENJURJERÍA MILAGROS). Without case/diacritic folding, D3-migrated sources would silently fail channel lookup. The D3 apply was previously blocked because provisionAssetShell had no powerradarChannel passthrough — this branch adds it as the minimal unblocking change.
Areas
services/metrics+14619scripts/metrics+17190infra/cdk+4041packages/integration-manifests+10602packages/database+25714domains/cross-domain+111domains/core+161domains/metrics+54.branch+184172
Blast
57 files, +5566/-204 across services/metrics (worker), scripts/metrics (migration tooling), infra/cdk (Lambda stack), packages (manifests + DB schema). domains/cross-domain touched minimally (11 lines added to provisionAssetShell). No frontend changes. No API contract changes.
All 4 SGs complete — branch is merge-ready pending findings Live validated: 3-site e2e on real legacy CSVs, 60/60 CDMX days exact parity, 31/31 Tijuana days ≤0.0112% Dev account IAM 1000-role quota hit during e2e (learning #6) — sibling branches should coordinate CDK sweeps
CI· No open PR — no CI checks availableCodeRabbit· No .coderabbit.yaml in repo

Findings · 12

correctness3

high

Reactivation path silently ignores updated powerradarChannel

domains/cross-domain/src/asset-provisioning.shells.ts:830

When ensureMetricSource finds an existing MetricSource with status='inactive', it reactivates via changeStatusWithVersion — a status-only flip that leaves source_config untouched. If a source was originally persisted without powerradarChannel (before this diff when D3 apply was blocked), re-provisioning with a powerradarChannel value reactivates the stale row and silently discards the channel. The worker then finds source_config.powerradarChannel undefined and emits channel-not-configured, collecting nothing. Fix: after reactivation, merge egaugeColumn/powerradarChannel into source_config when they differ from the stored value.

medium

Site + SiteLocation inserts not atomic in seed-migration-sites.ts

scripts/metrics/seed-migration-sites.ts:77

The sites insert and siteLocations insert are two separate awaits with no wrapping transaction. If the process crashes between them or siteLocations throws, the site row is committed but has no location. The metrics pipeline resolves site timezone via SiteLocation; a location-less site fails at collection time. Both inserts should be wrapped in db.transaction() to succeed or fail atomically.

low

Idempotency check in seed-migration-sites matches decommissioned/inactive sites

scripts/metrics/seed-migration-sites.ts:66

The existing-site lookup filters on (orgId, name) without requiring status='active'. A decommissioned site with the same name is reused and its publicId flows into migrate-legacy-registry's provisionAssetShell, which does not validate that the target site is active. Adding eq(sites.status, 'active') to the WHERE clause prevents silently mapping onto a non-operational site.

security3

low

seed-migration-sites: no env guard prevents accidental prod run

scripts/metrics/seed-migration-sites.ts:51

The script uses a bare drizzle connection (no RLS, no service-role restriction) and its own comment says 'no guard beyond the URL you point it at'. A POSTGRES_URL pointing to prod gives unrestricted write access to sites and site_locations across any org. Adding a check that POSTGRES_URL does not match a prod host pattern would prevent a misfire.

low

Spec-file JSON cast without runtime validation before DB writes

scripts/metrics/seed-migration-sites.ts:46

The spec JSON is parsed and immediately cast as Record<string, SiteSpec> with no runtime Zod validation. Fields flow directly into DB inserts. In a shared dev environment with an operator-controlled spec file this is low risk, but a Zod parse would catch malformed entries early (wrong timezone format, missing org) with a clear error instead of a DB constraint violation.

low

powerradarChannel stored in JSONB with no shell-layer schema validation

domains/cross-domain/src/asset-provisioning.shells.ts:891

powerradarChannel is spread into source_config JSONB without invoking validateSourceConfig (which enforces the powerradar manifest's sourceConfigSchema). This is the same pattern used for egaugeColumn. The gap means an empty or excessively long channel string is accepted at provision time and surfaces only as channel-not-found at collection time. No injection risk (parameterized JSONB), but validation at the boundary would shift failure earlier.

conventions2

high

Comment embeds branch name `feat/powerradar` — will rot on merge

scripts/metrics/lib/legacy-registry-core.ts:98

The JSDoc on applySupported reads 'powerradar became apply-supported on feat/powerradar: …'. Branch names are transient identifiers that become meaningless after merge. Project rules prohibit referencing branch names, task IDs, or callers in comments (CLAUDE.md). Drop the `feat/powerradar:` reference and replace with the permanent architectural reason — the field is now true for single-device powerradar because powerradarChannel is persisted in source_config, satisfying the manifest requirement.

low

JSDoc on powerradarChannel references 'D3 apply' — caller/task leak

domains/cross-domain/src/asset-provisioning.shells.ts:82

'this passthrough is what unblocks the D3 apply for the brand' — D3 is an internal milestone identifier that is meaningless after the migration completes. The comment should describe the permanent structural constraint: the powerradar manifest requires powerradarChannel in source_config and the intake worker has no fuzzy fallback.

tests2

medium

No test for multi-device powerradar site remaining apply-unsupported

scripts/metrics/__tests__/legacy-registry-core.test.ts

The diff removes the powerradar-specific applySkipReason guard, making multi-device collision the only remaining false path for powerradar. The existing multi-device test uses hoymilesRecord, not powerradarRecord. A regression that widens the multi-device check would silently make multi-powerradar-device migrations apply-supported without a test catching it.

low

No test for accent/case folding interacting with legacy renames (HM→H&M)

services/metrics/integrations/powerradar/src/__tests__/rows-to-batu.test.ts

channelMatchKey applies uppercase + diacritic strip on top of normalizeChannelName's legacy renames (HM→H&M, CA→C&A). No test exercises a case where both a rename AND case folding apply together. The LEGACY_RENAMES map keys are mixed-case and looked up before uppercasing, so a lowercase alias + rename interaction could produce a miss.

improvement2

info

channelMatchKey recomputed per column on every resolveChannelColumn call

services/metrics/integrations/powerradar/src/translation/rows-to-batu.ts

channelMatchKey(powerradarChannel) is constant for a given channel but computed per-call; channelMatchKey(c) is called for every column in the scan. Each call chains normalizeChannelName + NFD normalize + diacritic strip + toUpperCase. Negligible at today's CSV size but a module-level cache or memoization wrapper would eliminate the redundant work.

info

seed-migration-sites makes 3-4 sequential round-trips per site entry

scripts/metrics/seed-migration-sites.ts

Each spec entry runs: org lookup + existing-site check + site insert + siteLocation insert, all sequential. For a one-shot migration tool with small spec files this is fine. If the pattern is reused for larger spec files, batching the org lookups (WHERE publicId IN (...)) would reduce round-trips from O(4N) to O(2 + 2N).

History · 15 commits

  1. f34e4ceneeds attentionincremental0H · 1M · 3L2026-07-23 00:32
  2. 1bc8aedneeds attentionincremental0H · 2M · 3L2026-07-22 21:34
  3. 31e63bbneeds attentionincremental0H · 2M · 5L2026-07-22 20:53
  4. 482cb88safeincremental0H · 0M · 1L2026-07-22 17:56
  5. 9a73b79safeincremental0H · 0M · 0L2026-07-22 17:34
  6. 87f8298needs attentionincremental0H · 1M · 3L2026-07-22 16:59
  7. 1e80096safeincremental0H · 0M · 1L2026-07-22 16:42
  8. 6ed65b4safeincremental0H · 1M · 3L2026-07-22 00:00
  9. 74a1131needs attentionfull0H · 7M · 12L2026-07-21 19:06
  10. 563252bneeds attentionincremental4H · 9M · 8L2026-07-21 18:32
  11. beced58needs attentionincremental0H · 3M · 3L2026-07-21 01:13
  12. 7fa8684needs attentionincremental4H · 9M · 7L2026-07-20 22:56
  13. 230784fneeds attentionfull2H · 11M · 14L2026-07-10 00:15
  14. b876b54needs attentionincremental2H · 2M · 5L2026-07-08 04:44current
  15. ec4847fneeds attentionfull2H · 11M · 8L2026-07-08 03:32