feat/powerradar
needs attentionviewing older commit7fa8684 · incrementalpre-PRreviewed 2026-07-20 22:56 UTC4H · 9M · 7L- Purpose
- Replace the manual-CSV PowerRadar integration with an automatic, API-based pull worker using the portal's internal /eigw (energy-insights-gateway) API, enabling native 5/15/60-min collection via the existing Site Collection SFN.
- Goal
- Ship powerradar-api: a headless Cognito REFRESH_TOKEN_AUTH integration against /eigw chart-data, fitting the standard metrics pull model. The existing CSV powerradar worker remains as a deep-history fallback.
- Sub-goals
- SG-1: Auth + transport engine + live probe (Cognito REFRESH_TOKEN_AUTH, /eigw chart-data client, probe response shape/retention/granularities)
- SG-2: Manifest + catalog + provider enum (powerradar-api, requiresAuth:true, POWER→demand)
- SG-3: Pure translation + unit tests (chart-data JSON → MetricsPayload, §9 multi-granularity)
- SG-4: Handlers (metrics + explore) following INTEGRATION_STANDARDS §9
- SG-5: CDK stack (secrets-only IAM, victron shape) + wiring in paths.ts + main.ts
- SG-6: Seed routing + live end-to-end validation (SFN → Tinybird → parity check vs CSV baseline)
- SG-7: Framework learnings fold-back into INTEGRATION_STANDARDS + .claude/rules
- What
- scope.md rewritten from completed CSV-integration plan to new API-integration plan (SG-1..SG-7 all unchecked). intent.md cleared to reset the intent log for the new phase.
- Why
- The CSV-based phase (SG-1..SG-N, including 91/91 parity days) is complete. The branch now pivots to the real goal: automated pull collection without manual CSV uploads.
- Areas
- .branch/scope.md+142−119.branch/intent.md+0−200
- Blast
- 2 files, +142/−319 in .branch/ only. Zero production code changed this commit.
Findings · 20
correctness5
Access-token cache backend unspecified — stampede risk on multi-device fleet
Concurrent Lambda cold-starts for a multi-device site will each call REFRESH_TOKEN_AUTH if cache is in-memory. Clarify in SG-1: in-memory is acceptable for small fleet; DDB if Cognito throttle is a concern. Also specify exp-buffer margin (not raw exp).
CSV→API transition: disjoint-tier invariant not addressed in SG-6 runbook
Existing powerradar CSV MetricSources have open coverage windows. SG-6 runbook must include: deactivate the CSV source before the API source goes live on the same stream, per the splice invariant (site-metrics-seam.md). Otherwise priority-splice outcome depends on source priority, potentially preferring CSV over live API data.
POWER unit (W vs kW) pending SG-1 probe — gating SG-3 translation
SG-3 should gate on SG-1 probe results. Use a real fixture from the live probe, not a synthetic one. The existing CSV worker scales kW×1000→W; confirm if API also returns kW.
ENERGY measurementType ambiguity: seam rule prohibits collecting ENERGY alongside POWER
Adding an ENERGY collection source alongside POWER on the same stream creates non-disjoint sources, violating the seam rule. Clarify in SG-2: ENERGY measurementType is probe-only documentation; do not register as a collectible manifest variable.
x-panpwr-roleversion: 12 hardcoded — no diagnostic logging for breakage
A portal release bumping this to 13 will silently fail. Log the roleversion sent in error diagnostics so a future 403 is diagnosable without re-probing.
security5
169-day refresh token: no rotation runbook or revocation procedure documented
A stolen Cognito refresh token gives up to 169 days of portal-read access. Plan should include: (1) revocation runbook (cognito-idp:RevokeToken or password reset), (2) Secrets Manager rotation cadence ≤90 days or tied to a proactive re-login cycle, (3) anomaly alert for unexpected REFRESH_TOKEN_AUTH calls. Out-of-scope section says 'ship a runbook' — that runbook must cover revocation, not just initial login.
Cognito IDP egress not in IAM/VPC allowlist note
Lambda needs outbound HTTPS to cognito-idp.us-east-1.amazonaws.com. No IAM permission needed (public SPA client), but if NAT/security-group rules allowlist outbound HTTPS by host, add this endpoint alongside www.powerradar.energy. Note explicitly in SG-5 INFRA_DESIGN.md.
Using a SPA Cognito client server-side: ToS and stability risk
client_id 3kcr9l6iol4skjkao4768s1rtf is a public SPA client used server-side. Vendor can add adaptive auth, rotate the client, or block non-browser user-agents at any time. Document this risk in INFRA_DESIGN.md alongside the Export Jobs fallback.
Default AWS-managed KMS key for 169-day Secrets Manager credential
Hardening recommendation: use a CMK with key policy restricted to the Lambda's execution role ARN, providing defense-in-depth for this long-lived, high-access-scope credential.
Access token must never appear in CloudWatch logs
Add explicit constraint to SG-1 cognito-auth.ts design: never log AccessToken or RefreshToken values. Mirror the 'secret: true' credentialField convention.
conventions3
Secret path should use canonical batu/{env}/metrics/powerradar-api/* prefix
infra/cdk/src/stacks/services/metrics/integrations/powerradar-api/iam.ts
scope.md proposes {env}/powerradar-api/oid={oid}. The v2 canonical pattern (victron, egauge, growatt IAM stacks) is batu/{env}/metrics/{integrationCode}/*. Fix in SG-5 iam.ts before seeding dev secrets.
shelly-fleet cited as two-access-path precedent but does not exist in the codebase
services/metrics/integrations/shelly-fleet/ does not exist. The live precedent is helioscope (helioscope-csv + helioscope-api manifests). Cite helioscope-api instead.
ENERGY variable note in SG-2 is misleading given the seam rule
Clarify in the manifest file comment: POWER→demand only; ENERGY is probe artifact, not a registered collection variable. Prevents a future implementer from adding an ENERGY MetricSource.
tests3
Engine-layer tests absent: cognito-auth.ts and eigw-client.ts have no planned coverage
SG-3 scopes tests to pure translation only. Auth engine is the highest-risk new code (new auth archetype). Minimum additions: (a) TTL gate tests for cognito-auth.ts, (b) error-classification tests for eigw-client.ts (401→UpstreamAuthFailed, 429→UpstreamRateLimited, 5xx→UpstreamUnavailable). Reference: huawei/auth-and-explore.test.ts, enphase/token-manager.test.ts.
Cognito REFRESH_TOKEN_AUTH failure path: NotAuthorizedException → wrong error tag
services/metrics/integrations/powerradar-api/src/engine/cognito-auth.ts
A revoked/expired refresh token returns NotAuthorizedException. Without a test, this could be mis-classified as UpstreamUnavailable → silent retry storms instead of alert. Add unit test for the error→_tag mapping.
Malformed/empty chart-data response not in SG-3 test scenarios
services/metrics/integrations/powerradar-api/src/translation/chart-to-batu.ts
§9 requires resilient skip for failing channels. Test: empty data array → zero MetricsPayload items, null point value → point dropped, unexpected JSON shape → error-as-value (not thrown). Reference: rows-to-batu.test.ts channel-not-found case.
improvement4
Token cache TTL must subtract a safety buffer from exp (60–300s)
services/metrics/integrations/powerradar-api/src/engine/cognito-auth.ts
Using raw exp means a Lambda starting 2s before expiry passes the TTL gate but finds the token expired mid-request. Pattern: enphase EXPIRY_MARGIN_SEC=300. Specify exp−120s (minimum) in SG-1 cognito-auth.ts design.
In-process token cache lost on cold start — document the design decision
Module-level cache is acceptable for small fleets (SMA/Sol-Ark precedent). Document in INFRA_DESIGN.md: why in-memory chosen over DDB, and at what fleet size to switch. Prevents SG-7 re-litigating this.
@aws-sdk/client-cognito-identity-provider must be a runtime dep, not externalized
services/metrics/integrations/powerradar-api/package.json
Not pre-bundled in the Node.js 20 Lambda runtime. Must be in package.json dependencies and absent from CDK bundling externalModules. Missing → cold-start module-not-found crash. Note explicitly in SG-5.
SG-7 should add Cognito-refresh auth archetype to INTEGRATION_STANDARDS §8 checklist
services/metrics/integrations/INTEGRATION_STANDARDS.md
INTEGRATION_STANDARDS.md §8 only documents static-token and session/DDB-cache archetypes. Add headless Cognito REFRESH_TOKEN_AUTH as a third archetype. Prevents cargo-culting a DDB auth-cache stack for similar token-based integrations.
History · 15 commits
- f34e4ceneeds attentionincremental0H · 1M · 3L2026-07-23 00:32
- 1bc8aedneeds attentionincremental0H · 2M · 3L2026-07-22 21:34
- 31e63bbneeds attentionincremental0H · 2M · 5L2026-07-22 20:53
- 482cb88safeincremental0H · 0M · 1L2026-07-22 17:56
- 9a73b79safeincremental0H · 0M · 0L2026-07-22 17:34
- 87f8298needs attentionincremental0H · 1M · 3L2026-07-22 16:59
- 1e80096safeincremental0H · 0M · 1L2026-07-22 16:42
- 6ed65b4safeincremental0H · 1M · 3L2026-07-22 00:00
- 74a1131needs attentionfull0H · 7M · 12L2026-07-21 19:06
- 563252bneeds attentionincremental4H · 9M · 8L2026-07-21 18:32
- beced58needs attentionincremental0H · 3M · 3L2026-07-21 01:13
- 7fa8684needs attentionincremental4H · 9M · 7L2026-07-20 22:56current
- 230784fneeds attentionfull2H · 11M · 14L2026-07-10 00:15
- b876b54needs attentionincremental2H · 2M · 5L2026-07-08 04:44
- ec4847fneeds attentionfull2H · 11M · 8L2026-07-08 03:32