← all branches

feat/powerradar

needs attentionviewing older commit
7fa8684 · incrementalpre-PRreviewed 2026-07-20 22:56 UTC4H · 9M · 7L
The branch
Purpose
Replace the manual-CSV PowerRadar integration with an automatic, API-based pull worker using the portal's internal /eigw (energy-insights-gateway) API, enabling native 5/15/60-min collection via the existing Site Collection SFN.
Goal
Ship powerradar-api: a headless Cognito REFRESH_TOKEN_AUTH integration against /eigw chart-data, fitting the standard metrics pull model. The existing CSV powerradar worker remains as a deep-history fallback.
Sub-goals
  • SG-1: Auth + transport engine + live probe (Cognito REFRESH_TOKEN_AUTH, /eigw chart-data client, probe response shape/retention/granularities)
  • SG-2: Manifest + catalog + provider enum (powerradar-api, requiresAuth:true, POWER→demand)
  • SG-3: Pure translation + unit tests (chart-data JSON → MetricsPayload, §9 multi-granularity)
  • SG-4: Handlers (metrics + explore) following INTEGRATION_STANDARDS §9
  • SG-5: CDK stack (secrets-only IAM, victron shape) + wiring in paths.ts + main.ts
  • SG-6: Seed routing + live end-to-end validation (SFN → Tinybird → parity check vs CSV baseline)
  • SG-7: Framework learnings fold-back into INTEGRATION_STANDARDS + .claude/rules
The changes (whole branch)
What
scope.md rewritten from completed CSV-integration plan to new API-integration plan (SG-1..SG-7 all unchecked). intent.md cleared to reset the intent log for the new phase.
Why
The CSV-based phase (SG-1..SG-N, including 91/91 parity days) is complete. The branch now pivots to the real goal: automated pull collection without manual CSV uploads.
Areas
.branch/scope.md+142119.branch/intent.md+0200
Blast
2 files, +142/−319 in .branch/ only. Zero production code changed this commit.
planning-docs-only long-lived-credential new-auth-archetype internal-vendor-api
CI· no PR — pre-PR branchCodeRabbit· no .coderabbit.yaml

Findings · 20

correctness5

medium

Access-token cache backend unspecified — stampede risk on multi-device fleet

Concurrent Lambda cold-starts for a multi-device site will each call REFRESH_TOKEN_AUTH if cache is in-memory. Clarify in SG-1: in-memory is acceptable for small fleet; DDB if Cognito throttle is a concern. Also specify exp-buffer margin (not raw exp).

medium

CSV→API transition: disjoint-tier invariant not addressed in SG-6 runbook

Existing powerradar CSV MetricSources have open coverage windows. SG-6 runbook must include: deactivate the CSV source before the API source goes live on the same stream, per the splice invariant (site-metrics-seam.md). Otherwise priority-splice outcome depends on source priority, potentially preferring CSV over live API data.

low

POWER unit (W vs kW) pending SG-1 probe — gating SG-3 translation

SG-3 should gate on SG-1 probe results. Use a real fixture from the live probe, not a synthetic one. The existing CSV worker scales kW×1000→W; confirm if API also returns kW.

low

ENERGY measurementType ambiguity: seam rule prohibits collecting ENERGY alongside POWER

Adding an ENERGY collection source alongside POWER on the same stream creates non-disjoint sources, violating the seam rule. Clarify in SG-2: ENERGY measurementType is probe-only documentation; do not register as a collectible manifest variable.

low

x-panpwr-roleversion: 12 hardcoded — no diagnostic logging for breakage

A portal release bumping this to 13 will silently fail. Log the roleversion sent in error diagnostics so a future 403 is diagnosable without re-probing.

security5

high

169-day refresh token: no rotation runbook or revocation procedure documented

A stolen Cognito refresh token gives up to 169 days of portal-read access. Plan should include: (1) revocation runbook (cognito-idp:RevokeToken or password reset), (2) Secrets Manager rotation cadence ≤90 days or tied to a proactive re-login cycle, (3) anomaly alert for unexpected REFRESH_TOKEN_AUTH calls. Out-of-scope section says 'ship a runbook' — that runbook must cover revocation, not just initial login.

high

Cognito IDP egress not in IAM/VPC allowlist note

Lambda needs outbound HTTPS to cognito-idp.us-east-1.amazonaws.com. No IAM permission needed (public SPA client), but if NAT/security-group rules allowlist outbound HTTPS by host, add this endpoint alongside www.powerradar.energy. Note explicitly in SG-5 INFRA_DESIGN.md.

medium

Using a SPA Cognito client server-side: ToS and stability risk

client_id 3kcr9l6iol4skjkao4768s1rtf is a public SPA client used server-side. Vendor can add adaptive auth, rotate the client, or block non-browser user-agents at any time. Document this risk in INFRA_DESIGN.md alongside the Export Jobs fallback.

medium

Default AWS-managed KMS key for 169-day Secrets Manager credential

Hardening recommendation: use a CMK with key policy restricted to the Lambda's execution role ARN, providing defense-in-depth for this long-lived, high-access-scope credential.

low

Access token must never appear in CloudWatch logs

Add explicit constraint to SG-1 cognito-auth.ts design: never log AccessToken or RefreshToken values. Mirror the 'secret: true' credentialField convention.

conventions3

medium

Secret path should use canonical batu/{env}/metrics/powerradar-api/* prefix

infra/cdk/src/stacks/services/metrics/integrations/powerradar-api/iam.ts

scope.md proposes {env}/powerradar-api/oid={oid}. The v2 canonical pattern (victron, egauge, growatt IAM stacks) is batu/{env}/metrics/{integrationCode}/*. Fix in SG-5 iam.ts before seeding dev secrets.

low

shelly-fleet cited as two-access-path precedent but does not exist in the codebase

services/metrics/integrations/shelly-fleet/ does not exist. The live precedent is helioscope (helioscope-csv + helioscope-api manifests). Cite helioscope-api instead.

low

ENERGY variable note in SG-2 is misleading given the seam rule

Clarify in the manifest file comment: POWER→demand only; ENERGY is probe artifact, not a registered collection variable. Prevents a future implementer from adding an ENERGY MetricSource.

tests3

high

Engine-layer tests absent: cognito-auth.ts and eigw-client.ts have no planned coverage

SG-3 scopes tests to pure translation only. Auth engine is the highest-risk new code (new auth archetype). Minimum additions: (a) TTL gate tests for cognito-auth.ts, (b) error-classification tests for eigw-client.ts (401→UpstreamAuthFailed, 429→UpstreamRateLimited, 5xx→UpstreamUnavailable). Reference: huawei/auth-and-explore.test.ts, enphase/token-manager.test.ts.

medium

Cognito REFRESH_TOKEN_AUTH failure path: NotAuthorizedException → wrong error tag

services/metrics/integrations/powerradar-api/src/engine/cognito-auth.ts

A revoked/expired refresh token returns NotAuthorizedException. Without a test, this could be mis-classified as UpstreamUnavailable → silent retry storms instead of alert. Add unit test for the error→_tag mapping.

medium

Malformed/empty chart-data response not in SG-3 test scenarios

services/metrics/integrations/powerradar-api/src/translation/chart-to-batu.ts

§9 requires resilient skip for failing channels. Test: empty data array → zero MetricsPayload items, null point value → point dropped, unexpected JSON shape → error-as-value (not thrown). Reference: rows-to-batu.test.ts channel-not-found case.

improvement4

high

Token cache TTL must subtract a safety buffer from exp (60–300s)

services/metrics/integrations/powerradar-api/src/engine/cognito-auth.ts

Using raw exp means a Lambda starting 2s before expiry passes the TTL gate but finds the token expired mid-request. Pattern: enphase EXPIRY_MARGIN_SEC=300. Specify exp−120s (minimum) in SG-1 cognito-auth.ts design.

medium

In-process token cache lost on cold start — document the design decision

Module-level cache is acceptable for small fleets (SMA/Sol-Ark precedent). Document in INFRA_DESIGN.md: why in-memory chosen over DDB, and at what fleet size to switch. Prevents SG-7 re-litigating this.

medium

@aws-sdk/client-cognito-identity-provider must be a runtime dep, not externalized

services/metrics/integrations/powerradar-api/package.json

Not pre-bundled in the Node.js 20 Lambda runtime. Must be in package.json dependencies and absent from CDK bundling externalModules. Missing → cold-start module-not-found crash. Note explicitly in SG-5.

low

SG-7 should add Cognito-refresh auth archetype to INTEGRATION_STANDARDS §8 checklist

services/metrics/integrations/INTEGRATION_STANDARDS.md

INTEGRATION_STANDARDS.md §8 only documents static-token and session/DDB-cache archetypes. Add headless Cognito REFRESH_TOKEN_AUTH as a third archetype. Prevents cargo-culting a DDB auth-cache stack for similar token-based integrations.

History · 15 commits

  1. f34e4ceneeds attentionincremental0H · 1M · 3L2026-07-23 00:32
  2. 1bc8aedneeds attentionincremental0H · 2M · 3L2026-07-22 21:34
  3. 31e63bbneeds attentionincremental0H · 2M · 5L2026-07-22 20:53
  4. 482cb88safeincremental0H · 0M · 1L2026-07-22 17:56
  5. 9a73b79safeincremental0H · 0M · 0L2026-07-22 17:34
  6. 87f8298needs attentionincremental0H · 1M · 3L2026-07-22 16:59
  7. 1e80096safeincremental0H · 0M · 1L2026-07-22 16:42
  8. 6ed65b4safeincremental0H · 1M · 3L2026-07-22 00:00
  9. 74a1131needs attentionfull0H · 7M · 12L2026-07-21 19:06
  10. 563252bneeds attentionincremental4H · 9M · 8L2026-07-21 18:32
  11. beced58needs attentionincremental0H · 3M · 3L2026-07-21 01:13
  12. 7fa8684needs attentionincremental4H · 9M · 7L2026-07-20 22:56current
  13. 230784fneeds attentionfull2H · 11M · 14L2026-07-10 00:15
  14. b876b54needs attentionincremental2H · 2M · 5L2026-07-08 04:44
  15. ec4847fneeds attentionfull2H · 11M · 8L2026-07-08 03:32