feat/powerradar
needs attentionviewing older commit74a1131 · fullPR #327reviewed 2026-07-21 19:06 UTC0H · 7M · 12L · 11I- Purpose
- Add an automatic, API-based PowerRadar integration to replace the manual-CSV path with native 5/15/60-min headless collection on the existing Site Collection SFN.
- Goal
- PowerRadar-API integration worker — headless OAuth password-grant auth + /eigw chart-data pull; live-proven on ER devices
- Sub-goals
- SG-1: Auth + transport engine — OAuth password grant (Spring broker), WAF-mitigation browser headers, live-verified
- SG-2: Manifest + catalog + provider enum + ARN routing seeded
- SG-3: Pure chart-to-batu translation + 13 unit tests
- SG-4: Metrics handler (§9 group-by-granularity) + 18 tests; assetConfigSchema device addressing
- SG-5: CDK stack + paths + main.ts wiring + coordinator invoke grant
- SG-6/7: Live e2e (191 demand points at 15m/2d, solar correctly skipped on load-only) + integrations CLAUDE.md
- What
- New powerradar-api integration package (engine, translation, handler, tests), manifest registration, database schema/seed changes, CDK Lambda stack with secrets-only IAM, coordinator invoke-grant wiring. Also fixes a cross-stack SSM race: CsvExportLambdaStack now addDependency(CsvExportStorageStack).
- Why
- The manual-CSV path requires periodic export downloads; the /eigw API enables fully automatic 5/15/60-min collection with no human intervention once the org credential is seeded.
- Areas
- services/metrics/integrations/powerradar-api+1650−0infra/cdk/src+270−20packages/integration-manifests/src/manifests+100−0packages/database/src+80−2.branch+82−295
- Blast
- 24 files (excl. lockfile), +2182/−317. Self-contained new package; no existing integration modified. CDK wiring is additive. Database changes are text-enum additions (no migration). The CsvExport dependency fix is a correctness-only infra change.
Findings · 29
correctness4
grainMismatch detail silently dropped when fetchErrors[0] takes precedence
services/metrics/integrations/powerradar-api/src/handlers/metrics.lambda.ts
When payloads.length === 0 AND fetchErrors[0] is set, the returned InvocationError is fetchErrors[0] and the grain/unit-unknown detail is not surfaced (though it is logged). An operator sees the auth/rate-limit error with no indication a translation failure also occurred. Acceptable precedence, but worth a note in the log-line.
MAX_POINTS_PER_GROUP doc comment overstates 5-min budget (~9.7 weeks, not 10)
services/metrics/integrations/powerradar-api/src/handlers/metrics.lambda.ts
Comment says '~10 weeks of 5-min'. Actual: 20,000 / (7×24×12) = 9.72 weeks. A 10-week window at 5-min sends 20,160 points and is rejected. The cap logic is correct; only the comment misleads an operator sizing a backfill chunk.
getToken() concurrent calls for same credential: no in-flight deduplication
services/metrics/integrations/powerradar-api/src/engine/eigw-client.ts
Safe today because granularity groups are iterated sequentially. If groups are ever parallelised (Promise.all), two in-flight getToken calls for the same key could both miss the memo and issue independent password grants. No data corruption; noting for any future parallelism refactor.
resolveChannels() scalar fallback uses asset.externalId as metricSourcePublicId if params.metricSourcePublicId absent
services/metrics/integrations/powerradar-api/src/handlers/metrics.lambda.ts
A vendor device ID like '268165' is not a valid batu public ID. The coordinator always populates sources[], so this only affects legacy direct-invoke without params.metricSourcePublicId. If exercised, it produces a persist failure rather than a clear configuration error. Document the limitation in a comment.
security6
tokenMemo thundering-herd: concurrent invocations for same credential both miss and double-refresh
services/metrics/integrations/powerradar-api/src/engine/eigw-client.ts
The module-level tokenMemo Map persists across Lambda invocations in the same warm container. Node.js single-thread makes Map access safe, but the async gap between 'check expiry → fetch new token → write back' is not atomic. Two concurrent invocations for the same org can both observe a cache miss and independently trigger OAuth grants, adding WAF pressure. Current execution is sequential per-granularity so this is latent risk only. A per-key in-flight promise memo would deduplicate concurrent refreshes.
JWT exp trusted without a hard TTL cap — malformed/manipulated exp could cache token indefinitely
services/metrics/integrations/powerradar-api/src/engine/eigw-client.ts
jwtExpiryMs() reads the server-provided exp claim without signature verification or a maximum cap. A malformed or server-manipulated token with a far-future exp would be cached in tokenMemo indefinitely and never refreshed. Mitigation: cap expiresAtMs at Date.now() + some hard max (e.g. 2h) regardless of the decoded exp value.
Browser header spoofing — document WAF constraint so maintainers don't remove headers
services/metrics/integrations/powerradar-api/src/engine/eigw-client.ts
Sending forged sec-ch-ua, User-Agent, and Origin headers to impersonate Chrome is a ToS risk with the upstream provider. More practically, if a future maintainer 'cleans up' the headers not knowing they are required, the integration silently breaks with 403s. Add a comment naming which specific header the WAF gates on and why it is necessary.
IAM centrica/* secret prefix — confirm this Lambda legitimately needs it
infra/cdk/src/stacks/services/metrics/integrations/powerradar-api/iam.ts
The policy grants GetSecretValue on {env}/centrica/* to support reuse of existing portal-login secrets. Confirm that the Centrica path is genuinely needed at runtime (i.e. some orgs have secrets seeded there that this worker will read via secretConfigArn), not just a convenience catch-all. If the canonical path is batu/{env}/metrics/powerradar-api/*, the centrica/* grant is over-permissive.
ROPC grant is deprecated in OAuth 2.1 — document as vendor constraint
services/metrics/integrations/powerradar-api/src/engine/eigw-client.ts
OAuth 2.0 Resource Owner Password Credentials grant is disallowed for public clients in OAuth 2.1. This is acceptable because the vendor mandates it and credentials are in Secrets Manager, but it should be documented as a vendor constraint so future reviewers don't try to modernise without vendor support.
e.message logged in fail.exception — confirm upstream SDK errors don't embed credential-bearing URLs
services/metrics/integrations/powerradar-api/src/handlers/metrics.lambda.ts
Logging e.message only (not the Error object) is the correct practice. Query params here are timestamps/resolution — no credentials. Confirm no upstream library (e.g. AWS SDK) embeds the full request context in error messages.
conventions4
InvocationResult/InvocationError pattern is correct for this integration layer
services/metrics/integrations/powerradar-api/src/handlers/metrics.lambda.ts
@batu/result is not used here; InvocationResult<MetricsPayload[]> discriminated on `status` is the canonical integration-layer pattern consistent with all 17 existing workers.
coordinatorLambdaStack.addDependency(powerradarApiLambdaStack) wired in correct location
infra/cdk/src/app/main.ts
Dependency is declared in the post-instantiation block (~line 573), consistent with every other cross-stack SSM dependency. CDK addDependency is not order-sensitive relative to construct instantiation.
Three IAM secret prefixes consistent with SMA/victron precedent for legacy secret reuse
infra/cdk/src/stacks/services/metrics/integrations/powerradar-api/iam.ts
centrica/* covers existing portal-login secrets seeded under that path. This three-prefix pattern exists in at least one other worker (SMA) for the same legacy-path reason.
No exploration seed entry — correct for device-addressed integrations
packages/database/src/seed-integration-arns.ts
Only 3 of 17 integrations carry an exploration job (cloud-portal discovery integrations). powerradar-api is device-addressed (assetConfigSchema), matching the eGauge/solark pattern. Deferring exploration is architecturally correct.
tests7
eigw-client.ts has zero unit tests — retry, token-memo, WAF detection all unverified
services/metrics/integrations/powerradar-api/src/engine/eigw-client.ts
The handler mocks eigw-client wholesale via vi.hoisted(), so none of the transport layer branches are exercised: token memo warm-reuse, forceRefresh bypass, the 401-retry path in fetchChartDataWithRetry, isCloudFrontBlock HTML detection, jwtExpiryMs, or loadCredentialFromArn error paths. At minimum: (1) warm-reuse vs expired, (2) forceRefresh, (3) 401-retry executes once and surfaces UpstreamAuthFailed on second 401, (4) isCloudFrontBlock identifies WAF HTML, (5) loadCredentialFromArn on ResourceNotFoundException and missing fields.
loadCredentialFromArn error branches not independently tested
services/metrics/integrations/powerradar-api/src/engine/eigw-client.ts
The handler suite covers 'unreadable secret' as a single mock case but does not exercise: ResourceNotFoundException (→ null), malformed JSON body (→ null), or a secret that parses but lacks username/password (→ null). These produce the same MissingSecret downstream tag today but could diverge if error handling is refined.
MW unit scaling path not exercised in translation tests
services/metrics/integrations/powerradar-api/src/__tests__/chart-to-batu.test.ts
The unit tests cover kW→W (×1000) and unknown-unit failure, but not MW→W (×1,000,000). If the vendor ever emits MW (unlikely given the API always returns POWER in W/kW), the branch is untested. Worth adding one test or a comment confirming the unit set is closed to {W, kW}.
Empty dataRecord array and all-null values not tested at the translation layer
services/metrics/integrations/powerradar-api/src/__tests__/chart-to-batu.test.ts
The handler-level test covers both-series-null at the invocation level, but chartToBatu() is not called directly with (a) an empty dataRecord array or (b) a series where every value is null. These could expose an off-by-one in the clip/dedupe or return an unexpected shape.
Single-point series not tested end-to-end through chartToBatu grain guard
services/metrics/integrations/powerradar-api/src/__tests__/chart-to-batu.test.ts
dominantSpacingSeconds returns null for <2 points, meaning the grain guard is skipped for single-point series. This is intentional (no spacing to check) but not exercised end-to-end through the handler.
mapFetchError exhaustiveness — add never-branch guard in the switch
services/metrics/integrations/powerradar-api/src/handlers/metrics.lambda.ts
All 8 reason branches are covered in one parameterised test block, which is good. A compile-time exhaustiveness check (switch default: const _exhaustive: never = result; throw ...) inside mapFetchError would catch a new reason at TypeScript compile time rather than at test runtime.
No integration test — intentional for Lambda workers, worth a brief comment
services/metrics/integrations/powerradar-api/src/__tests__/
Absence of an integration test is appropriate for a Lambda coordinator worker where e2e is the primary verification layer (documented in the branch CLAUDE.md). A brief comment in the test directory or README would signal the intentional gap to future contributors.
improvement8
BROWSER_HEADERS will silently go stale — document which headers are load-bearing
services/metrics/integrations/powerradar-api/src/engine/eigw-client.ts
Headers were manually snapshotted from Chrome 148 DevTools. There is no test or CI gate that would surface a breakage. If the WAF changes its fingerprint requirements, requests will silently start returning 403/block with no indication why. Document which specific headers the WAF gates on (e.g. User-Agent, sec-ch-ua) so a maintainer knows what is load-bearing vs noise and can prune or update without guessing.
grainMismatch last-write-wins silently drops earlier mismatch details
services/metrics/integrations/powerradar-api/src/handlers/metrics.lambda.ts
The variable is overwritten on each channel with a grain/unit mismatch, so if both demand and solar_generation disagree only the final channel's detail appears in the error payload. Collecting into an array and joining would surface the full picture.
resolveChannels() scalar fallback uses unvalidated `as MetricChannelSpec[]` cast
services/metrics/integrations/powerradar-api/src/handlers/metrics.lambda.ts
When sources is present but is an unexpected shape, the cast hides the mismatch and the error surfaces later as a confusing downstream failure. A narrow runtime check or narrow Zod parse on the scalar path would make the failure loud and local. (The coordinator always populates sources[] correctly, so this is a legacy-path robustness concern.)
Three separate lookup tables keyed on same types — risk of sync drift as channel set grows
services/metrics/integrations/powerradar-api/src/translation/chart-to-batu.ts
CHANNEL_SERIES, GRANULARITY_RESOLUTION, and SLOT_SECONDS are keyed on PowerradarChannel/PowerradarGranularity but kept separate. Adding a new channel requires updating three independent objects. A unified record per key would make omissions a type error.
dominantSpacingSeconds makes two passes — one-pass accumulator possible
services/metrics/integrations/powerradar-api/src/translation/chart-to-batu.ts
Builds a frequency Map then iterates it to find the max. A single-pass accumulator tracking bestVal/bestCount alongside the Map halves the constant factor. Not a hot path at current scale but a straightforward cleanup.
MAX_POINTS_PER_GROUP is undocumented — cite whether this is an API limit or a memory/timeout guard
services/metrics/integrations/powerradar-api/src/handlers/metrics.lambda.ts
20,000 appears without a reference to whether it is a PowerRadar API hard cap or an internal memory/timeout budget. Future maintainers have no way to know if this can be relaxed when the API changes, or if it must stay fixed. A comment citing the source would cost nothing and prevent silent under-fetching.
secretsClient() lazy singleton — check consistency with other workers
services/metrics/integrations/powerradar-api/src/engine/eigw-client.ts
Uses `let cachedSecretsClient: SecretsManagerClient | null = null` initialized at call time. If the established pattern elsewhere is eager module-load construction (common Lambda cold-start pattern), aligning here makes the cold-start profile predictable.
jwtExpiryMs() throws synchronously on malformed token — wrap in try/catch
services/metrics/integrations/powerradar-api/src/engine/eigw-client.ts
If the upstream token is malformed (wrong number of segments, non-JSON payload), Buffer.from or JSON.parse may throw. The call site in parseTokenResponse does not guard for a thrown error — it would propagate as an unhandled exception from getToken. The function already returns null on missing exp; wrapping the whole body in try/catch and returning null (fallback TTL) on any parse error would make it total.
History · 15 commits
- f34e4ceneeds attentionincremental0H · 1M · 3L2026-07-23 00:32
- 1bc8aedneeds attentionincremental0H · 2M · 3L2026-07-22 21:34
- 31e63bbneeds attentionincremental0H · 2M · 5L2026-07-22 20:53
- 482cb88safeincremental0H · 0M · 1L2026-07-22 17:56
- 9a73b79safeincremental0H · 0M · 0L2026-07-22 17:34
- 87f8298needs attentionincremental0H · 1M · 3L2026-07-22 16:59
- 1e80096safeincremental0H · 0M · 1L2026-07-22 16:42
- 6ed65b4safeincremental0H · 1M · 3L2026-07-22 00:00
- 74a1131needs attentionfull0H · 7M · 12L2026-07-21 19:06current
- 563252bneeds attentionincremental4H · 9M · 8L2026-07-21 18:32
- beced58needs attentionincremental0H · 3M · 3L2026-07-21 01:13
- 7fa8684needs attentionincremental4H · 9M · 7L2026-07-20 22:56
- 230784fneeds attentionfull2H · 11M · 14L2026-07-10 00:15
- b876b54needs attentionincremental2H · 2M · 5L2026-07-08 04:44
- ec4847fneeds attentionfull2H · 11M · 8L2026-07-08 03:32