feat/enphase
blockedviewing older commitf6e795d · fullpre-PRreviewed 2026-07-08 03:42 UTC4H · 14M · 12L · 3I- Purpose
- Port the Enphase Enlighten v4 integration (2 devices, org 33) into the v2 metrics engine with a deliberate auth redesign: legacy rewrote Secrets Manager on every OAuth refresh (secrets mutable, rotation hazard); new design keeps secrets READ-ONLY and caches the rotating access/refresh token pair in an auth-cache DynamoDB table (growatt/huawei precedent). Critical constraint: Enphase refresh tokens are single-use — each refresh mints a new refresh token, so the DDB row is the live SSOT after first use.
- Goal
- Deliver a production-ready Enphase Lambda worker + CDK stacks (auth-cache DDB + lambda) that collects solar_generation (channel e), grid_import (i_e), and grid_export (o_e) at 15m native granularity via Enlighten v4, with correct token-rotation state machine, unit tests green, synth clean, and a verified live data path (SG-3/SG-4).
- Sub-goals
- SG-1: Probe (legacy-liveness, token rotation semantics, finest grain, secret keys) + worker + token-rotation design + unit tests
- SG-2: CDK + auth-cache stack + main.ts + coordinator grant; synth clean
- SG-3: Preview data plane (proven recipe)
- SG-4: Live Site-SFN validation + Tinybird + legacy parity
- SG-N: Framework learnings fold-back
- What
- Added the complete Enphase v4 integration package (services/metrics/integrations/enphase/) including: metrics.lambda.ts handler (15m telemetry fetch with 7d chunking, channel fan-out, rate-limit pacing), enlighten-client.ts (Enphase API client with 401→refresh→retry-once), token-manager.ts (OAuth token rotation state machine with DDB conditional writes), token-store.ts (DDB r/w with lock/CAS), credentials.ts (Secrets Manager credential loader), points-to-batu.ts (Wh→W translation for all 3 channels + daily lifetime fallback), domain types, and tests. CDK: auth-cache DDB stack + Lambda stack + IAM + coordinator invoke grant wired in main.ts. Supporting: enphase manifest in integration-manifests, seed-metrics-catalog rows, seed-integration-arns entries, paths.ts handler entry. Scripts: legacy-registry-core + legacy-secrets-core migration utilities with tests.
- Why
- Legacy enphase integration rewrites Secrets Manager on each refresh (a shared grant contention hazard). This port removes that hazard, migrates to v2 metrics engine conventions (no bell-curve synthesis, native 15m cadence, FCIS error shape), and delivers the DDB-cached token-pair design validated against the 2-device fleet with a grant-contention safety constraint (branch collections must ride the copied access_token, never exercise the refresh path against the shared prod grant until cutover).
- Areas
- services/metrics/integrations/enphase+2308−0infra/cdk/src/stacks/services/metrics/integrations/enphase+338−0infra/cdk/src/app/main.ts+45−0infra/cdk/src/lib/paths.ts+185−0scripts/metrics+1375−0packages/integration-manifests/src/manifests+690−11packages/database/src+160−11packages/integration-manifests/src/__tests__+178−0
- Blast
- 56 files, +6274/-199 (excluding lockfile). New package: services/metrics/integrations/enphase (fully additive). CDK stacks: 2 new stacks (auth-cache DDB, lambda) + wiring in main.ts. Integration manifests: enphase added, 11 other manifests updated with live-verified credential fields. Database seeds: catalog rows + ARN seeds for enphase. Scripts: migration utilities for legacy registry/secrets. No existing production code modified.
Findings · 32
correctness6
Lock not released on CAS-loss + no-replacement-pair path (line 227)
services/metrics/integrations/enphase/src/lib/token-manager.ts:222
When casWritePair returns false (ConditionalCheckFailedException), rotateAsClaimHolder re-reads the row. If the concurrent winner's row has no accessToken (postRow?.accessToken absent), the function returns 'refresh-unavailable' at line 227 WITHOUT calling releaseLock. The lock remains held for up to 90 seconds, blocking all concurrent invocations from rotating. Fix: call await ports.releaseLock(hash) before returning at line 227.
Rotation retry after 401 not routed through paced() — rate-limit exposure
services/metrics/integrations/enphase/src/handlers/metrics.lambda.ts:212
The retry at line 212 after a 401→rotate is a direct fn(accessToken) call, not routed through paced(). So callCount is not incremented for the retry, and in multi-chunk scenarios the retry + the next paced call can land back-to-back, potentially violating the 10/min Watt-plan budget. Fix: route the retry through paced(), or add await sleep(deps.callSpacingMs) + callCount++ before line 212.
bad-secret reason in TokenAcquireFailure is dead — never emitted but handled
services/metrics/integrations/enphase/src/domain/types.ts:148
TokenAcquireFailure includes { reason: 'bad-secret' } and mapTokenFailure handles it, but acquireAccessToken never produces this reason. Dead branch creates a false safety impression. Either emit bad-secret from acquireAccessToken when the credential is structurally invalid, or remove the variant.
Stale secret access_token cold-start costs one extra vendor call
services/metrics/integrations/enphase/src/lib/token-manager.ts:113
On cold start with no DDB row, acquireAccessToken returns the secret's access_token without checking expiry. If the token is already expired (>24h since last rotation), the first vendor call 401s, triggers rotation, and retries — one extra call. This is the designed fallback path but is worth documenting as the known cold-start behavior.
Grid chunk: transient total_devices=1 on a truly meter-less system stores fabricated zeros
services/metrics/integrations/enphase/src/handlers/metrics.lambda.ts:324
mergeTelemetryChunks takes Math.max(total_devices) across chunks to prevent a transient-0 from vetoing the channel. If a system genuinely has no meter but one chunk transiently reports total_devices=1, zeros are stored. The inverse risk (missing data) is correctly considered worse. No fix required, but an explicit test for the mixed-devices case would document the chosen semantics.
401→rotate→retry not counted in vendorCalls telemetry (undercount by 1)
services/metrics/integrations/enphase/src/handlers/metrics.lambda.ts:306
The retry call at line 212 does not increment callCount so vendorCalls in the success log undercounts by 1 when a rotation occurs. Does not affect correctness but misleads rate-limit diagnostics.
security5
releaseLock is unconditional — can release a concurrent winner's lock after expiry
services/metrics/integrations/enphase/src/lib/token-store.ts:85
releaseLock unconditionally sets lockUntil = 0 with no ConditionExpression. If the original lock holder's refreshGrant call takes longer than LOCK_SECONDS=90s (network stall), the lock expires and a second invocation can acquire it. When the first invocation finally returns and calls releaseLock, it zeros out the second holder's active lock, opening a window for a third concurrent refresh — the exact double-spend race the lock was designed to prevent. Fix: add ConditionExpression: 'lockUntil = :myExpiry' to releaseLock so only the rightful holder can release it.
DDB auth-cache table has no encryption at rest
infra/cdk/src/stacks/services/metrics/integrations/enphase/auth-cache.stack.ts:70
The DynamoDB table storing live OAuth token pairs uses default AWS-owned key encryption, inconsistent with the CFE session auth-cache which uses TableEncryption.AWS_MANAGED. This table is classified batu:dataClass=confidential. Fix: add encryption: TableEncryption.AWS_MANAGED to the Table construct.
DDB auth-cache table has no point-in-time recovery
infra/cdk/src/stacks/services/metrics/integrations/enphase/auth-cache.stack.ts:70
The stack comment says rows must NEVER be reaped because losing a row orphans the rotation chain and forces full CS re-auth. Despite this, pointInTimeRecovery is not enabled. If data is accidentally deleted, recovery requires a manual Enphase re-authorization. Fix: add pointInTimeRecovery: true.
refresh_token embedded in fallback OAuth URL — may appear in CloudWatch error logs
services/metrics/integrations/enphase/src/engine/enlighten-client.ts:54
The fallback OAuth token request (attempt index 1) appends the refresh token as a URL query param: ?grant_type=refresh_token&refresh_token=<value>. On network-level errors, the undici runtime may include the full URL in e.message, which propagates to the dead-grant error detail logged to CloudWatch. Fix: send refresh_token as POST body for both attempts, or redact it in error messages.
IAM wildcard also matches /portal human-login secrets
infra/cdk/src/stacks/services/metrics/integrations/enphase/iam.ts:44
The GetSecretValue grant uses ${env}/enphase/* which matches /portal/oid=33 (human username/password) in addition to the API credential secrets. The credential loader rejects the wrong payload, but the Lambda role could read human portal credentials. Tighten path to ${env}/enphase/oid=* to exclude /portal subtree.
conventions4
Manifest externalVariableId uses legacy vendor codes — SFN invocations will 100% fail
packages/integration-manifests/src/manifests/enphase.ts:49
The manifest declares externalVariableId as 'e', 'i_e', 'o_e' (legacy vendor codes). The provisioning shell stores MetricSource.sourceConfig.externalVariableId from this value and passes it to the worker via SFN. The worker's channel routing only accepts 'solar_generation', 'grid_import', 'grid_export' — so every SFN-path invocation fails with TranslationFailed: unsupported Enphase channel "e". Fix: change externalVariableId to the canonical variable names (solar_generation, grid_import, grid_export) in the manifest. The vendor-to-endpoint mapping ('e' → production_meter, etc.) belongs inside the worker, not the manifest field. NOTE: unit tests mask this because they invoke the worker directly with canonical names.
EnphaseFetchFailure uses `reason` discriminant instead of `_tag`
services/metrics/integrations/enphase/src/domain/types.ts:125
Per canonical-form.md, discriminated union errors use _tag. EnphaseFetchFailure and TokenAcquireFailure use reason as discriminant. These are internal types that never cross the package boundary (mapped to InvocationError._tag before return), but the dual-convention inside the package can mislead future contributors. Recommend adding a JSDoc note or aligning to _tag.
deploy-order comment does not mention Enphase stack dependencies
infra/cdk/src/app/main.ts:299
The deploy-order comment block only mentions Growatt. The Enphase addDependency wiring (lambda→authCache, coordinator→lambda) is correct but undocumented in the comment. Future engineers adding new integrations will miss the Enphase pattern. Minor doc gap.
Manifest externalVariableId comment describes legacy codes that will become a bug
packages/integration-manifests/src/manifests/enphase.ts:15
After fixing the critical finding, the header comment should be updated to clarify that the vendor channel mapping ('e'→production_meter etc.) is internal to the worker, not stored on the manifest field.
tests12
forceRotate: no test for refreshGrant throwing (vs returning unavailable)
services/metrics/integrations/enphase/src/__tests__/token-manager.test.ts:304
token-manager.ts:194-199 catches a thrown exception from refreshGrant and maps it to { ok: false, reason: 'refresh-unavailable' }. The existing test exercises the path where the vendor returns { ok: false } but not where it throws. Add a test where ports.refreshGrant throws (e.g. new Error('ECONNRESET')) and assert result.reason === 'refresh-unavailable' and store.row?.lockUntil === 0.
Second-chunk 401 after rotation (rotationSpent guard) untested
services/metrics/integrations/enphase/src/__tests__/metrics-handler.test.ts:251
metrics.lambda.ts:203 sets rotationSpent = true after a rotation. If a second chunk gets a 401, no re-rotation happens — result converts to auth-expired → UpstreamAuthFailed. This is correct behavior (rotate-once) but is completely untested. Add: 20-day window, first chunk succeeds, second chunk returns 401 (rotation already spent), expect UpstreamAuthFailed.
acquireAccessToken: expired-row-with-same-refresh-token path untested
services/metrics/integrations/enphase/src/__tests__/token-manager.test.ts:124
The branch where a row exists but its refreshToken hasn't yet diverged from the secret's AND the secret's access token is expired falls through to source: 'secret'. No test covers this. Add: row present, accessTokenExpiresAt in the past, row.refreshToken === credential.refresh_token — expect source: 'secret' without a vendor call.
projectLifetime: grid_import/grid_export channels never tested
services/metrics/integrations/enphase/src/__tests__/points-to-batu.test.ts:215
The projectLifetime describe block only exercises channel: 'solar_generation'. The grid_import → import and grid_export → export field projections (LIFETIME_FIELD constant) are not tested. Add a test where the response includes an import array and channel: 'grid_import' to confirm the correct field is picked.
Handler: multi-source fan-out 401 after solar completes untested
services/metrics/integrations/enphase/src/__tests__/metrics-handler.test.ts:184
paced() shares rotationSpent and accessToken across ALL channels. If solar succeeds (consuming the rotation budget), then a grid channel gets a 401, rotationSpent is true so retry is skipped and the result is auth-expired → UpstreamAuthFailed. This cross-channel interaction is completely untested.
Handler: credential load failure path untested
services/metrics/integrations/enphase/src/__tests__/metrics-handler.test.ts:141
The loaded.ok === false path (metrics.lambda.ts:184-186) is never exercised. Add a test where deps.loadCredential resolves { ok: false, details: 'permission denied' } and assert MissingSecret error tag.
Handler: mergeTelemetryChunks total_devices max-across-chunks logic untested
services/metrics/integrations/enphase/src/__tests__/metrics-handler.test.ts:199
The max-devices merge logic at metrics.lambda.ts:324 (preventing a transiently-zero chunk from vetoing a channel) is never asserted. Add a test where chunk 1 has total_devices=1 and chunk 2 has total_devices=0, asserting the channel is still projected.
Translation: cadence guard single-interval boundary case untested
services/metrics/integrations/enphase/src/__tests__/points-to-batu.test.ts:120
dominantSpacingSec returns null when endAts.length < 2, making cadenceMismatch return null (valid). A single-slot response is valid but produces one point. No test covers this boundary. Add: response with exactly 1 interval for a 15m channel, assert ok: true with 1 point.
Translation: grid response with non-array inner chunk untested
services/metrics/integrations/enphase/src/__tests__/points-to-batu.test.ts:162
intervalsOf returns null when an inner chunk is not an array (!Array.isArray(inner)). No test sends intervals: [null] or intervals: [42] to assert malformed-response.
Translation: projectLifetime invalid start_date guard untested
services/metrics/integrations/enphase/src/__tests__/points-to-batu.test.ts:215
points-to-batu.ts:289-295 validates that start_date matches YYYY-MM-DD. No test exercises this guard. Add: response.start_date = 'Jul 1 2026', assert ok: false, reason: 'malformed-response'.
credentialHashOf: api_key change not tested as a non-re-keying case
services/metrics/integrations/enphase/src/__tests__/token-manager.test.ts:323
token-manager.ts:71-74 hashes client_id:refresh_token (api_key intentionally excluded). The hash test verifies refresh_token change re-keys, but there's no negative test confirming api_key change does NOT re-key. This design contract is worth asserting.
FakeStore hash assertion bypassed in patched concurrent-rotation tests
services/metrics/integrations/enphase/src/__tests__/token-manager.test.ts:60
In tests that patch ports.casWritePair, the expect(hash) assertion in FakeStore.casWritePair is bypassed because the patch calls origCas only after mutating state. Cosmetic — real semantic behavior is still tested.
improvement5
success/failure envelope helpers copy-pasted across every integration
services/metrics/integrations/enphase/src/handlers/metrics.lambda.ts:387
The success<T>() and failure() functions are structurally identical across growatt, egauge, and enphase handlers. A pair of factory helpers (makeSuccess<T> / makeFailure) belongs in services/metrics/engine/src/lib/invocation-result.ts. This will be replicated in every new integration (10+ brands still to be ported).
Timezone/cadence helpers copy-pasted verbatim from victron
services/metrics/integrations/enphase/src/translation/points-to-batu.ts:59
The comment explicitly says '— victron `points-to-batu.ts` verbatim'. Functions tzOffsetMs, localDateOf, localMidnightUtcMs, addDays, dominantSpacingSec are copied across integrations. Should be extracted to services/metrics/integrations/shared/translation-utils.ts or extended in @batu/metrics-domain before additional brands are ported.
No-op .map(inner => inner) in mergeTelemetryChunks grid branch
services/metrics/integrations/enphase/src/handlers/metrics.lambda.ts:340
gridChunks.flatMap((c) => (c.intervals ?? []).map((inner) => inner)) — the .map(inner => inner) is a no-op identity transform. The comment says 'keep nesting' but the flatMap already preserves it correctly. Simplify to gridChunks.flatMap((c) => c.intervals ?? []).
Duplicate sleep() definition in enlighten-client.ts and metrics.lambda.ts
services/metrics/integrations/enphase/src/engine/enlighten-client.ts:34
const sleep = ... is defined identically in both files. The client's copy could be removed in favor of a single shared definition, or the client could accept the sleep function through dependency injection like the handler does.
windowDays computed inside per-channel loop but is window-wide
services/metrics/integrations/enphase/src/handlers/metrics.lambda.ts:172
The (toUtc - fromUtc) / 86_400_000 computation runs on every iteration of the channels loop but the window is fixed. Minor clarity improvement: move it above the loop.