← all branches

feat/shelly

blocked
4350d25 · fullPR #273reviewed 2026-07-08 02:36 UTC2H · 11M · 7L
The branch
Purpose
Port the Shelly Cloud integration (216 shellypro3em grid meters) into the v2 metrics engine at native 1h granularity
Goal
Metrics worker Lambda + CDK stack delivering 1h native collection with 30/30 local-day kWh parity vs legacy DDB
Sub-goals
  • SG-1: API probe + session strategy decision + worker + 50 unit tests
  • SG-2: CDK stack + main.ts + coordinator grants; synth clean
  • SG-3: Preview data plane via victron recipe (3 sites seeded)
  • SG-4: Live Site-SFN validation + Tinybird + 30/30 legacy parity (max Δ 0.085%)
  • SG-N: 9 framework learnings → scope.md + CLAUDE.md
The changes (whole branch)
What
Adds Shelly Cloud worker (per-invocation login + warm-container memo, 48h chunking, device-tz self-heal, grain verification); CDK Lambda + IAM stack; shelly-cloud manifest + 11 other brand manifests (foundation seam); seed catalog expanded with 12 Makes + Integration rows + 3 MetricType rows; seed-integration-arns REGISTRY + --only scoping; D2/D3 migration scripts
Why
Legacy prod fleet (115 Shelly devices) still collecting via batu-monorepo. Port to v2 engine enables unified monitoring, Tinybird storage, and future decommission of the legacy stack
Areas
services/metrics+18546scripts/metrics+17190packages/integration-manifests+10682infra/cdk+3601packages/database+25714.branch/scope.md+67167domains/core+161domains/metrics+54
Blast
8 areas touched, +4562/−198 net. Core risk surface: new Lambda worker (services/metrics) + CDK stack (infra/cdk). Seed expansions affect catalog + ARN routing but are idempotent upserts. Migration scripts are dry-run by default.
foundation-seam: 11 other brand manifests + seeds added for future ports
ci· No CI checks reported on PR 273coderabbit· No .coderabbit.yaml in repo

Findings · 22

correctness1

critical

shelly-cloud Integration seed has wrong credentialFields (api_key/access_token → must be user/password)

packages/database/src/seed-metrics-catalog.ts:165

Seed declares [api_key, access_token] but the manifest, worker loadCredentialFromArn, domain types, and probe evidence all require [user, password]. The seed comment explicitly says 'MIRRORS the manifest — keep the two in lockstep'. Impact: UI wizard prompts operators for wrong field names; secrets stored with wrong keys; loadCredentialFromArn returns null → MissingSecret on every collection for newly provisioned Shelly orgs. Fix: replace the two credentialFields entries with [{ name: 'user', label: 'Shelly Cloud account email', secret: false }, { name: 'password', label: 'Shelly Cloud password', secret: true }].

security4

medium

JWT user_api_url drives all stats fetches with only https:// prefix check — no hostname pinning

services/metrics/integrations/shelly/src/engine/shelly-client.ts:141

The shard URL extracted from the JWT user_api_url claim is only validated for startsWith('https://') before being used to construct all statistics fetch URLs. A compromised Shelly Cloud login endpoint could return a JWT with an arbitrary https URL and redirect the Lambda's fetches to any HTTPS endpoint. Mitigation: add new URL(apiBaseUrl).hostname.endsWith('.shelly.cloud') validation alongside the https:// check.

low

Vendor-controlled timezone string passed unvalidated to Intl.DateTimeFormat

services/metrics/integrations/shelly/src/translation/points-to-batu.ts:92

A malformed timezone from the Shelly stats response throws in tzOffsetMs; top-level catch converts to UpstreamUnavailable 502. No credential leak; effective collection DoS for that device. Mitigate with a try/catch probe on Intl.DateTimeFormat instantiation.

low

SHA1 password hashing — weak crypto, but Shelly Cloud API protocol requirement

services/metrics/integrations/shelly/src/engine/shelly-client.ts:97

SHA1 is cryptographically broken but this is what the Shelly Cloud API mandates (probe-verified). Accept as-is; document constraint explicitly in the comment.

low

Module-scope session memo accumulates tokens without size eviction bound

services/metrics/integrations/shelly/src/engine/shelly-client.ts:153

One entry per org-ARN accumulates until container recycles. At 115 Shelly orgs this is not a practical concern. Inherent to in-process session caching.

conventions3

medium

device-not-found mapped to TranslationFailed (statusCode 500) → JSend error instead of fail

services/metrics/integrations/shelly/src/handlers/metrics.lambda.ts:355

A decommissioned device is a known provisioning fact (4xx-class), not a server error. TranslationFailed with statusCode 500 classifies it as error in the JSend envelope, which may trigger unnecessary SFN retries and mislead monitoring. The InvocationError contract ties TranslationFailed to statusCode:500, so a fix requires either a new _tag (AssetDecommissioned) or a contract change.

medium

Missing mandatory CDK tags batu:env, batu:owner, batu:costCenter on Shelly stack

infra/cdk/src/stacks/services/metrics/integrations/shelly/lambda.stack.ts:49

infrastructure.md lists batu:env, batu:owner, batu:costCenter as mandatory. The Shelly stack applies only domain/subdomain/provider/service/dataClass. Same gap exists in growatt/egauge peers — systemic issue, but each new stack is an opportunity to fix it.

low

mapFetchError switch lacks exhaustiveness never-guard

services/metrics/integrations/shelly/src/handlers/metrics.lambda.ts:345

Adding a new ShellyFetchFailure.reason will cause a TypeScript error at the unhandled case, but no explicit default: never assertion makes this a guaranteed compile-time check.

tests9

high

No direct tests for shelly-client.ts login, JWT parsing, and session memo

services/metrics/integrations/shelly/src/engine/shelly-client.ts

login(), decodeJwtClaims(), and getSession() memo (expiry-margin hit/miss, forceRefresh, memo eviction) have zero independent test coverage. The handler tests mock getSession/fetchStatistics entirely. The wrong-shard → silent-200-all-missing failure mode (documented as a critical operational hazard) is not pinned by any test.

high

Initial getSession failure (login fails before any fetch) not covered by handler tests

services/metrics/integrations/shelly/src/handlers/metrics.lambda.ts:272

Existing auth tests only exercise invalid-token mid-collection (fetchStatistics fails). The case where the initial getSession call returns ok:false (e.g. auth-failed at login, rate-limited at login) is not tested. A broken credential that prevents initial login would produce an error of unpredictable tag.

medium

tz-correction across multiple chunks not verified

services/metrics/integrations/shelly/src/handlers/metrics.lambda.ts:280

The single-chunk tz test confirms the corrective refetch fires. It does not verify chunks 2+ use the device tz in their wall-clock strings. Real fleet has Europe/London devices in Mexican orgs. A bug where requestTz were re-initialized per chunk would corrupt UTC anchoring on chunks 2+.

medium

Mixed-granularity invocation (1h + 1d channels) not tested

services/metrics/integrations/shelly/src/handlers/metrics.lambda.ts:164

Handler explicitly supports multiple granularities in one call via the granularities Set. No test covers this path. A bug in the fetches Map iteration would silently return data from the wrong granularity's entries.

medium

Mid-window timezone change defensive guard (hard failure) not tested

services/metrics/integrations/shelly/src/handlers/metrics.lambda.ts:322

The guard at line 322 that hard-fails when a device reports a different timezone on chunk N vs chunk 1 is untested. A refactor removing it would be invisible.

medium

All-channels-unsupported handler path (payloads.length === 0 via pointsToBatu failure) not tested

services/metrics/integrations/shelly/src/handlers/metrics.lambda.ts:222

The handler path where ALL channels fail with unsupported-channel and payloads.length === 0 → failure is not exercised. Distinct from the all-missing-device path tested at line 262.

low

mapFetchError coverage missing for http-error, vendor-rejected, unparseable → UpstreamUnavailable

services/metrics/integrations/shelly/src/handlers/metrics.lambda.ts:273

The tag test covers rate-limited, timeout, unreachable, auth-failed. Three remaining cases (http-error, vendor-rejected, unparseable) map to UpstreamUnavailable and are not pinned.

low

Auth retry on non-first chunk (token expires mid-collection) not tested

services/metrics/integrations/shelly/src/handlers/metrics.lambda.ts:302

Existing test exercises invalid-token on the first and only chunk. A realistic scenario where chunks 1–2 succeed but chunk 3 returns invalid-token (JWT expired mid-collection) is not covered.

low

Session memo warm-container hit (no re-login on second invocation) not tested

services/metrics/integrations/shelly/src/engine/shelly-client.ts:165

clearSessionMemo() export exists as a test hook but is unused in tests. The memo saving ~250ms per warm invocation is not pinned.

improvement5

medium

alignWindowStart called twice with identical args per channel — second call is redundant

services/metrics/integrations/shelly/src/handlers/metrics.lambda.ts:211

alignedFrom is already computed at line 177 for the current granularity and passed into fetchGranularity. The identical call at line 211 (pointsToBatu fromUtc: arg) should reference alignedFrom directly.

medium

authRetried/tzCorrected flags are function-scope (per-granularity) but comments say 'per-chunk'

services/metrics/integrations/shelly/src/handlers/metrics.lambda.ts:280

Comment at line 301: 'One forced re-login on an expired/revoked token, then retry the chunk' implies per-chunk scope. Code is per-granularity (across all chunks). If a token expires between chunks, the second invalid-token returns UpstreamAuthFailed instead of refreshing again. Add a clarifying comment or restructure.

low

Content-Type: application/json header on GET statistics request

services/metrics/integrations/shelly/src/engine/shelly-client.ts:223

GET requests have no body; Content-Type is semantically meaningless and some strict proxies reject it. Only Authorization header is needed.

low

Tz helpers copied verbatim from victron — should be extracted to _shared/tz-helpers.ts

services/metrics/integrations/shelly/src/translation/points-to-batu.ts:86

12 more ports are planned. Each that uses wall-clock vendor APIs will copy these helpers and diverge. Extract to services/metrics/integrations/_shared/tz-helpers.ts (or @batu/tz-helpers).

low

INTERVALS array manually mirrors ShellyInterval union without type-level enforcement

services/metrics/integrations/shelly/src/engine/shelly-client.ts:186

Use `(['minute', 'hour', 'day'] as const satisfies ReadonlyArray<ShellyInterval>)` to make TypeScript enforce sync between the union and the runtime array.

History · 2 commits

  1. 4350d25blockedfull2H · 11M · 7L2026-07-08 02:36current
  2. 6c2d095safefull0H · 0M · 2L2026-07-07 21:09