feat/int-base

needs attentionviewing older commit

dd403fe · incrementalpre-PRreviewed 2026-07-07 20:02 UTC12H · 22M · 12L · 2I

The branch

Purpose

Land every shared seam the 12 device-integration ports depend on — manifests, catalog seeds, ARN-seed registry entries, CDK path entries, and the legacy-registry/credential migration scripts — so each per-brand branch only touches its own vendor directories.

Goal

All 6 sub-goals (SG-1 through SG-N) complete: 12 manifests, catalog seeds, ARN REGISTRY, CDK paths.ts, D3 migration script, D2 credential-copy script, and framework doc improvements.

Sub-goals

SG-1: 12 IntegrationManifests registered (done — commit 0380b0ca)
SG-2: Catalog seed rows for 12 Makes + Integrations + MetricTypes battery/load (done — 648aafa2)
SG-3: ARN-seed REGISTRY entries + --only scoping (done — fb2c36af)
SG-4: CDK paths.ts handler entries for 12 vendors (done — 486fb9ed)
SG-5: D3 registry-migration script, dry-run default, validated on prod snapshot (done — db23e8ff)
SG-6: D2 credential-copy script + SecretProvider widening (done — 9cc267ed)
SG-N: Framework docs — retire stale manifest co-location guidance + porting runbook (done — 9e2a53c8)

The changes (whole branch)

What: Incremental from e663ae90: SG-3 ARN seed REGISTRY (12 entries + --only scoping), SG-4 CDK paths.ts (12 vendor blocks), SG-5 D3 migrate-legacy-registry.ts + tests, SG-6 D2 copy-legacy-secrets.ts + SecretProvider widening + tests, SG-N framework doc improvements.
Why: Pre-flight base for 12 brand branches. Each port branch will only need to add its own vendor Lambda/CDK stack; all shared seams (manifests, seeds, CDK paths, migration scripts) are here.
Areas: scripts/metrics/+1602−0packages/integration-manifests/+1060−2packages/database/+257−14infra/cdk/+185−0services/metrics/integrations/+38−6domains/core/+16−1
Blast: 34 files, +3268/−187 across 6 areas; migration scripts are dry-run by default; no production data written by this branch

D2-before-D3-ordering-critical dangling-ARN-on-full-seed-without---only portal-variant-live-prod-finding

➖CI· no self-hosted runner — workflow gated on LOOP_REVIEW_ENABLED➖coderabbit· no .coderabbit.yaml

Findings · 29

correctness8

high

egaugeColumn not set on direct-match path

scripts/metrics/lib/legacy-registry-core.ts:320

eGauge streams always need egaugeColumn but the field is only set on the canonical-fallback path (step 2). If an eGauge variable has a direct externalVariableId match (step 1), deviceLocalColumn is false and egaugeColumn is absent, causing downstream provisioning failure. Test only exercises fallback path.

high

No warning when --only omitted — dangling ARNs for all 12 undeployed integrations

packages/database/src/seed-integration-arns.ts:146

Seeding without --only writes dangling ARNs for every undeployed integration, turning clean WorkerArnMissing rejections into opaque Lambda ResourceNotFound errors at invocation time. No confirmation prompt or warning log exists.

high

oid sort assumes numeric but relies solely on regex guard

scripts/metrics/lib/legacy-secrets-core.ts:57

buildSecretCopyPlan sorts unmappedOids with Number(a)-Number(b). Non-numeric oids return NaN, corrupting sort order. Fragile if parseLegacySecretName regex is ever relaxed.

medium

--overwrite flag does not update secretArn on existing SC rows

scripts/metrics/copy-legacy-secrets.ts:251

--overwrite updates the Secrets Manager value but reuses existing scId without updating secretArn, desyncing SecretConfiguration.secretArn from actual ARN.

medium

Devices with zero resolvable streams silently skipped in apply

scripts/metrics/migrate-legacy-registry.ts:152

applySupported:true devices with all variables unmapped appear planned but are silently skipped. Undetectable from exit code.

medium

DescribeSecret ARN cast as string without null guard

scripts/metrics/copy-legacy-secrets.ts:153

existing.ARN cast as string without null check — undefined ARN would be persisted to SecretConfiguration.secretArn if secret is in pending-deletion state.

low

correlationId collision in device loop (Date.now())

scripts/metrics/migrate-legacy-registry.ts:222

Fast consecutive iterations share the same millisecond timestamp as correlationId.

low

correlationId collision in secrets loop

scripts/metrics/copy-legacy-secrets.ts:258

Same Date.now() collision risk.

security5

high

Unvalidated file path in --org-mapping / --snapshot — path traversal

scripts/metrics/copy-legacy-secrets.ts:289

readFileSync on user-supplied CLI arg with no path restriction. Admin-only context lowers risk but the pattern is unsafe. Same issue in migrate-legacy-registry.ts --snapshot (line 71).

high

DB credentials may appear in CI logs on connection failure

scripts/metrics/copy-legacy-secrets.ts:185

postgres() errors can include the connection string; console.error(e) in catch block could leak POSTGRES_URL with embedded credentials. Same risk in migrate-legacy-registry.ts line 149.

medium

AWS account IDs hardcoded in source

packages/database/src/seed-integration-arns.ts:30

Hardcoded dev/stg/prod account IDs enable account enumeration. Load from SSM/env instead.

medium

No source==target account guard in copy-legacy-secrets

scripts/metrics/copy-legacy-secrets.ts:63

Running with same source and target account could overwrite prod secrets with no guard.

low

Binary secrets silently skipped — partial migration without clear operator awareness

scripts/metrics/copy-legacy-secrets.ts:129

Errors logged to stderr only; an operator not reading stderr could believe migration completed cleanly.

conventions6

medium

SECRET_PROVIDERS constant duplicates SecretProvider union — dual SSOT

packages/database/src/schema/secret-configurations.ts:134

Three copies of the provider list must be kept in sync manually. Types should flow in one direction per canonical-form.md.

medium

correlationId non-unique in shell loops (Date.now() collision)

scripts/metrics/migrate-legacy-registry.ts:223

Date.now() inside device loop can produce duplicate correlationIds for fast consecutive iterations. Same issue in copy-legacy-secrets.ts:259.

low

Raw Drizzle / no createRLSDb in migration shells

scripts/metrics/migrate-legacy-registry.ts:149

Works under service_role but deviates from RLS convention. Double-cast as unknown as Database suppresses type safety.

low

Same raw Drizzle / no createRLSDb in copy-legacy-secrets

scripts/metrics/copy-legacy-secrets.ts:186

Same pattern as migrate-legacy-registry.ts.

info

INTEGRATION_STANDARDS §1 folder tree still shows retired manifest.ts

services/metrics/integrations/INTEGRATION_STANDARDS.md:49

Tree entry for {integrationCode}/src/manifest.ts remains even though prose in same doc says it's retired.

info

paths.ts pre-declares handlers for undeployed integrations — acknowledged by design

infra/cdk/src/lib/paths.ts:236

Pure join() data, no synth impact until a brand stack references it. Flagged info only for visibility.

tests5

high

No test for brands filter + multi-device collision interaction

scripts/metrics/__tests__/legacy-registry-core.test.ts:231

When only one of two co-sited devices passes the brands filter, the collision is missed and the filtered-in device is incorrectly marked applySupported.

high

No test for portal/non-portal coexistence for same oid+brand

scripts/metrics/__tests__/legacy-secrets-core.test.ts:79

Key live-prod invariant: both prod/huawei/oid=11 AND prod/huawei/portal/oid=11 must emit distinct SecretCopyPlanItems. Not tested.

medium

solis/hoymiles/victron/enphase portal variants missing from parseLegacySecretName tests

scripts/metrics/__tests__/legacy-secrets-core.test.ts:24

Portal-variant test only covers shelly/huawei. Branch context lists these 4 brands as portal-capable.

medium

growatt-cons test missing granularity assertion

scripts/metrics/__tests__/legacy-registry-core.test.ts:161

Unlike hoymiles/powerradar tests, growatt-cons has no granularity assertion.

low

helioscope-discovery test validates call counts, not query conditions

domains/metrics/src/metric-source/__tests__/helioscope-discovery.queries.test.ts:56

Chain-mock approach: join count assertions pass even if conditions swap. Billing-grade query deserves integration test.

improvement5

high

Duplicated arg()/flag() CLI helpers across both scripts

scripts/metrics/copy-legacy-secrets.ts:67

Identical helper functions in both scripts; will be repeated in 12 brand-branch scripts. Extract to scripts/metrics/lib/cli.ts.

high

LEGACY_INTEGRATION_TO_MANIFEST_ID and SECRET_BRAND_TO_INTEGRATION_ID are near-duplicate maps

scripts/metrics/lib/legacy-registry-core.ts:41

Both map brand names to integration IDs for the same set with slightly different key names. Single shared integration-registry.ts would eliminate the silent maintenance trap.

medium

Per-device site lookup in apply loop — N queries instead of 1 batch

scripts/metrics/migrate-legacy-registry.ts:192

db.select().from(sites) runs once per device. Use inArray() like the make/integration/metricType lookups above.

medium

attr() DynamoDB unmarshaller reimplements @aws-sdk/util-dynamodb unmarshall()

scripts/metrics/lib/legacy-registry-core.ts:169

SDK ships unmarshall() which handles SS/NS/BS/NULL/nested M/L. Custom implementation has untested edge cases.

low

paths.ts: 26× repeated join() pattern — integrationPaths() helper

infra/cdk/src/lib/paths.ts:39

Each integration entry is 8-12 lines of identical structure. A helper would enforce the path convention and save ~160 lines.

History · 11 commits

f7b9554safeincremental0H · 0M · 0L2026-07-13 20:18
9ca23adneeds attentionincremental0H · 3M · 3L2026-07-13 19:17
595484fneeds attentionincremental1H · 3M · 7L2026-07-13 04:29
2ed62f4safeincremental0H · 0M · 0L2026-07-13 04:13
2812f54needs attentionincremental1H · 3M · 10L2026-07-11 00:19
1794b23needs attentionincremental2H · 2M · 5L2026-07-10 21:59
e5674dcneeds attentionincremental3H · 7M · 6L2026-07-09 19:29
ca45a96needs attentionincremental0H · 3M · 3L2026-07-09 18:56
5d19484safeincremental0H · 0M · 1L2026-07-08 02:04
dd403feneeds attentionincremental12H · 22M · 12L2026-07-07 20:02current
e663ae9needs attentionincremental2H · 7M · 7L2026-07-07 19:02