← all branches

feat/int-base

needs attentionviewing older commit
e663ae9 · incrementalpre-PRreviewed 2026-07-07 19:02 UTC2H · 7M · 7L · 3I
The branch
Purpose
Land every shared seam the 12 device-integration ports depend on — manifests, catalog seeds, ARN-seed registry entries, CDK path entries, and legacy-registry/credential migration scripts — so each per-brand branch only touches its own vendor directories.
Goal
All 12 integration manifests + catalog seed rows complete; SG-3 through SG-6 remain
Sub-goals
  • SG-1 ✓ — 12 manifests + registration (commit 0380b0ca)
  • SG-2 ✓ — Catalog seed rows: +11 Makes, +12 Integrations, +3 MetricTypes (commit 648aafa2)
  • SG-3 — ARN-seed REGISTRY entries + missing-Lambda tolerance
  • SG-4 — CDK paths.ts handler entries
  • SG-5 — D3 registry-migration script
  • SG-6 — D2 credential-copy script
  • SG-N — Framework improvements
The changes (whole branch)
What
SG-2 complete: seed-metrics-catalog.ts gained +11 Make rows (hoymiles, shelly, huawei, solark, fronius, apsystems, abb, victron, enphase, sma, powerradar), +12 Integration rows (names = manifest ids, credentialFields mirroring manifests), +3 MetricType rows (battery_charge, battery_discharge, load — W/mean). .branch/scope.md marked SG-1 and SG-2 done. .branch/intent.md added SG-2 iteration log.
Why
Catalog rows must exist before ARN seeding (SG-3) and the D3 migration (SG-5) can proceed — manifests validate supportedMakes against Make.name and metricTypeVariable against MetricType.variable; unseeded rows cause every provision to reject.
Areas
packages/database+16011packages/integration-manifests+10642.branch+114159
Blast
18 files, +1282/-15 lines. Entirely additive: new manifests (pure data, no runtime coupling) + seed rows (additive upserts, idempotent). Zero impact on existing Growatt/eGauge/HelioScope paths. egauge.test.ts fix (pre-existing red) is the only change to a prior test.
solis Make comment stale — contradicts live Integration row key_id/AccessKeyId secret classification needs decision before SG-6
CI/PR checks· No open PR for feat/int-baseCodeRabbit· No .coderabbit.yaml in repotypecheck (per intent log)· pnpm --filter @batu/database typecheck passed (recorded in .branch/intent.md)lint+test (per intent log)· pnpm turbo run lint test --filter='@batu/database' — 4 tasks pass (recorded in .branch/intent.md)seed idempotency (per intent log)· Run 1: 13 Makes + 17 Integrations + 9 MetricTypes created. Run 2: 41 exists, 0 created — proven live on feat/int-base preview DB

Findings · 21

correctness4

high

Stale MAKE_SEEDS comment falsely states solis has no IntegrationManifest

packages/database/src/seed-metrics-catalog.ts:62

Lines 62–64 comment: 'Solis intentionally has no matching IntegrationManifest registered today — the wizard surfaces a "not available yet" state when the user picks it.' This is no longer true: solis IS in INTEGRATION_SEEDS and solisManifest is fully registered in manifests/index.ts. Any UI code or future developer reading this comment as authoritative will incorrectly treat solis as unavailable. Remove or replace with the accurate status.

medium

Victron externalVariableId 'solarGeneration' camelCase — forward-compat risk for brand branch

packages/integration-manifests/src/manifests/victron.ts:42

victron.ts sets externalVariableId: 'solarGeneration' (camelCase). This is intentional per the prod registry snapshot. The risk is that the brand-branch worker may normalize to snake_case and mismatch. The test locks this value, so a normalization bug in the worker would be silent at manifest level. The brand branch must verify the exact VRM API response field casing.

low

APsystems header comment documents legacy key as 'solarGeneration' (camelCase) vs actual key 'solar_generation'

packages/integration-manifests/src/manifests/apsystems.ts:13

The header comment documents the legacy mapping as 'solarGeneration→produced'. The actual manifest variable key is 'solar_generation'. The externalVariableId 'produced' is correct. Documentation-only inconsistency but could mislead brand-branch developer looking up the legacy channel.

low

powerradar seed has credentialFields: [] but manifest has no credentialFields property — absent vs empty inconsistency

packages/database/src/seed-metrics-catalog.ts:259

The powerradar seed stores credentialFields: [] while the manifest declares no credentialFields property. Any FE code guarding on credentialFields !== undefined vs credentialFields.length > 0 will behave differently for powerradar vs other requiresAuth:false manifests. Verify the wizard handles both representations.

security4

medium

solis key_id marked secret: false — HMAC credential-pair ID should be secret

packages/integration-manifests/src/manifests/solis.ts:42

SolisCloud HMAC auth uses (KeyID, KeySecret). The KeyID is not a public identifier — it's the credential's username equivalent, sufficient to enumerate the Batu account's integration. Marking secret: false stores it unredacted in JSONB and renders it in cleartext in the FE wizard. Same classification question applies to Fronius AccessKeyId. Recommendation: mark both secret: true.

medium

Fronius AccessKeyId marked secret: false — same HMAC-pair argument as solis

packages/integration-manifests/src/manifests/fronius.ts:39

Solar.web AccessKeyId is the public half of a (AccessKeyId, AccessKeyValue) pair. Like Solis's KeyID, it is not a public registry value and could be used to enumerate the integration owner. Stored unredacted in integration.metadata JSONB and shown in cleartext in the wizard.

low

dotenv config() silently no-ops when POSTGRES_URL already set — missing error surfaces as cryptic DB failure

packages/database/src/seed-metrics-catalog.ts:25

Guard `if (!process.env.POSTGRES_URL) { config(...) }` skips dotenv when POSTGRES_URL is set, even if .env.local is missing other required vars. dotenv's error is also swallowed (config() returns error without throwing). Add `const result = config(...); if (result.error) throw result.error;` for actionable failures.

low

Seed↔manifest credentialFields drift is an operational security risk

packages/database/src/seed-metrics-catalog.ts:151

If a manifest's credentialFields are updated (e.g. adding a field, flipping secret: true) without re-running the seed against each environment, the FE wizard renders the old classification. A field intended to be secret:true could be shown in cleartext until the seed is redeployed.

conventions5

medium

Redundant side-effect import in test file

packages/integration-manifests/src/__tests__/device-integrations.test.ts:27

Line 27 does `import '../manifests'` as an explicit side-effect import, but lines 11-24 already import named exports from the same module. A named import triggers top-level side effects (the registerManifest() calls). The bare import on line 27 is redundant and implies the named imports are insufficient — which is false. Remove it.

medium

shelly-cloud.ts secret path says 'portal' but integration id is 'shelly-cloud' — D2 copier mismatch risk

packages/integration-manifests/src/manifests/shelly-cloud.ts:21

The JSDoc Auth section documents the legacy secret path as `{env}/shelly/portal/oid={oid}`. The word 'portal' mirrors Growatt convention but the integration id is 'shelly-cloud', not 'shelly-portal'. The D2 secret copier must map '/shelly/portal/oid=' → '/shelly/cloud/oid='. This mismatch should be explicitly called out in the comment so the brand branch doesn't silently copy to the wrong path.

medium

ABB consumption/demand variable semantic mismatch: MetricType is W/mean but source is kWh-interval

packages/integration-manifests/src/manifests/abb.ts:44

The ABB manifest JSDoc states both channels 'derive from the meter's ActiveEnergy series' (cumulative kWh). The granularities ['15m', '1h'] are consistent with interval-energy, not demand polling. But MetricType.unit is 'W' and aggregator is 'mean'. The brand branch must clarify whether the ABB worker emits converted W or raw kWh, and add a note explaining the derivation — otherwise the ingest layer will mis-aggregate.

low

Inline variable comments describe what, not why

packages/integration-manifests/src/manifests/shelly-cloud.ts:43

Comments like '// Grid import — the meter's forward energy channel (vendor 'consumption').' restate what's visible from externalVariableId. Same pattern in huawei-portal.ts and abb.ts. Comments should carry the why — e.g. for Shelly, note that 'consumption' is Shelly Cloud's API field name for forward energy (not the eGauge convention), and 'reversed' is their field for export without a sign convention.

info

All supportedMakes values and export constant names conform to conventions

packages/integration-manifests/src/manifests/index.ts

Positive: every manifest's supportedMakes contains the hardware brand name (plain, no access-path suffix); export constants follow {BRAND}_INTEGRATION_ID; battery/load MetricType seeds use unit:'W' and aggregator:'mean'; INTEGRATION_SEEDS names match manifest ids exactly.

tests5

high

No test enforcing seed↔manifest credentialFields mirror invariant

packages/integration-manifests/src/__tests__/device-integrations.test.ts

The seed manually duplicates each manifest's credentialFields (12 brands; enphase has 5 fields, abb has 3, fronius has non-obvious casing 'AccessKeyId'/'AccessKeyValue'). No test asserts seed.metadata.credentialFields deep-equals manifest.credentialFields. A mismatch would silently give the credential wizard a wrong field set. Add a cross-package test or seed-package test that imports both the seed data and the manifests package.

medium

powerradar sourceConfigSchema shape not fully verified

packages/integration-manifests/src/__tests__/device-integrations.test.ts:166

Test asserts `sourceConfigSchema?.required` contains 'powerradarChannel' but doesn't verify the properties shape (type: 'string'), schema type ('object'), or that no extra required fields crept in. Compare with egauge.test.ts which checks schema.properties.egaugeColumn.type. Since powerradarChannel is the only channel mapping, a schema regression would cause silent intake failures.

medium

externalVariableId spot-checks missing for abb, enphase grid channels, fronius grid channels

packages/integration-manifests/src/__tests__/device-integrations.test.ts:128

The spot-check test has gaps: (1) abb — neither demand nor consumption externalVariableId is checked; (2) enphase — only solar_generation is tested; (3) fronius — only solar_generation; (4) sma grid_import ('gridConsumption') is missing from both the main spot-check and the battery test block.

low

toBeGreaterThan(0) assertions on variable count and credentialFields count are too loose

packages/integration-manifests/src/__tests__/device-integrations.test.ts:104

Line 104 asserts fields.length > 0 and line 113 asserts entries.length > 0. Enphase dropping 3 of 5 credential fields or sma losing battery_charge would pass. Asserting exact per-brand counts would catch silent regressions.

info

egauge.test.ts confirms 15m granularity fix is in place

packages/integration-manifests/src/__tests__/egauge.test.ts:32

Pre-existing egauge test asserts granularities: ['5m', '15m']. The fix from SG-1 is correctly in place. egauge.test.ts is a good benchmark for assertion depth that the newer brand-specific tests should match.

improvement3

low

credentialFields duplicated in seed — import or factory would prevent drift

packages/database/src/seed-metrics-catalog.ts:151

The seed could import @batu/integration-manifests and derive credentialFields from manifest.credentialFields ?? [] directly, making the copy structurally impossible. The 'DB package stays free of that dep' constraint should be re-evaluated against the maintenance cost of 12 hand-kept mirrors. A shared zero-dep sub-path export would serve as a middle ground.

low

CATALOG_VARIABLES in test is a hand-enumerated copy of METRIC_TYPE_SEEDS

packages/integration-manifests/src/__tests__/device-integrations.test.ts:49

Lines 49–58 define a local CATALOG_VARIABLES set identical to METRIC_TYPE_SEEDS.variable values. A future branch adding a new MetricType must update both places or tests fail unexpectedly. Export a CATALOG_VARIABLE_NAMES constant from the seed or a shared types location.

info

Spot-check test is a flat list — a data table would make additions a one-liner

packages/integration-manifests/src/__tests__/device-integrations.test.ts:128

The 22 individual expect() calls in 'carries the legacy vendor channel ids' have no structure. A CHANNEL_SPOT_CHECKS data table iterated with a single for-loop would make brand-branch additions a one-liner and surface the full channel registry at a glance.

History · 11 commits

  1. f7b9554safeincremental0H · 0M · 0L2026-07-13 20:18
  2. 9ca23adneeds attentionincremental0H · 3M · 3L2026-07-13 19:17
  3. 595484fneeds attentionincremental1H · 3M · 7L2026-07-13 04:29
  4. 2ed62f4safeincremental0H · 0M · 0L2026-07-13 04:13
  5. 2812f54needs attentionincremental1H · 3M · 10L2026-07-11 00:19
  6. 1794b23needs attentionincremental2H · 2M · 5L2026-07-10 21:59
  7. e5674dcneeds attentionincremental3H · 7M · 6L2026-07-09 19:29
  8. ca45a96needs attentionincremental0H · 3M · 3L2026-07-09 18:56
  9. 5d19484safeincremental0H · 0M · 1L2026-07-08 02:04
  10. dd403feneeds attentionincremental12H · 22M · 12L2026-07-07 20:02
  11. e663ae9needs attentionincremental2H · 7M · 7L2026-07-07 19:02current