← all branches

feat/energy-api

needs attentionviewing older commit
c24738e · incrementalPR #207reviewed 2026-07-04 05:12 UTC3H · 10M · 9L · 4I
The branch
Purpose
Port the legacy electricity-data-api (billing + savings + metrics + webhooks + PDFs, Pulumi/DDB/Athena stack in batu-monorepo) into the platform. Domain logic and contracts port; AWS plumbing dies.
Goal
Deliver the functional core of the end-to-end port (Phases 0–8 + pure cores of 4/5/6/7): metrics adapter → tariff/TOU reference data → CFE bill calculators → calculated-bill persistence → PDF. Inc-2 infra (cron/SFN estate) and the legacy A/B are follow-ups.
Sub-goals
  • SG-1: Phase 0 — Port spec + decisions (D1–D5)
  • SG-2: Phase 1 — Tinybird→SiteMetrics adapter, DQ, KK validated to 1e-6
  • SG-3: Phase 2 — tariff_rate_values storage + CFE TOU classifier (97 live schedules, conservation 1e-9)
  • SG-4: Phase 3 — CFE bill calculators + calculated bill source
  • SG-5: Phase 4 — HelioScope CSV-upload intake + API-fetch + materialize-persist
  • SG-6: Phase 5 — HelioScope daily rolling top-up job
  • SG-7: Phase 6 — QuickSight-parity synthesizers (powerFactorAdjustment, billingFrequency)
  • SG-8: Phase 7/8 — CDK stacks (DevOpsStack, IntegrationHelioscopeLambdaStack) + preview-provision wiring
The changes (whole branch)
What
This incremental window (94b95a05→c24738e9) adds: (1) HelioScope intake contract + handler routing in the ts-rest router; (2) QuickSight-parity bill synthesizers (powerFactorAdjustment + billingFrequency) in bill.decisions + queries + shells; (3) egaugeColumn field threading through asset-management-validation and the provision shell; (4) Expect<AssertEqual> drift-guard hardening across all 25+ type-check files in core + utility domains; (5) Phase 7 CI hardening (catch-all no-op detection) + HelioScope SSM params in preview-provision; (6) DevOpsStack + IntegrationHelioscopeLambdaStack CDK registration; (7) CollectOptionsFields shared UI component replacing duplicated advanced-options code in AddContractDrawer and InlineContractForm.
Why
Completes the HelioScope integration wire-up (Phases 3–5 infra glue), adds QuickSight analytics parity for CFE billing columns, and hardens type-safety tooling that was previously a silent no-op.
Areas
domains/utility/src+770apps/platform/src/api+310apps/platform/src/app+300domains/cross-domain/src+250domains/core/src+230packages/api/src + packages/database+240domains/metrics/src+120.github/workflows + infra/cdk+80
Blast
66 files in incremental window, +49760/−4697 across the full branch (482 files). Core areas: utility-domain bill calculators (billing-critical), metrics-domain queries (daily-job discovery), cross-domain provision shell (onboarding path), CDK infra (preview deploy).
billing-critical: bill synthesizers write line items that feed savings/PDF/reporting daily-job: findHelioscopeSourcesDueForTopUp drives materialize-persist dispatching infra: DevOpsStack + HelioscopeLambdaStack added to CDK app
ci· No CI checks visible via gh pr checks at review timecoderabbit· No .coderabbit.yaml in repository

Findings · 26

correctness5

high

findHelioscopeSourcesDueForTopUp: duplicate rows when org has multiple active secrets

domains/metrics/src/metric-source/metric-source.queries.ts

LEFT JOIN to integrationSecrets uses (orgId, integrationId, status='active') with no DISTINCT or LIMIT 1. An org with multiple active IntegrationSecret rows for the same integration (multiple secretConfigIds) produces one row per secret, so the same MetricSource appears multiple times. The daily top-up job dispatches each candidate independently — this causes double materialize-persist invocations for the same source, racing to overwrite coverage state. Fix: add DISTINCT ON metricSources.publicId or a LIMIT 1 subquery on the credential join.

medium

synthesizePowerFactorAdjustmentFromCommand: zero-valued charge emits numericValue 0 instead of blank

domains/utility/src/bill/bill.decisions.ts

The null-guard `if (charge == null && bonus == null) return null` fires only when both are strictly null. A CFE lineItem with numericValue=0 (zero charge present in XML) makes charge=0, so the condition is false and adjustment=0+0=0 emits a concrete line item contradicting the 'blank, not misleading $0' contract.

medium

synthesizeBillingFrequencyFromCommand: new Date(dateString) UTC parsing reliance

domains/utility/src/bill/bill.decisions.ts

new Date('YYYY-MM-DD').getTime() is parsed as UTC midnight per ECMA-262. This is correct in V8/Node.js but was implementation-defined in ES5. A non-ISO date string (e.g. from a CFE data anomaly) returns NaN (guarded) — but making UTC intent explicit with 'YYYY-MM-DDT00:00:00.000Z' removes the environment dependency. Low blast radius, but worth a one-liner fix.

low

mergeSourceConfigS3JsonRefByPublicId: unlocked updatedAt bumps without version — breaks optimistic-lock invariant

domains/metrics/src/metric-source/metric-source.queries.ts

The self-heal update bumps updatedAt without bumping version. If a concurrent optimistic-locked update (operator editing sourceConfig via API) commits between this write's SELECT and UPDATE, the operator's next save sees a stale version but still succeeds because only updatedAt advanced. Minor inconsistency, not data loss.

info

Migration ordering 0053→0054→0055 is correct — no DROP-before-backfill risk

packages/database/drizzle

Journal confirms: 0053 adds sites.operation_start_date, 0054 backfills from site_locations, 0055 drops site_locations.operation_start_date. Sequential order is sound.

security7

medium

Lambda ARNs echoed to GITHUB_OUTPUT without ::add-mask:: masking

.github/workflows/preview-provision.yml

HS_INTAKE_ARN, HS_API_FETCH_ARN, HS_MATERIALIZE_ARN are written to GITHUB_OUTPUT via bare echo statements. GHA does not auto-mask step outputs; ARNs appear in plain text in workflow logs, exposing AWS account ID, region, and function names to everyone with read access to the repo. Add '::add-mask::$VAR' immediately after each SSM read. Pre-existing pattern for CFE ARNs — this extends its scope.

low

No file-size cap on HelioScope presigned PUT URL

apps/platform/src/api/handlers/helioscope-intake.handler.ts

The presigned PUT has no ContentLengthRange condition. An authenticated member can upload arbitrarily large files. The intake Lambda reads the entire CSV into memory before parsing — a very large upload triggers OOM/timeout. Authenticated-member-only but worth adding a max-size check in the Lambda before full parse.

low

simulationId: no length or format validation

apps/platform/src/api/handlers/helioscope-intake.handler.ts

parseIntakeBody accepts any non-null string as simulationId with no maxLength or numeric format check. The value is stored in source_config.simulation_id (JSONB) and forwarded to the intake Lambda. A very long string inflates the Lambda payload. Low risk (authenticated member, JSONB tolerates large values), but a 256-char guard closes the attack surface.

low

HELIOSCOPE_INTAKE_BUCKET_NAME set as plain type in Vercel env

.github/workflows/preview-provision.yml

The bucket name is type:plain (visible in Vercel UI/logs) while Lambda ARNs are type:encrypted. Bucket names in presigned URLs already reach the browser so this is not a new exposure, but it's inconsistent with defense-in-depth. Matches CFE_BILLS_BUCKET_NAME precedent — worth noting for future hardening.

info

tariff_jobs SELECT USING(true) exposes step_function_execution_arn to all authenticated users

packages/database/drizzle/0053_third_hammerhead.sql

All authenticated Supabase sessions can SELECT every tariff_jobs row, including step_function_execution_arn (reveals AWS account ID + region). Likely intentional since tariff rates are global reference data, but the execution ARN feels internal. Consider excluding it from the API response mapper even if the RLS stays open.

info

savings_configs/savings_reports DELETE granted to all authenticated members

packages/database/drizzle/0053_third_hammerhead.sql

Both tables allow DELETE by any authenticated org member via RLS, even though no application handler currently exposes DELETE. If deletion should be admin/service-role only, restrict the policy now before a handler accidentally exposes it.

info

site_exception_states RLS join-based pattern is correct but creates indirect trust dependency

packages/database/drizzle/0053_third_hammerhead.sql

RLS uses `site_id IN (SELECT id FROM sites WHERE org_id IN (SELECT get_user_org_ids()))`. Functionally correct and consistent with the established pattern for other site-linked tables. Worth flagging in future audits of get_user_org_ids().

conventions3

medium

Domain constants + 6-table JOIN coordinator logic embedded in metric-source.queries.ts

domains/metrics/src/metric-source/metric-source.queries.ts

Two violations: (1) HELIOSCOPE_INTEGRATION_NAMES constant array and re-exports of HELIOSCOPE_API_INTEGRATION_ID/HELIOSCOPE_CSV_INTEGRATION_ID from @batu/integration-manifests are authored in queries.ts — a thin-DB-wrapper file that should not relay constants. (2) findHelioscopeSourcesDueForTopUp performs a 6-table JOIN spanning core and metrics domains (metric_sources→integrations→metric_streams→sites→integration_secrets→secret_configurations) and embeds a business-rule cadence predicate. Both belong in a coordinator shell or a dedicated read-model query, not queries.ts.

medium

Executable statement interleaved between import declarations

domains/cross-domain/src/site-with-devices-provisioning.shells.ts

const { resolveStartDates } = MetricSourceFCIS; appears at line ~59, between two import blocks. While syntactically valid, it violates the ESM convention that all imports precede executable statements and confuses linters and readers. Move it to the module-level destructure block after all imports.

medium

@batu/metrics-domain in devDependencies makes the seam drift guard invisible in stripped CI passes

packages/api/package.json

SiteMetricsFCIS.SiteMetrics is used type-only in energy-metrics.schemas.ts for the Expect<AssertEqual<...>> drift guard. Correct as devDep — but if a CI step skips devDependencies during typecheck, the guard compiles away silently with no error. Consider keeping it as a regular dependency to make the constraint durable, or document the CI gate that ensures devDeps are always available.

tests8

high

mergeSourceConfigS3JsonRefByPublicId: zero test coverage — jsonb_set clobber safety unverified

domains/metrics/src/metric-source/metric-source.queries.ts

The function's doc comment claims it does not clobber other sourceConfig keys (simulation_id, start, max_years_coverage), but there is no unit or integration test verifying this. A regression in the jsonb_set path or a future refactor could silently wipe billing-critical sourceConfig fields.

high

ProvisionSiteWithDevicesInvalidConfigError: source-config (egaugeColumn) invalid path has no integration test

domains/cross-domain/src/site-with-devices-provisioning.shells.ts

The shell validates both assetConfig and sourceConfig before the transaction. The assetConfig branch has integration test coverage; the sourceConfig (egaugeColumn) invalidConfig branch does not. This new error path can silently regress.

medium

No test exercises all three synths (averageFee + powerFactor + billingFrequency) together

domains/utility/src/bill/__tests__/bill.decisions.test.ts

Each synth is tested in isolation. No test constructs a BatchCreateBillsState with all six conceptId fields populated. Interaction bugs (e.g. one synth stomping another's output via the appendSynth chain) would not be caught.

medium

synthesizePowerFactorAdjustmentFromCommand: bonus with positive numericValue (data error) untested

domains/utility/src/bill/__tests__/bill.decisions.test.ts

PF-2 only tests a correctly-negative bonus. A malformed CFE lineItem where the bonus has a positive numericValue sums as a penalty contribution. Behavior is undocumented and untested.

medium

loadDerivedSynthConceptIds: concept name→id mapping not directly tested

domains/utility/src/bill/bill.queries.ts

The function is only reachable through persistBatchBillsShell. No unit test stubs the DB and asserts the four concept names (powerFactorCharge, powerFactorBonus, powerFactorAdjustment, billingFrequency) map to the correct IDs. A mistyped concept name would surface only at runtime.

medium

synthesizeBillingFrequencyFromCommand: invalid date string (NaN guard) untested

domains/utility/src/bill/__tests__/bill.decisions.test.ts

FREQ-5 covers null period dates but not invalid date strings (e.g. 'not-a-date'). The Number.isNaN guard is present but its branch is untested; a future refactor could silently break null-safety.

low

Non-string egauge_column input type (number, null) untested in parseProvisionBody

apps/platform/src/api/utils/__tests__/asset-management-validation.test.ts

Existing tests cover string acceptance and empty-string rejection but not { egauge_column: 42 } or { egauge_column: null } from a JSON body. The type guard `typeof s.egauge_column !== 'string'` handles these but the branch is unexercised.

low

findHelioscopeSourcesDueForTopUp: no real DB integration test for complex JOIN

domains/metrics/src/metric-source/metric-source.queries.ts

The 6-table JOIN with LEFT JOINs and cadence cutoff predicates is only covered by mock-chain builder tests. Schema-name mismatches or incorrect join conditions only surface against a real DB. An integration test with seeded data would provide stronger guarantees for this billing-critical query.

improvement3

low

validateSourceConfig pre-check fires only when egaugeColumn is truthy — guard too narrow

domains/cross-domain/src/site-with-devices-provisioning.shells.ts

The pre-transaction sourceConfig validation is gated on `if (metricStream.egaugeColumn)`. If the manifest's sourceConfigSchema evolves to require additional keys beyond egaugeColumn, those keys will bypass this check when egaugeColumn is absent. Passing the full prospective sourceConfig to validateSourceConfig unconditionally is more robust.

low

CDK Phase 7 no-op detection grep pattern is fragile to upstream wording changes

.github/workflows/preview-provision.yml

The grep pattern (no stacks match|no stacks to deploy|matched no stacks) is correct for CDK v2's current CLI output but CDK output messages are unversioned. A CDK CLI upgrade that rewrites these strings silently converts the benign no-op path back into a failing job. Consider pinning a comment with the CDK version verified against, or using a dedicated CDK exit code when available.

low

HELIOSCOPE_INTEGRATION_NAMES exported from queries.ts but has no external consumer yet

domains/metrics/src/metric-source/metric-source.queries.ts

The array simplifies the WHERE clause internally but is exported from the module barrel with no current external consumer. The daily-job Lambda branches on HELIOSCOPE_API_INTEGRATION_ID directly, not this array. Keep it unexported until there is an actual external caller.

History · 8 commits

  1. d6b4a35needs attentionincremental4H · 4M · 5L2026-07-05 20:29
  2. 678cb93safeincremental0H · 0M · 1L2026-07-05 17:40
  3. d219403needs attentionincremental1H · 3M · 4L2026-07-05 05:37
  4. 5ecd80cneeds attentionincremental1H · 2M · 1L2026-07-05 04:59
  5. 3fd1d03needs attentionincremental0H · 3M · 5L2026-07-05 04:36
  6. 1446f7eneeds attentionincremental2H · 4M · 7L2026-07-05 04:08
  7. ca73a1bneeds attentionfull4H · 10M · 14L2026-07-05 02:46
  8. c24738eneeds attentionincremental3H · 10M · 9L2026-07-04 05:12current