← all branches

feat/metrics-agg

needs attentionviewing older commit
6580b91 · incrementalpre-PRreviewed 2026-07-09 02:06 UTC2H · 9M · 11L · 1I
The branch
Purpose
Deliver the read-time derivation engine (feat/metrics-agg) that makes a site-level total (e.g. total_demand = feeder_a + feeder_b) a first-class readable stream without materializing it, satisfying the cross-device totals requirement from the electricity-data-api port (Phase 0–1).
Goal
SG-1: pure formula evaluator (restricted arithmetic → AST → slot-wise eval, strict/partial policy). SG-2: read coordinator refactor (StreamWindowReader port, recursive resolveWindowWithReader with cycle/depth guards).
Sub-goals
  • SG-1: Pure derivation evaluator in domains/metrics/src/derivation/ — parseDerivationConfig, parseFormula, evaluateDerivation
  • SG-2: Coordinator refactor in metric.queries.ts — StreamWindowReader port, resolveWindowWithReader, makePgTbReader adapter
  • SG-3 (seam guard): derived-total-seam.test.ts pins coincident peak and honest DQ through the SiteMetrics billing boundary
  • SG-4 (not yet): FE formula-authoring wizard and preview e2e
The changes (whole branch)
What
Added pure derivation evaluator (derivation.decisions.ts, derivation.errors.ts, index.ts), refactored metric.queries.ts behind a StreamWindowReader port to enable recursive derivation resolution and unit testing without I/O, added 2 new error types to metric.errors.ts, exported DerivationFCIS from the metrics domain barrel. Comprehensive test suite: 3 new test files covering evaluator, coordinator, and seam guard.
Why
Site-level totals require summing across device streams without materializing derived rows (late corrections self-heal via dedupe; materialized rows go stale). The derivation engine resolves each input at the same interval so inputs share a deterministic slot grid, then evaluates the formula slot-wise. strict policy ensures billing-grade honesty: a slot emitted only when all inputs have data.
Areas
domains/metrics/src/derivation/+9320domains/metrics/src/metric/+35070.claude/rules/site-metrics-seam.md+230domains/metrics/CLAUDE.md+140
Blast
11 files, +1732/−70 lines. All changes are in domains/metrics/ plus rule docs. No schema migration, no API surface change, no UI change. Existing metric resolution path is preserved (resolveMetricStreamWindow delegates to makePgTbReader, same queries). New derivation path is additive.
SG-4 (FE wizard + preview e2e) not yet shipped — derivation creation is API/seed-only Write-time guards non-functional due to pre-existing array/object mismatch in walkDerivationGraph
ci· No open PR — CI signals not available for pre-PR branchescoderabbit· No .coderabbit.yaml in repo

Findings · 23

correctness4

medium

Write-time derivation walk reads inputs as array — cycle/depth write-time guards are dead

domains/metrics/src/metric-source/metric-source.queries.ts:487

walkDerivationGraph does `Array.isArray(row.inputs) ? row.inputs : []` but source_config.inputs is always a JSON object, never an array. Walk never descends, so derivationDepth=0 and hasCycle=false always. Both write-time guards are non-functional. Only read-time guards in metric.queries.ts protect. Mitigated by derivation creation being FE-v2-only (not yet user-reachable).

medium

Depth check `> MAX_DERIVATION_DEPTH` allows MAX+1 levels — off-by-one vs constant name

domains/metrics/src/metric/metric.queries.ts:211

Check `ctx.depth > MAX_DERIVATION_DEPTH` (MAX=10) allows depths 0–10 inclusive — 11 levels before the error fires. The constant name implies a cap of 10. Write-time uses the same `>` so guards are internally consistent and tests match, but the constant name communicates a cap of 10 while behavior is 11.

low

Dead-code guard `physical.length === 0` is unreachable after derivation filtering

domains/metrics/src/metric/metric.queries.ts:163

After the early-return derivation branch, sources[0] is guaranteed physical and survives the filter. The guard never fires and misleads future maintainers.

info

Diamond dependency causes repeated Tinybird reads — correct but redundant

domains/metrics/src/metric/metric.queries.ts:227

When two derivation inputs share a sub-stream, its window is fetched from Tinybird twice. Results are identical. No correctness impact under current invariant (one stream per site/MetricType) but worth noting for future multi-device topologies.

security5

high

Handler error switch missing 5 new derivation error types — TypeError/500 at runtime

apps/platform/src/api/handlers/metric-stream-window.handler.ts:94

The switch on result.error._tag handles only 3 of 8 members in ResolveMetricStreamWindowError. The five new derivation errors (MetricDerivationCycle, MetricDerivationDepthExceeded, DerivationMalformedFormula, DerivationInputNotFound, DerivationMalformedConfig) fall through to `return success(toMetricStreamWindowResponse(..., result.value))` where result.value is undefined — throws TypeError at runtime. TypeScript doesn't catch this because the return type is inferred.

medium

No request window size limit enables unbounded Tinybird fan-out

apps/platform/src/api/handlers/metric-stream-window.handler.ts:67

No maximum window duration enforced. A single authenticated request with 6-year window at 5m interval against a 10-deep derivation graph triggers O(inputs^depth) Tinybird pipe calls. ~630k points per input per year. Realistic cost/compute amplification from one API call.

medium

No sourceConfigSchema on internal:derivation — formula not validated at write time

packages/integration-manifests/src/manifests/internal-derivation.ts:25

validateSourceConfig() is a no-op for this integration. Malformed formula accepted verbatim into source_config and only fails at read time, causing billing reads to return errors instead of data. Add parseFormula() check to the manifest's sourceConfigSchema or decideCreateMetricSource.

low

RLS enforcement on recursive input reads relies on caller convention, not type system

domains/metrics/src/metric/metric.queries.ts:227

As currently wired (handler wraps in createRLSDb transaction), RLS IS in effect for recursive reads. But if a future internal caller passes a non-RLS db, recursive input resolution silently reads cross-org streams. Consider a JSDoc note or typed RLS-scoped db.

low

No formula string length cap before tokenization

domains/metrics/src/derivation/derivation.decisions.ts:164

tokenize() iterates with no upfront length check. A multi-MB formula from JSONB allocates a proportional tokens[] array. A length cap (e.g. 4096 chars) eliminates this as a resource-exhaustion vector. Low severity: write requires authentication + RBAC.

conventions6

high

Coordinator + I/O-port logic in metric.queries.ts violates FCIS

domains/metrics/src/metric/metric.queries.ts:91

canonical-form.md defines queries files as 'thin DB wrappers (DbOrTx first param, Entity | null returns).' metric.queries.ts now exports StreamWindowReader (I/O port), resolveWindowWithReader (recursive coordinator), resolveWindowInner (full orchestrator with 5 phases), and resolveDerivationWindow. None are thin DB wrappers. Per ADR-016, the imperative shell owns I/O + orchestration. Correct home: metric.shells.ts or metric.read-coordinator.ts.

medium

Multi-paragraph file header comments violate one-line-max rule

domains/metrics/src/derivation/derivation.decisions.ts:1

35-line JSDoc block at file top. CLAUDE.md: 'don't write multi-paragraph docstrings or multi-line comment blocks — one short line max.' Grammar BNF belongs in .claude/rules/site-metrics-seam.md.

medium

Multi-line block comments in derivation.errors.ts and derivation/index.ts

domains/metrics/src/derivation/derivation.errors.ts:1

14-line JSDoc block in errors.ts, 7-line block in index.ts, plus JSDoc on each error type. All violate the one-line-max commenting rule.

medium

Multi-line styled banner comments in resolveWindowInner violate commenting rule

domains/metrics/src/metric/metric.queries.ts:149

5-line banner with box-drawing characters. Should be a single inline comment, e.g. `// P7: derivation wins; never splice a computed total with raw channels.`

low

Decision functions not named `decide{Operation}`

domains/metrics/src/derivation/derivation.decisions.ts:89

parseDerivationConfig, parseFormula, evaluateDerivation deviate from the canonical `decide{Operation}` naming. Defensible (sub-operations, not CQRS decisions) but not aligned with the naming table.

low

`resolveWindowWithReader` exported from queries file but not from metric barrel

domains/metrics/src/metric/metric.queries.ts:126

metric/index.ts exports resolveMetricStreamWindow but not resolveWindowWithReader or StreamWindowReader. Tests import directly from metric.queries. Either surface through the barrel (public) or remove the export from the file (internal).

tests4

medium

`partial` policy not exercised through the coordinator

domains/metrics/src/metric/__tests__/metric.derivation.queries.test.ts

partial is tested at the evaluateDerivation unit level but not through resolveWindowWithReader. The full call chain (parseDerivationConfig → coordinator → evaluator with partial policy) has no integration coverage.

medium

Aggregator non-propagation into derivation inputs not asserted

domains/metrics/src/metric/__tests__/metric.derivation.queries.test.ts:134

The same-interval test captures reads[] including aggregator but only asserts interval. Adding `reads.every(r => r.aggregator === 'mean')` when calling with `aggregator: 'sum'` would pin the aggregator non-propagation contract.

low

Window clamp (openStartedAt) never exercised in new tests

domains/metrics/src/metric/__tests__/metric.derivation.queries.test.ts

Every ActiveSourceRow sets openStartedAt: null. The window-clamp step (trim from forward to earliest open-coverage start) is never exercised by the new test suite.

low

Multi-source priority-ordered splice not regression-tested post-refactor

domains/metrics/src/metric/__tests__/metric.derivation.queries.test.ts

All test helpers create single-source streams. No test verifies that a stream with two physical sources passes both sourceIds to readWindow in priority order — a core pre-refactor correctness property.

improvement4

low

Sequential input resolution adds O(N) latency for N-feeder derivations

domains/metrics/src/metric/metric.queries.ts:227

childCtx is immutable so Promise.all is safe. For a 3-feeder site total: 3×(1 PG + 1 Tinybird call) in series vs parallel. Tradeoff: lose early-exit on first error (all requests fire). At 2–4 feeders delta is modest but grows linearly.

low

Redundant `.sort()` in strict-intersection — Tinybird already returns rows ordered

domains/metrics/src/derivation/derivation.decisions.ts:485

resolve_stream_window pipe emits ORDER BY bucket_ts. Map keys iterate in insertion order (ascending). The sort at line 485 is a no-op in practice — O(n log n) work already paid by Tinybird. Documenting the ordering assumption on SeriesPoint[] inputs captures the implicit contract.

low

`MAX_DERIVATION_DEPTH` belongs in derivation module, not metric-source.errors

domains/metrics/src/metric-source/metric-source.errors.ts

An errors file is wrong home for a configuration constant governing two layers. Moving to derivation/derivation.decisions.ts (alongside MAX_FORMULA_NESTING) removes a cross-layer dependency.

low

Two PG queries per `resolveStream` could be one join

domains/metrics/src/metric/metric.queries.ts:252

resolveStream makes two sequential PG round-trips (metricStreams+metricTypes, then metricSources+integrations). A single LEFT JOIN reduces round-trip latency by ~50% per call. For a 3-feeder derivation: 6 PG calls today vs 3.

History · 5 commits

  1. 6faff4bneeds attentionincremental0H · 2M · 6L2026-07-10 00:01
  2. d75e738needs attentionincremental0H · 2M · 5L2026-07-09 19:39
  3. 17b6060needs attentionfull2H · 10M · 8L2026-07-09 18:48
  4. 6580b91needs attentionincremental2H · 9M · 11L2026-07-09 02:06current
  5. 4849d9aneeds attentionfull6H · 8M · 6L2026-07-07 18:54