feat/metrics-agg
needs attentionviewing older commitd75e738 · incrementalPR #291reviewed 2026-07-09 19:39 UTC0H · 2M · 5L · 2I- Purpose
- Introduce read-time derivation engine so site totals (e.g. total_demand = feeder_A + feeder_B) are computable as first-class streams without materializing derived rows that go stale.
- Goal
- feat(metrics): read-time derivation engine — site totals as derived streams
- Sub-goals
- SG-1: pure arithmetic evaluator (restricted grammar, strict/partial policy, typed errors)
- SG-2: read-path integration — route internal:derivation source through recursive resolver
- SG-3: seam guard — derived total flows through SiteMetrics seam; coincident peak + honest DQ
- SG-N: docs — CLAUDE.md + site-metrics-seam.md
- Hardening (this commit): exhaustive switch, fan-out cap, unary depth fix, >= depth guard, new tests
- What
- Hardening commit: exhaustive handler switch case, MAX_DERIVATION_INPUTS=32 fan-out cap in parseDerivationConfig, unary depth increment fix in parseFactor, MAX_DERIVATION_DEPTH check tightened from > to >= at write-time and read-time, new tests for all guards, stale HelioScope join-count expectation fix.
- Why
- Follow-up hardening after initial PR review. The unary depth guard existed but didn't fire for unary chains (real bug). The >= change closes an off-by-one allowing one extra derivation level. The fan-out cap prevents unbounded Postgres+Tinybird fan-out per derived stream.
- Areas
- domains/metrics/src+1267−70apps/platform/src/api+19−22packages/integration-manifests/src+4−4
- Blast
- 16 files (+1942/−99 cumulative branch); 7 files in this increment. Metric window read path, derivation decisions, one API handler.
Findings · 10
correctness4
MAX_DERIVATION_INPUTS declared 60 lines after first use
domains/metrics/src/derivation/derivation.decisions.ts:167
parseDerivationConfig at ~line 108 references MAX_DERIVATION_INPUTS declared at ~line 167. Safe at runtime (module evaluates before any caller invokes the function) but unconventional. Move alongside MAX_FORMULA_NESTING / MAX_FORMULA_LENGTH.
Write-time vs read-time depth guard: verify fence-post is intentional
domains/metrics/src/metric-source/metric-source.decisions.ts:205
Both guards now use >=. Write-time receives derivationDepth (pre-insertion depth of existing inputs); read-time ctx.depth starts at 0. Consistent IF derivationDepth is the pre-existing max (adding the new stream creates depth N+1, blocked when N >= MAX). Worth confirming shell computation to rule out a fence-post that writes a stream which immediately fails at read.
Exhaustive default: case pattern is correct and sound
apps/platform/src/api/handlers/metric-stream-window.handler.ts:113
const _exhaustive: never = result.error correctly causes a compile error when a new variant is added without a matching case. Runtime cast to {_tag: string} is safe since all error variants carry _tag.
Unary depth fix was a real prior bug, now correctly patched
domains/metrics/src/derivation/derivation.decisions.ts:323
The prior parseFactor called parseFactor(depth) for unary operators, meaning ----x would recurse N times at depth 0 — effectively unbounded. Fixed by passing depth+1 + adding the guard at parseFactor entry. The test with 40 unary minuses proves it.
security2
Derivation DAG: shared sub-streams resolved exponentially (no global memoization)
domains/metrics/src/metric/metric.queries.ts:133
resolveWindowInner tracks ancestors (current path) for cycle detection but not globally-visited nodes. A diamond DAG resolves shared sub-streams exponentially. At MAX_DERIVATION_DEPTH=10 and MAX_DERIVATION_INPUTS=32, worst-case is 32^10 visits each triggering Postgres + Tinybird calls. Requires operator-level DB access to exploit. Fix: add a visited memo map to DerivationCtx before authoring UI ships.
serverError() embeds formula content in HTTP 500 body
apps/platform/src/api/handlers/metric-stream-window.handler.ts:111
Pre-existing error cases call serverError(result.error.message). Messages may include up to 1024-char formula strings and stream public-ids. No credentials or PII; operator-only config. Low risk but worth stripping from 500 bodies on the next error-mapping pass.
tests3
Fan-out cap test description says 'read fan-out cap' but tests declared-input count
domains/metrics/src/derivation/__tests__/derivation.decisions.test.ts:347
parseDerivationConfig rejects inputEntries.length > 32. Test uses formula 'm0 + m1' (2 vars) with 33 declared inputs. Guard fires correctly on declared count, not formula-referenced count. Description misleads: 'read fan-out cap' implies fan-out = referenced vars. Consider renaming + adding a 33-var formula companion test.
Tinybird failure test: { message: 'boom' } as never skips TinybirdError shape validation
domains/metrics/src/metric/__tests__/metric.derivation.queries.test.ts:340
MetricErrors.tinybirdReadFailed expects a TinybirdError (_tag, statusCode, message). The `as never` cast silences TypeScript for an incomplete object. Test only asserts on the outer _tag so it passes, but masks any downstream inspection of tinybirdError._tag. Use a properly-typed fixture.
Helioscope join test verifies aggregate count, not which specific joins remain
domains/metrics/src/metric-source/__tests__/helioscope-discovery.queries.test.ts:56
The count assertion (leftJoin: 2) is a structural proxy. Two joins swapped for one join + one innerJoin would pass while changing query logic. Acceptable given Drizzle builder opacity; the comment documenting why it's 2 (not 3) is the important part.
improvement1
parseFactor and parseExpr both guard depth — interaction needs a comment
domains/metrics/src/derivation/derivation.decisions.ts:286
Both guards are intentional and not redundant: parseExpr catches paren nesting (caller passes depth+1 on lparen); parseFactor catches unary chains (now correctly depth+1). A brief comment prevents a future maintainer from removing one thinking it's duplicate.
History · 5 commits
- 6faff4bneeds attentionincremental0H · 2M · 6L2026-07-10 00:01
- d75e738needs attentionincremental0H · 2M · 5L2026-07-09 19:39current
- 17b6060needs attentionfull2H · 10M · 8L2026-07-09 18:48
- 6580b91needs attentionincremental2H · 9M · 11L2026-07-09 02:06
- 4849d9aneeds attentionfull6H · 8M · 6L2026-07-07 18:54