← all branches

docs/metrics-ret

needs attention
f060efc · fullPR #318reviewed 2026-07-16 18:43 UTC4H · 6M · 5L · 2I
The branch
Purpose
Record empirically verified per-vendor retention boundaries and investigation runbooks from the Niko (oid=33) device migration sweep, so the next on-call engineer does not have to rediscover the same failure modes.
Goal
Add a per-integration granularity-retention matrix (fact-base) + four read-only investigation runbooks (hoymiles / huawei / shelly / solis) covering the full dead-plant diagnostic path from CloudWatch logs → portal probe → classification → deliverable.
Sub-goals
  • SG-1: metrics-granularity-retention.md — retention matrix with empirically verified boundaries (bisected 2026-07-16 on prod)
  • SG-2: site-metrics-seam.md update — cross-reference to matrix, backfill choreography subsection, inversion-hazard fix (#311) noted as order-independent
  • SG-3: Four investigate-* skills — hoymiles, huawei, shelly, solis — read-only runbooks with decision trees
The changes (whole branch)
What
1 new .claude/rules file (118 lines), 1 modified .claude/rules file (+23/-21 lines), 4 new .claude/skills runbooks (238+236+213+213 lines). No application code, no infra, no migrations.
Why
The fine retention tier varies 15 days to plant lifetime by vendor — the team was about to settle for daily history on vendors that serve 5-minute data to install. The investigation runbooks encode 10+ trap patterns that produced false boundaries during the 2026-07-16 probes.
Areas
.claude/rules+14121.claude/skills+9000
Blast
6 files, +1041/-21. Docs only — no runtime impact.
docs-only no-code-change no-infra-change investigation-runbooks
CI· CI check API not accessible from runner tokencoderabbit· no .coderabbit.yaml in repo

Findings · 18

correctness7

high

_try used before definition — NameError if copy-pasted

.claude/skills/investigate-hoymiles/SKILL.md:77

Snippet A (the heredoc block) calls _try(m[i:]) inside the for-loop but defines def _try(s) after it. Python raises NameError at runtime. Snippet B (lines 86–106) is the correct working replacement. Remove Snippet A entirely or add a clear 'do not run this' callout.

high

Month probe window exceeds hoymiles 2-year per-request span cap

.claude/skills/investigate-hoymiles/SKILL.md:163

The mon probe uses start_date='2024-01-01' and end_date='2026-07-15' — a ~926-day span. The retention matrix explicitly documents a 2-year request-span cap for the 1d endpoint ('Requested date range exceeds two years'). If this applies to monthly calls too, the probe returns an error instead of data, quietly sabotaging the GONE-vs-STALE classification. Cap the window to ≤24 months (e.g. start_date='2024-07-01').

high

Stray closing triple-backtick at end of site-metrics-seam.md

.claude/rules/site-metrics-seam.md:161

Line 161 is a standalone ``` with no matching opening fence in the 'Rates are reference data' section. In GitHub's renderer this opens an unterminated code block that swallows whatever follows, making the Rates section unreadable. Remove the stray fence.

high

Python snippets use literal string 'SCRATCH/...' — FileNotFoundError at runtime

.claude/skills/investigate-shelly/SKILL.md:74

Lines 74, 123, and 147 pass 'SCRATCH/shelly-logs-raw.json' and 'SCRATCH/.shelly_session.json' as literal Python strings. The shell variable $SCRATCH is never exported into the Python interpreter, so open() looks for a directory literally named 'SCRATCH' in cwd and raises FileNotFoundError. Fix: use f"{os.environ['SCRATCH']}/..." (import os), or pass SCRATCH via sys.argv as the hoymiles skill does.

medium

Shelly 1m retention stated as '~120–150 d' — conflicts with matrix's verified 129 d

.claude/skills/investigate-shelly/SKILL.md:181

The error taxonomy gives a loose '~120–150 d' range for Shelly 1m retention; the retention matrix records a bisected empirical boundary of 'today − 129d ≈ 130d ✅'. The matrix's verified number supersedes the estimate. Update to '~129–130 d (bisected; see metrics-granularity-retention.md)' for consistency.

low

Huawei fine-grain bracket notation '(12, 25]' implies >12 months; empirical data shows present at 365 d (exactly 12 mo)

.claude/rules/metrics-granularity-retention.md:26

Mathematical open-lower-bound notation (12, 25] means strictly greater than 12 months. The empirical check found data present at 365 days (~12 months) — so the lower bound should be closed: [12, 25] or accompanied by a prose note 'present at 12mo (365d), absent at 25mo (761d)'.

info

'VRM 31-day' span cap cited but Victron has no API token — source is theoretical

.claude/rules/site-metrics-seam.md:75

Line 75 lists 'VRM 31-day' as an active operational span cap alongside ABB/hoymiles. The retention matrix marks Victron as 'credential is portal-only (no VRM API token) — unverifiable'. The 31-day cap is from VRM API docs, not empirical. Qualify: 'VRM 31-day (theoretical — no API token yet)' or remove until a token is available.

security3

high

Hoymiles api_key written to disk with no cleanup step

.claude/skills/investigate-hoymiles/SKILL.md:141

The runbook writes the raw Secrets Manager payload (containing the live api_key) to $SCRATCH/.sec.json but has no rm step afterward. On a shared or persistent directory (/tmp, a dev home, CI workspace) the secret persists indefinitely. Add an explicit rm -f "$SCRATCH/.sec.json" at step 5 (Deliverable), matching the Shelly skill's cleanup pattern.

medium

Cleanup rm uses bare 'SCRATCH' literal instead of shell variable $SCRATCH

.claude/skills/investigate-shelly/SKILL.md:212

rm -f SCRATCH/.shelly_session.json treats SCRATCH as a directory name, not the shell variable. The bearer-token session file is therefore never deleted. Fix: rm -f "$SCRATCH/.shelly_session.json" with dollar sign and quotes.

low

Shelly session file written with default permissions (potentially 0644)

.claude/skills/investigate-shelly/SKILL.md:123

open('SCRATCH/.shelly_session.json','w') uses the process umask. On systems with umask 022, the file is mode 0644 — readable by all local users. After the write, add os.chmod(path, 0o600) or use os.open with O_CREAT|O_WRONLY|O_TRUNC and mode 0o600.

conventions3

medium

investigate-shelly missing ## 5. Deliverable and ## Gotchas recap sections

.claude/skills/investigate-shelly/SKILL.md:185

All three sibling skills (hoymiles, huawei, solis) have a numbered '## 5. Deliverable' describing the expected output (category breakdown with counts, per-device table, evidence, bottom line, recommendations) and a '## Gotchas recap' for quick reference. investigate-shelly skips both, going straight from Error taxonomy to Cleanup. Add a Deliverable section (write investigation-shelly.md to scratch with the same structure) and a Gotchas recap (wrong shard = silent all-missing; narrow-grain sweep ≠ dead; GHOST = present but zero history).

medium

Four new investigate-* skills not registered in CLAUDE.md skills table

CLAUDE.md:60

The skills index table lists all operator-facing skills and their triggers. investigate-hoymiles, investigate-huawei, investigate-shelly, and investigate-solis are absent. Without a table entry, an operator asking 'why is Hoymiles returning no data?' has no hint the skill exists. Add one line per skill with trigger description.

info

metrics-granularity-retention.md path scope may miss billing/metrics domain code

.claude/rules/metrics-granularity-retention.md:1

The frontmatter scopes auto-loading to services/metrics/integrations/**, scripts/metrics/**, packages/integration-manifests/**. But site-metrics-seam.md (always-loaded) cross-links to this file as relevant to billing/savings consumers in domains/utility/** and domains/metrics/**. Consider adding those paths to the frontmatter so the retention rules are visible when editing billing code.

improvement5

medium

SWEEP date hardcoded to 2026-07-14 — investigator must update manually, not told to

.claude/skills/investigate-hoymiles/SKILL.md:158

SWEEP="2026-07-14" is a specific authoring-session date. An investigator copy-pasting the snippet later will probe the wrong day. This also contradicts retention-matrix trap 11: 'never hardcode a date'. Replace with SWEEP=$(date -d 'yesterday' +%F) and add a comment: '# change to the day you are investigating'.

medium

SAMPLE variable used in probe loop but never defined

.claude/skills/investigate-hoymiles/SKILL.md:159

for plant in SAMPLE: references a variable that appears nowhere else in the skill. An investigator must guess that SAMPLE comes from the dead list assembled in step 2. Add two lines before the loop: SAMPLE = [pid for pid, _ in dead] # or a subset; and note that ~15-20 plants at 4 calls × 300ms ≈ 20-25 min.

low

No investigation runbook for growatt despite documented 95-day gap

.claude/rules/metrics-granularity-retention.md:47

The matrix documents growatt's unrecoverable >95d history with two endpoint gotchas (year key vs date; autoEnergy summation). No investigate-growatt skill exists and the matrix doesn't note this absence. Add a note: 'No investigation runbook — see gotchas above.' so an operator hitting dead growatt plants has clear instructions.

low

investigate-shelly worker code paths omit /src/ segment

.claude/skills/investigate-shelly/SKILL.md:44

The inputs table lists engine/shelly-client.ts, handlers/metrics.lambda.ts, translation/points-to-batu.ts — but the actual layout is services/metrics/integrations/shelly/src/engine/..., services/metrics/integrations/shelly/src/handlers/..., etc. All other integrations use the same src/ layout. A developer following the path literally won't find the files.

low

CloudWatch Insights sleep 6 is fragile — no retry shown

.claude/skills/investigate-huawei/SKILL.md:77

A fixed 6-second wait before get-query-results can return status=Running for large log groups (large fleets may take 30+ s). Add a polling note: 'If status is Running, re-run get-query-results after 10 s; allow up to 60 s total.'