← all branches

feat/sites-v2

needs attentionviewing older commit
6bfc5bc · incrementalpre-PRreviewed 2026-08-12 02:49 UTC4H · 7M · 6L · 5I
The branch
Purpose
Build a 'sites & devices demo' for feat/sites-v2 with realistic production-fidelity data — real cadences, device IDs, and energy curves from prod, replacing synthetic demo telemetry.
Goal
Demo org seeded with real prod fleet (5 sites, real monitoring graph, real Tinybird series) so the demo presentation reflects actual production behavior.
Sub-goals
  • SG-1: Provision the feat/sites-v2 preview environment
  • SG-2: Seed the demo org with realistic sites/devices/streams end-to-end
  • SG-3: Fix preview to read from the DEV Tinybird workspace (not staging/prod)
  • SG-4: Import the REAL production fleet and readings, replacing the modelled data
The changes (whole branch)
What
Adds `import-real-fleet.ts` — a one-shot script that reads the real production monitoring graph (5 sites) from prod Postgres and Tinybird, then writes it verbatim to the dev environment. Replaces modelled demo telemetry with production-actual data.
Why
The demo presentation needed authentic energy curves and device IDs to be credible. Synthetic data doesn't reflect the real system's cadence or coverage patterns.
Areas
packages/database/src/demo+3400packages/database/package.json+10docs/development/sites-devices-demo+2550.github/workflows/preview-provision.yml+125packages/database/src/demo/seed-demo-org.ts+8910packages/database/src/demo/seed-demo-metrics.ts+2890packages/database/src/demo/extract-prod-fixture.ts+2540
Blast
8 files across packages/database/src/demo and supporting infra; +2046 lines total. No production code paths or domain packages touched — entirely demo tooling and preview provisioning.
data-privacy prod-read-access
CI· No PR open — no CI runCodeRabbit· No .coderabbit.yaml in repotypecheck· Not run in this review pass

Findings · 17

correctness5

high

Wrong PublicIdPrefix for `makes.public_id` — stores `ast_` instead of `mke_`

packages/database/src/demo/import-real-fleet.ts:196

`generatePublicId(PublicIdPrefix.Asset)` used when inserting into `makes` table. Correct prefix is `PublicIdPrefix.Make` (`mke_`). Makes rows get wrong id prefix, breaking prefix-keyed lookups and audit logs.

high

`from_ts`/`to_ts` use ISO T-format — Tinybird `DateTime` requires space-separated format

packages/database/src/demo/import-real-fleet.ts:284

`toISOString().slice(0,19)` produces `YYYY-MM-DDTHH:MM:SS`. ClickHouse/Tinybird `DateTime` requires `YYYY-MM-DD HH:MM:SS`. The existing `toIso()` helper in `metric.queries.ts` does `.replace('T',' ')` for exactly this reason. Without the replace, the pipe returns zero or wrong rows — silently producing an empty import.

medium

Null-asset sources skip duplicate-binding check → DB unique constraint error

packages/database/src/demo/import-real-fleet.ts:227

Duplicate check at line 227 is guarded by `if (assetId)`. Sources with no asset skip it. The DB's `UNIQUE (metric_stream_id) WHERE asset_id IS NULL` partial index means a second null-asset source throws PG 23505, aborting mid-import.

medium

Missing `'1M'` in GRAIN_SECONDS — monthly sources queried at wrong interval

packages/database/src/demo/import-real-fleet.ts:58

`GRAIN_SECONDS` covers 1m/5m/15m/1h/1d but not `1M` (calendar-month). Monthly prod sources fall to `?? 3600` fallback, querying at wrong resolution.

low

`stats.assets` and `stats.streams` over-count on ON CONFLICT DO UPDATE

packages/database/src/demo/import-real-fleet.ts:181

Both upserts return the row on conflict, so counters fire even when no new row was inserted. Cosmetic only.

security6

high

Real customer RPU numbers hardcoded in committed source

packages/database/src/demo/import-real-fleet.ts:49

Five production RPU numbers (Mexican CFE account identifiers) are committed to git. RPUs persist in git history. If the repo is made public, customer account numbers are exposed.

high

Real production customer energy data copied to development environment

packages/database/src/demo/import-real-fleet.ts:280

Script copies actual customer meter readings verbatim from prod Tinybird to dev. Under LFPDPPP and general privacy principles, real customer data should not populate dev environments.

medium

No Tinybird token direction guard — token swap risk

packages/database/src/demo/import-real-fleet.ts:99

The prod-write guard checks POSTGRES_URL but not the Tinybird tokens. If swapped, the script could append to prod Tinybird (append-only, no easy rollback).

medium

URL injection via unencoded `source_pid` in Tinybird query param

packages/database/src/demo/import-real-fleet.ts:288

`r.source_pid` interpolated in URL without `encodeURIComponent`. ULIDs are safe in practice but a corrupted record with metacharacters could manipulate the query string.

medium

No guard validating PROD_POSTGRES_URL is actually a prod host

packages/database/src/demo/import-real-fleet.ts:93

Guard checks devUrl for prod refs but not that prodUrl is actually prod. Swapped env vars bypass the guard.

low

Tinybird append error messages include raw response body

packages/database/src/demo/import-real-fleet.ts:317

Tinybird error responses can include token metadata. Low risk in a dev script.

conventions4

medium

Non-atomic wipe + recreate — broken state on interruption

packages/database/src/demo/import-real-fleet.ts:138

DELETE (step 2) and INSERT (step 3) are separate awaits. Interruption between them leaves demo org with no streams. Wrap in `dev.begin(tx => ...)` for atomicity.

medium

`dev.json({} as never)` — unsafe `as never` TypeScript cast

packages/database/src/demo/import-real-fleet.ts:205

`as never` is the bottom type — semantically wrong. Use `dev.json({} as unknown)` or just `dev.json({})`. Same pattern at line 261.

low

`(asset?.id as string) ?? null` — redundant double cast

packages/database/src/demo/import-real-fleet.ts:210

`as string` before `?? null` is a compile-time lie. Use `asset?.id ?? null` directly.

low

Prod secret vars not documented in `.env.example`

packages/database/src/demo/import-real-fleet.ts:42

`PROD_POSTGRES_URL` and `TB_PROD_TOKEN` are required but absent from `.env.example`. First-time users get a runtime error with no guidance.

improvement2

low

Sequential Tinybird fetches — could be parallelized for speed

packages/database/src/demo/import-real-fleet.ts:280

Step 4 fetches each source sequentially. Bounded parallel (p-limit at 5–10) would reduce wall-clock significantly for larger mappings.

low

Coverage query returns one row per source — multiple windows silently dropped

packages/database/src/demo/import-real-fleet.ts:126

If a source has multiple coverage windows, only the first matched row's cov_from/cov_to is used. Worth a code comment.

History · 7 commits

  1. bf960baneeds attentionincremental5H · 12M · 4L2026-08-12 05:48
  2. eb284feneeds attentionincremental0H · 5M · 4L2026-08-12 05:10
  3. 6bfc5bcneeds attentionincremental4H · 7M · 6L2026-08-12 02:49current
  4. 3d49126safeincremental0H · 0M · 2L2026-08-11 23:15
  5. df18b6bneeds attentionincremental2H · 5M · 5L2026-08-11 23:08
  6. 8b42c1eneeds attentionincremental0H · 2M · 3L2026-08-11 23:01
  7. 5c64ea3needs attentionfull3H · 4M · 5L2026-08-11 19:15