feat/sites-v2
needs attentionviewing older commit6bfc5bc · incrementalpre-PRreviewed 2026-08-12 02:49 UTC4H · 7M · 6L · 5I- Purpose
- Build a 'sites & devices demo' for feat/sites-v2 with realistic production-fidelity data — real cadences, device IDs, and energy curves from prod, replacing synthetic demo telemetry.
- Goal
- Demo org seeded with real prod fleet (5 sites, real monitoring graph, real Tinybird series) so the demo presentation reflects actual production behavior.
- Sub-goals
- SG-1: Provision the feat/sites-v2 preview environment
- SG-2: Seed the demo org with realistic sites/devices/streams end-to-end
- SG-3: Fix preview to read from the DEV Tinybird workspace (not staging/prod)
- SG-4: Import the REAL production fleet and readings, replacing the modelled data
- What
- Adds `import-real-fleet.ts` — a one-shot script that reads the real production monitoring graph (5 sites) from prod Postgres and Tinybird, then writes it verbatim to the dev environment. Replaces modelled demo telemetry with production-actual data.
- Why
- The demo presentation needed authentic energy curves and device IDs to be credible. Synthetic data doesn't reflect the real system's cadence or coverage patterns.
- Areas
- packages/database/src/demo+340−0packages/database/package.json+1−0docs/development/sites-devices-demo+255−0.github/workflows/preview-provision.yml+12−5packages/database/src/demo/seed-demo-org.ts+891−0packages/database/src/demo/seed-demo-metrics.ts+289−0packages/database/src/demo/extract-prod-fixture.ts+254−0
- Blast
- 8 files across packages/database/src/demo and supporting infra; +2046 lines total. No production code paths or domain packages touched — entirely demo tooling and preview provisioning.
Findings · 17
correctness5
Wrong PublicIdPrefix for `makes.public_id` — stores `ast_` instead of `mke_`
packages/database/src/demo/import-real-fleet.ts:196
`generatePublicId(PublicIdPrefix.Asset)` used when inserting into `makes` table. Correct prefix is `PublicIdPrefix.Make` (`mke_`). Makes rows get wrong id prefix, breaking prefix-keyed lookups and audit logs.
`from_ts`/`to_ts` use ISO T-format — Tinybird `DateTime` requires space-separated format
packages/database/src/demo/import-real-fleet.ts:284
`toISOString().slice(0,19)` produces `YYYY-MM-DDTHH:MM:SS`. ClickHouse/Tinybird `DateTime` requires `YYYY-MM-DD HH:MM:SS`. The existing `toIso()` helper in `metric.queries.ts` does `.replace('T',' ')` for exactly this reason. Without the replace, the pipe returns zero or wrong rows — silently producing an empty import.
Null-asset sources skip duplicate-binding check → DB unique constraint error
packages/database/src/demo/import-real-fleet.ts:227
Duplicate check at line 227 is guarded by `if (assetId)`. Sources with no asset skip it. The DB's `UNIQUE (metric_stream_id) WHERE asset_id IS NULL` partial index means a second null-asset source throws PG 23505, aborting mid-import.
Missing `'1M'` in GRAIN_SECONDS — monthly sources queried at wrong interval
packages/database/src/demo/import-real-fleet.ts:58
`GRAIN_SECONDS` covers 1m/5m/15m/1h/1d but not `1M` (calendar-month). Monthly prod sources fall to `?? 3600` fallback, querying at wrong resolution.
`stats.assets` and `stats.streams` over-count on ON CONFLICT DO UPDATE
packages/database/src/demo/import-real-fleet.ts:181
Both upserts return the row on conflict, so counters fire even when no new row was inserted. Cosmetic only.
security6
Real customer RPU numbers hardcoded in committed source
packages/database/src/demo/import-real-fleet.ts:49
Five production RPU numbers (Mexican CFE account identifiers) are committed to git. RPUs persist in git history. If the repo is made public, customer account numbers are exposed.
Real production customer energy data copied to development environment
packages/database/src/demo/import-real-fleet.ts:280
Script copies actual customer meter readings verbatim from prod Tinybird to dev. Under LFPDPPP and general privacy principles, real customer data should not populate dev environments.
No Tinybird token direction guard — token swap risk
packages/database/src/demo/import-real-fleet.ts:99
The prod-write guard checks POSTGRES_URL but not the Tinybird tokens. If swapped, the script could append to prod Tinybird (append-only, no easy rollback).
URL injection via unencoded `source_pid` in Tinybird query param
packages/database/src/demo/import-real-fleet.ts:288
`r.source_pid` interpolated in URL without `encodeURIComponent`. ULIDs are safe in practice but a corrupted record with metacharacters could manipulate the query string.
No guard validating PROD_POSTGRES_URL is actually a prod host
packages/database/src/demo/import-real-fleet.ts:93
Guard checks devUrl for prod refs but not that prodUrl is actually prod. Swapped env vars bypass the guard.
Tinybird append error messages include raw response body
packages/database/src/demo/import-real-fleet.ts:317
Tinybird error responses can include token metadata. Low risk in a dev script.
conventions4
Non-atomic wipe + recreate — broken state on interruption
packages/database/src/demo/import-real-fleet.ts:138
DELETE (step 2) and INSERT (step 3) are separate awaits. Interruption between them leaves demo org with no streams. Wrap in `dev.begin(tx => ...)` for atomicity.
`dev.json({} as never)` — unsafe `as never` TypeScript cast
packages/database/src/demo/import-real-fleet.ts:205
`as never` is the bottom type — semantically wrong. Use `dev.json({} as unknown)` or just `dev.json({})`. Same pattern at line 261.
`(asset?.id as string) ?? null` — redundant double cast
packages/database/src/demo/import-real-fleet.ts:210
`as string` before `?? null` is a compile-time lie. Use `asset?.id ?? null` directly.
Prod secret vars not documented in `.env.example`
packages/database/src/demo/import-real-fleet.ts:42
`PROD_POSTGRES_URL` and `TB_PROD_TOKEN` are required but absent from `.env.example`. First-time users get a runtime error with no guidance.
improvement2
Sequential Tinybird fetches — could be parallelized for speed
packages/database/src/demo/import-real-fleet.ts:280
Step 4 fetches each source sequentially. Bounded parallel (p-limit at 5–10) would reduce wall-clock significantly for larger mappings.
Coverage query returns one row per source — multiple windows silently dropped
packages/database/src/demo/import-real-fleet.ts:126
If a source has multiple coverage windows, only the first matched row's cov_from/cov_to is used. Worth a code comment.
History · 7 commits
- bf960baneeds attentionincremental5H · 12M · 4L2026-08-12 05:48
- eb284feneeds attentionincremental0H · 5M · 4L2026-08-12 05:10
- 6bfc5bcneeds attentionincremental4H · 7M · 6L2026-08-12 02:49current
- 3d49126safeincremental0H · 0M · 2L2026-08-11 23:15
- df18b6bneeds attentionincremental2H · 5M · 5L2026-08-11 23:08
- 8b42c1eneeds attentionincremental0H · 2M · 3L2026-08-11 23:01
- 5c64ea3needs attentionfull3H · 4M · 5L2026-08-11 19:15