feat/tb-deploy
blockedviewing older commit95b4484 · fullpre-PRreviewed 2026-07-06 05:36 UTC3H · 8M · 4L · 3I- Purpose
- Wire Tinybird schema-CD into all three CI lanes and establish a least-privilege token model declared as code.
- Goal
- Eliminate manual schema drift, enforce per-env token isolation (runtime vs CI-deploy), and provision ephemeral Tinybird branches for metrics-relevant PRs.
- Sub-goals
- SG-1: Declare metrics_runtime token as code (APPEND on metric.datasource, READ on resolve_stream_window.pipe)
- SG-2: Per-env runtime token seeding to SSM + Vercel with legacy fallback
- SG-3: Per-PR ephemeral Tinybird branch provisioning gated on metrics-pipeline path changes
- SG-4: Schema-CD via tb deploy --cloud in staging (auto) and prod (gated)
- SG-5: Cleanup ephemeral branches on PR close (protect 4-branch workspace cap)
- SG-6: tinybird-deployment.md as SSOT rule for token topology, schema-CD, and operator runbook
- What
- Added TOKEN declarations to metric.datasource and resolve_stream_window.pipe; per-PR Tinybird branch provisioning/deprovision; tinybird-staging job in staging-update.yml; tinybird-prod job in release-promote.yml; new .claude/rules/tinybird-deployment.md; updated infrastructure.md + migration-pipeline.md.
- Why
- Close the schema-CD gap (Tinybird schema was applied manually), enforce least-privilege tokens (no admin tokens at rest), and bring Tinybird lifecycle to the same three-lane standard as DB migrations.
- Areas
- .claude/rules+267−2.github/workflows+302−44infra/tinybird+72−43
- Blast
- 10 files, +641/-89 across CI rules and infra. No application code changed.
Findings · 17
correctness4
if: env.TB_TOKEN guard always false — tb_provision and cleanup steps never run
.github/workflows/preview-provision.yml:294
TB_TOKEN is declared only in the step's own env: block, but GHA evaluates if: conditions against job-level env context only. env.TB_TOKEN is always empty, so tb_provision (provision.yml:294) and Delete Tinybird branch (deprovision.yml:61) are permanently dead code. Fix: move the check into the run script or promote the secret to a job-level env var.
Branch cap discrepancy: rule says 4, workflow comment says 6
.github/workflows/preview-provision.yml:222
tinybird-deployment.md says 4 branches/workspace (hard). The workflow comment says 6. The rule is SSOT — align the comment.
tb --cloud branch rm syntax unverified against the Forward CLI
.github/workflows/preview-deprovision.yml:77
With set +e active, a wrong-syntax error is swallowed and branches accumulate. Verify the exact Forward CLI branch deletion command.
metrics_runtime token does not exist on dev workspace until schema is first deployed
.github/workflows/preview-provision.yml
The token is only created by tb deploy. Operators need to bootstrap dev post-merge by running tb deploy against the dev workspace.
security4
Branch token extraction selects admin-named token — seeds privileged token into Vercel and SSM
.github/workflows/preview-provision.yml:321
The jq expression extracts the first token whose name matches 'admin'. After tb --cloud preview, the branch inherits workspace admin tokens. The extracted token seeds TINYBIRD_BRANCH_TOKEN (Vercel) and SSM metrics/tinybird/token — inverse of intended design. Fix: run tb deploy into the branch to materialize TOKEN metrics_runtime, then extract with tb --cloud token copy metrics_runtime.
TINYBIRD_TOKEN_DEV (admin token) seeds the Lambda runtime SSM path as fallback
.github/workflows/preview-provision.yml:800
SSM seed fallback writes the admin TINYBIRD_TOKEN_DEV to /batu/{slug}/dev/metrics/tinybird/token as a plain SSM String. Admin token grants full workspace access. The token split (TINYBIRD_DEPLOY_TOKEN_DEV for branch creation, demoted scoped TINYBIRD_TOKEN_DEV for runtime) should be done before merge.
curl -fsSL https://tinybird.co | sh — unpinned supply-chain risk in 4 workflow files
.github/workflows/staging-update.yml:223
All four modified workflows install the Tinybird CLI without version pinning or checksum. In staging-update.yml and release-promote.yml this runs with AWS OIDC credentials in scope. Pin to a specific GitHub release SHA.
dorny/paths-filter@v3 not SHA-pinned in pipelines with cloud credentials
.github/workflows/preview-provision.yml:230
Mutable tag reference on an action controlling which jobs run in preview, staging, and release pipelines. Pin to a commit SHA.
conventions3
TINYBIRD_DEPLOY_TOKEN_DEV missing from infrastructure.md secrets manifest
.claude/rules/infrastructure.md:256
tinybird-deployment.md declares TINYBIRD_DEPLOY_TOKEN_DEV as the CI-only deploy token for branch creation. Absent from the SSOT manifest.
tinybird-deployment.md paths: frontmatter missing release-promote.yml
.claude/rules/tinybird-deployment.md:8
The rule auto-loads for the other 3 workflow files but not for release-promote.yml — the prod lane and highest-stakes context.
Vercel precedence in rule does not match the workflow single-key approach
.claude/rules/tinybird-deployment.md:128
Rule shows TINYBIRD_BRANCH_TOKEN ⊳ TINYBIRD_TOKEN as a two-key model. The workflow collapses the selection in CI and writes a single TINYBIRD_BRANCH_TOKEN. Update the diagram.
tests2
No tb deploy --check in pr-checks.yml for infra/tinybird/** changes
.github/workflows/pr-checks.yml
Schema errors surface only after merge to staging. Add a path-gated tinybird-checks job in pr-checks.yml running tb --cloud deploy --check.
Stale test fixture tests a stub pipe interface that no longer exists
infra/tinybird/tests/resolve_stream_window.test.yaml
The fixture sends stream_id and interval='1h' for the old stub pipe. The live pipe accepts metric_source_ids (Array(String)), interval_seconds (Int32), etc. tb test is never invoked in any CI pipeline. Rewrite the fixture and wire tb test into CI.
improvement4
Tinybird SSM seed is inside the CDK gate — metrics-only PRs skip the SSM seed
.github/workflows/preview-provision.yml
The SSM seed step is gated on CDK's should_deploy. A PR touching only infra/tinybird/** fires tb_decision but not CDK, so SSM params are never seeded and Lambda stacks fail. Decouple the SSM seed from the CDK gate.
tb --cloud preview idempotency on re-push unverified
.github/workflows/preview-provision.yml:316
If the CLI returns non-zero for 'already exists', every re-push after the first loses branch isolation. Add an idempotency check.
Silent CLI install hides error details on failure
.github/workflows/preview-provision.yml
curl ... | sh >/dev/null 2>&1 eats all install errors. Use a temp log file and print it on failure.
quality column suppressed in resolve_stream_window without a note on metric.datasource
infra/tinybird/pipes/resolve_stream_window.pipe
raw_dedup drops quality with 'not exposed yet'. No corresponding note on metric.datasource. Add a cross-reference to prevent future confusion.
History · 10 commits
- 5d30429safeincremental0H · 0M · 1L2026-07-08 23:04
- 19bac85needs attentionincremental0H · 5M · 6L2026-07-08 22:46
- 92e890aneeds attentionincremental0H · 1M · 3L2026-07-08 21:46
- e81a781needs attentionincremental0H · 4M · 4L2026-07-08 21:31
- 2eca91cneeds attentionfull5H · 13M · 12L2026-07-08 19:18
- cf63bb1needs attentionincremental4H · 4M · 7L2026-07-07 19:47
- f43d034needs attentionincremental2H · 3M · 3L2026-07-07 18:35
- b7800c0needs attentionincremental1H · 2M · 7L2026-07-07 03:31
- 5aef105needs attentionfull6H · 9M · 2L2026-07-06 17:52
- 95b4484blockedfull3H · 8M · 4L2026-07-06 05:36current