feat/tb-deploy
needs attentionviewing older commit2eca91c · fullPR #261reviewed 2026-07-08 19:18 UTC5H · 13M · 12L · 1I- Purpose
- Define Batu Tinybird deployment strategy with hard per-environment isolation and least-privilege token model.
- Goal
- Three long-lived workspaces (dev/stg/prod), ephemeral Forward branches per PR, per-env token split, schema-CD lanes via tb deploy.
- Sub-goals
- SG-1: Strategy rule doc + README de-stale; token split
- SG-2: Per-env metrics-SSM seed (preview/stg/prod)
- SG-4: Ephemeral per-PR Tinybird branch + cleanup
- SG-5: Schema-CD tinybird-staging/tinybird-prod jobs
- SG-6: Fold into infrastructure.md/migration-pipeline.md
- What
- New tinybird-deployment.md rule; preview-provision.yml gains TB branch + SSM seed; staging-update.yml + release workflow gain schema-CD; pr-checks.yml adds tinybird-check; infra/tinybird datasource+pipe+tests updated.
- Why
- Single shared TINYBIRD_TOKEN was admin-scoped with no env isolation. Forward workspace model enables per-env schema versioning and least-privilege tokens.
- Areas
- .claude/rules/infrastructure.md+10−2.claude/rules/migration-pipeline.md+13−0.claude/rules/tinybird-deployment.md+255−0.github/workflows/pr-checks.yml+96−2.github/workflows/preview-deprovision.yml+27−0.github/workflows/preview-provision.yml+165−36.github/workflows/release-workflow.yml+62−5.github/workflows/staging-update.yml+66−5infra/tinybird/README.md+60−43infra/tinybird/datasources/metric.datasource+13−0infra/tinybird/fixtures/metric.ndjson+7−0infra/tinybird/pipes/resolve_stream_window.pipe+10−1+2 more
- Blast
- 14 files, +821/-108; CI/infra only, no TypeScript. Write path (CDK Lambda SSM), read path (Vercel env), schema-CD lane, and operator runbook all affected.
Findings · 31
correctness10
README contradicts workflow: branch SSM token seeding status
infra/tinybird/README.md:100
README states the branch token is NOT seeded to SSM by preview-provision.yml, but the workflow code does seed it. If README is authoritative, CDK Lambdas write to shared dev workspace while Vercel reads from the ephemeral branch.
Branch cap conflict: SSOT says 4, README says 6
infra/tinybird/README.md:87
The declared SSOT (tinybird-deployment.md) states 4 branches/workspace in three places. README says 6. The cleanup urgency and capacity-guard logic are anchored to 4.
tinybird-prod runs in parallel to cdk-deploy with no ordering gate
.github/workflows/release-workflow.yml:313
Both jobs depend only on preflight and run concurrently. A breaking schema change could reach the prod Tinybird workspace before CDK Lambda code catches up.
README operator runbook uses wrong secret name for branch auth
infra/tinybird/README.md:97
README says TINYBIRD_TOKEN_DEV (runtime token). The workflow uses TINYBIRD_DEPLOY_TOKEN_DEV (ADMIN scope). Operator following README will configure the wrong secret.
TB branch provisioned but SSM token seed gated on cdk_decision - split on TB-only PRs
.github/workflows/preview-provision.yml:776
If a PR touches only infra/tinybird/ but not CDK paths, the branch is provisioned but its token is never written to SSM. CDK Lambda uses stale dev-workspace token while Vercel reads from the ephemeral branch.
staging-update: tb --check failure does not block tb deploy
.github/workflows/staging-update.yml:230
The deploy step runs tb --cloud deploy --check then tb --cloud deploy in the same shell without set -e. A failing --check does not prevent the real deploy.
Empty TINYBIRD_BRANCH_TOKEN silently written to Vercel when both fallback paths fail
.github/workflows/preview-provision.yml:454
The fallback chain tb_branch_token || workspace_token yields empty string if extraction fails and the tinybird step is skipped. Metrics silently disabled in preview.
Docker health poll matches on container name tinybird-local - brittle across CLI versions
.github/workflows/pr-checks.yml:398
grep times out if the CLI names the container differently. Use docker ps --filter health=healthy.
summary gate allows skipped for tinybird-check but the job never skips
.github/workflows/pr-checks.yml:436
The job has no job-level if: so GitHub never marks it skipped. Dead branch in the summary check.
Deprovision uses tb branch rm, provision uses tb preview - unpinned CLI may diverge
.github/workflows/preview-deprovision.yml:80
Both workflows install CLI without version pinning. If a release renames these subcommands they could diverge silently.
security8
Unpinned curl-pipe-sh Tinybird CLI install in 4 workflows - RCE surface on CI
.github/workflows/pr-checks.yml:416
curl -fsSL https://tinybird.co | sh with no version pin or checksum. A compromised CDN executes arbitrary code in the CI runner that holds TINYBIRD_DEPLOY_TOKEN_DEV (ADMIN). Appears in 4 workflow files.
tinybird-check inherits pull-requests: write on fork-facing pull_request trigger
.github/workflows/pr-checks.yml:389
No job-level permissions block. Inherits workflow-level pull-requests: write. GITHUB_TOKEN with PR write is available to fork PRs, allowing comment injection.
Branch token written to GITHUB_OUTPUT before masking
.github/workflows/preview-provision.yml:336
BRANCH_TOKEN held in shell before add-mask is emitted. A future set -x for debugging would expose the token in logs before masking.
Legacy TINYBIRD_TOKEN echoed into GITHUB_OUTPUT without masking
.github/workflows/preview-provision.yml:282
GitHub auto-masks secrets in env: but not in output expressions. A step logging the expression value would expose the token.
Dev OIDC role may grant SSM writes to /batu/* - crafted branch name could overwrite sensitive paths
.github/workflows/preview-provision.yml:830
SSM writes to /batu/SLUG/dev/... where SLUG derives from the branch name. If IAM policy grants ssm:PutParameter on /batu/* rather than /batu/*/dev/*, a crafted slug could target prod paths.
Best-effort branch cleanup leaves orphaned tokens valid if deprovision fails
.github/workflows/preview-deprovision.yml:469
set +e active for entire TB cleanup block. A failed branch rm exits 0 with a warning. Leaked branches exhaust the 4-branch cap; orphaned tokens may remain valid indefinitely.
Working-directory: infra/tinybird could load a .tinyenv credential from checkout
.github/workflows/preview-provision.yml:306
Forward CLI reads .tinyenv from CWD. A malicious commit introducing a .tinyenv file would redirect CLI auth to an attacker-controlled workspace.
Comment incorrectly labels TINYBIRD_TOKEN_DEV as admin token
.github/workflows/preview-provision.yml:821
Inline comment says admin token (TINYBIRD_TOKEN_DEV). That secret is the runtime token. A future operator may substitute the actual ADMIN token, seeding an over-privileged token into SSM.
conventions4
tinybird-check job missing permissions at job level
.github/workflows/pr-checks.yml:389
All other jobs in pr-checks.yml that need write access declare permissions at job level. Should declare permissions: contents: read.
preview-deprovision.yml has no permissions block - defaults to wide write access
.github/workflows/preview-deprovision.yml:1
Repo convention is to declare explicit permissions. preview-deprovision.yml has none, running with GitHub defaults.
SSM path notation in tinybird-deployment.md conflates branch-preview and env formats
.claude/rules/tinybird-deployment.md:126
Documented path makes /dev/ look like a fixed segment. For stg/prod the actual path has no /dev/ segment.
tinybird-staging job missing job-level permissions
.github/workflows/staging-update.yml:198
Other narrowly-scoped jobs in staging-update.yml follow the pattern of declaring job-level permissions.
tests3
Aggregators min/max/last untested in resolve_stream_window.yaml
infra/tinybird/tests/resolve_stream_window.yaml:37
Only sum and mean are exercised. The min/max/last branches use structurally distinct SQL and could silently break.
Sub-hourly interval buckets (300s, 900s) untested - primary billing read path
infra/tinybird/tests/resolve_stream_window.yaml:37
All 5 tests use interval_seconds=3600. The 5-min/15-min bucket alignment is the primary billing input per site-metrics-seam.md.
quality field absence implicitly covered by golden output schema
infra/tinybird/tests/resolve_stream_window.yaml:1
Existing goldens assert output without quality column. Adequate coverage.
improvement6
Tinybird CLI install block duplicated in 4 workflows
.github/workflows/staging-update.yml:223
curl install + PATH export appears in 4 files with slight error-handling variations. A composite action at .github/actions/tb-install/ would collapse all four.
tinybird-check uses hand-rolled git diff for path detection instead of dorny/paths-filter
.github/workflows/pr-checks.yml:400
Every other path-gated job uses dorny/paths-filter at a pinned SHA. The hand-rolled filter only watches infra/tinybird/; provision gate also covers services/metrics/**, packages/tinybird/**, and handler file.
tinybird-prod not in rollback job needs list - prod TB failure invisible to rollback
.github/workflows/release-workflow.yml:846
If TB prod schema deploy fails after CDK has already run, rollback does not trigger. Should appear in rollback needs list and if: condition.
TB_HOST default URL hardcoded in 5+ places across 4 workflows
.github/workflows/preview-provision.yml:608
The literal API URL appears as a fallback in every workflow. A workflow-level env: would make it a single source of truth.
tinybird-staging runs --check then --deploy in same step
.github/workflows/staging-update.yml:230
Splitting into two named steps would give separate pass/fail visibility in the GitHub Actions UI.
Token table duplicated in two rule docs
.claude/rules/tinybird-deployment.md:130
The GitHub Actions secrets table appears in both tinybird-deployment.md and infrastructure.md with near-identical content. One should cross-reference the other.
History · 10 commits
- 5d30429safeincremental0H · 0M · 1L2026-07-08 23:04
- 19bac85needs attentionincremental0H · 5M · 6L2026-07-08 22:46
- 92e890aneeds attentionincremental0H · 1M · 3L2026-07-08 21:46
- e81a781needs attentionincremental0H · 4M · 4L2026-07-08 21:31
- 2eca91cneeds attentionfull5H · 13M · 12L2026-07-08 19:18current
- cf63bb1needs attentionincremental4H · 4M · 7L2026-07-07 19:47
- f43d034needs attentionincremental2H · 3M · 3L2026-07-07 18:35
- b7800c0needs attentionincremental1H · 2M · 7L2026-07-07 03:31
- 5aef105needs attentionfull6H · 9M · 2L2026-07-06 17:52
- 95b4484blockedfull3H · 8M · 4L2026-07-06 05:36