feat/ci-speed
needs attention167f02a · incrementalpre-PRreviewed 2026-08-12 17:47 UTC1H · 2M · 2L · 2I- Purpose
- Reduce PR-gate wall-clock time by eliminating the dominant costs in the CI pipeline
- Goal
- Quantify the latency improvement from Turbo remote cache + deployment-triggered E2E + Blacksmith bare-metal runners, then merge the proven speedups
- Sub-goals
- SG-1: Turbo remote cache — typecheck/lint/test return cached results in seconds instead of recomputing (~143s cold typecheck → seconds warm)
- SG-2: deployment_status E2E trigger — start from a READY deployment instead of polling for build completion (~287s wait eliminated)
- SG-3: Blacksmith 4-vCPU runner — measure bare-metal vs GitHub 2-vCPU hosted for the CPU-bound validate job
- SG-4: TURBO_FORCE=true A/B — measure Blacksmith raw compute without cache hits (temporary, must revert)
- What
- Four commits: (1) Turbo remote cache + deployment_status E2E trigger, (2) empty commit for warm-cache baseline measurement, (3) Blacksmith runner switch for A/B, (4) TURBO_FORCE=true for cold-compute A/B measurement
- Why
- PR gate was timing out at 10min on cold runs (320s typecheck + 195s lint); even with the 15min budget, runs were slow. The branch measures each optimization incrementally before committing to it
- Areas
- .github/workflows/e2e.yml+111−55.github/workflows/pr-checks.yml+22−1.claude/rules/e2e-testing.md+29−19docs/ci-cd-diagnosis.html+127−0
- Blast
- 4 files, +289/−75 total. CI pipeline only — no application code, no schema, no domain logic. Risk is limited to PR gate reliability.
Findings · 7
correctness1
TURBO_FORCE: "true" — active on branch, must be reverted before merge
.github/workflows/pr-checks.yml:45
TURBO_FORCE=true forces a full recompute on every CI run, defeating the remote cache that is the primary latency win on this branch. The commit message says 'revert before merge' and an inline comment acknowledges this, but there is no automated enforcement — nothing prevents an accidental merge. Every PR on this branch currently runs at full cold-cache cost (~320–522s typecheck per the branch's own measurements).
security2
TURBO_TOKEN promoted to job-level env — exposed to pnpm install lifecycle scripts
.github/workflows/pr-checks.yml:40
Moving TURBO_TOKEN (the Vercel token) from step-level to job-level means every step in the validate job — including `pnpm install --frozen-lockfile` — inherits it as an env var. A malicious package postinstall script could exfiltrate it. The prior step-level scoping was safer. Fix: keep TURBO_TEAM at job level (non-sensitive) and move TURBO_TOKEN back to the three Turbo-invoking steps (typecheck, lint, test).
Blacksmith is third-party runner infra — verify org policy
.github/workflows/pr-checks.yml:25
blacksmith-4vcpu-ubuntu-2404 is a Blacksmith (useblacksmith.com) hosted runner, not a GitHub-managed runner. Secrets injected into this job (TURBO_TOKEN, implicit GITHUB_TOKEN) are processed on Blacksmith infrastructure. Confirm: (a) Blacksmith is approved in the org's Actions runner policy, (b) runner is registered via the official GitHub App, (c) Blacksmith's runner isolation guarantees are acceptable for a Vercel token. The tinybird-check job already stays on GitHub runners for this same isolation reason.
conventions1
No machine-scannable guard against accidental merge of TURBO_FORCE
.github/workflows/pr-checks.yml:43
The removal reminder is a YAML comment only — no TODO/FIXME prefix, no CI step that fails if TURBO_FORCE is set on a PR to main. A reviewer who misses the comment would merge it. Consider: a step that runs `if grep -q 'TURBO_FORCE' .github/workflows/pr-checks.yml; then echo '::error::TURBO_FORCE benchmark flag must be removed'; exit 1; fi`, or at minimum a # FIXME: prefix.
tests2
No fallback if Blacksmith runner is unavailable
.github/workflows/pr-checks.yml:25
runs-on: blacksmith-4vcpu-ubuntu-2404 is the sole runner label with no fallback. If Blacksmith is at capacity, offline, or the label is misconfigured, the validate job queues indefinitely — GitHub does not fail immediately on unknown/unavailable labels. Since the summary job needs validate to succeed, an unavailable Blacksmith would silently block all PRs. Prior ubuntu-latest had guaranteed availability.
A/B benchmark results not captured in a durable form
.github/workflows/pr-checks.yml
The workflow emits no structured timing artifact — job durations exist only in the GitHub Actions UI and Blacksmith's dashboard. Nothing writes elapsed times to a step summary, uploads a JSON artifact, or posts a PR comment. The two data points (warm-cache baseline at ca9f9158 vs cold-compute at 167f02a4) must be manually read from two separate run pages. Easy to lose.
improvement1
Local .turbo cache step is dead weight while TURBO_FORCE=true
.github/workflows/pr-checks.yml:67
The actions/cache step restoring .turbo is fully bypassed by TURBO_FORCE=true (Turbo ignores all cached output). The restore leg wastes ~5-10s and the save leg writes a bloated full-compute cache on every run. Not a problem for the benchmark, but the local cache step may also be safely dropped post-merge once remote cache hit rates are proven.