← all branches

fix/cdk-deprov

needs attentionviewing older commit
5ec41e1 · incrementalPR #193reviewed 2026-07-07 03:23 UTC3H · 4M · 6L · 5I
The branch
Purpose
Fix critical IAM role accumulation in the dev AWS account — every preview branch deployed 8–14 CloudFormation stacks with 1–15 IAM roles each, but branch deletion had no AWS teardown. Account hit 999/1000 IAM roles, blocking new preview deploys with ServiceLimitExceeded.
Goal
Add OIDC-authenticated AWS teardown to preview-deprovision.yml and harden preview-provision.yml against flaky transient failures.
Sub-goals
  • SG-1: Derive slug from deleted branch using exact same slugification rule as provision, enumerate Batu*-{slug}-dev stacks, delete each and wait for completion
  • SG-2: Best-effort remove /batu/{slug}/dev/** SSM parameter namespace on branch delete
  • SG-3: Add curl retry hardening to preview-provision.yml for flaky network calls (this commit)
The changes (whole branch)
What
This incremental commit (5ec41e13) adds `--retry 5 --retry-all-errors --retry-delay 2 --connect-timeout 15 --max-time N` to 6 curl calls in preview-provision.yml: the Supabase CLI version-fetch and binary download, the Supabase auth PATCH, and 4 Vercel API POSTs.
Why
Flaky transient network failures in the provision workflow were causing CI runs to fail non-deterministically. The main PR adds a full CDK stack teardown on branch delete (preview-deprovision.yml, +214/-1 lines); this commit hardens the provision side against the same class of failures.
Areas
.github/workflows/preview-deprovision.yml+2141.github/workflows/preview-provision.yml+77
Blast
2 files, +221/−8 across .github/workflows. CI-only change — no application code, no database, no CDK stacks, no API contracts.
ci-only no-app-code-changed
ci· GitHub check-runs not accessible via PAT (HTTP 403)coderabbit· No .coderabbit.yaml present

Findings · 18

correctness4

high

--retry-all-errors on Supabase PATCH retries 4xx auth failures — up to 370s wasted

.github/workflows/preview-provision.yml:272

--retry-all-errors retries on every HTTP error including 401 Unauthorized and 403 Forbidden. If SUPABASE_ACCESS_TOKEN is missing or expired, the PATCH /config/auth will be retried 5 times with --max-time 60 per attempt (worst case 6×60 + 5×2 = 370s) before the step fails. The actual error (bad token vs. transient network failure) is masked — only the last retry's response is visible. For an authenticated PATCH, --retry-connrefused or --retry with explicit HTTP code filtering (500,502,503,504) would be more appropriate than retrying on all errors including 4xx.

high

--retry-all-errors on Vercel deployments POST may create duplicate deployments on 5xx

.github/workflows/preview-provision.yml:475

POST /v13/deployments is retried on 5xx by --retry-all-errors. If Vercel returns a 5xx after the deployment was actually created server-side (a common pattern with creation APIs), a retry triggers a second deployment with the same gitSource.sha. Vercel's deduplication behavior is not documented as a hard guarantee. The same pattern appears at line 1058. A safer approach: retry only on connection errors (--retry-connrefused) or check for an existing deployment before creating.

medium

Worst-case retry timing can exhaust the 45-minute job timeout with CDK

.github/workflows/preview-provision.yml:440

--max-time is per attempt, not total. With --retry 5 and --retry-delay 2, each --max-time 90 call can take up to 5×(90+2)=460s (~7.7 min). There are 5 such calls in the provision workflow (lines 272, 440, 475, ~1023, 1058). In the adversarial case where multiple calls each hit all retries, total curl time can exceed 45 min — especially combined with CDK deploy (~25 min). Consider --max-time 30 for API calls (not binary downloads) or reducing to --retry 3.

low

GitHub API version check lacks User-Agent — rate-limited IPs retry 5× before failing

.github/workflows/preview-provision.yml:70

GitHub's REST API requires a User-Agent header and rate-limits unauthenticated requests to 60/hour per IP. GitHub Actions runners share IP pools. If the runner IP is rate-limited, the API returns 403. With --retry-all-errors and --max-time 30, this retries 5× (up to 190s) before failing. If LATEST_VERSION ends up empty, the download URL becomes malformed (v//supabase...) and tar fails. Pre-existing issue; retry addition widens the failure window. Fix: add -H 'User-Agent: batu-preview-provision' and authenticate with the $GITHUB_TOKEN.

security4

medium

Supabase CLI binary downloaded without checksum or signature verification

.github/workflows/preview-provision.yml:71

The binary is downloaded from GitHub Releases and extracted directly with no integrity check (sha256sum, cosign, or GPG). The retry logic introduces a subtle new risk: LATEST_VERSION is resolved in one curl call; the tarball is downloaded in a second. If GitHub releases a new version between the two requests, the installed binary silently diverges from the resolved version string. There is no checksum gate to detect this. Mitigation: pin to a fixed version or verify the tarball against the release's .sha256 file before extraction.

low

--retry-all-errors re-sends Bearer tokens on every 4xx retry

.github/workflows/preview-provision.yml:272

All three authenticated curl calls carry --retry-all-errors, causing the full Authorization: Bearer header to be sent on each retry attempt including 401/403 responses. This increases the observable footprint in server-side access logs and delays detection of a misconfigured token. Not an exploitable vulnerability (the token is already in memory and used on the first attempt), but --retry-http-codes 429,500,502,503,504 would be a more precise and secure alternative.

low

LATEST_VERSION extracted via grep/sed from unvalidated API response — no format guard

.github/workflows/preview-provision.yml:70

The version string from the GitHub API is used directly in a URL path segment without a format validation. The sed regex ([^"]+) restricts to non-quote characters but doesn't enforce semver format. A [[ $LATEST_VERSION =~ ^[0-9]+\.[0-9]+\.[0-9]+$ ]] guard before the download would fail fast on a malformed response. Pre-existing issue; flagging as defense-in-depth.

info

No new credential exposure introduced by this diff

.github/workflows/preview-provision.yml:272

All secrets (SUPABASE_ACCESS_TOKEN, VERCEL_TOKEN) are sourced from secrets.* context and passed as HTTP headers, not as environment variables visible in process listings. The retry flags do not change how credentials are stored or logged within GitHub Actions. The -sS flag suppresses progress but retains error messages — response bodies are not echoed. No new secrets introduced.

conventions5

high

preview-deprovision.yml has 6 bare curl calls not hardened

.github/workflows/preview-deprovision.yml:43

The companion deprovisioner (the main PR work — AWS teardown) has 6 bare curl calls (lines 43, 44, 76, 99, 128, 145) with no --retry, --connect-timeout, or --max-time. Lines 43–44 mirror the exact Supabase CLI install pattern hardened in provision.yml. Lines 76, 99, 128, 145 are Vercel API calls using bare `curl -s` (silent — swallows errors). Since deprovision runs on branch deletion, transient failures leave orphaned Vercel resources. The hardening applied to provision.yml should be applied symmetrically.

medium

Inconsistent flag ordering across curl calls

.github/workflows/preview-provision.yml:272

On line 272, retry flags are inserted between -sS and -o/-w output flags. On lines 440/475/1023/1058, they appear before -w and -X. No codified convention, but the de-facto grouping (mode flags → retry/timeout → output/method flags → URL) is not consistently followed. Minor but worth standardising if a CURL_RETRY env var approach is adopted.

low

--retry-all-errors behavior on 4xx is non-intuitive for readers

.github/workflows/preview-provision.yml:440

When used with -w '%{http_code}', a 4xx response does NOT set curl's error exit code — so retry won't actually fire on 409/404/422 from Vercel. The flag is therefore safer than it looks, but a comment explaining 'retries on connection errors and 5xx only (4xx don't trigger curl error exit code with -w)' would prevent future confusion when someone reads --retry-all-errors and assumes all HTTP errors are retried.

info

--max-time values are appropriate per use-case

.github/workflows/preview-provision.yml:70

30s for GitHub API JSON (version check), 120s for a ~50MB binary download, 60s for Supabase management PATCH, 90s for Vercel API POSTs — all appropriate. No anomalies.

info

--retry-all-errors is safe on current GHA runners (curl 7.81+)

.github/workflows/preview-provision.yml:70

--retry-all-errors requires curl >= 7.71.0 (June 2020). ubuntu-latest ships curl 7.81+ (Ubuntu 22.04) / 8.5+ (Ubuntu 24.04). The self-hosted runner has curl 8.17.0. No compatibility risk.

tests1

info

No unit tests — acceptable for CI workflow hardening

.github/workflows/preview-provision.yml:1

GHA workflows are not unit-testable in a meaningful way. The retry behavior is a curl built-in and trust in it is reasonable. No act-based test infrastructure exists in the repo (normal for this scale). Correct validation is observing flaky-failure rates drop in production runs.

improvement4

medium

DRY: identical curl retry flags repeated 7 times

.github/workflows/preview-provision.yml:70

The string `--retry 5 --retry-all-errors --retry-delay 2 --connect-timeout 15` appears identically on all 7 curl invocations (lines 70, 71, 272, 440, 475, 1023, 1058). Maintenance debt: changing the retry count or timeout requires updating every line. Idiomatic GHA fix: a job-level env var `CURL_RETRY: "--retry 5 --retry-all-errors --retry-delay 2 --connect-timeout 15"` used as `curl $CURL_RETRY ...`. Centralises all instances and makes the intent explicit.

low

Missing --retry-max-time to bound total retry window

.github/workflows/preview-provision.yml:440

--max-time caps a single attempt but the total retry sequence is unbounded except by the job timeout. Adding --retry-max-time 300 would stop retrying after 5 minutes total regardless of remaining attempts. Useful for surfacing stuck steps sooner without changing per-attempt behavior.

low

Fixed --retry-delay 2 suboptimal for API rate-limit (429) scenarios

.github/workflows/preview-provision.yml:272

A 2-second fixed delay is fine for transient TCP errors but too short for API rate-limit (HTTP 429) scenarios where the server's retry-after may be 60s+. curl doesn't natively support exponential backoff, but a shell wrapper loop with sleep $((2 ** attempt)) would handle 429s more gracefully for the Vercel/Supabase calls. Minor — 429 is not the primary failure mode being fixed here.

info

--retry-all-errors compatibility confirmed — no action needed

.github/workflows/preview-provision.yml:70

GHA ubuntu-latest and the self-hosted runner both have curl 8.x, well above the 7.71.0 minimum for --retry-all-errors. Document if runner images are ever pinned to older Ubuntu versions.

History · 6 commits

  1. d159446needs attentionincremental1H · 5M · 7L2026-07-07 03:47
  2. 5ec41e1needs attentionincremental3H · 4M · 6L2026-07-07 03:23current
  3. 9814d4dneeds attentionincremental1H · 3M · 5L2026-07-05 05:43
  4. 9e87f75needs attentionincremental0H · 1M · 2L2026-07-05 05:17
  5. 02936b3blockedincremental1H · 0M · 4L2026-07-05 04:53
  6. 4407258needs attentionfull0H · 3M · 4L2026-07-05 04:27