fix/cdk-deprov
blockedviewing older commit02936b3 · incrementalPR #193reviewed 2026-07-05 04:53 UTC1H · 0M · 4L · 3I- Purpose
- Fix branch-scoped CDK stack teardown on branch delete: stacks were not being deleted and failures were silent, risking IAM role quota exhaustion.
- Goal
- Make preview-deprovision.yml CDK teardown fail loudly and be observable via GitHub Step Summary + blocking-event diagnostics.
- Sub-goals
- SG-1: Convert sequential stack-delete waiters to parallel background processes
- SG-2: Add bounded waiter timeout to prevent the job being killed mid-wait
- SG-3: Emit describe-stack-events diagnostics when a stack fails to delete
- SG-4: Write a structured GitHub Step Summary with a deleted/failed table
- SG-5: exit 1 when any stack fails — turn silent partial failure into a visible error
- What
- Single file: .github/workflows/preview-deprovision.yml — sequential waiters converted to parallel background procs with pids; DELETED/FAILED counters; bounded waiter; describe-stack-events diagnostics; GitHub Step Summary; exit 1 on failure; SSM delete now warns on failure.
- Why
- Previous teardown silently ignored waiter failures (warning only, no exit 1), so orphaned IAM roles from failed deletes were invisible.
- Areas
- .github/workflows/preview-deprovision.yml+189−1
- Blast
- 1 file, +189/-1 lines. CI workflow only — no production code, no package changes.
Findings · 9
correctness2
`--delay` and `--max-attempts` are invalid AWS CLI v2 flags for `cloudformation wait`
.github/workflows/preview-deprovision.yml:284
Confirmed on this runner (aws-cli/2.33.15): `aws cloudformation wait stack-delete-complete --delay 20 --max-attempts 45` exits 252 with 'Unknown options: --delay, --max-attempts, 45, 20'. Because stderr is redirected with `2>/dev/null`, the error is invisible. The `if` condition sees non-zero, branches immediately to the else-path, prints the warning + describe-stack-events, then exit 1 — for every stack, every run, regardless of actual deletion outcome. Fix: remove the invalid flags and use `timeout 900 aws cloudformation wait stack-delete-complete --stack-name "$s"` to bound the wait, OR pass waiter config as JSON `--waiter-config '{"Delay":20,"MaxAttempts":45}'` (the v2-supported form — needs verification against this AWS CLI version).
Failed `delete-stack` still spawns a waiter, causing misleading 15-min wait
.github/workflows/preview-deprovision.yml:272
When `aws cloudformation delete-stack` fails (the `|| echo ::warning::` path), the waiter loop still spawns a background process for that stack. The waiter sees the stack in its pre-delete state and runs until max-attempts or until the stack happens to not exist. The describe-stack-events output shows no DELETE_FAILED events, confusing the operator. A `continue` on delete-stack failure would skip the pointless wait and give a cleaner error message.
security1
Raw BRANCH_NAME passed to Supabase CLI without slug-normalization (pre-existing, not in this diff)
.github/workflows/preview-deprovision.yml
Pre-existing: BRANCH_NAME from github.event.ref is passed directly to `supabase branches delete` without the [a-z0-9-] normalization applied to SLUG. Double-quoting prevents word-splitting but a crafted branch name could theoretically inject CLI flags. GitHub's branch naming rules limit real-world exploitability.
conventions4
Step name 'Configure AWS Credentials' deviates from all other steps in repo
.github/workflows/preview-deprovision.yml:189
Every other 'Configure AWS credentials' step (infra-cdk.yml, pr-checks.yml, staging-update.yml x2, release-promote.yml x2, preview-provision.yml) uses lowercase 'credentials'. This step is now the only outlier. Revert to lowercase for consistency.
`${GITHUB_STEP_SUMMARY:-/dev/null}` fallback pattern inconsistent with rest of repo
.github/workflows/preview-deprovision.yml:330
Every other GITHUB_STEP_SUMMARY write in the repo uses the bare form (no fallback). The local-run guard is technically correct but creates a style inconsistency. Either adopt it uniformly or drop it here to match existing convention.
`describe-stack-events` JMESPath `contains(ResourceStatus, 'FAILED')` matches historical events
.github/workflows/preview-deprovision.yml:288
The filter matches any FAILED event in the stack's full history — including historical UPDATE_ROLLBACK_FAILED events from before the delete attempt. On stacks with prior failed updates, this surfaces noise alongside the deletion-blocking event. More precise: `ResourceStatus == 'DELETE_FAILED'`.
`set -uo pipefail` (no `-e`) is intentional but undocumented
.github/workflows/preview-deprovision.yml:201
Intentional (the `|| echo ::warning::` chains require no -e), but all other `set` invocations in the repo use `set -euo pipefail`. A brief inline comment would make the deliberate choice clear.
improvement2
No SIGTERM trap — cancelled jobs leave background waiters as orphans
.github/workflows/preview-deprovision.yml
If the GHA job is cancelled, background waiter subprocesses receive SIGTERM but no cleanup trap is set. A `trap 'kill $pids 2>/dev/null; exit 130' INT TERM` before the waiter loop would make cancellation clean.
Concurrent waiter stdout may interleave — cosmetic, not operational
.github/workflows/preview-deprovision.yml
With multiple background waiters writing to stdout concurrently, log lines may interleave. Individual echo calls are atomic at the OS level so GHA annotation parsing should not be corrupted. Not worth adding complexity to fix at current scale.
History · 6 commits
- d159446needs attentionincremental1H · 5M · 7L2026-07-07 03:47
- 5ec41e1needs attentionincremental3H · 4M · 6L2026-07-07 03:23
- 9814d4dneeds attentionincremental1H · 3M · 5L2026-07-05 05:43
- 9e87f75needs attentionincremental0H · 1M · 2L2026-07-05 05:17
- 02936b3blockedincremental1H · 0M · 4L2026-07-05 04:53current
- 4407258needs attentionfull0H · 3M · 4L2026-07-05 04:27