fix/cdk-deprov
needs attentionviewing older commit9e87f75 · incrementalPR #193reviewed 2026-07-05 05:17 UTC0H · 1M · 2L- Purpose
- Prevent IAM-role quota exhaustion (999/1000 hit in dev account) caused by orphaned CloudFormation stacks from deleted preview branches
- Goal
- Add OIDC-authenticated AWS teardown to preview-deprovision.yml that deletes all Batu*-{slug}-dev stacks when a branch is deleted
- Sub-goals
- SG-1: Add permissions + bump timeout 10→20 min
- SG-2: Implement slug derivation matching preview-provision exactly
- SG-3: Enumerate and delete branch-scoped stacks with exact slug-component match
- SG-4: Wait on deletions concurrently; fail loudly on partial failure
- SG-5: Fix invalid --delay/--max-attempts waiter flags (this commit)
- What
- Replaced invalid `aws cloudformation wait --delay 20 --max-attempts 45` flags (aws-cli exits 252 on unrecognized flags) with `timeout 900 aws cloudformation wait`. Added REQUESTED tracking to only wait on successfully-submitted deletes. Narrowed describe-stack-events filter to DELETE_FAILED. Added SIGTERM/INT trap to reap background waiters on job cancellation. Added explanatory comment for intentional -e omission.
- Why
- The prior waiter invocation used --delay/--max-attempts flags that do not exist on aws cloudformation wait — the CLI would exit 252 (usage error) with stderr silenced, making every stack appear failed on every run. Author verified on aws-cli 2.31.5.
- Areas
- .github/workflows/preview-deprovision.yml+199−1
- Blast
- 1 file, +199/-1 lines (cumulative branch). Change scope is CI-only (on:delete workflow); no application code or schema affected.
Findings · 3
correctness1
Timeout-induced wait failure shows empty DELETE_FAILED events
.github/workflows/preview-deprovision.yml:296
When `timeout 900` fires before the waiter resolves (stack still in DELETE_IN_PROGRESS at 15 min), the else-branch queries `StackEvents[?ResourceStatus=='DELETE_FAILED']` — but the stack never reached that state, so zero rows are returned. Operator sees the warning but no blocking events. A fallback to show DELETE_IN_PROGRESS resources when no DELETE_FAILED events are found would improve diagnostics.
tests1
No shell unit tests for slug-extraction and guard logic
.github/workflows/preview-deprovision.yml
Slug computation, exact-match filter, and protected-scope guard are non-trivial shell logic with real blast radius if wrong (bad slug match could delete the wrong stacks). These are straightforwardly testable with bats-core; no tests exist. The new REQUESTED/pids loop adds additional untested branching paths.
improvement1
`2>/dev/null` on `wait` suppresses all waiter stderr
.github/workflows/preview-deprovision.yml:292
`timeout 900 aws cloudformation wait ... 2>/dev/null` silences all stderr from both timeout and the waiter. When the waiter exits due to a transient API error (not DELETE_FAILED), the diagnostic block runs but may find no matching events — silent failure. Consider capturing stderr to a variable for the failure branch rather than suppressing it wholesale.
History · 6 commits
- d159446needs attentionincremental1H · 5M · 7L2026-07-07 03:47
- 5ec41e1needs attentionincremental3H · 4M · 6L2026-07-07 03:23
- 9814d4dneeds attentionincremental1H · 3M · 5L2026-07-05 05:43
- 9e87f75needs attentionincremental0H · 1M · 2L2026-07-05 05:17current
- 02936b3blockedincremental1H · 0M · 4L2026-07-05 04:53
- 4407258needs attentionfull0H · 3M · 4L2026-07-05 04:27