chore/ph-alerts
needs attention6b815eb · fullPR #360reviewed 2026-07-30 20:49 UTC0H · 1M · 2LThe branch
- Purpose
- Stop alert fatigue from 4 broken PostHog alerts firing false daily signals
- Goal
- Disable 4 lying alerts while keeping healthy absence-detection alerts enabled
- Sub-goals
- SG-1: Disable job_success_rate_low (dead Lambda event)
- SG-2: Disable job_failure_spike (dead Lambda event)
- SG-3: Disable new_subscription (inverted threshold_lower)
- SG-4: Disable subscription_churned (inverted threshold_lower)
- SG-5: Add file-level comment distinguishing absence vs positive-event shapes
The changes (whole branch)
- What
- infra/posthog/alerts.tf: added enabled=false to 4 alert resources + explanatory comments
- Why
- 4 alerts were generating daily false pages obscuring real signals
- Areas
- infra/posthog/alerts.tf+28−0
- Blast
- 1 file, +28/-0. IaC-only: dev on merge, prod on next gated dispatch.
prod-propagation-deferred
➖ci· CI check rollup not accessible via token➖coderabbit· No .coderabbit.yaml present
Findings · 3
improvement3
medium
Prod keeps paging until next IaC apply — not tracked
infra/posthog/alerts.tf:61
enabled=false lands in dev (245284) on merge but prod (334265) only applies on the next gated dispatch. No sub-task tracks this. A BAT-286 sub-task for the prod apply would close the loop.
low
login_success_rate_low denominator question not tracked
infra/posthog/alerts.tf:10
PR flags that login.attempted may not fire on every auth path but alert stays enabled with no BAT issue tracking verification.
low
No grep-findable sentinel for disabled alerts
infra/posthog/alerts.tf:70
A consistent inline tag (e.g. # TODO(BAT-286): re-enable) on the enabled=false line would let grep surface all disabled alerts.