fix/ph-insights
needs attentionviewing older commit6b58b01 · fullPR #366reviewed 2026-07-31 18:10 UTC0H · 3M · 5L · 7I- Purpose
- Fix four PostHog dashboard tiles that permanently show 'no matching events' because they query cfe_job.succeeded/failed events that the prod Lambda pipeline never emits
- Goal
- Restore meaningful analytics signal on the PostHog Activation and Platform Health dashboards without waiting for BAT-286 (Lambda terminal event wiring)
- Sub-goals
- Repoint file/data consumption funnels from dead cfe_job.succeeded to live cfe_job.created
- Add time_to_value_funnel_no_credentials for member-level activation view (bypassing admin-only credentials step)
- Detach permanently-empty job health insights from Platform Health dashboard
- Document which cfe_job.* events are live vs dead to prevent future mis-builds
- What
- infra/posthog/insights.tf — +94/-16. New funnel insight, renamed/redescribed 4 existing insights, 2 insights detached from dashboards, extensive comment documentation added.
- Why
- cfe_job.succeeded/failed/partial_success were only ever emitted by the retired Vercel HTTP handler; the Lambda pipeline (transition.lambda.ts) never ported them. Tiles have been permanently empty since the Lambda migration, misleading product analytics.
- Areas
- infra/posthog/insights.tf+94−16
- Blast
- 1 file, +94/-16. PostHog analytics only — no application code, no database, no API surface. Dev dashboard auto-updated by pr-checks; prod applies on manual workflow_dispatch.
Findings · 13
correctness4
New 'excl. credentials' funnel still has person-level admin problem at contract.created step
infra/posthog/insights.tf
time_to_value_funnel_no_credentials removes credentials.saved (admin-only) but contract.created is also emitted by whoever creates the contract — typically a billing admin, not necessarily the invited member. PostHog funnels are per-person, so an invited member who never personally fires contract.created (a colleague did it) still drops at step 2. The bias is probably smaller than the credentials step (more members create contracts), but the structural issue isn't fully resolved. The description says it answers 'how long until a member reaches value' — that claim should be softened to acknowledge this limitation.
Verify terraform plan shows in-place update for dashboard_ids = [] — destroy+recreate would break alerts.tf
infra/posthog/insights.tf
job_success_rate and job_failure_by_error are referenced by posthog_alert resources in alerts.tf (lines 62, 89) by Terraform ID. If the PostHog provider treats dashboard_ids as a ForceNew attribute, setting [] would destroy+recreate these insights with new IDs, silently breaking the alert resources. Likely safe (dashboard association is typically a mutable attribute), but requires terraform plan confirmation before merge. The PR notes that terraform isn't installed locally — this is the one pre-merge check that matters.
bill_files.downloaded_csv as terminal step misses ZIP-only users in both funnel variants
infra/posthog/insights.tf
Users who reach value via ZIP download (bill_files.downloaded_zip) but never export CSV appear as non-converters in both the original and new funnels. Not a regression — the original had the same limitation — but since time_to_value_funnel_no_credentials is specifically intended as the accurate member-level view, the description should note this constraint.
All referenced live events confirmed present in platform source
infra/posthog/insights.tf
invitation.accepted, contract.created, bill_files.downloaded_csv, bill_files.downloaded_zip, cfe_job.created are all tracked in the platform. The dead events (cfe_job.succeeded/failed/partial_success) are confirmed emitted only by the retired Vercel endpoint, not the Lambda pipeline — validating the rationale.
security1
No security concerns — dashboard detach and event renames are safe
infra/posthog/insights.tf
Setting dashboard_ids=[] removes visibility of insights from dashboards without deleting underlying data or changing project-level access. Event names (cfe_job.created, invitation.accepted, etc.) are behavioral telemetry identifiers — no PII embedded.
conventions1
New insight follows established structural pattern
infra/posthog/insights.tf
time_to_value_funnel_no_credentials matches InsightVizNode > FunnelsQuery pattern, funnelWindowIntervalUnit/funnelWindowInterval/funnelVizType, dateRange, dashboard_ids, depends_on. Detached insights correctly omit depends_on since they no longer reference the dashboard.
tests1
No automated tests expected for PostHog Terraform IaC
infra/posthog/insights.tf
PostHog IaC is validated by pr-checks auto-apply to dev project 245284. terraform plan output is the gate — the PR correctly identifies this as the verification mechanism.
improvement6
Delete the disabled alerts instead of retaining dead insights as anchor objects indefinitely
infra/posthog/insights.tf
job_success_rate and job_failure_by_error are retained solely because alerts.tf references their Terraform IDs. Both referenced alerts (job_success_rate_low, job_failure_by_error — lines 61-70, 88-97) have enabled=false and are never firing. Deleting the two disabled alert resources in alerts.tf removes the only blocker to also deleting the dead insights. Retaining permanently-empty insights as anchor objects creates a misleading audit trail and sets a precedent that accumulates. BAT-286 tracks re-wiring job health — the dead Terraform objects should not wait for that work.
Two funnel variants share no configuration — window/date range divergence risk
infra/posthog/insights.tf
time_to_value_funnel and time_to_value_funnel_no_credentials share funnelWindowInterval=30, funnelWindowIntervalUnit='day', funnelVizType='steps', and dateRange='-90d'. A future edit that changes the window on one but not the other creates a silent measurement inconsistency. Add a comment co-located with both resources explicitly stating they must be kept in sync: 'Both funnels use a 30-day window and -90d range — change both together.'
org_activation_funnel has the same uncaveated credentials.saved admin-gating problem
infra/posthog/insights.tf
org_activation_funnel (line 463) includes credentials.saved as step 2 and carries no disclaimer. The same admin-only gating analysis that motivated time_to_value_funnel_no_credentials applies here: a member who accepted an invite and ran through the rest of the flow still drops at step 2 if a colleague configured credentials. A future reader consulting this funnel for activation rates will see a structurally-bounded low number with no explanation.
Comment-only documentation of dead events will silently drift when BAT-286 lands
infra/posthog/insights.tf
The DEAD TERMINAL JOB EVENTS block is good, but prose comments are invisible to grep-for-event-name lookups. Consider adding a locals block listing the dead events referencing BAT-286, so a future search for 'cfe_job.succeeded' hits both the locals definition and every resource that references it — making cleanup automatically discoverable.
data_consumption_funnel cross-references file_consumption_funnel — too indirect if resources are reordered
infra/posthog/insights.tf
The comment above data_consumption_funnel says 'Same dead-step-1 problem and the same fix as file_consumption_funnel above'. This cross-reference breaks if the resources are ever reordered or file_consumption_funnel is deleted/renamed. Repeat the essential rationale inline (one sentence: 'cfe_job.succeeded is never emitted by the Lambda pipeline') rather than delegating to a sibling resource.
WIP follow-up scope: BAT-286 + org_activation_funnel variant
infra/posthog/insights.tf
Two likely next commits on this branch: (1) add a disclaimer/variant for org_activation_funnel, (2) delete the two disabled alerts in alerts.tf + the dead insight resources. Both are small, self-contained, and well-scoped for this branch.