← all branches

fix/ph-insights

needs attentionviewing older commit
6b58b01 · fullPR #366reviewed 2026-07-31 18:10 UTC0H · 3M · 5L · 7I
The branch
Purpose
Fix four PostHog dashboard tiles that permanently show 'no matching events' because they query cfe_job.succeeded/failed events that the prod Lambda pipeline never emits
Goal
Restore meaningful analytics signal on the PostHog Activation and Platform Health dashboards without waiting for BAT-286 (Lambda terminal event wiring)
Sub-goals
  • Repoint file/data consumption funnels from dead cfe_job.succeeded to live cfe_job.created
  • Add time_to_value_funnel_no_credentials for member-level activation view (bypassing admin-only credentials step)
  • Detach permanently-empty job health insights from Platform Health dashboard
  • Document which cfe_job.* events are live vs dead to prevent future mis-builds
The changes (whole branch)
What
infra/posthog/insights.tf — +94/-16. New funnel insight, renamed/redescribed 4 existing insights, 2 insights detached from dashboards, extensive comment documentation added.
Why
cfe_job.succeeded/failed/partial_success were only ever emitted by the retired Vercel HTTP handler; the Lambda pipeline (transition.lambda.ts) never ported them. Tiles have been permanently empty since the Lambda migration, misleading product analytics.
Areas
infra/posthog/insights.tf+9416
Blast
1 file, +94/-16. PostHog analytics only — no application code, no database, no API surface. Dev dashboard auto-updated by pr-checks; prod applies on manual workflow_dispatch.
wip-more-commits-expected terraform-plan-needed-before-merge draft-pr
ci/typecheck· Terraform IaC — no typecheck applicableci/tests· No automated tests for PostHog Terraformterraform/plan· pr-checks auto-applies to dev (245284) — plan output not yet availablecoderabbit· No .coderabbit.yaml in repo

Findings · 13

correctness4

medium

New 'excl. credentials' funnel still has person-level admin problem at contract.created step

infra/posthog/insights.tf

time_to_value_funnel_no_credentials removes credentials.saved (admin-only) but contract.created is also emitted by whoever creates the contract — typically a billing admin, not necessarily the invited member. PostHog funnels are per-person, so an invited member who never personally fires contract.created (a colleague did it) still drops at step 2. The bias is probably smaller than the credentials step (more members create contracts), but the structural issue isn't fully resolved. The description says it answers 'how long until a member reaches value' — that claim should be softened to acknowledge this limitation.

low

Verify terraform plan shows in-place update for dashboard_ids = [] — destroy+recreate would break alerts.tf

infra/posthog/insights.tf

job_success_rate and job_failure_by_error are referenced by posthog_alert resources in alerts.tf (lines 62, 89) by Terraform ID. If the PostHog provider treats dashboard_ids as a ForceNew attribute, setting [] would destroy+recreate these insights with new IDs, silently breaking the alert resources. Likely safe (dashboard association is typically a mutable attribute), but requires terraform plan confirmation before merge. The PR notes that terraform isn't installed locally — this is the one pre-merge check that matters.

low

bill_files.downloaded_csv as terminal step misses ZIP-only users in both funnel variants

infra/posthog/insights.tf

Users who reach value via ZIP download (bill_files.downloaded_zip) but never export CSV appear as non-converters in both the original and new funnels. Not a regression — the original had the same limitation — but since time_to_value_funnel_no_credentials is specifically intended as the accurate member-level view, the description should note this constraint.

info

All referenced live events confirmed present in platform source

infra/posthog/insights.tf

invitation.accepted, contract.created, bill_files.downloaded_csv, bill_files.downloaded_zip, cfe_job.created are all tracked in the platform. The dead events (cfe_job.succeeded/failed/partial_success) are confirmed emitted only by the retired Vercel endpoint, not the Lambda pipeline — validating the rationale.

security1

info

No security concerns — dashboard detach and event renames are safe

infra/posthog/insights.tf

Setting dashboard_ids=[] removes visibility of insights from dashboards without deleting underlying data or changing project-level access. Event names (cfe_job.created, invitation.accepted, etc.) are behavioral telemetry identifiers — no PII embedded.

conventions1

info

New insight follows established structural pattern

infra/posthog/insights.tf

time_to_value_funnel_no_credentials matches InsightVizNode > FunnelsQuery pattern, funnelWindowIntervalUnit/funnelWindowInterval/funnelVizType, dateRange, dashboard_ids, depends_on. Detached insights correctly omit depends_on since they no longer reference the dashboard.

tests1

info

No automated tests expected for PostHog Terraform IaC

infra/posthog/insights.tf

PostHog IaC is validated by pr-checks auto-apply to dev project 245284. terraform plan output is the gate — the PR correctly identifies this as the verification mechanism.

improvement6

medium

Delete the disabled alerts instead of retaining dead insights as anchor objects indefinitely

infra/posthog/insights.tf

job_success_rate and job_failure_by_error are retained solely because alerts.tf references their Terraform IDs. Both referenced alerts (job_success_rate_low, job_failure_by_error — lines 61-70, 88-97) have enabled=false and are never firing. Deleting the two disabled alert resources in alerts.tf removes the only blocker to also deleting the dead insights. Retaining permanently-empty insights as anchor objects creates a misleading audit trail and sets a precedent that accumulates. BAT-286 tracks re-wiring job health — the dead Terraform objects should not wait for that work.

medium

Two funnel variants share no configuration — window/date range divergence risk

infra/posthog/insights.tf

time_to_value_funnel and time_to_value_funnel_no_credentials share funnelWindowInterval=30, funnelWindowIntervalUnit='day', funnelVizType='steps', and dateRange='-90d'. A future edit that changes the window on one but not the other creates a silent measurement inconsistency. Add a comment co-located with both resources explicitly stating they must be kept in sync: 'Both funnels use a 30-day window and -90d range — change both together.'

low

org_activation_funnel has the same uncaveated credentials.saved admin-gating problem

infra/posthog/insights.tf

org_activation_funnel (line 463) includes credentials.saved as step 2 and carries no disclaimer. The same admin-only gating analysis that motivated time_to_value_funnel_no_credentials applies here: a member who accepted an invite and ran through the rest of the flow still drops at step 2 if a colleague configured credentials. A future reader consulting this funnel for activation rates will see a structurally-bounded low number with no explanation.

low

Comment-only documentation of dead events will silently drift when BAT-286 lands

infra/posthog/insights.tf

The DEAD TERMINAL JOB EVENTS block is good, but prose comments are invisible to grep-for-event-name lookups. Consider adding a locals block listing the dead events referencing BAT-286, so a future search for 'cfe_job.succeeded' hits both the locals definition and every resource that references it — making cleanup automatically discoverable.

low

data_consumption_funnel cross-references file_consumption_funnel — too indirect if resources are reordered

infra/posthog/insights.tf

The comment above data_consumption_funnel says 'Same dead-step-1 problem and the same fix as file_consumption_funnel above'. This cross-reference breaks if the resources are ever reordered or file_consumption_funnel is deleted/renamed. Repeat the essential rationale inline (one sentence: 'cfe_job.succeeded is never emitted by the Lambda pipeline') rather than delegating to a sibling resource.

info

WIP follow-up scope: BAT-286 + org_activation_funnel variant

infra/posthog/insights.tf

Two likely next commits on this branch: (1) add a disclaimer/variant for org_activation_funnel, (2) delete the two disabled alerts in alerts.tf + the dead insight resources. Both are small, self-contained, and well-scoped for this branch.

History · 5 commits

  1. ba4f438safeincremental0H · 0M · 1L2026-08-01 00:54
  2. f9ad657needs attentionincremental0H · 1M · 2L2026-08-01 00:35
  3. 1475bfbsafeincremental0H · 0M · 0L2026-07-31 20:34
  4. 6b58b01needs attentionfull0H · 3M · 5L2026-07-31 18:10current
  5. a97e2f0needs attentionfull1H · 2M · 2L2026-07-31 17:23