chore/ph-annot

needs attention

70e9fcb · incrementalPR #359reviewed 2026-07-30 20:43 UTC0H · 2M · 5L · 6I

The branch

Purpose

Wire a non-blocking PostHog deploy annotation so every prod release drops a vertical marker on the PostHog timeline, making metric changes attributable to specific deploys

Goal

Eliminate the 'something changed but when?' ambiguity on PostHog charts; each prod promotion now writes a dated marker with SHA, actor, and stable prod URL

Sub-goals

SG-1: Add scripts/posthog-annotate.mjs — resilient API caller (exits 0 on any error)
SG-2: Add posthog-annotate job in release-promote.yml Phase 5b — non-gating, continue-on-error
SG-3: Document the convention in .claude/rules/posthog-annotations.md as SSOT

The changes (whole branch)

What: Added PostHog annotation script + non-gating CI job; added skip_annotation boolean input escape hatch; tightened job permissions to contents:read; switched DEPLOY_URL from rotating per-deploy Vercel URL to stable secrets.PRODUCTION_URL; scrubbed annotation content from runner log
Why: Previous codebase had no deploy markers on PostHog charts — impossible to see 'a deploy happened here' when debugging metric cliffs
Areas: .claude/rules/posthog-annotations.md+81−0.github/workflows/release-promote.yml+53−0scripts/posthog-annotate.mjs+109−0
Blast: 3 files, +243 lines. CI-only changes: no application code, no database schema, no API contracts. The annotation job is non-gating (continue-on-error) so it cannot break a release.

ci-only non-gating analytics-telemetry

➖ci· CI check rollup not accessible via PAT➖coderabbit· No .coderabbit.yaml in repo

Findings · 13

correctness3

medium

skip_annotation boolean coercion — verify on first real dispatch

.github/workflows/release-promote.yml:751

GitHub Actions has coerced boolean-typed workflow_dispatch inputs correctly since late 2021, so !inputs.skip_annotation with default: false should work. Worth a one-time manual confirm on next dispatch since the default (false → annotation runs) is the hot path.

low

DEPLOY_URL silently empty when PRODUCTION_URL secret is unset

.github/workflows/release-promote.yml:764

Previously production_deployment_url was always populated from preflight output. secrets.PRODUCTION_URL resolves to empty string if not configured, producing annotations with no URL. Script handles it gracefully but no warning is emitted in the runner log.

low

Log drops content entirely — reduces triage visibility

scripts/posthog-annotate.mjs:97

The rationale (avoid echoing operator-supplied ANNOTATION_CONTENT) is sound. Logging content.length chars or a content=<default|custom> type-flag would preserve triage signal without echoing sensitive values.

security4

info

POSTHOG_HOST from vars (not secrets) — low injection surface

.github/workflows/release-promote.yml:757

Repo vars are changeable by anyone with repo write access. Risk is bounded: only repo admins can set vars, the job is continue-on-error. No change required.

info

permissions: contents:read tightening is correct — no OIDC needed

.github/workflows/release-promote.yml:754

Job no longer inherits top-level id-token:write. It only reads the repo and makes an outbound HTTPS call with a pre-loaded secret. Least-privilege correctly applied.

info

ANNOTATION_CONTENT override still stored verbatim in PostHog — by design

scripts/posthog-annotate.mjs:97

Log scrubbing is correct. But if an operator passes a sensitive string via ANNOTATION_CONTENT, it will be stored in PostHog's annotation store. This is the intended design — annotations are metadata for humans. Operators should be aware.

info

github.actor in annotation content — intentional, not a concern

.github/workflows/release-promote.yml:762

DEPLOY_ACTOR=${{ github.actor }} is a trusted, authenticated context value matching the documented annotation format. No action needed.

conventions1

info

skip_annotation could group with other skip-style inputs

.github/workflows/release-promote.yml:51

Minor ordering: skip_annotation is appended after sha rather than grouped with dry_run/skip_preflight. Not functional.

tests1

low

No tests cover skip_annotation guard or updated log format

.github/workflows/release-promote.yml:752

Consistent with existing posture (pull-metrics.mjs also untested). If a test suite is added, assert log does NOT include body.content and script exits 0 when POSTHOG_PERSONAL_API_KEY absent.

improvement4

medium

skip_annotation description missing 'when to use' context

.github/workflows/release-promote.yml:51

Current: 'Skip the PostHog deploy annotation marker'. In a timed incident-response dispatch the operator may not know when to reach for this. Suggested: 'Skip the PostHog annotation (e.g. re-deploying an already-annotated SHA or during rollbacks)'.

low

PRODUCTION_URL silent fallback invisible in runner log

.github/workflows/release-promote.yml:764

Comment says 'Omitted if the secret is unset' but no runner log warns. A step-level echo or script-side console.warn would surface the gap before someone reads a bare PostHog marker.

low

Log line omits project id — harder to grep in multi-project setups

scripts/posthog-annotate.mjs:97

Given three projects (prod 334265, dev 245284, staging 366142), logging project=${POSTHOG_PROJECT_ID} would immediately confirm which environment the annotation landed in — especially useful for staging backfills.

info

skip_annotation and !inputs.dry_run could share a line in the if: expression

.github/workflows/release-promote.yml:752

Moving && !inputs.skip_annotation adjacent to && !inputs.dry_run groups the two operator opt-out flags and separates them visually from the dependency-result check.

History · 3 commits

70e9fcbneeds attentionincremental0H · 2M · 5L2026-07-30 20:43current
5d8dfd6needs attentionfull0H · 1M · 5L2026-07-30 17:33
3a88710needs attentionfull0H · 2M · 7L2026-07-30 01:43