Add analytics page telemetry + QA data correctness runbook

CHANGELOG.md

@@ -20,6 +20,8 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
   - project analytics aggregation API contract (`GET /api/projects/{project_id}/analytics/aggregation`) with validated date-range controls, normalized overview/source/source-health response blocks, cursor-aware partial-source health metadata, and short-lived cache hydration
   - v1 detailed analytics experience on the dedicated Analytics page: date-range presets/custom picker with refresh, API-driven KPI cards, source health badges (connected/stale/missing), sessions trend bars, provider breakdown table, and top-page breakdown table
   - project analytics aggregation API now includes `daily_trend` and `page_breakdown` blocks for UI chart/table rendering on date-range changes
+  - analytics page v1 interaction/error telemetry via PostHog: `analytics_page_viewed`, `analytics_date_range_changed`, `analytics_refresh_clicked`, `analytics_source_error_shown`
+  - Analytics page QA data-correctness checklist + production troubleshooting runbook (`docs/analytics-page-data-correctness-checklist.md`)
   - Project Home analytics simplified to summary-only snippets + clear "View detailed analytics" CTA to avoid duplicate dense analytics blocks
 - Pages
   - added changelog page

core/analytics/event_taxonomy.json

@@ -310,6 +310,48 @@
     "outcome_attribution_recorded": {
       "stage": "lifecycle",
       "description": "A workflow contribution was linked to a measurable project outcome metric."
+    },
+    "analytics_page_viewed": {
+      "stage": "engagement",
+      "description": "User opened the project Analytics page shell.",
+      "required_properties": [
+        "project_id",
+        "date_range_start",
+        "date_range_end",
+        "range_days"
+      ]
+    },
+    "analytics_date_range_changed": {
+      "stage": "engagement",
+      "description": "Analytics page date range was changed before a new query load.",
+      "required_properties": [
+        "project_id",
+        "date_range_start",
+        "date_range_end",
+        "range_days",
+        "change_source"
+      ]
+    },
+    "analytics_refresh_clicked": {
+      "stage": "engagement",
+      "description": "User clicked refresh on the Analytics page.",
+      "required_properties": [
+        "project_id",
+        "date_range_start",
+        "date_range_end",
+        "range_days"
+      ]
+    },
+    "analytics_source_error_shown": {
+      "stage": "risk",
+      "description": "Analytics page surfaced a provider or API error to the user.",
+      "required_properties": [
+        "project_id",
+        "source",
+        "source_status",
+        "error_message",
+        "result_status"
+      ]
     }
   },
   "deprecated_aliases": {

core/tests/test_analytics_dashboard_telemetry.py

@@ -0,0 +1,28 @@
+from pathlib import Path
+
+from core.analytics import EVENT_TAXONOMY
+
+
+def test_analytics_dashboard_controller_tracks_v1_analytics_events():
+    controller_path = Path(__file__).resolve().parents[2] / "frontend" / "src" / "controllers" / "analytics_dashboard_controller.js"
+    source = controller_path.read_text(encoding="utf-8")
+
+    assert "window.posthog.capture" in source
+    assert '"analytics_page_viewed"' in source
+    assert '"analytics_date_range_changed"' in source
+    assert '"analytics_refresh_clicked"' in source
+    assert '"analytics_source_error_shown"' in source
+    assert "project_id" in source
+    assert "date_range_start" in source
+    assert "date_range_end" in source
+    assert "range_days" in source
+
+    assert source.index("this.captureDateRangeChangedIfNeeded({") < source.index(
+        'this.captureEvent("analytics_refresh_clicked", {'
+    )
+    assert "if (end.getTime() < start.getTime())" in source
+
+
+def test_analytics_source_error_event_requires_source_status_property():
+    required_properties = EVENT_TAXONOMY["events"]["analytics_source_error_shown"]["required_properties"]
+    assert "source_status" in required_properties

core/tests/test_posthog_event_coverage.py

@@ -36,6 +36,10 @@ def test_p1_event_coverage_matrix_events_exist_in_taxonomy():
         "backlink_discovery_failed",
         "opportunities_viewed",
         "contact_method_copied",
+        "analytics_page_viewed",
+        "analytics_date_range_changed",
+        "analytics_refresh_clicked",
+        "analytics_source_error_shown",
     }
 
     taxonomy_events = set(EVENT_TAXONOMY["events"].keys())

docs/analytics-page-data-correctness-checklist.md

@@ -0,0 +1,98 @@
+# Analytics page v1: QA data-correctness checklist + production troubleshooting
+
+## Scope
+
+This checklist validates the v1 Analytics page (`/project/<id>/analytics/`) end-to-end:
+
+- API payload correctness (`/api/projects/<id>/analytics/aggregation`)
+- UI rendering correctness (KPIs, source health, trend, breakdowns)
+- telemetry reliability for interaction/error signals:
+  - `analytics_page_viewed`
+  - `analytics_date_range_changed`
+  - `analytics_refresh_clicked`
+  - `analytics_source_error_shown`
+
+## QA checklist (pre-ship + regression)
+
+### 1) Access and baseline shell
+
+- [ ] Logged-out user is redirected to login.
+- [ ] Logged-in user cannot open another user's project analytics page (404).
+- [ ] Owner sees all expected sections: Overview KPIs, Sessions trend, Source state, Source breakdown, Top pages.
+
+### 2) Date range controls + query behavior
+
+- [ ] Default load uses Last 30d and a valid inclusive date window.
+- [ ] Last 7d and Last 90d presets update both date inputs correctly.
+- [ ] Custom start/end + Refresh returns data for the exact selected range.
+- [ ] Validation error appears when either start/end is missing.
+
+### 3) Data correctness cross-checks
+
+Run API and compare to rendered UI values for the same date range:
+
+```bash
+curl -sS "http://localhost:8000/api/projects/<PROJECT_ID>/analytics/aggregation?start_date=YYYY-MM-DD&end_date=YYYY-MM-DD" \
+  -H "Cookie: sessionid=<SESSION>"
+```
+
+- [ ] KPI totals match API `overview` (`clicks`, `impressions`, `sessions`, `users`, `conversions`).
+- [ ] CTR and conversion rate match API percent fields (2 decimal places).
+- [ ] Source breakdown table rows map 1:1 to API `source_breakdown` rows.
+- [ ] Trend bars presence/empty-state follows API `daily_trend` content.
+- [ ] Top pages table matches API `page_breakdown` ordering and values.
+
+### 4) Partial/missing integration behavior
+
+- [ ] Missing integration shows `Missing` badge and explanatory copy.
+- [ ] Connected but stale integration shows `Stale` badge.
+- [ ] Connected with healthy sync metadata shows `Connected` badge.
+- [ ] Page does not crash when only one provider has data.
+
+### 5) Telemetry correctness (PostHog)
+
+Open PostHog Live Events (project 105300) and verify properties are attached.
+
+- [ ] Page load emits `analytics_page_viewed` once with: `project_id`, `date_range_start`, `date_range_end`, `range_days`.
+- [ ] Preset switch emits `analytics_date_range_changed` with `change_source=preset_click`.
+- [ ] Custom date change + Refresh emits `analytics_date_range_changed` with `change_source=custom_date_refresh`.
+- [ ] Refresh click emits `analytics_refresh_clicked`.
+- [ ] API/source error surfaced in UI emits `analytics_source_error_shown` with `source`, `error_message`, `result_status=shown`.
+
+## Production troubleshooting notes
+
+### Symptom: Analytics page appears empty
+
+1. Check source health card:
+   - `Missing` => integration not connected.
+   - `Stale` + error detail => provider sync issue.
+2. Inspect latest sync cursor rows for the project in admin (`AnalyticsSyncCursor`).
+3. Confirm ingestion snapshots exist (`AnalyticsSourceSnapshot`) and are recent.
+4. Verify date range isn't excluding known data window.
+
+### Symptom: KPI mismatch vs expected provider dashboard
+
+1. Call aggregation API directly for same date range and compare to UI.
+2. Confirm canonical metric ownership assumptions:
+   - clicks/impressions from search scope
+   - sessions/users/conversions from traffic scope
+3. Check for stale provider cursor and last sync errors.
+4. Validate timezone/date-boundary assumptions (inclusive day counts).
+
+### Symptom: Repeated error toasts/messages
+
+1. Inspect browser network call to aggregation endpoint and response status.
+2. Use PostHog event `analytics_source_error_shown` to identify:
+   - `source` (`dashboard_api` or provider from source health)
+   - `source_status`
+   - `error_message`
+3. If provider errors repeat, run/inspect provider sync task logs and retry after fixing upstream credentials/rate limits.
+
+## Operational note
+
+When changing Analytics page event names or required properties, update in the same PR:
+
+1. `core/analytics/event_taxonomy.json`
+2. `frontend/src/controllers/analytics_dashboard_controller.js`
+3. event docs (`docs/posthog-event-coverage-matrix.md`, this checklist)
+4. test coverage for taxonomy and controller references

docs/event-taxonomy.md

@@ -31,7 +31,7 @@ For critical outcomes, capture is done server-side (not client-only).
 
 ## Canonical events (v2 highlights)
 
-### P1 funnel coverage
+### P1/P2 funnel + product coverage
 
 - `signup_completed`
 - `login_succeeded`
@@ -49,6 +49,10 @@ For critical outcomes, capture is done server-side (not client-only).
 - `link_exchange_toggled`
 - `plan_upgraded`
 - `plan_cancelled`
+- `analytics_page_viewed`
+- `analytics_date_range_changed`
+- `analytics_refresh_clicked`
+- `analytics_source_error_shown`
 
 See `event_taxonomy.json` for full list + required properties per event.
 

docs/posthog-event-coverage-matrix.md

@@ -1,6 +1,6 @@
 # PostHog Event Coverage Matrix (TuxSEO)
 
-Last updated: 2026-03-17
+Last updated: 2026-03-19
 
 ## Scope
 
@@ -26,6 +26,7 @@ Critical P1 funnel and product actions requested for reliable conversion/product
 | Onboarding complete | `onboarding_completed` | Server (`Profile.get_or_create_project`, first project only) | (taxonomy optional) |
 | First content generated | `first_content_generated` | Server (`BlogPostTitleSuggestion.generate_content`, first generated post) | (taxonomy optional) |
 | Subscription start | `subscription_created`, `subscription_started`, `paid_conversion` | Server webhook (`handle_created_subscription`) | (taxonomy optional) |
+| Analytics page usage telemetry | `analytics_page_viewed`, `analytics_date_range_changed`, `analytics_refresh_clicked`, `analytics_source_error_shown` | Frontend (`frontend/src/controllers/analytics_dashboard_controller.js`) | page viewed: `project_id`, `date_range_start`, `date_range_end`, `range_days`; date range changed: + `change_source`; refresh clicked: same as page viewed; source error shown: `project_id`, `source`, `source_status`, `error_message`, `result_status` |
 
 ## Audit summary (before this change)