fix(analytics): improve PostHog attribution for automations and fix missing user profiles#3179
Open
fix(analytics): improve PostHog attribution for automations and fix missing user profiles#3179
Conversation
…arding Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Contributor
🧪 BenchmarkShould we run the Virtual MCP strategy benchmark for this PR? React with 👍 to run the benchmark.
Benchmark will run on the next push after you react. |
Contributor
Release OptionsSuggested: Patch ( React with an emoji to override the release type:
Current version:
|
…ostHog Use `automation_<id>` as PostHog distinctId for chat and tool events triggered by automations, so they show up under the automation persona instead of the owner user. Also propagates automation metadata (id, name, trigger_id, user_agent) to all captured events and fixes abort event naming to distinguish user-cancelled from server-aborted runs. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
… self-heal Add posthog.identify() for the user and posthog.groupIdentify() for the organization on every streamCore run. This refreshes the person/group profile each session so legacy users and orgs that predate the signup/creation hooks get backfilled automatically. Also drop the redundant `organization_name` / `organization_slug` event properties — PostHog resolves them via the organization group. Carry `virtual_mcp_name` on chat/tool events since virtual MCPs are not a PostHog group type. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Frontend already emits chat_message_stopped when the user clicks the stop button (apps/mesh/src/web/components/chat/input.tsx). The server side was firing the same event for abortReason="cancelled", producing two events per user cancel. Now we skip the server event for user cancels and only fire chat_message_aborted for genuine non-user aborts (tab close, registry race, force-fail, reaped, ghost). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
rafavalls
changed the title
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What is this contribution about?
Several PostHog attribution issues were found and fixed:
Missing user profiles —
posthog.identify()was never called server-side, so users appeared as raw IDs (e.g.VWPJPnX0DXgPDMIo5J6CBNTbLOPI6XVS) instead of their name/email in PostHog dashboards.Automation events attributed to users — Chats and tool calls triggered by automations were captured under the owner user's
distinctId, making it impossible to distinguish user-initiated from automation-initiated activity in analytics.Duplicate/wrong abort events — The server was always firing
chat_message_abortedon stream abort, even for user-initiated stops where the frontend already fireschat_message_stopped, causing duplicate events. Non-user aborts also lacked context on why they were aborted.Missing event context — Events were missing
trigger_id,user_agent,virtual_mcp_name,user_id, and automation metadata, making filtering and debugging harder.Changes
User identity
posthog.identify()in theuser.createhook so every new user gets their name + email linked to their PostHog profile at signupnameto the existingposthog.identify()call in/domain-setupgroupIdentify) on every stream run to self-heal legacy users/orgs created before PostHog integrationAutomation attribution
automation_<id>as the PostHogdistinctIdinstead of the owner's user IDposthog.identify()'d with its name, owner, and org so it appears as a named persona in PostHogis_automation,automation_id,automation_name, andtrigger_idpropertiesAbort event semantics
RunRegistrynow passes typed abort reasons ("cancelled","server-aborted","ghost") viaAbortController.abort(reason)stream-corereadsregistrySignal.reasonand skips the server event whenreason === "cancelled"(user stop button) to avoid duplicating the frontend'schat_message_stoppedeventchat_message_abortedwith anabort_reasonproperty for debuggingRicher event properties
trigger_id,is_automation,automation_id,automation_name,user_id,user_agent,virtual_mcp_namekey_limitandtotal_limitpatternsScreenshots/Demonstration
N/A — analytics-only change, no UI impact.
How to Test
automation_<id>persona, not the userchat_message_stoppedevent (from frontend), no server duplicatechat_message_abortedfires withabort_reasonReview Checklist