Skip to content

fix: drop connection/render-triggered events, gate MCP tool calls#509

Merged
Ehesp merged 2 commits into
mainfrom
analytics/count-real-usage
Jul 3, 2026
Merged

fix: drop connection/render-triggered events, gate MCP tool calls#509
Ehesp merged 2 commits into
mainfrom
analytics/count-real-usage

Conversation

@claude

@claude claude Bot commented Jul 2, 2026

Copy link
Copy Markdown
Contributor

Requested by Elliot Hesp · Slack thread

Summary

Several PostHog events were being emitted by connections and page renders rather than by real usage, which distorted the analytics (idle MCP connections and every doc render were inflating the top-line numbers). This trims those events so each metric counts genuine activity.

Before / After

  • mcp:server_view — Before: fired on every GET to the MCP endpoint, so a connection just sitting in someone's MCP config counted as a "view" and inflated it to the top event. After: removed entirely; real MCP usage is measured by mcp:tool_call.
  • mcp:tool_call — Before: fired on every POST to the MCP endpoint, including JSON-RPC protocol chatter (initialize, tools/list, notifications/*). After: captured from inside the individual tool handlers, so it only fires when a tool actually executes (read_doc_page / list_doc_files) and carries the tool name directly. Protocol chatter no longer counts.
  • agent:session_create — Before: fired on every server-side render of an agent-enabled repo page, whether or not anyone used the agent. After: removed; the session is still minted and the cookie is still set, we just stop emitting the event. First-use is now signalled on the message event instead.
  • agent:message_send — Before: captured message_count only. After: also captures is_first_message (messages.length === 1) and session_id, so the first message of a session is identifiable without a separate render-triggered event.
  • docs:bundle_view — Before: fired on every successful bundle fetch, double-counting views alongside docs:page_view. After: removed. docs:page_view remains the canonical view metric and docs:bundle_fail is untouched.

Scope

  • app/ (hosted site, MCP, Ask AI)
  • packages/cli/
  • packages/mdx-bundler/
  • docs/ (product documentation)
  • Repo / CI / other

Type of change

  • Bug fix
  • New feature
  • Documentation
  • Refactor / chore

How

  • app/src/app/api/[owner]/[repo]/mcp/route.ts — dropped the mcp:server_view capture from GET. The POST handler no longer peeks at the request body; it just resolves context and delegates to handleMcpPost, so there is no req.clone().json() double read of the request stream.
  • app/src/server/mcp/server.ts — moved the mcp:tool_call capture into the per-request tool handlers. createMcpServer now builds a small captureToolCall(tool) helper (over the request's route context and the shared PostHog client) and each tool handler calls it at the top — read_doc_page and list_doc_files. This fires the event only when a tool actually executes, attaches the tool name directly, and avoids parsing JSON-RPC or cloning the request; protocol chatter (initialize, tools/list, notifications/*) is naturally excluded because those never reach a tool handler.
  • app/src/pages/[[...path]].tsx — removed the agent:session_create capture; createAgentSession(...) and the Set-Cookie logic are unchanged.
  • app/src/app/api/agent/route.ts — added is_first_message and session_id: session.sid to the existing agent:message_send capture; distinctId is unchanged.
  • app/src/app/api/bundle/route.ts — removed the docs:bundle_view capture; docs:bundle_fail and the rest of the handler are unchanged.

Test plan

  • next build (from app/) passes — compiles, TypeScript check clean, static generation succeeds.
  • Biome check clean on the changed files.

Notes for reviewers

No behavioural change beyond analytics: sessions are still minted, cookies still set, and MCP requests are still handled identically. The MCP POST handler no longer clones or peeks at the request body — mcp:tool_call is now captured inside the tool handlers themselves, so the event fires on real tool execution rather than on protocol traffic.

…gate MCP tool calls

- Remove mcp:server_view (fired on every GET to the MCP endpoint; idle
  connections in configs inflated it).
- Gate mcp:tool_call to actual JSON-RPC tools/call requests, ignoring
  protocol chatter (initialize, tools/list, notifications/*). Peek at a
  cloned request body so the handler still reads an intact stream; handle
  batched arrays and attach the tool name.
- Remove agent:session_create (fired on every render for agent-enabled
  repos); keep session minting and cookie logic intact. Add
  is_first_message and session_id to agent:message_send instead.
- Remove docs:bundle_view; docs:page_view remains the canonical view
  metric. docs:bundle_fail is untouched.
@CLAassistant

Copy link
Copy Markdown

CLA assistant check
Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.
You have signed the CLA already but the status is still pending? Let us recheck it.

@railway-app railway-app Bot temporarily deployed to docs.page / docs.page-pr-509 July 2, 2026 21:28 Destroyed
@railway-app

railway-app Bot commented Jul 2, 2026

Copy link
Copy Markdown

🚅 Deployed to the docs.page-pr-509 environment in docs.page

Service Status Web Updated (UTC)
docs.page ✅ Success (View Logs) Web Jul 3, 2026 at 9:21 am

Move the mcp:tool_call analytics capture out of the POST route's
JSON-RPC body peek and into the per-request MCP tool handlers. This
removes the double body read (req.clone().json()) and ensures the
event only fires when a tool actually executes.
@railway-app railway-app Bot temporarily deployed to docs.page / docs.page-pr-509 July 3, 2026 09:19 Destroyed
@Ehesp Ehesp marked this pull request as ready for review July 3, 2026 14:30
@Ehesp Ehesp changed the title Count real analytics usage: drop connection/render-triggered events, gate MCP tool calls fix: drop connection/render-triggered events, gate MCP tool calls Jul 3, 2026
@Ehesp Ehesp merged commit 6989a54 into main Jul 3, 2026
3 checks passed
@Ehesp Ehesp deleted the analytics/count-real-usage branch July 3, 2026 14:31
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants