Skip to content

[CmdPal] Opt-in, privacy-safe search telemetry#49197

Open
michaeljolley wants to merge 6 commits into
dev/mjolley/relevance-harnessfrom
dev/mjolley/search-telemetry
Open

[CmdPal] Opt-in, privacy-safe search telemetry#49197
michaeljolley wants to merge 6 commits into
dev/mjolley/relevance-harnessfrom
dev/mjolley/search-telemetry

Conversation

@michaeljolley

@michaeljolley michaeljolley commented Jul 8, 2026

Copy link
Copy Markdown
Contributor

Warning

This PR is one in a series of PRs focused on rearchitecting the search/scoring logic of the MainListPage. An explanation of the entire search/scoring logic can be found in the first PR of the change (#49189).

This PR should not be merged until PR #49246 is merged into it.

Phase 6 of the stacked CmdPal search overhaul (FINAL)

Stacked PR: base branch is dev/mjolley/relevance-harness (phase 5), NOT main. This is the last phase of the chain.

Adds opt-in, privacy-safe telemetry to the main-page search path so we can measure relevance and perceived speed of the ranking overhaul, without ever capturing user content.

What is emitted (the ONLY fields captured)

CmdPal_SearchResults - fired once when a search query settles (debounced, not per keystroke):

  • QueryLength (int) - character count of the query, NOT the text
  • ResultCount (int) - number of deterministic first-paint results (commands + capped apps)
  • NoResults (bool) - true when ResultCount == 0
  • LatencyMs (ulong) - time to produce the deterministic first-paint results

CmdPal_SearchResultSelected - fired when the user invokes a result during an active search:

  • QueryLength (int) - character count of the query at selection, NOT the text
  • SelectedIndex (int) - zero-based visible rank of the invoked result
  • SelectedTier (string) - ranker tier name (e.g. ExactTitle, Prefix, Fuzzy), derived from the packed score via MainListRanker.TierOf. This is a fixed enum name, not user content.

Privacy affirmation

No raw query text, command titles, subtitles, file paths, app names, aliases, ids, or any user content / PII is captured anywhere. Only the non-identifying aggregates listed above are emitted. The telemetry messages carry only int / bool / ulong / a RankTier enum - there are no string fields that could carry query text (unit-tested reflectively). The tier name string is materialized only at the sink boundary from a fixed enum.

Consent gating & privacy tags (reused, not reinvented)

Emission flows through the existing PowerToys path: the ViewModels layer sends WeakReferenceMessenger messages, and the host-side TelemetryForwarder calls PowerToysTelemetry.Log.WriteEvent(...), which respects the existing PowerToys data-diagnostics consent gate. No new ungated path is introduced - if the user has not consented, nothing is emitted. Both event classes follow the exact existing CmdPal event pattern: [EventData], [DynamicallyAccessedMembers(PublicProperties)], extend EventBase, IEvent, and expose PartA_PrivTags => PartA_PrivTags.ProductAndServiceUsage, matching sibling events (CmdPalInvokeResult, CmdPalSessionDuration, etc.).

Hot-path discipline

All measurement happens at boundaries, never inside the per-item scoring loop:

  • Query-settle metrics are captured once after filtering completes and debounced (600 ms trailing edge) so keystrokes do not each emit.
  • Selection metrics are resolved only on invoke (a deliberate, infrequent action), reading snapshots of the last rendered results.
  • Rendered-order and packed-score snapshots are plain reference assignments (no extra allocation on the render path).

Files

  • Microsoft.CmdPal.UI/Events/CmdPalSearchResults.cs, CmdPalSearchResultSelected.cs - event definitions with privacy tags
  • Microsoft.CmdPal.UI.ViewModels/Messages/TelemetrySearchResultsMessage.cs, TelemetrySearchResultSelectedMessage.cs - messenger DTOs (no string fields)
  • Microsoft.CmdPal.UI/Helpers/TelemetryForwarder.cs - two new recipients that write events via the consent-gated sink
  • Microsoft.CmdPal.UI.ViewModels/Commands/MainListPage.cs - debounced query-settle firing, on-invoke selection firing, testable payload/tier/index helpers
  • Tests/.../SearchTelemetryTests.cs - payload builders, no-string-field privacy assertions, tier derivation, no-results flag, visible-index resolution

Build & test

  • Build: build.ps1 -Path src\modules\cmdpal -Platform x64 -Configuration Debug - exit code 0, errors log empty.
  • ViewModels unit tests (incl. new SearchTelemetryTests + phase-5 golden harness): 201/201 pass.
  • All CmdPal *.UnitTests green (WindowWalker "Zero tests ran" is the known pre-existing exception).
  • No XAML changed.

Draft for human privacy review before marking ready.

Add opt-in, privacy-safe main-page search telemetry that captures only

non-identifying aggregates via the existing PowerToys telemetry consent gate.

Query-settle event (CmdPal_SearchResults): query LENGTH, result count,

no-results flag, latency ms. Debounced so it fires once when a search

settles, not per keystroke.

Selection event (CmdPal_SearchResultSelected): query LENGTH, selected

visible rank, and ranker tier (via MainListRanker.TierOf). Fired on invoke.

Raw query text, titles, subtitles, paths, and app names are never logged.

Emission flows through PowerToysTelemetry.Log.WriteEvent (consent-gated)

and events are tagged PartA_PrivTags.ProductAndServiceUsage.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
@github-actions github-actions Bot added the Product-Command Palette Refers to the Command Palette utility label Jul 8, 2026
@github-actions

github-actions Bot commented Jul 11, 2026

Copy link
Copy Markdown

Thank you for contributing to PowerToys. We've detected that this PR might include a new or modified telemetry event. After this PR is merged, please follow these next steps:

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Product-Command Palette Refers to the Command Palette utility

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant