[CmdPal] Opt-in, privacy-safe search telemetry#49197
Open
michaeljolley wants to merge 6 commits into
Open
Conversation
Add opt-in, privacy-safe main-page search telemetry that captures only non-identifying aggregates via the existing PowerToys telemetry consent gate. Query-settle event (CmdPal_SearchResults): query LENGTH, result count, no-results flag, latency ms. Debounced so it fires once when a search settles, not per keystroke. Selection event (CmdPal_SearchResultSelected): query LENGTH, selected visible rank, and ranker tier (via MainListRanker.TierOf). Fired on invoke. Raw query text, titles, subtitles, paths, and app names are never logged. Emission flows through PowerToysTelemetry.Log.WriteEvent (consent-gated) and events are tagged PartA_PrivTags.ProductAndServiceUsage. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
…nto dev/mjolley/search-telemetry
…nto dev/mjolley/search-telemetry
…nto dev/mjolley/search-telemetry
…nto dev/mjolley/search-telemetry
|
Thank you for contributing to PowerToys. We've detected that this PR might include a new or modified telemetry event. After this PR is merged, please follow these next steps:
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Warning
This PR is one in a series of PRs focused on rearchitecting the search/scoring logic of the
MainListPage. An explanation of the entire search/scoring logic can be found in the first PR of the change (#49189).This PR should not be merged until PR #49246 is merged into it.
Phase 6 of the stacked CmdPal search overhaul (FINAL)
Stacked PR: base branch is
dev/mjolley/relevance-harness(phase 5), NOT main. This is the last phase of the chain.Adds opt-in, privacy-safe telemetry to the main-page search path so we can measure relevance and perceived speed of the ranking overhaul, without ever capturing user content.
What is emitted (the ONLY fields captured)
CmdPal_SearchResults- fired once when a search query settles (debounced, not per keystroke):QueryLength(int) - character count of the query, NOT the textResultCount(int) - number of deterministic first-paint results (commands + capped apps)NoResults(bool) - true whenResultCount == 0LatencyMs(ulong) - time to produce the deterministic first-paint resultsCmdPal_SearchResultSelected- fired when the user invokes a result during an active search:QueryLength(int) - character count of the query at selection, NOT the textSelectedIndex(int) - zero-based visible rank of the invoked resultSelectedTier(string) - ranker tier name (e.g.ExactTitle,Prefix,Fuzzy), derived from the packed score viaMainListRanker.TierOf. This is a fixed enum name, not user content.Privacy affirmation
No raw query text, command titles, subtitles, file paths, app names, aliases, ids, or any user content / PII is captured anywhere. Only the non-identifying aggregates listed above are emitted. The telemetry messages carry only
int/bool/ulong/ aRankTierenum - there are no string fields that could carry query text (unit-tested reflectively). The tier name string is materialized only at the sink boundary from a fixed enum.Consent gating & privacy tags (reused, not reinvented)
Emission flows through the existing PowerToys path: the ViewModels layer sends
WeakReferenceMessengermessages, and the host-sideTelemetryForwardercallsPowerToysTelemetry.Log.WriteEvent(...), which respects the existing PowerToys data-diagnostics consent gate. No new ungated path is introduced - if the user has not consented, nothing is emitted. Both event classes follow the exact existing CmdPal event pattern:[EventData],[DynamicallyAccessedMembers(PublicProperties)], extendEventBase, IEvent, and exposePartA_PrivTags => PartA_PrivTags.ProductAndServiceUsage, matching sibling events (CmdPalInvokeResult, CmdPalSessionDuration, etc.).Hot-path discipline
All measurement happens at boundaries, never inside the per-item scoring loop:
Files
Microsoft.CmdPal.UI/Events/CmdPalSearchResults.cs,CmdPalSearchResultSelected.cs- event definitions with privacy tagsMicrosoft.CmdPal.UI.ViewModels/Messages/TelemetrySearchResultsMessage.cs,TelemetrySearchResultSelectedMessage.cs- messenger DTOs (no string fields)Microsoft.CmdPal.UI/Helpers/TelemetryForwarder.cs- two new recipients that write events via the consent-gated sinkMicrosoft.CmdPal.UI.ViewModels/Commands/MainListPage.cs- debounced query-settle firing, on-invoke selection firing, testable payload/tier/index helpersTests/.../SearchTelemetryTests.cs- payload builders, no-string-field privacy assertions, tier derivation, no-results flag, visible-index resolutionBuild & test
build.ps1 -Path src\modules\cmdpal -Platform x64 -Configuration Debug- exit code 0, errors log empty.*.UnitTestsgreen (WindowWalker "Zero tests ran" is the known pre-existing exception).Draft for human privacy review before marking ready.