perf: viewport-scoped search highlight pre-computation#25
Open
kdkavanagh-agent wants to merge 2 commits intomasterfrom
Open
perf: viewport-scoped search highlight pre-computation#25kdkavanagh-agent wants to merge 2 commits intomasterfrom
kdkavanagh-agent wants to merge 2 commits intomasterfrom
Conversation
Parametrized tests measuring scroll performance with active search highlights vs baseline across different search cardinalities (single match, sparse, adjacent block, dense) and screen sizes (120x30, 800x200, 3000x1000). Tests fail if search scrolling is >15% slower. Configured pytest-xdist with -n 2 for parallel execution. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Replace the O(n) linear scan on active_search_queue list and the per-render is_in(full_200k_list) Polars expression with a two-part optimization: 1. Convert search queue to frozenset for O(1) membership tests 2. Pre-compute a __search_bg column in the viewport dataframe once per viewport rebuild, using only the visible indices intersected with the search set. This avoids rebuilding any Polars is_in expression during partial scroll updates (up/down keys). 3. Add _resolve_row_bgcolor hook in CustomTable so subclasses can apply row-level bgcolor adjustments in Python without modifying the Polars expression chain. The search highlight is now read from the pre-computed column during the segment iteration loop, eliminating all per-row overhead during the hot scroll path. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Optimizes search highlight performance with two complementary techniques:
is_in()— Instead of passing the entire search queue (up to 200k items) tois_in()on every render, pre-computes a__search_bgcolumn by intersecting only the visible row indices with the search set. This reduces theis_in()input to at most ~1000 items (viewport height) instead of 200k.frozensetfor O(1) membership — Convertsactive_search_queueto afrozensetfor O(1) lookups in_get_sel_col_bg_color()instead of O(n) linear scan on a Python list._resolve_row_bgcolorhook — Moves search highlight resolution out of the Polars expression tree and into a pre-computed column, eliminating per-row expression overhead during the hot scroll path.Performance Results
Slowdown = (search_scroll_time - baseline_scroll_time) / baseline_scroll_time. Threshold: 15%.
All 12 tests pass (threshold: 15%). Previously dense_most_rows at 800x200 was intermittently failing at 15-16%.
Test plan
Generated with Claude Code