perf(gfql): seeded typed-hop fast paths use resident indexes (#1658 × #1755)#1768
Merged
Merged
Conversation
lmeyerov
force-pushed
the
perf/gfql-seeded-fastpath-resident-index
branch
from
July 22, 2026 01:28
4cbaf3b to
c7419a3
Compare
…1755) With gfql_index_all() resident, the seeded helpers replace their three full-frame scans (O(N) seed row, O(E) frontier isin, O(N) endpoint gather) with node-id searchsorted + CSR adjacency gathers. Decline-gated to the identical scan body: absent/stale index (get_valid fingerprint + identity), non-numeric id families, seed not id-filtered. Local 50k/200k covered shape: pandas 3.11->1.85ms, polars 1.74->1.10ms; win scales with graph size. Pinned: parity+engagement (pandas/polars), string-id decline, stale-index decline. Stacked on perf/gfql-index-preserve-polars-frames (#1767) — polars index residency requires it. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Y6dQEcjdazEnzuvuwf73ZL
…f device path, polars-gpu tags, serve-asserted tests Review findings (plans/review-pr-1768/review.md): B1 int64<->uint64 promotes to float64 and collapses ids >= 2^53 into false matches (repro'd: seed 2^62+1 matched node 2^62) — integral pairs whose promote is float now DECLINE to the exact-compare scan path. M2 spies count SERVES (non-None), not calls, so a gate regression that declines everything fails the suite. M1 native-chain hop indexed branch + reverse/EDGE_IN_ADJ now tested (serve-asserted, parity). M3 cudf ids via .dropna().values (device array; to_numpy raised on nulls and round-tripped host). M4 polars frames also accept polars-gpu-tagged indexes (same numpy sidecars + polars gather). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Y6dQEcjdazEnzuvuwf73ZL
lmeyerov
force-pushed
the
perf/gfql-seeded-fastpath-resident-index
branch
from
July 22, 2026 02:14
c7419a3 to
a929331
Compare
lmeyerov
changed the base branch from
perf/gfql-index-preserve-polars-frames
to
master
July 22, 2026 02:14
… gate Review feedback: _ids_to_key_array & friends were Any-typed — now NodeIdIndex/AdjacencyIndex/ArrayLike/ArrayNamespace/SeriesT protocol types (isnan added to the ArrayNamespace protocol), localized ignores only at the cudf module-check narrows. New GFQL_PERF_TESTS=1 opt-in wall-clock gate (indexed must beat scan >=1.5x at 500k edges) for the benchmarking-rig protocol; CI keeps the deterministic engagement asserts. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Y6dQEcjdazEnzuvuwf73ZL
This was referenced Jul 22, 2026
pull Bot
pushed a commit
to admariner/pygraphistry
that referenced
this pull request
Jul 22, 2026
New docs/source/gfql/indexing.rst: lifecycle guide to GFQL resident indexes (gfql_index_all / gfql_index_edges / create_index / show_indexes / drop_index) — node-id + CSR in/out adjacency sidecars, what engages them on 0.58.0 (seeded typed-hop fast paths incl. property RETURNs and property-seeded lookups per graphistry#1768/graphistry#1770, direct g.hop(); general polars chain traversal honestly noted as not yet covered), the identity + fingerprint staleness contract with safe declines, engine notes (polars needs gfql_index_all(engine='polars') until graphistry#1767), 0.58.0-tag measured numbers, and a runnable end-to-end example (executed). Wired into the GFQL toctree + recommended paths next to index_adjacency; CHANGELOG entry under Development > Documentation. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Y6dQEcjdazEnzuvuwf73ZL
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
With
gfql_index_all()resident, the #1755 seeded fast-path helpers (_seeded_typed_hop_pandas_cudf,_seeded_typed_return_dst_pandas_cudf,_seeded_typed_return_dst_polars) replace their three full-frame scans — O(N) seed-row lookup, O(E) frontierisin, O(N) endpoint gather — with positional index lookups (lookup_node_rowssearchsorted +lookup_edge_rowsCSR gather viatake_rows). A seeded Cypher lookup stops paying graph-size costs; this is the lever aimed at the C2 covered-shape gap vs Kuzu (2.60 vs 1.06ms) and the IS5 seed-scan residue.Decline contract (falls back to the byte-for-byte scan body)
get_valid)node not in n0f)Results identical either way (row order may differ — node-id index implies unique ids; value-identical contract). Null endpoints never link (float-NaN dropped pre-lookup; numeric-only gate excludes object nulls).
Measured (DGX Spark GB10, container polars 1.35.2, warm median N=30)
Covered shape (seeded whole-row
MATCH (m {id})-[:T]->(p) RETURN p):The 50k covered-shape pair vs Kuzu's same-box 1.06ms: GFQL moves from 2.5× behind to ahead. The pandas SF1 residue (15.7ms) is outside the three replaced scans — open question tracked in the campaign plan; polars is the shipping engine for this surface.
Tests
TestResidentIndexSeededFastPath: indexed-vs-plain parity with engagement spies (pandas + polars), string-id decline, stale-index (rebound edges) decline — plus the full seeded fast-path + index suites green. Test file already inPOLARS_TEST_FILES.Independence from #1767 (retargeted to master)
Rebased onto master (now incl. #1764/#1766) and independently landable. The code never depended on #1767; the coupling was behavioral: without #1767,
gfql_index_all()under AUTO swaps polars frames to pandas, so polars graphs get index service only via an explicitgfql_index_all(engine='polars')(works on master today; test adapted accordingly). pandas/cuDF flows are fully independent. When #1767 lands, bare AUTO indexing on polars frames starts engaging this path too — no changes needed here.🤖 Generated with Claude Code
https://claude.ai/code/session_01Y6dQEcjdazEnzuvuwf73ZL