perf(topology): bound Sigma work on large graphs#71
Merged
Conversation
Warm the dispatch surface once, keep a timed-out background warmup alive explicitly, and release async-write and lifecycle resources safely when startup cancellation escapes the inner run loop. Designed by Marsu — Refined by Codex. Co-Authored-By: Codex <codex@openai.com>
Remove workstation-only and process-history assumptions, keep version checks source-aware, use the standard library for plist parsing, and preserve correctness signals without unstable global monkeypatches or hard timing gates. Designed by Marsu — Refined by Codex. Co-Authored-By: Codex <codex@openai.com>
Recognize exact iai_mcp and iai-mcp path components while preserving Desktop precedence and rejecting substring lookalikes. Designed by Marsu — Refined by Codex. Co-Authored-By: Codex <codex@openai.com>
Replay an edge only when both endpoints are active graph nodes, so stale storage edges cannot recreate deleted records or inflate topology work in either full or incremental rebuilds. Bump the graph-cache version once so polluted assignments and rich-club data cannot survive the fix. Designed by Marsu — Refined by Codex. Co-Authored-By: Codex <codex@openai.com>
Keep exact APSL through the frozen 2,500-node fixtures, then estimate from 512 deterministic sources in batches of 32 and reuse the observed pass so production topology audits cannot materialize repeated N-by-N matrices. Designed by Marsu — Refined by Codex. Co-Authored-By: Codex <codex@openai.com>
9217f48 to
1af832b
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Draft / stacked PR
Do not merge before #67 through #70. This branch is stacked on those fixes, so the GitHub diff includes dependency commits until they land. It will then be rebased onto
mainto leave only the bounded Sigma topology work.Problem
The exact native average-shortest-path kernel materializes an
N x Nfloat64distance matrix. A Sigma observation computed it for the observed graph and
three random references, while
compute_topology_snapshotalso repeated workalready returned by
fast_sigma. The snapshot then ran exact betweennesscentrality solely to count the top ten percent of nodes, even though that count
is fixed by definition.
Solution
sparse unweighted Dijkstra in batches of 32;
fast_sigma's observed Sigma, clustering, and path-length result inthe topology snapshot;
ceil(0.10 * N) / N, without adiscarded exact centrality ranking.
Guarantees and scope
Small and frozen fixtures keep exact native parity. Large-graph APSL becomes a
deterministic estimator; retrieval, embeddings, and HNSW ranking are untouched.
The helper continues to operate on the largest connected component, matching
the existing Sigma contract.
Validation
single-pass snapshot, and centrality-cold tests;
seconds without an
N x Nallocation or an exact centrality call;that snapshot;
Designed by Marsu — Refined by Codex.
Public CI result
The cumulative macOS correctness gate is fully green: 5,130 passed, 113 skipped, 107 deselected, and 4 xfailed in 1:16:17. The universal2 wheel and secret scan also pass.