Problem
PersonalizedPageRank in internal/graph/graph.go iterates over every node in the KB graph for up to 20 iterations (power iteration). Each iteration:
- Iterates all nodes to distribute rank to neighbors —
O(V + E)
- Iterates all nodes again for teleport + convergence check —
O(V)
- Previously allocated a fresh
newRank map per iteration (fixed — now reuses allocation)
For a KB with 50K nodes and 200K edges, one PPR call does 20 × O(V + E) = ~5M map reads on the search hot-path. This runs in a single goroutine with no ctx.Done() cancellation check.
Reachable from
- MCP
memex_search with graph_scorer=pagerank
- HTTP
GET /search with graph_scorer=pagerank
- CLI
memex search --graph-scorer pagerank
Current mitigation
- Per-KB graphs are typically <100K nodes where this runs in milliseconds
- PPR constants are now named (
pprAlpha=0.15, pprMaxIter=20, pprEpsilon=1e-6)
Proposed improvements
- Add
ctx.Done() check inside the PPR iteration loop to respect request timeouts/cancellation
- Local PPR: Instead of iterating the full graph, first compute the BFS neighborhood of seed nodes (e.g., 3-hop), then run PPR only on that subgraph. This is the standard "local PPR" approximation used in production graph systems and would bound the computation to the relevant neighborhood
- Pre-compute
allNodes and degree: These are rebuilt from scratch on every PPR call. They could be cached as part of Graph and updated incrementally in addEdge/removeEdge
Problem
PersonalizedPageRankininternal/graph/graph.goiterates over every node in the KB graph for up to 20 iterations (power iteration). Each iteration:O(V + E)O(V)newRankmap per iteration (fixed — now reuses allocation)For a KB with 50K nodes and 200K edges, one PPR call does
20 × O(V + E)= ~5M map reads on the search hot-path. This runs in a single goroutine with noctx.Done()cancellation check.Reachable from
memex_searchwithgraph_scorer=pagerankGET /searchwithgraph_scorer=pagerankmemex search --graph-scorer pagerankCurrent mitigation
pprAlpha=0.15,pprMaxIter=20,pprEpsilon=1e-6)Proposed improvements
ctx.Done()check inside the PPR iteration loop to respect request timeouts/cancellationallNodesanddegree: These are rebuilt from scratch on every PPR call. They could be cached as part ofGraphand updated incrementally inaddEdge/removeEdge