query: single generic term exact-matching a short leaf label hijacks seed selection in multi-term queries

[fkhawajagh]

Summary

In a multi-term query, a single generic term that exactly equals a short, unrelated leaf node's label receives the _EXACT_MATCH_BONUS (1000×IDF) and outscores every genuinely relevant node by ~10×. _pick_seeds (gap_ratio 0.2) then drops all other candidates, so the traversal starts from one irrelevant leaf and returns a tiny dead-end subgraph.

Observed on graphifyy 0.8.49.

Repro (real graph, Phoenix/Elixir codebase, ~5,400 nodes)

Query:

graphify query "dashboard home page LiveView content KPI cards alerts today grid"

The graph contains a rich cluster for the dashboard feature (IrisWeb.DashboardLive, kpi_card(), alert_row(), plus design docs whose labels match 3 of the query terms). Expected seeds somewhere in that cluster.

Actual _score_nodes output (top of the ranking):

6037.89  home()                          <- "home" == bare label -> exact tier
 680.74  grid_view()                     <- "grid" prefix tier
 621.57  Home / Today Dashboard (Iris)   <- 3 substring hits
 616.01  Home Today Mock

_pick_seeds keeps only home() because every other node is below 20% of the top score. home() is a 2-neighbor leaf (a controller action), so BFS depth 2 returns 3 nodes and the answer collapses:

Traversal: BFS depth=2 | Start: ['home()'] | 3 nodes found

The failure generalizes: any generic English word in a concept query that happens to equal (or prefix) a short function name hijacks the seed. Same graph, "dashboard liveview kpi alerts today grid" seeds at grid_view() / gridlines() for the same reason, while "dashboard KPI alerts" (no colliding generic words) seeds correctly.

Why this feels like a scorer bug rather than user error

The exact-match tier is clearly right for identifier queries (graphify query "FooBarService"). But in a 10-term query, an exact hit on one term says much less than substring hits on three terms. The current scoring has no notion of term coverage, so one lucky collision dominates, and the aggressive seed gap then removes all diversity.

Possible directions

• Weight the exact/prefix bonuses by the fraction of query terms the node matches, so a 1-of-10 exact hit can't bury a 3-of-10 multi-term match.
• Or relax _pick_seeds when the top seed's score comes from a single term of a many-term query — keep at least one seed from the best multi-term-coverage node.
• Or treat very short labels (home, grid) as low-information exact matches, e.g. dampen the exact bonus by label-token IDF.

Happy to provide more score traces if useful.

[fkhawajagh]

A second repro that shows this isn't just poorly-formed queries. On the same graph:

graphify query "ClientLive.Index clients list columns"

The query contains the exact identifier (ClientLive.Index, a real node), yet seeds at three unrelated list() nodes:

4892  list()              <- token "list" == bare label -> exact tier
4892  list()
4892  list()
 596  ClientLive.Index    <- the intended node, dropped by the 20% seed gap

Dropping the single word "list" fixes it ("ClientLive.Index clients columns" seeds at ClientLive.Index/ClientLive.Form/ClientLive.Show).

The problematic words are hard to avoid in practice: in an Elixir/Phoenix codebase, list(), get(), new(), show(), index() are ubiquitous short function labels, and "list"/"new"/"show"/"index" are exactly the words a natural question about a list/show/index page contains. A term-coverage weight (the exact bonus scaled by how many of the query's terms the node matches) would resolve both repros: list() matches 1 of 5 terms, ClientLive.Index matches 2 plus the prefix tier.

[fkhawajagh]

For what it's worth, we've been running a local patch that resolves both repros on our graph (5,421 nodes) without regressing identifier or well-formed queries:

1. Scale the per-term exact/prefix tier contributions by squared term coverage — tiered * (matched_terms / total_terms) ** 2. Squaring matters because the exact tier is 10x the prefix tier: at linear coverage, a 1-of-10-terms exact hit (score 6038 -> 604) still edges out a 3-of-10-terms prefix+substring match (621 -> ~199 after its own scaling). Squared, the lone-term hit drops to ~241 and multi-term matches win. Single-term and full-coverage queries are unchanged (coverage == 1), so query "FooBarService" keeps its full bonus.
2. In _pick_seeds, dedupe seeds by label — otherwise N same-named tiny functions (three list() nodes in our graph, all tied at 4892) fill every seed slot before the first differently-named relevant node.

Happy to send this as a PR if the approach looks right to you.

[safishamsi]

Confirmed the mechanism. In _score_nodes, each query term that exactly equals a label adds _EXACT_MATCH_BONUS (1000) × IDF(term) (serve.py:335), and there's no term-coverage signal — so in a 10-term query, one exact hit on a generic word (home == home()) scores ~6000 and outranks a node that substring-matches three genuinely relevant terms. _pick_seeds (gap_ratio 0.2) then drops every candidate below 20% of that top, so BFS starts from the one irrelevant leaf. Exactly as you diagnosed: one lucky collision dominates because nothing rewards matching more of the query.

Two things worth noting:

• The recently-merged fix(serve): drop question/filler stopwords from query terms #1597 (drop question/filler stopwords) removes the filler-word variant of this (how/the/work), but not your core case — home/grid/today are content-ish words that happen to equal short leaf labels, and fix(serve): drop question/filler stopwords from query terms #1597 won't drop those.
• The fix has to be careful not to regress single-term identifier queries (query "FooBarService"), where the exact-match tier is exactly right.

Proposed direction (term coverage), which I'd want to land with a before/after eval rather than blind:

1. Track per node the count of distinct query terms matched (matched_terms).
2. Scale the score by coverage in multi-term queries only — e.g. multiply by (matched_terms / len(terms)), or add a coverage bonus proportional to matched_terms. Single-term queries are unaffected (coverage is 1/1), so identifier lookups keep exact-match dominance.
3. Optionally soften _pick_seeds when the top score comes from a single-term exact hit but runners-up cover more terms (ties into the Seed selection: tied _EXACT_MATCH_BONUS scores aren't degree/community-diversified (residual to #897) #1303 seed-diversity thread).

This is a core-ranking change, so I want to implement it against a set of representative queries (identifier, natural-language, and your multi-term collision case) and diff the seeds before/after, not tune it blind. Scoping it as a focused follow-up. Does the coverage-scaling direction match what you had in mind, or were you leaning toward capping the exact tier / diversifying seeds instead?

[safishamsi]

Partially addressed on v8 by #1596 (d56ee83): _pick_seeds now guarantees at least one seed per distinct query term (ties broken by graph degree), so a single term's incidental exact-match collision can no longer starve out the other terms' seeds — the seed-hijack you described. Verified on a #1445-shaped graph: seeds went from ['home'] to the full relevant cluster.

That's the seed-selection half. The scoring half from my earlier proposal — term-coverage weighting so one exact hit on a generic word doesn't outscore substring matches on three relevant terms — is still open and worth doing on top; #1596 mitigates the downstream symptom, coverage-scoring would fix the root ranking. Leaving this open for that.

[fkhawajagh]

First off, thank you. Graphify has become genuinely critical for us; it's the knowledge graph behind an internal "brain" we query constantly over a ~5,400-node Phoenix/Elixir codebase. And thank you for the quick response here, much appreciated.

On the substance:

#1596's per-term seed guarantee is a cleaner fix for the seed half than the label-dedupe we'd been carrying locally, and it also covers the lone-collision case (home()) that label-dedupe alone doesn't. We'll drop our seed patch and rely on #1596. (Quick check: which release carries it? We're on graphifyy 0.8.49, whose _pick_seeds doesn't have the per-term guarantee yet, want to make sure we upgrade to the right one before removing our local patch.)

On your question from the earlier comment: yes, coverage-scaling is the direction, not capping the exact tier or seed-diversification alone. Concretely, the scoring patch we've been running in production (5,421-node graph) is:

• Scale only the exact/prefix tier contributions by (matched_terms / total_terms) ** 2; leave substring, source-file, and the whole-query full-label tier unscaled.
• Squared, not linear, on purpose: the exact tier is 10x the prefix tier, so at linear coverage a 1-of-N exact hit still edges out a 3-of-N prefix/substring match. Squaring pushes the lone hit below the multi-term matches while leaving single-term and full-coverage queries untouched (coverage == 1), so query "FooBarService" keeps full exact dominance.

It resolves both repros here without regressing our identifier or well-formed natural-language queries. With #1596 now handling the seed side, the two changes are complementary; #1596 stops the starvation, coverage-scoring fixes the underlying ranking so the irrelevant leaf stops topping _score_nodes in the first place.

We'll put this together as a PR against v8, with a before/after seed diff across the three query classes you mentioned (identifier / natural-language / multi-term collision), run on top of #1596 so the eval reflects the combined behavior.

[safishamsi]

100%
Feel free to send a pr

[fkhawajagh]

PR is up: #1724. It implements the term-coverage scaling discussed above, on top of #1596, with before/after rankings and seed diffs from the same real graph both repros came from. Both repros resolve; identifier and natural-language queries produce identical seeds before and after. Details and trade-offs are in the PR body.

[safishamsi]

Fixed on v8 in 2f9deae (credited to @fkhawajagh from #1724). _score_nodes now scales the per-term exact/prefix tiers by squared term coverage, so a lone generic term that exact-matches a short leaf label no longer hijacks the top of the ranking / seed selection in multi-term queries. Single-term and full-coverage queries are unchanged. Thanks @fkhawajagh.