feat(graph): connected-components split + O(E) modularity + deterministic detection (mache-e1fa1f, mache-ff7e31)#492
Conversation
…E) modularity DetectCommunities (Phase-1 Louvain) can emit internally-disconnected communities (Traag et al. 2019) — a hazard for the sheaf cache-invalidation unit. Defensive fix (0 such communities on today's dense token projection; becomes load-bearing when the graph sparsifies under the code-fact IR). - splitDisconnectedCommunities: member-gated connected-components pass between local-moving and result collection; relabels any community whose induced subgraph has >1 component into one community per component. Split-only (never merges) → monotonically finer + deterministic (no Leiden aggregation/randomization that would thrash the sheaf cache key). - computeModularity rewritten O(N^2) -> O(E): Q = sum_c[inW_c/m2 - (commDegree_c/m2)^2], one adj scan; recomputed AFTER the split. - cmd/pack.go: dropped the 5000-ref guardrail (detection is near-linear now). - Tests: invariant (0 communities span >1 component), planted-disconnection barbell split, O(E)==O(N^2) modularity parity on small graphs. Local gate green: gofumpt clean, go vet clean, internal/graph + cmd Pack pass.
find_smells (advisory)Scoped to files changed in this PR. Rules below run on the standalone (no-LLO) cross-ref tables; duplicate_definitions — 2 finding(s) in changed files
Rules: the registry is |
Three pre-existing map-iteration-order dependencies made DetectCommunities non-deterministic run-to-run on identical input (flagged reviewing #492). Harmless while the partition is only compared as a set, but load-bearing once it is content-addressed (the CAS/merkle direction). - buildProjection: assign node indices iterating tokens in SORTED order, so node→index (which the whole partition is keyed on) is a pure function of refs. - phase-1: iterate candidate communities sorted with strict `>`, so ΔQ ties break deterministically (lowest community index) not by map order. - final sort: total order (size desc, then first-member lexicographic) so equal-sized communities get stable, input-order-independent IDs. Test: 40× DetectCommunities on an equal-sized-communities graph → byte-identical IDs + Membership every run. Existing split/modularity/community tests green.
jamestexas
changed the title
|
Added |
1 participant
What
DetectCommunities(Phase-1-only local-moving Louvain) can, per Traag et al. 2019, emit internally-disconnected communities — a hazard because the community is the sheaf cache-invalidation unit. Defensive fix + regression gate: 0 such communities on today's dense token projection (it masks the pathology), but it becomes load-bearing when the graph sparsifies under the code-fact IR.splitDisconnectedCommunities— member-gated connected-components pass between local-moving and result collection. Relabels any community whose induced subgraph (in-community edges only) has >1 component into one community per component. Split-only (never merges) → monotonically finer + deterministic. Not full Leiden (its aggregation coarsens + its randomization would thrash the deterministic sheaf cache key).computeModularityO(N²) → O(E) —Q = Σ_c[inW_c/m2 − (commDegree_c/m2)²], one adjacency scan; recomputed on the post-split partition.cmd/pack.go— dropped the 5000-ref guardrail (detection is near-linear now; the O(N²) modularity was its only reason).Correctness — reviewed by the theoretical-foundations-analyst (clean)
m2 = totalWeight = 2m(no factor-of-2 slip),inW_c= ordered double-sum pairing correctly with1/2m, and the algebra from the canonicalQ = (1/2m)Σ_ij[A_ij − k_ik_j/2m]δto the O(E) form. New ≡ old ≡ canonical. The parity test provably catches a factor-2 error (all-one partition → −2 vs 0).Tests
Invariant (0 communities span >1 connected component), planted-disconnection barbell split, O(E)==O(N²) modularity parity across weighted/asymmetric/edgeless fixtures. Local gate green: gofumpt / go vet / golangci-lint (pre-commit) /
go test -race ./internal/graph/all pass.Follow-up (not this PR)
Pre-existing pipeline nondeterminism (
buildProjectionindex order, Louvain tie-breaking, non-stablesort.Sliceby size) can swap equal-sized community IDs across independent runs — harmless today, but relevant if the partition is ever content-addressed (the CAS/merkle direction). Worth a separate bead.Bead mache-e1fa1f. Draft — no auto-merge.
🤖 Generated with Claude Code