Skip to content

feat(math): Clojure hash-map iteration order for the in-conv greedy tie-break (legacy)#2650

Draft
jucor wants to merge 1 commit into
spr/edge/0add28f3from
spr/edge/d5f1f51d
Draft

feat(math): Clojure hash-map iteration order for the in-conv greedy tie-break (legacy)#2650
jucor wants to merge 1 commit into
spr/edge/0add28f3from
spr/edge/d5f1f51d

Conversation

@jucor

@jucor jucor commented Jul 22, 2026

Copy link
Copy Markdown
Collaborator

The greedy floor (conversation.clj:259-268) sorts the user-vote-counts
hash-map with a STABLE sort by count desc — so equal-count ties follow
Clojure's PersistentHashMap ITERATION ORDER, which is deterministic:
sort by successive 5-bit chunks (low first) of Murmur3 hashLong
(Clojure hasheq for Long keys). Validated against three recorded-blob
oracles (user-vote-counts raw key order: vw front-loaded6 n=18, vw
uniform8 n=30, biodiversity n=98 — all exact).

The PR-E port assumed this order was non-deterministic and substituted
matrix row order; on vw front-loaded6 step 0 that admits pid 14 where
Clojure admits pid 17 (five-way 1-vote tie at the floor boundary),
seeding an in-conv/base-cluster membership divergence that cascades
through every downstream key — the battery's last diverging entry.

Adds polismath/utils/clj_hash.py (hashLong + HAMT key order; int keys
only, with a documented row-order fallback otherwise) and applies it to
the legacy greedy candidate order. Array-map (≤8 entries, insertion
order) cannot affect the pick: ties only matter with ≥16 participants,
which guarantees hash-map. Improved mode unchanged.

commit-id:d5f1f51d


Stack:


⚠️ Part of a stack created by spr. Do not merge manually using the UI - doing so may have unexpected results.

This was referenced Jul 22, 2026
@jucor
jucor force-pushed the spr/edge/0add28f3 branch from f5a2c36 to aa5a214 Compare July 22, 2026 02:31
@jucor
jucor force-pushed the spr/edge/d5f1f51d branch from 16a2920 to 1f33fd2 Compare July 22, 2026 02:31
@jucor jucor changed the title feat(math): Clojure hash-map iteration order for the in-conv greedy tie-break (legacy) feat(math): Clojure hash-map iteration order for the in-conv greedy t… Jul 22, 2026
@jucor
jucor changed the base branch from spr/edge/0add28f3 to edge July 22, 2026 03:54
@jucor
jucor force-pushed the spr/edge/d5f1f51d branch from 1f33fd2 to 288f984 Compare July 22, 2026 03:54
@jucor jucor changed the title feat(math): Clojure hash-map iteration order for the in-conv greedy t… feat(math): Clojure hash-map iteration order for the in-conv greedy tie-break (legacy) Jul 22, 2026
@jucor
jucor changed the base branch from edge to spr/edge/0add28f3 July 22, 2026 03:54
…ie-break (legacy)

The greedy floor (conversation.clj:259-268) sorts the user-vote-counts
hash-map with a STABLE sort by count desc — so equal-count ties follow
Clojure's PersistentHashMap ITERATION ORDER, which is deterministic:
sort by successive 5-bit chunks (low first) of Murmur3 hashLong
(Clojure hasheq for Long keys). Validated against three recorded-blob
oracles (user-vote-counts raw key order: vw front-loaded6 n=18, vw
uniform8 n=30, biodiversity n=98 — all exact).

The PR-E port assumed this order was non-deterministic and substituted
matrix row order; on vw front-loaded6 step 0 that admits pid 14 where
Clojure admits pid 17 (five-way 1-vote tie at the floor boundary),
seeding an in-conv/base-cluster membership divergence that cascades
through every downstream key — the battery's last diverging entry.

Adds polismath/utils/clj_hash.py (hashLong + HAMT key order; int keys
only, with a documented row-order fallback otherwise) and applies it to
the legacy greedy candidate order. Array-map (≤8 entries, insertion
order) cannot affect the pick: ties only matter with ≥16 participants,
which guarantees hash-map. Improved mode unchanged.

commit-id:d5f1f51d
@jucor jucor changed the title feat(math): Clojure hash-map iteration order for the in-conv greedy tie-break (legacy) feat(math): Clojure hash-map iteration order for the in-conv greedy t… Jul 22, 2026
@jucor
jucor changed the base branch from spr/edge/0add28f3 to edge July 22, 2026 06:16
@jucor
jucor force-pushed the spr/edge/d5f1f51d branch from 288f984 to 5bf8560 Compare July 22, 2026 06:16
@jucor jucor changed the title feat(math): Clojure hash-map iteration order for the in-conv greedy t… feat(math): Clojure hash-map iteration order for the in-conv greedy tie-break (legacy) Jul 22, 2026
@jucor
jucor changed the base branch from edge to spr/edge/0add28f3 July 22, 2026 06:16
@github-actions

Copy link
Copy Markdown

Delphi Coverage Report

File Stmts Miss Cover
init.py 2 0 100%
benchmarks/bench_pca.py 128 107 16%
benchmarks/bench_repness.py 81 65 20%
benchmarks/bench_update_votes.py 38 28 26%
benchmarks/benchmark_utils.py 34 24 29%
components/init.py 1 0 100%
components/config.py 165 133 19%
conversation/init.py 2 0 100%
conversation/conversation.py 1300 261 80%
conversation/manager.py 131 42 68%
database/init.py 1 0 100%
database/dynamodb.py 395 189 52%
database/postgres.py 310 166 46%
pca_kmeans_rep/init.py 5 0 100%
pca_kmeans_rep/clusters.py 257 21 92%
pca_kmeans_rep/corr.py 98 17 83%
pca_kmeans_rep/group_k_smoother.py 28 0 100%
pca_kmeans_rep/legacy_kmeans.py 151 6 96%
pca_kmeans_rep/pca.py 130 18 86%
pca_kmeans_rep/repness.py 223 6 97%
poller/init.py 4 0 100%
poller/math_writer.py 41 1 98%
poller/service.py 235 62 74%
poller/worker_pool.py 75 0 100%
regression/init.py 4 0 100%
regression/clojure_comparer.py 188 20 89%
regression/comparer.py 887 649 27%
regression/datasets.py 135 27 80%
regression/recorder.py 36 27 25%
regression/utils.py 130 85 35%
replay/init.py 2 0 100%
replay/certify.py 452 121 73%
replay/crosslang.py 128 3 98%
replay/driver.py 51 2 96%
replay/prodclone.py 184 16 91%
replay/real_data.py 19 2 89%
replay/schedule.py 125 6 95%
replay/stepcompare.py 77 1 99%
replay/store.py 120 11 91%
replay/types.py 65 8 88%
run_math_pipeline.py 261 114 56%
umap_narrative/500_generate_embedding_umap_cluster.py 210 109 48%
umap_narrative/501_calculate_comment_extremity.py 112 53 53%
umap_narrative/502_calculate_priorities.py 135 135 0%
umap_narrative/700_datamapplot_for_layer.py 502 502 0%
umap_narrative/701_static_datamapplot_for_layer.py 310 310 0%
umap_narrative/702_consensus_divisive_datamapplot.py 432 432 0%
umap_narrative/801_narrative_report_batch.py 785 785 0%
umap_narrative/802_process_batch_results.py 268 268 0%
umap_narrative/803_check_batch_status.py 183 183 0%
umap_narrative/llm_factory_constructor/init.py 2 2 0%
umap_narrative/llm_factory_constructor/model_provider.py 192 192 0%
umap_narrative/polismath_commentgraph/init.py 1 0 100%
umap_narrative/polismath_commentgraph/cli.py 270 270 0%
umap_narrative/polismath_commentgraph/core/init.py 3 3 0%
umap_narrative/polismath_commentgraph/core/clustering.py 108 108 0%
umap_narrative/polismath_commentgraph/core/embedding.py 104 104 0%
umap_narrative/polismath_commentgraph/lambda_handler.py 219 219 0%
umap_narrative/polismath_commentgraph/schemas/init.py 2 0 100%
umap_narrative/polismath_commentgraph/schemas/dynamo_models.py 160 9 94%
umap_narrative/polismath_commentgraph/tests/conftest.py 17 17 0%
umap_narrative/polismath_commentgraph/tests/test_clustering.py 74 74 0%
umap_narrative/polismath_commentgraph/tests/test_embedding.py 55 55 0%
umap_narrative/polismath_commentgraph/tests/test_storage.py 87 87 0%
umap_narrative/polismath_commentgraph/utils/init.py 3 0 100%
umap_narrative/polismath_commentgraph/utils/converter.py 283 237 16%
umap_narrative/polismath_commentgraph/utils/group_data.py 354 336 5%
umap_narrative/polismath_commentgraph/utils/storage.py 584 518 11%
umap_narrative/reset_conversation.py 159 50 69%
umap_narrative/run_pipeline.py 453 312 31%
utils/clj_hash.py 44 0 100%
utils/engine_mode.py 9 0 100%
utils/env_flags.py 13 0 100%
utils/general.py 62 41 34%
utils/serialization.py 9 5 44%
Total 12903 7654 41%

@jucor jucor changed the title feat(math): Clojure hash-map iteration order for the in-conv greedy tie-break (legacy) feat(math): Clojure hash-map iteration order for the in-conv greedy t… Jul 22, 2026
@jucor
jucor changed the base branch from spr/edge/0add28f3 to edge July 22, 2026 06:50
@jucor jucor changed the title feat(math): Clojure hash-map iteration order for the in-conv greedy t… feat(math): Clojure hash-map iteration order for the in-conv greedy tie-break (legacy) Jul 22, 2026
@jucor
jucor changed the base branch from edge to spr/edge/0add28f3 July 22, 2026 06:50
@jucor jucor changed the title feat(math): Clojure hash-map iteration order for the in-conv greedy tie-break (legacy) feat(math): Clojure hash-map iteration order for the in-conv greedy t… Jul 22, 2026
@jucor
jucor changed the base branch from spr/edge/0add28f3 to edge July 22, 2026 08:11
@jucor jucor changed the title feat(math): Clojure hash-map iteration order for the in-conv greedy t… feat(math): Clojure hash-map iteration order for the in-conv greedy tie-break (legacy) Jul 22, 2026
@jucor
jucor changed the base branch from edge to spr/edge/0add28f3 July 22, 2026 08:11
@jucor jucor changed the title feat(math): Clojure hash-map iteration order for the in-conv greedy tie-break (legacy) feat(math): Clojure hash-map iteration order for the in-conv greedy t… Jul 22, 2026
@jucor
jucor changed the base branch from spr/edge/0add28f3 to edge July 22, 2026 09:58
@jucor jucor changed the title feat(math): Clojure hash-map iteration order for the in-conv greedy t… feat(math): Clojure hash-map iteration order for the in-conv greedy tie-break (legacy) Jul 22, 2026
@jucor
jucor changed the base branch from edge to spr/edge/0add28f3 July 22, 2026 09:58
@jucor jucor changed the title feat(math): Clojure hash-map iteration order for the in-conv greedy tie-break (legacy) feat(math): Clojure hash-map iteration order for the in-conv greedy t… Jul 22, 2026
@jucor
jucor changed the base branch from spr/edge/0add28f3 to edge July 22, 2026 11:20
@jucor jucor changed the title feat(math): Clojure hash-map iteration order for the in-conv greedy t… feat(math): Clojure hash-map iteration order for the in-conv greedy tie-break (legacy) Jul 22, 2026
@jucor
jucor changed the base branch from edge to spr/edge/0add28f3 July 22, 2026 11:21
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant