Add DHT benchmark simulation and finger-table override#656
Conversation
|
Docker cluster benchmark update from the 16-node runs:
|
|
Paper-facing review, scoped to whether this PR can support PR #655's evaluation claims. This PR is useful for the paper, but it supports two different objects and the data should stay separated: The dummy simulator can back the formal For the paper, I need the PR to attach or paste reproducible JSONL result sets, not only summary bullets:
With those artifacts, PR #655 can safely say: Chord baseline is measured by |
|
Addressing the paper-facing benchmark artifact request from the earlier review comment. I added reproducible result artifacts in
I also updated Net interpretation for PR #655 / the paper: use the dummy simulator JSONL for the Chord baseline; use Docker/WebRTC artifacts as Rings implementation evidence. The current Docker ring convergence artifact exposes a real limitation: the fresh 16-node ring did not converge within the sampled window, so a steady-state Docker lookup table still needs either a successfully converged run or a separate fix to cluster stabilization behavior. |
|
Follow-up after the benchmark artifact update. This now gives PR #655 usable paper inputs, with one important boundary: The dummy baseline is now paper-grade for the formal Chord baseline object: it has the requested The Docker transport artifact is also now usable as Rings implementation evidence. It has repeated payload sizes, source and destination counters, zero send/receive failures, and no flush timeouts. The paper can cite it as application-payload-through-WebRTC evidence, with the explicit caveat already documented for 64 KiB payloads: The Docker convergence artifact should not be used as a steady-state lookup result. Across 30 samples / about 309.5 seconds it stays flat at For the paper side, the remaining missing artifact is a valid Docker/WebRTC steady-state lookup window if we want to make any steady-state Rings lookup claim. Without that, we can still write the evaluation section accurately as: |
|
Update after opening #658:
Result summary from the regenerated dummy baseline:
Important conclusion: the wrapping fix corrects a real Chord semantic mismatch, but it does not reduce the stable aggregate avg hops; the stable baseline remains Validation run locally:
Docker/WebRTC artifacts were not regenerated in this commit; the artifact metadata now explicitly scopes them to the existing |
|
Formal hop-accounting pass: Let So for With our successor list terminal condition ( Independent rank-model calculation for
Independent reimplementation over the benchmark's splitmix DIDs and deterministic lookup targets gives:
This exactly reproduces the committed dummy result ( Recommended follow-up: rename or split this metric in the benchmark output, e.g. keep |
|
Additional note on the finger-table-size intuition:
This is why the comparable theoretical scale is still |
|
Follow-up on the new hop-accounting notes. The formal accounting resolves the main paper-comparison concern: the committed That changes the interpretation materially. Stable For paper safety, I think this PR should make the metric split explicit in the emitted JSON and README before PR #655 consumes it: Then PR #655 can use The churn reliability conclusion does not change: failures/10k and timeout growth are still much worse than the original Chord table under the current churn model, so the paper should present churn as a scoped baseline/obligation, not as a superiority claim. Docker/WebRTC lookup also remains convergence evidence only until a valid steady-state run is produced. |
|
Added the What changed:
Regenerated
This confirms the earlier diagnosis: the large timeout/failure numbers belong to the no-maintenance stale-snapshot stress model, not to a maintained Chord baseline. The README now says to use Validation:
|
|
Follow-up on the remaining hop-count gap versus the original Chord table: The maintained K=8 maintained-only probe:
The failed rows are lower there because a fully maintained active ring has fewer live nodes and no stale pointers. The original Chord failure table appears closer to a stale-but-robust failure model: stale routing entries still exist, but a large enough successor list lets lookups route around them. Running
So the previously reported K=3 maintained rows are a Rings-configuration baseline, not the closest reproduction of the Chord paper table. For paper-table reproduction, we should run/report a separate I also added |
|
Correction on the Chord paper successor-list parameter: The original Chord paper does not use So the direct parameter mapping to this benchmark is: My previous K=8 probe is still useful as a sensitivity check, and it happens to put the stable row near |
|
Added a paper-aligned simulator artifact for the original Chord paper coverage. New commit: What changed:
Coverage now maps the original Chord paper's numbered tables and data figures:
Important boundary: Rings does not currently implement Chord-style virtual nodes, i.e. one physical node owning multiple unrelated ring positions. The old Validation:
|
|
Updated this PR to keep the benchmark implementation pure Rust. Changes:
Validation:
|
2b6f2ef to
aa89fc4
Compare
|
Rebased this PR onto What changed:
Fig. 9 runtime storage-vnode data (
Validation run:
|
|
Follow-up cleanup: the branch no longer commits benchmark result data directly. I removed the tracked Current cleanup commit: |
Summary
dht_network_simfor small-scale DHT evaluation using full native WebRTC nodes rather than the dummy transport.webrtc_dhtreports for stable, failed-node, and churn scenarios, including convergence, connected WebRTC edges, topology/full finger matches, lookup success/correctness, average hops, lookup timeouts, and lookup failures per 10k.webrtc_transportreports for WebRTC data-channel throughput through Rings messages, including payload size, message count, send/receive elapsed time, and Mbps.dht_finger_table_sizethrough native config, serialized processor config, andProcessorBuilder, with--dht-finger-table-sizeandRINGS_DHT_FINGER_TABLE_SIZEruntime overrides.Validation
CARGO_TARGET_DIR=/private/tmp/rings-dht-benchmark-653-target cargo check -p rings-node --no-default-features --features node --libCARGO_TARGET_DIR=/private/tmp/rings-dht-benchmark-653-target cargo check -p rings-core --bench dht_network_simCARGO_TARGET_DIR=/private/tmp/rings-dht-benchmark-653-target RINGS_DHT_BENCH_NODES=3 RINGS_DHT_BENCH_FINGER_TABLE_SIZES=4 RINGS_DHT_BENCH_FAILED_NODE_PCTS=0 RINGS_DHT_BENCH_CHURN_RATE_PCTS=0 RINGS_DHT_BENCH_MAX_ROUNDS=5 RINGS_DHT_BENCH_THROUGHPUT_MESSAGES=2 RINGS_DHT_BENCH_THROUGHPUT_PAYLOAD_BYTES=1024 RINGS_DHT_BENCH_WAIT_TIMEOUT_MS=30000 cargo bench -p rings-core --bench dht_network_simCARGO_TARGET_DIR=/private/tmp/rings-dht-benchmark-653-target cargo test -p rings-node --lib dht_finger_table_sizeCARGO_TARGET_DIR=/private/tmp/rings-dht-benchmark-653-target cargo test -p rings-node --lib test_deserialization_with_missed_fieldCARGO_TARGET_DIR=/private/tmp/rings-dht-benchmark-653-target cargo clippy -p rings-core --bench dht_network_sim -- -D warnings -D clippy::unwrap_used -D clippy::expect_used -D clippy::panic -D clippy::todo -D clippy::unimplemented -D clippy::indexing_slicingCARGO_TARGET_DIR=/private/tmp/rings-dht-benchmark-653-target cargo clippy -p rings-node --no-default-features --features node --lib -- -D warnings -D clippy::unwrap_used -D clippy::expect_used -D clippy::panic -D clippy::todo -D clippy::unimplemented -D clippy::indexing_slicinggit diff --checkFixes #653