QUIC-Based P2P File Sharing with DHT vs Non-DHT Peer Discovery

[ItshMoh]

Summary

Build a practical file-sharing MVP on py-libp2p for a small peer set (target: 5 peers, file sizes up to 5 GB), and evaluate two peer discovery strategies:

1. Kademlia DHT-based discovery (baseline)
2. Non-DHT discovery (fast path, app-level)

This issue defines the scope, architecture, and measurable outcomes for comparing both approaches.

Why This Issue

• We are trying to build a Lean Pylibp2p for developer, ( developer friendy) , we will not be giving a lot of features, but features that they need in default , which make it easy to use.
• for testing this we will make MVP that works for real file transfer workloads (multi-GB files).
• We need to quantify discovery latency and reliability, not assume one method is better.
• We need a baseline (DHT) and a controlled alternative (non-DHT) for design decisions.

Goals

• Implement an end-to-end file sharing flow using py-libp2p + Bitswap over QUIC.
• Support direct peer-to-peer download using share metadata (ticket/direct connect).
• Implement and compare:
  • DHT discovery path: file_hash -> provider peer(s) via Kademlia.
  • Non-DHT discovery path: application-managed provider lookup.
• Produce empirical measurements for both approaches.

Scope

1. Transfer Path

• Use QUIC transport for all peers.
• Chunking and CID verification through current Bitswap/DAG flow.
• Resume-friendly download behavior for large files.
• Chunking strategy experiments:
  • Parallel Chunk Fetch
  • Disk Streaming Reassembly

2. Discovery Path A: Kademlia DHT

• Publish provider records for shared files.
• Query providers from DHT before dialing.
• Measure cold and warm lookup behavior.

3. Discovery Path B: Non-DHT

• Implement app-level provider discovery with one of:
  • Share ticket direct-connect path (peer_id + addrs + file_root_cid)
  • Lightweight provider index service for file_hash -> providers
• Query this path first for fast lookup.

4. Comparison Framework

• Same topology and workload for both paths.
• Repeat test runs and report p50/p95 values.

Expected Outcomes

• A measurable baseline for DHT discovery in small-peer file sharing.
• A measurable non-DHT alternative for low-latency provider resolution.
• Clear recommendation for default discovery path in MVP and fallback strategy.

Note

• QUIC-only transport is intentional for this MVP to reduce handshake complexity and transport-level HOL issues.

[ItshMoh]

The work is under progress here https://github.com/AspiringDevelopers/js-libp2p-examples/tree/leanpylibp2p/examples/py-libp2p-example-file-sharing