mesh: upgrade to mesh-llm v0.72.2, relay tunneling on, membership-derived admission (WIP)#1656
Draft
michaelneale wants to merge 7 commits into
Draft
mesh: upgrade to mesh-llm v0.72.2, relay tunneling on, membership-derived admission (WIP)#1656michaelneale wants to merge 7 commits into
michaelneale wants to merge 7 commits into
Conversation
… iroh relay tunneling Bump the mesh-llm pin from ebc3ab4c (0.68.0) to c209d177 (v0.72.2, latest release) in both the desktop crate and buzz-relay dev-deps. One API break: initialize_host_runtime() is now async — awaited in the desktop runtime gate and the hardware smoke example. Ergonomics adopted from the sibling mesh-console integration: - startup_timeout(180s) on both serve and client builders. The SDK default (30s) times out during first model load (multi-GB download + Metal warmup). - BUZZ_MESH_IROH_RELAYS env (default: disabled — unchanged behavior). "1"/"default" enables iroh's default relay servers as a transport fallback for symmetric-NAT peers; any other value is a comma-separated list of custom iroh relay URLs. Iroh relay traffic is end-to-end encrypted QUIC — relays forward ciphertext only. This is transport-only: publish(false) stays hardcoded, the Nostr relay list stays empty, and mesh presence is never announced publicly regardless of this setting. - mesh_serve_client_smoke exits via std::process::exit(0) after PASS to skip C++ static destructors — the embedded ggml Metal runtime can GGML_ASSERT in its global teardown (same crash as mesh-console issue #8). Verified: cargo check (desktop --features mesh-llm, relay --all-targets), fmt, clippy (no new lints over baseline), 27 mesh unit tests pass, and the hardware smoke proves real serve→client→inference over the mesh on 0.72.2. Native runtime and CI need no changes: ensure-mesh-native-runtime.sh and the release/ci llama cache both derive the version from the Cargo metadata.
Flip BUZZ_MESH_IROH_RELAYS default from disabled to enabled: unset/empty now means iroh's default relay servers are available as transport fallback, so member pairs behind symmetric NATs can connect. "0" opts out (direct QUIC only); a comma-separated URL list selects custom relays. Iroh relays forward end-to-end encrypted QUIC only — no admission role. publish(false) and the empty Nostr relay list remain hardcoded; mesh presence is never announced publicly. Reachability is no longer a de-facto membership gate, so explicit mesh admission (owner allowlist from relay membership) follows in the next commits. Docs connectivity section updated to match.
…t the mesh layer With iroh relay tunneling on by default, reachability no longer acts as an accidental membership gate: anyone holding a dial pointer could reach a node. This makes admission explicit using mesh-llm's native trust layer, with Buzz relay membership as the single source of truth. Desktop: - mesh_llm/identity.rs: per-machine mesh owner keypair (ed25519, distinct from the Nostr identity) at mesh-llm's default keystore path (~/.mesh-llm/owner-keystore.json); load-or-generate, process-cached. - Both node modes present a signed ownership attestation (.owner_key). - Status reports now carry the reporter's ownerId; the relay republishes it in the sanitized kind:30621 note (relay-signed, member-gated), which makes the set of owner ids in those notes the admission roster. - StartMeshNodeRequest.trusted_owner_ids: when a roster is resolved, nodes enforce TrustPolicy::Allowlist + owner_required over exactly those owners (self always included so a solo sharer and a fresh relay's first member are never locked out). None = no roster resolved (open relay / tests / relay outage) -> no enforcement, matching relay-side gating. - mesh_start_node and both client start paths resolve the roster from the relay before starting. - Coordinator roster watcher (60s poll): membership changed -> restart the node with the fresh roster. The SDK trust store is fixed at node start, so restart is how join/leave takes effect; an ex-member's access ends within one poll interval + restart. Relay outages leave the node as-is. Relay: - BuzzMeshStatus.owner_id (camelCase + snake_case accepted, omitted when absent) carried through sanitize_mesh_status. Security outcome: a leaked EndpointAddr or relay reachability admits nobody; leaving the Buzz server revokes mesh access. Consumption-side trust (dialing a member's real machine) is unchanged and remains anchored in relay-signed discovery notes. Verified: mesh unit tests 30 passed (desktop) + 24 passed (relay), clippy clean over baseline, non-mesh build unaffected, hardware smoke PASS.
Replaces guess-into-a-text-field with mesh-console's diagnose pattern: survey the machine (GPU, AI memory), rank mesh-llm's curated MODEL_CATALOG by fit (comfortable/tight/tradeoff/too_large, same thresholds as mesh-llm's fit_code_for_size_label), mark installed models from the HF cache, and recommend the best fit via auto_model_pack. - New optional deps (mesh-llm feature only): mesh-llm-client (catalog + auto_model_pack), mesh-llm-node (HF cache scan), mesh-llm-system (hardware survey), same pinned rev. - mesh_llm/catalog.rs: model_catalog() -> MeshModelCatalog (recommended first, then by fit, larger first within a fit; draft-only speculative models excluded). - New Tauri command mesh_model_catalog (+ stub for non-mesh builds); survey + cache scan run on spawn_blocking. - MeshComputeSettingsCard: "Suggested for this machine" list above the model field — click to fill, fit labels, Recommended/Installed badges, too-large entries visible but disabled, show-all toggle. Free-text field unchanged; catalog failure degrades to the old UI. - e2e bridge mocks mesh_model_catalog. Verified: 33 mesh unit tests (3 new: fit thresholds, ranking order, installed matching), tsc + biome clean, non-mesh build clean, all 7 mesh-compute Playwright specs pass.
Fixes the two worst Share-compute UX failures: 1. Starting a serve node with an uncached model froze the card greyed-out for the whole multi-GB download. Now the model is pre-downloaded *before* serve::start() (ensure_model_downloaded, mesh-console's ensure_model pattern), and byte progress flows out of mesh-llm's process-global OutputSink (mesh_llm/progress.rs, installed at setup) as `mesh-download-progress` Tauri events. The card renders a live progress bar: model name, N.N GB of N.N GB, percent. Already-cached models skip straight to node start. 2. The recommendation was ghost placeholder text in an empty field. Now the catalog fetch pre-fills the recommended model as a real value (draft persisted); placeholder only shows when there is genuinely no recommendation. - mesh-llm-events added as an optional dep (same pinned rev); ConsoleSessionMode::InteractiveDashboard is what makes byte-level progress flow through the sink instead of stderr ANSI bars. - recursion_limit=256 on buzz_lib: the ensure→download→start async chain under Tauri command futures exceeded the default query depth. - useMeshDownloadProgress hook + DownloadProgressBar; progress resets when the start action settles or errors. Verified: 33 mesh unit tests, tsc + biome clean, non-mesh build clean, all 7 mesh-compute Playwright specs pass.
…poses upstream token-holder inference hole
Three-process harness (mesh_admission_smoke): allowlist serve node, a
trusted client (allowlisted owner key), and a stranger (different owner
key, SAME leaked invite token). One process per node because mesh-llm
keeps process-global state (~/.mesh-llm node identity + attestation);
each role gets an isolated HOME.
Results on v0.72.2 (run on hardware, real inference):
- PASS: trusted client admitted, model visible, inference routes ("PONG!")
- PASS: serve rejects the stranger at gossip — "Rejecting peer due to
owner policy: UntrustedOwner" — it is never promoted to the peer set
- FAIL (upstream, deliberate): the stranger can STILL route an inference
with the leaked token. mesh-llm's quarantine gate exempts
STREAM_TUNNEL_HTTP (0x04) and STREAM_ROUTE_REQUEST (0x05) from
admission ("passive SDK inference path for callers that have an invite
token" — stream_allowed_before_admission, mesh/mod.rs:2550).
signed_join_tokens(true) does not help: the leak IS the signed token.
So on current upstream, the owner allowlist gates mesh *membership*
(peer promotion, gossip, raw tunnel) but not *inference serving* to any
holder of a valid invite token. Buzz's membership-derived admission
therefore hardens the pool, and the leaked-dial-pointer inference hole
needs an upstream change (gate 0x04/0x05 behind admission whenever
trust_policy != Off). The harness asserts today's reachable guarantee
(gossip rejection) and documents the routing gap; flip the inference
assertion to load-bearing once upstream closes it.
… upstream inference-path gap Verdict handling now matches what v0.72.2 actually enforces: - hard PASS/FAIL on membership gating (trusted admitted + routes, stranger rejected at gossip as UntrustedOwner) - the stranger's token-holder inference via the admission-exempt STREAM_TUNNEL_HTTP path is reported as a KNOWN UPSTREAM GAP, not a harness failure; flip to hard-fail once upstream gates 0x04/0x05 behind admission when trust_policy != Off. Full run: exit 0, PASS 1/3 + 2/3 + 3/3(with gap noted).
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
WIP — draft.
Upgrades the embedded mesh-llm and hardens compute sharing to the Buzz-server membership circle:
BUZZ_MESH_IROH_RELAYS=0opts out) — transport-only encrypted fallback so member pairs behind symmetric NATs can connect; mesh presence is still never published publiclyTrustPolicy::Allowlistover that roster. Leaving the server revokes mesh access (roster watcher restarts the node on membership change, ~60s)Verified: mesh unit tests (30 desktop + 24 relay), clippy clean over baseline, hardware smoke (real serve→client→inference) PASS after each commit.
Still to do:
just cion a machine with VPN + mobile toolchain (local hooks were env-blocked; Rust gates all green)