Expose bounded route diagnostics#804
Open
IvGolovach wants to merge 1 commit into
Open
Conversation
Validation * Validation tier: Tier 3 - shared runtime routing/status behavior; target ordering, context-fit rejection diagnostics, routing metrics status, and operator diagnostics changed. * git fetch --no-tags origin main:refs/remotes/origin/main: PASS, origin/main at 95101ce. * git diff --check: PASS, no output. * git diff --cached --check: PASS, no output. * cargo fmt --all: PASS. * cargo fmt --all -- --check: PASS. * LLAMA_STAGE_BUILD_DIR=/Users/Funtland/Downloads/mesh-llm/.deps/llama-build/build-stage-abi-metal cargo test -p mesh-llm-host-runtime route_diagnostics --lib -- --test-threads=1: PASS, 1 passed. * LLAMA_STAGE_BUILD_DIR=/Users/Funtland/Downloads/mesh-llm/.deps/llama-build/build-stage-abi-metal cargo test -p mesh-llm-host-runtime reorder_candidates --lib -- --test-threads=1: PASS, 9 passed. * LLAMA_STAGE_BUILD_DIR=/Users/Funtland/Downloads/mesh-llm/.deps/llama-build/build-stage-abi-metal cargo test -p mesh-llm-host-runtime route_model_target_reason_codes --lib -- --test-threads=1: PASS, 2 passed. * LLAMA_STAGE_BUILD_DIR=/Users/Funtland/Downloads/mesh-llm/.deps/llama-build/build-stage-abi-metal cargo test -p mesh-llm-host-runtime status_payload_exposes_bounded_route_decision_diagnostics --lib -- --test-threads=1: PASS, 1 passed. * LLAMA_STAGE_BUILD_DIR=/Users/Funtland/Downloads/mesh-llm/.deps/llama-build/build-stage-abi-metal cargo test -p mesh-llm-host-runtime routing_metric --lib -- --test-threads=1: PASS, 10 passed. * LLAMA_STAGE_BUILD_DIR=/Users/Funtland/Downloads/mesh-llm/.deps/llama-build/build-stage-abi-metal cargo test -p mesh-llm-host-runtime test_api_proxy_rejects_request_when_all_known_contexts_too_small --lib -- --test-threads=1: PASS, 1 passed. * LLAMA_STAGE_BUILD_DIR=/Users/Funtland/Downloads/mesh-llm/.deps/llama-build/build-stage-abi-metal cargo test -p mesh-llm-host-runtime request_budget_tokens --lib -- --test-threads=1: PASS, 3 passed. * LLAMA_STAGE_BUILD_DIR=/Users/Funtland/Downloads/mesh-llm/.deps/llama-build/build-stage-abi-metal cargo check -p mesh-llm: PASS. * LLAMA_STAGE_BUILD_DIR=/Users/Funtland/Downloads/mesh-llm/.deps/llama-build/build-stage-abi-metal /opt/homebrew/bin/cargo-clippy clippy -p mesh-llm-host-runtime --all-targets -- -D warnings: PASS. * Ledger: not applicable - not required for selected validation tier/change family. * Version: not applicable - no release/version sync required for this non-release runtime diagnostics change. * Not run: live multi-node mesh startup/public-mesh smoke/agent harness - no local multi-node runtime mesh/model endpoint was available; targeted route ordering, diagnostics, status, proxy, shipped-binary check, and clippy cover the changed paths before mandatory remote CI/reviewer validation. Rollback * git revert HEAD
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Expose bounded local route-decision diagnostics in
/api/statusand make gossiped throughput hints safer for routing decisions.This PR adds a small operator-facing diagnostics window that explains recent model-routing choices: which targets were considered, which target was selected, and why other targets were rejected or deprioritized. It also prevents very low-sample gossiped throughput hints from reordering otherwise eligible targets.
Why
When routing fails because of context fit, target health, affinity, or throughput ordering, the current status payload does not explain enough to debug the decision. This makes routing behavior easier to inspect without changing mesh protocol compatibility or exposing request contents.
Diff scope
routing_metrics.recent_route_decisionsto the local/api/statusrouting metrics payload.Compatibility and safety
Validation
Validation tier: Tier 3 - shared runtime routing/status behavior.
Local validation passed: