feat(switchyard-server): Record routing overhead metric

[grahamking]

This is how much time Switchyard added on top of the backend call. This
can be very low (passthrough algo) or potentially quite high (classifier
algo), so a broad range.

Assisted-by: Claude:Opus 5 high
Signed-off-by: Graham King grahamk@nvidia.com

Summary by CodeRabbit

• New Features

  • Added routing overhead metrics to show time spent outside the routed model call.
  • Exposed routing overhead as a Prometheus histogram labeled by algorithm.
  • Added explicit histogram buckets for more consistent metric reporting.

• Documentation

  • Documented the new metric, calculation, labels, recording behavior, and streaming considerations.

• Tests

  • Added coverage for successful and failed runs, delayed calls, and exported metric buckets.

[grahamking]

@CodeRabbit review

[coderabbitai]

✅ Action performed

Review finished.

Note: CodeRabbit is an incremental review system and does not re-review already reviewed commits. This command is applicable only when automatic reviews are paused.

[coderabbitai]

Walkthrough

Changes

Routing overhead metrics

Layer / File(s)	Summary
Driver timing and observability wiring crates/libsy/src/core/algorithm.rs, crates/libsy/src/observability.rs	Driver records successful routed-call duration and supplies it to run observability.
Overhead calculation and behavioral tests crates/libsy/src/observability.rs, crates/libsy/tests/observability.rs	Run overhead is computed with saturating subtraction and tested for successful, failed, and classifier-routed calls.
Prometheus export and documentation crates/switchyard-server/src/metrics.rs, crates/switchyard-server/README.md, crates/switchyard-server/tests/server.rs	The new histogram receives explicit buckets, documented semantics, and integration coverage.

Estimated code review effort: 3 (Moderate) | ~20 minutes

Poem

I’m a rabbit with a stopwatch bright,
Measuring hops through the routing night.
Calls leave traces, buckets grow,
Overhead blooms in metrics’ glow—
Hop, hop, ship the stats just right!

🚥 Pre-merge checks | ✅ 5

✅ Passed checks (5 passed)

Check name	Status	Explanation
Docstring Coverage	✅ Passed	Docstring coverage is 100.00% which is sufficient. The required threshold is 80.00%.
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Title check	✅ Passed	The title accurately captures the main change: adding a routing overhead metric, even though related support spans more than switchyard-server.

---

_{Comment @coderabbitai help to get the list of available commands.}

[coderabbitai]

Actionable comments posted: 2

🤖 Prompt for all review comments with AI agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@crates/libsy/src/core/algorithm.rs`:
- Around line 119-120: Replace the shared routed_call completion slot with
per-response timing attribution in create_run_task, carrying each routed call’s
duration alongside its response and using the duration belonging to the response
ultimately selected to serve the run. Remove the last-completion overwrite
behavior and update overhead accounting accordingly. Add a regression test
covering multiple or concurrent routed calls where the selected response is not
the last to complete.

In `@crates/switchyard-server/README.md`:
- Around line 129-132: Update the README explanation of run and routed-call
clock behavior to distinguish their stopping points: the routed-call clock stops
when the streamed handle arrives, while observe_run stops when create_run_task
returns and therefore includes post-call algorithm work in routing overhead.
Remove the inaccurate “Both clocks” wording and preserve the clarification about
SSE relay time and stream completion.

🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

• Push a commit to this branch (recommended)
• Create a new PR with the fixes

---

ℹ️ Review info

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: b077ffc5-567f-4aa7-851d-8bf436b840d6

📥 Commits

Reviewing files that changed from the base of the PR and between 1f5d173 and 1993cf7.

📒 Files selected for processing (6)

• crates/libsy/src/core/algorithm.rs
• crates/libsy/src/observability.rs
• crates/libsy/tests/observability.rs
• crates/switchyard-server/README.md
• crates/switchyard-server/src/metrics.rs
• crates/switchyard-server/tests/server.rs