Skip to content

Add Skippy MTP runtime, metrics, and benchmark harness#540

Draft
i386 wants to merge 5 commits into
mainfrom
feat/mtp
Draft

Add Skippy MTP runtime, metrics, and benchmark harness#540
i386 wants to merge 5 commits into
mainfrom
feat/mtp

Conversation

@i386
Copy link
Copy Markdown
Collaborator

@i386 i386 commented May 13, 2026

Summary

  • Add the native MTP runtime path across Skippy server, runtime, FFI, protocol, and transport layers.
  • Wire telemetry and metrics-server storage so MTP runs emit usable run artifacts and counters.
  • Add local docs and benchmark corpora for one-stage MTP validation, including longer structured/code-heavy prompts.
  • Record the staged MTP experiment outputs under experiments/mtp/ for comparison and follow-up analysis.

Testing

  • Ran Rust build/format validation for the touched Skippy and metrics crates.
  • Exercised local one-stage benchmark runs against studio54 and verified metrics export and JSON artifact capture.
  • Validated benchmark corpus files and guarded runner scripts on the local host.

i386 added 4 commits May 13, 2026 06:59
# Conflicts:
#	crates/skippy-server/src/frontend/generation_flow.rs
#	crates/skippy-server/src/frontend/local_generation.rs
#	crates/skippy-server/src/frontend/prompting.rs
@i386 i386 marked this pull request as ready for review May 14, 2026 02:57
@i386 i386 marked this pull request as draft May 14, 2026 02:59
@ndizazzo
Copy link
Copy Markdown
Collaborator

ndizazzo commented May 16, 2026

@i386 Suggest we re-evaluate this since MTP support in llamacpp is now merged

@i386
Copy link
Copy Markdown
Collaborator Author

i386 commented May 17, 2026

@ndizazzo the version I am working on makes use of that PR, I've been tracking it religiously :)

@ndizazzo
Copy link
Copy Markdown
Collaborator

Gotcha gotcha - I thought we had just pulled certain changes in on through the patch stack. I figured we could drop that and just rely on a SHA bump, plus the skippy changes

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants