Skip to content

ci(bench): self-heal a bench box that lost rustup from PATH#926

Merged
stubbi merged 2 commits into
mainfrom
ci/bench-rustup-selfheal
Jul 16, 2026
Merged

ci(bench): self-heal a bench box that lost rustup from PATH#926
stubbi merged 2 commits into
mainfrom
ci/bench-rustup-selfheal

Conversation

@stubbi

@stubbi stubbi commented Jul 16, 2026

Copy link
Copy Markdown
Contributor

Thinking Path

  • Mitos boots Firecracker microVMs and forks them; its public performance numbers must be reproducible from bench/ (the no-unverified-claims rule), which the bare-metal Bench workflow produces on a self-hosted [bare-metal, kvm, bench] runner.
  • That workflow builds the Rust guest agent (guest/agent-rs) static-musl as the first real step, using rustup.
  • The bench box lost the Rust toolchain from PATH, so the 'Build guest agent' step died with "rustup: command not found" (exit 127), and every bare-metal bench (fork-exec, exec-rt, metering, and the CoW density sweep ci(bench): add a dispatch-only CoW density sweep #766/docs(plan): the winning fork primitive, cheap CoW forks per tenant on multi-node #764) stopped producing evidence.
  • It needs addressing now because it blocks the density-sweep evidence for docs(plan): the winning fork primitive, cheap CoW forks per tenant on multi-node #764 and any refresh of the published bare-metal numbers; the issue itself proposes a guarded install in the build step.
  • This pull request makes the build step self-heal: source the cargo env first (recover a PATH-only loss), then install rustup only when it is genuinely absent.
  • The benefit is that a stripped box repairs itself on the next run instead of hard-failing, and a healthy box is unchanged.

Linked Issues or Issue Description

Closes #767.

What Changed

  • In .github/workflows/bench.yaml 'Build guest agent': before rustup target add, source $HOME/.cargo/env if present, and if rustup is still not on PATH, install it with the official installer pinned to a minimal stable profile, then source the env. Idempotent.

Verification

  • YAML change is confined to the existing run: | shell block at matching indentation; the repo's actionlint and shellcheck-install checks validate it in CI.
  • The self-heal path exercises on the next scheduled/dispatched Bench run on the self-hosted box; a healthy box takes the no-op branch (rustup already present).

Risks

Low risk. The install runs only when rustup is absent, so a healthy runner is untouched. The workflow is not one of the eight required checks and does not gate other PRs; worst case a network hiccup during install fails the same step that already fails today.

Model Used

Claude Opus 4.8 (claude-opus-4-8), 1M context, extended thinking.

Summary by CodeRabbit

  • Chores
    • Improved build workflow reliability in environments where the Rust toolchain is not already available.
    • Automatically configures the required stable Rust toolchain before building.

The bare-metal bench workflow died at 'Build guest agent' with
"rustup: command not found" (exit 127) when the self-hosted box lost the
Rust toolchain from PATH, blocking every bare-metal bench (fork-exec,
exec-rt, metering, and the CoW density sweep #766/#764).

Guard the step: source $HOME/.cargo/env first in case only PATH was lost,
and install rustup (minimal stable profile) only when it is genuinely
absent. Idempotent, so a healthy box is unchanged and a stripped box
self-heals instead of failing the run.

Closes #767

Signed-off-by: Jannes Stubbemann <jannes@openclaw.rocks>
@coderabbitai

coderabbitai Bot commented Jul 16, 2026

Copy link
Copy Markdown

Review Change Stack

📝 Walkthrough

Walkthrough

The bare-metal benchmark workflow now restores the Cargo environment and conditionally installs a minimal stable Rust toolchain before adding targets and building the guest agent.

Changes

Bench workflow Rust setup

Layer / File(s) Summary
Prepare Rust for guest-agent build
.github/workflows/bench.yaml
The build step sources $HOME/.cargo/env when available, installs stable Rust through rustup when rustup is absent, and re-sources the environment before continuing.

Estimated code review effort: 2 (Simple) | ~10 minutes

🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Title check ✅ Passed The title is conventional, concise, and accurately reflects the bench workflow self-heal for missing rustup.
Description check ✅ Passed The description includes the required thinking path, linked issue, changes, verification, risks, and model details; only the checklist is missing.
Linked Issues check ✅ Passed The workflow change directly addresses #767 by recovering PATH and installing rustup only when absent, matching the reported failure.
Out of Scope Changes check ✅ Passed The changes stay confined to the bench workflow fix and do not introduce unrelated scope.
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch ci/bench-rustup-selfheal

Comment @coderabbitai help to get the list of available commands.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In @.github/workflows/bench.yaml:
- Around line 130-133: Update the Cargo environment sourcing block in the
workflow to run `. "$HOME/.cargo/env"` through a non-fatal guard, allowing
sourcing failures to continue to the existing rustup recovery or installation
path while retaining the file-existence check.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: ASSERTIVE

Plan: Pro Plus

Run ID: f76ba11a-bfc2-41ba-b53d-1a86012878cb

📥 Commits

Reviewing files that changed from the base of the PR and between 1fb5929 and 1884699.

📒 Files selected for processing (1)
  • .github/workflows/bench.yaml

Comment on lines +130 to +133
if [ -f "$HOME/.cargo/env" ]; then
# shellcheck disable=SC1091
. "$HOME/.cargo/env"
fi

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🩺 Stability & Availability | 🟠 Major | ⚡ Quick win

🧩 Analysis chain

🏁 Script executed:

#!/bin/bash
set -euo pipefail

printf '\n--- bench.yaml excerpt ---\n'
nl -ba .github/workflows/bench.yaml | sed -n '118,150p'

printf '\n--- kvm-test.yaml excerpt ---\n'
nl -ba .github/workflows/kvm-test.yaml | sed -n '118,150p'

Repository: mitos-run/mitos

Length of output: 221


🏁 Script executed:

python3 - <<'PY'
from pathlib import Path

for path in [Path('.github/workflows/bench.yaml'), Path('.github/workflows/kvm-test.yaml')]:
    print(f"\n--- {path} excerpt ---")
    lines = path.read_text().splitlines()
    for i in range(117, 151):
        if i <= len(lines):
            print(f"{i:4d}: {lines[i-1]}")
PY

Repository: mitos-run/mitos

Length of output: 4209


🏁 Script executed:

python3 - <<'PY'
from pathlib import Path

for path in [Path('.github/workflows/kvm-test.yaml'), Path('.github/workflows/bench.yaml')]:
    text = path.read_text()
    for needle in ['.cargo/env', 'rustup', 'source', 'shellcheck disable=SC1091']:
        if needle in text:
            print(f'\n--- {path} matches {needle!r} ---')
            for i, line in enumerate(text.splitlines(), 1):
                if needle in line:
                    print(f'{i:4d}: {line}')
PY

Repository: mitos-run/mitos

Length of output: 6771


Make Cargo env sourcing failure-tolerant

set -euo pipefail makes . "$HOME/.cargo/env" abort the step if that file is unreadable or broken, so the rustup recovery path never runs. Wrap the source in a non-fatal guard like the one in .github/workflows/kvm-test.yaml so a stale env file can still fall back to installation.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In @.github/workflows/bench.yaml around lines 130 - 133, Update the Cargo
environment sourcing block in the workflow to run `. "$HOME/.cargo/env"` through
a non-fatal guard, allowing sourcing failures to continue to the existing rustup
recovery or installation path while retaining the file-existence check.

@stubbi
stubbi merged commit 6d856b3 into main Jul 16, 2026
32 checks passed
@stubbi
stubbi deleted the ci/bench-rustup-selfheal branch July 16, 2026 11:25
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

ci(bench): bare-metal bench runner lost rustup, guest-agent build fails

1 participant