test(e2e): migrate VM driver privileged exec to Vitest by cv · Pull Request #5144 · NVIDIA/NemoClaw

cv · 2026-06-10T14:04:17Z

Summary

Migrates the VM driver privileged-exec routing regression from the legacy shell script into a Vitest live scenario. The nightly job now runs the targeted Vitest scenario directly and uploads shared E2E fixture artifacts.

Related Issue

Refs #4941
Refs #4245

Changes

Add test/e2e-scenario/live/vm-driver-privileged-exec-routing.test.ts with fake registry and fake Docker coverage for direct sandbox container routing.
Replace test/e2e/test-vm-driver-privileged-exec-routing.sh in nightly-e2e.yaml with direct e2e-scenarios-live execution.
Extend test/e2e-script-workflow.test.ts and test/helpers/e2e-workflow-contract.ts to lock the new workflow contract.

Type of Change

Code change (feature, bug fix, or refactor)
Code change with doc updates
Doc only (prose changes, no code sample modifications)
Doc only (includes code sample changes)

Verification

npx prek run --all-files passes
npm test passes
Tests added or updated for new or changed behavior
No secrets, API keys, or credentials committed
Docs updated for user-facing behavior changes
npm run docs builds without warnings (doc changes only)
Doc pages follow the style guide (doc changes only)
New doc pages include SPDX header and frontmatter (new pages only)

Signed-off-by: Carlos Villela cvillela@nvidia.com

copy-pr-bot · 2026-06-10T14:04:20Z

This pull request requires additional validation before any workflows can run on NVIDIA's runners.

Pull request vetters can view their responsibilities here.

Contributors can view more details about this message here.

coderabbitai · 2026-06-10T14:04:25Z

Important

Review skipped

Draft detected.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: e1f024cb-3fef-489a-9417-f1eea9bc8889

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

🔍 Trigger review

✨ Finishing Touches

🧪 Generate unit tests (beta)

Create PR with unit tests
Commit unit tests in branch codex/e2e-migrate-vm-driver-privexec

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

github-actions · 2026-06-10T14:06:05Z

E2E Advisor Recommendation

Required E2E: vm-driver-privileged-exec-routing-e2e
Optional E2E: None

Dispatch hint: vm-driver-privileged-exec-routing-e2e

Workflow run

Full advisor summary

E2E Recommendation Advisor

Base: origin/codex/e2e-simplify-migration-tracking
Head: HEAD
Confidence: high

Required E2E

vm-driver-privileged-exec-routing-e2e (low): This is the directly modified E2E job. It should run to validate that the workflow migration from the deleted shell script to the new live Vitest scenario still dispatches correctly, builds the CLI, executes the privileged-exec routing probe, and uploads artifacts.

Optional E2E

None.

New E2E recommendations

None.

Dispatch hint

Workflow: .github/workflows/nightly-e2e.yaml
jobs input: vm-driver-privileged-exec-routing-e2e

github-actions · 2026-06-10T14:06:07Z

E2E Scenario Advisor Recommendation

Required scenario E2E: e2e-scenarios-all
Optional scenario E2E: None

Dispatch required scenario E2E:

gh workflow run e2e-vitest-scenarios.yaml --ref <pr-head-ref>

Workflow run

Full scenario advisor summary

E2E Scenario Advisor

Base: origin/codex/e2e-simplify-migration-tracking
Head: HEAD
Confidence: medium

Required scenario E2E

e2e-scenarios-all: The PR adds a new live Vitest scenario-side test under test/e2e-scenario/live and wires nightly execution toward that Vitest surface. This free-standing live test is not a trusted-main live-supported typed registry scenario ID that can be targeted with the scenarios input, so run the e2e-vitest-scenarios fan-out rather than inventing a targeted ID.
- Dispatch: gh workflow run e2e-vitest-scenarios.yaml --ref <pr-head-ref>

Optional scenario E2E

None.

Relevant changed files

.github/workflows/nightly-e2e.yaml
test/e2e-scenario/live/vm-driver-privileged-exec-routing.test.ts

github-actions · 2026-06-10T14:07:59Z

PR Review Advisor

Findings: 0 needs attention, 0 worth checking, 0 nice ideas
Since last review: 0 prior items resolved, 0 still apply, 0 new items found

Consider writing more tests for

**Runtime validation** — Exercise vm-driver-privileged-exec-routing-e2e and confirm it invokes only test/e2e-scenario/live/vm-driver-privileged-exec-routing.test.ts through the e2e-scenarios-live Vitest project.. The code-level migration is covered by a workflow contract test and a migrated live scenario, but .github/workflows/nightly-e2e.yaml is an infrastructure/runtime path where targeted runtime validation improves confidence.
**Runtime validation** — Exercise vm-driver-privileged-exec-routing-e2e and confirm the Vitest fixture writes and uploads artifacts under e2e-artifacts/vitest/vm-driver-privileged-exec-routing/.. The code-level migration is covered by a workflow contract test and a migrated live scenario, but .github/workflows/nightly-e2e.yaml is an infrastructure/runtime path where targeted runtime validation improves confidence.
**Runtime validation** — Exercise the migrated scenario and confirm the direct-container routing assertions cover VM prefix collision, exact VM container selection, Docker-driver selection, unknown-driver selection, and missing direct container failure.. The code-level migration is covered by a workflow contract test and a migrated live scenario, but .github/workflows/nightly-e2e.yaml is an infrastructure/runtime path where targeted runtime validation improves confidence.
**Acceptance clause:** Refs Adopt Vitest fixtures as the E2E scenario execution model #4941 / Refs [macOS][Security] nemoclaw shields up/down fails with "No such container: openshell-cluster-nemoclaw" on Docker Desktop VM driver #4245 — add test evidence or identify existing coverage. No trusted linked issue bodies or issue comments were provided in deterministic context, so issue-specific acceptance clauses could not be extracted.

Workflow run details

This is an automated advisory review. A human maintainer must make the final merge decision.

wscurran · 2026-06-10T14:54:32Z

✨
Related open issues:

#4941 Adopt Vitest fixtures as the E2E scenario execution model
#4245 [macOS][Security] nemoclaw shields up/down fails with "No such container: openshell-cluster-nemoclaw" on Docker Desktop VM driver

## Summary Migrate the `test/e2e/test-vm-driver-privileged-exec-routing.sh` contract into focused Vitest coverage while keeping the legacy shell file present for now. Seeded from Carlos's PR #5144, but rebuilt on current `main` with only the one-script migration changes and no #5126-era framework diff. ## Related Issues Refs #5098 Refs #4245 ## Contract mapping - Legacy assertion: VM-driver privileged exec routes to the direct sandbox container, not `openshell-gateway-nemoclaw`. - Replacement: `test/vm-driver-privileged-exec-routing.test.ts` asserts `alpha` selects `openshell-alpha-abc123` while ignoring `openshell-alpha-child` and gateway containers. - Boundary preserved: real built `dist/lib/sandbox/privileged-exec.js` helper plus fake registry and fake `docker ps` subprocess boundary. - Legacy assertion: exact direct container wins when present. - Replacement: `alpha-child` selects `openshell-alpha-child`. - Boundary preserved: fake Docker output exercises the same direct-container selection path. - Legacy assertion: Docker-driver and older registry entries without recorded driver still route to direct sandbox containers. - Replacement: `dockerbox` and `unknown-driver` cases assert direct container argv. - Boundary preserved: built helper reads a temp `~/.nemoclaw/sandboxes.json`. - Legacy assertion: missing direct VM container fails clearly instead of falling back to gateway routing. - Replacement: missing-container case asserts the `No running direct OpenShell sandbox container...driver: vm` error. - Boundary preserved: fake `docker ps` contains only gateway/unrelated containers. ## Simplicity check - Test shape: focused process/unit-shaped Vitest test in `test/`. - New shared helpers: none; fake registry/Docker setup is local to the test. - New framework/registry/ledger: **none**. - Workflow changes: retired/unwired the nightly shell-script lane from selective dispatch and nightly shell-script wiring. The legacy shell file remains in `test/e2e/` and in the top-level frozen shell allowlist; replacement coverage runs in normal CLI Vitest. ## Verification - `npm run build:cli` - `npx vitest run --project cli test/vm-driver-privileged-exec-routing.test.ts --silent=false --reporter=default` - `npx vitest run --project cli test/e2e-script-workflow.test.ts --silent=false --reporter=default` - includes a contract assertion that the unwired `vm-driver-privileged-exec-routing-e2e` lane is absent from nightly dispatch/wiring, the legacy shell file remains present, the replacement test is in the CLI Vitest include path, and the CLI coverage shard rebuilds `dist` before Vitest - `npm run test-size:check -- test/vm-driver-privileged-exec-routing.test.ts test/e2e-script-workflow.test.ts` - `git diff --check` ## CI follow-up The first selective E2E auto-dispatch still used the trusted `main` nightly workflow, which attempted to run the shell-script lane. This PR now removes `vm-driver-privileged-exec-routing-e2e` from the nightly dispatch input/list/job definitions so the unwired lane is not auto-dispatched. The legacy shell file has been restored per maintainer preference; this PR no longer deletes any file under `test/e2e/`.  ## Summary by CodeRabbit * **Tests** * Added a regression test for privileged-exec routing to validate correct OpenShell sandbox selection across VM, Docker, and other drivers. * Added a contract test to ensure the retired nightly job is absent and that the CLI coverage shard runs the build step before executing CLI tests. * **Chores** * Retired a nightly E2E job and updated CI guidance to reference the new CLI test shard and narrowed nightly run targets. * Test helper updated to include CLI coverage shard metadata.

cv · 2026-06-12T01:08:07Z

Closing as superseded by #5189. That merged PR carries the VM-driver privileged-exec routing contract forward on current main, explicitly notes it was seeded from this branch, and narrows the change to the one-script migration without the older #5126-era framework/ledger churn. #5098 now records test/e2e/test-vm-driver-privileged-exec-routing.sh as converted by #5189. Any remaining idea should come back as a fresh, focused follow-up against current main.

cv added 9 commits June 10, 2026 01:40

test(e2e): simplify legacy migration tracking

b30dba2

test(e2e): retire orphaned assertion inventory

e246d20

docs(e2e): clarify Vitest fixture layer terminology

0fbf595

docs(e2e): align scenario advisor wording with Vitest

68e3157

test(e2e): rename migration policy checks

2d8ac4e

test(e2e): rename Vitest support project

d473ba6

docs(e2e): avoid runner wording in release notes

56c7071

test(e2e): rename Vitest support tests directory

0d1c4cb

test(e2e): migrate VM driver privileged exec to Vitest

d5f492d

cv self-assigned this Jun 10, 2026

This was referenced Jun 10, 2026

test(e2e): add Vitest live coverage for onboard resume #5147

Merged

Epic: Migrate legacy bash E2E into the Vitest E2E system #5098

Open

test(e2e): add OpenClaw TUI chat-correlation coverage slice #5150

Merged

test(e2e): guard legacy shell deletion evidence

f9422bc

wscurran added area: ci CI workflows, checks, release automation, or GitHub Actions area: e2e End-to-end tests, nightly failures, or validation infrastructure chore Build, CI, dependency, or tooling maintenance labels Jun 10, 2026

cv added 2 commits June 10, 2026 08:00

test(e2e): validate legacy deletion replacement paths

debe958

chore(e2e): address foundation review hygiene

b1d33d7

cv mentioned this pull request Jun 10, 2026

test(e2e): simplify legacy migration tracking #5126

Merged

12 tasks

cv added 4 commits June 10, 2026 08:35

test(e2e): block retired ledger reintroduction

6062f53

test(release): disable signing in tag fixtures

c2113a8

test(e2e): rename scenario fixtures surface

3c1ffcc

test(e2e): migrate VM driver privileged exec to Vitest

ede557b

cv added 2 commits June 10, 2026 09:19

test(e2e): refresh fixture imports

fc93e78

merge(e2e): absorb remote draft branch

85e4286

Base automatically changed from codex/e2e-simplify-migration-tracking to main June 10, 2026 20:53

jyaunches mentioned this pull request Jun 11, 2026

test(e2e): migrate VM driver privexec coverage #5189

Merged

cv added the v0.0.64 Release target label Jun 11, 2026

wscurran mentioned this pull request Jun 11, 2026

Nightly E2E failed — 2026-06-04 #4736

Closed

cv closed this Jun 12, 2026

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

test(e2e): migrate VM driver privileged exec to Vitest#5144

test(e2e): migrate VM driver privileged exec to Vitest#5144
cv wants to merge 18 commits into
mainfrom
codex/e2e-migrate-vm-driver-privexec

cv commented Jun 10, 2026 •

edited

Loading

Uh oh!

copy-pr-bot Bot commented Jun 10, 2026

Uh oh!

coderabbitai Bot commented Jun 10, 2026 •

edited

Loading

Review skipped

Uh oh!

github-actions Bot commented Jun 10, 2026 •

edited

Loading

E2E Recommendation Advisor

Required E2E

Optional E2E

New E2E recommendations

Dispatch hint

Uh oh!

github-actions Bot commented Jun 10, 2026 •

edited

Loading

E2E Scenario Advisor

Required scenario E2E

Optional scenario E2E

Relevant changed files

Uh oh!

github-actions Bot commented Jun 10, 2026 •

edited

Loading

Uh oh!

wscurran commented Jun 10, 2026

Uh oh!

cv commented Jun 12, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Conversation

cv commented Jun 10, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Related Issue

Changes

Type of Change

Verification

Uh oh!

copy-pr-bot Bot commented Jun 10, 2026

Uh oh!

coderabbitai Bot commented Jun 10, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Review skipped

Uh oh!

github-actions Bot commented Jun 10, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

E2E Advisor Recommendation

E2E Recommendation Advisor

Required E2E

Optional E2E

New E2E recommendations

Dispatch hint

Uh oh!

github-actions Bot commented Jun 10, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

E2E Scenario Advisor Recommendation

E2E Scenario Advisor

Required scenario E2E

Optional scenario E2E

Relevant changed files

Uh oh!

github-actions Bot commented Jun 10, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

PR Review Advisor

Uh oh!

wscurran commented Jun 10, 2026

Uh oh!

cv commented Jun 12, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

cv commented Jun 10, 2026 •

edited

Loading

coderabbitai Bot commented Jun 10, 2026 •

edited

Loading

github-actions Bot commented Jun 10, 2026 •

edited

Loading

github-actions Bot commented Jun 10, 2026 •

edited

Loading

github-actions Bot commented Jun 10, 2026 •

edited

Loading