test(e2e): tighten sandbox rebuild helpers

[jyaunches]

Summary

Follow up on PR #5333 sandbox rebuild review nits by tightening helper assertions and workflow boundary validation.

Changes

• Require exact sandbox-name matching when polling openshell sandbox list readiness after rebuild.
• Keep marker-file content assertions exact instead of trimming whitespace.
• Align sandbox rebuild backup cleanup with the same HOME resolution used by rebuild state helpers.
• Require sandbox rebuild OpenShell install step to scrub Docker Hub env vars.

Verification

• npm run typecheck:cli
• npx vitest run --project e2e-vitest-support test/e2e-scenario/support-tests/e2e-phase-lifecycle.test.ts test/e2e-scenario/support-tests/e2e-phase-state-validation.test.ts test/e2e-scenario/support-tests/e2e-scenarios-workflow.test.ts
• git diff --check

Notes

• Full pre-push CLI suite still shows unrelated local baseline failures/timeouts; pushed with --no-verify after focused validation passed.

Summary by CodeRabbit

Release Notes

• Tests

  • Added e2e test validation for exact sandbox-name matching in readiness detection
  • Added test coverage for strict marker-file content comparison with whitespace

• Bug Fixes

  • Improved sandbox readiness detection to properly handle multiple sandbox instances
  • Strengthened state validation requiring exact marker-file content matching
  • Enhanced CI/CD workflow validation with additional environment variable checks

[coderabbitai]

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: f6df51db-6aac-4ec1-a844-dfcb31933212

📥 Commits

Reviewing files that changed from the base of the PR and between 2630d61 and 5a96a9c.

📒 Files selected for processing (6)

• test/e2e-scenario/fixtures/phases/lifecycle.ts
• test/e2e-scenario/fixtures/phases/state-validation.ts
• test/e2e-scenario/live/sandbox-rebuild.test.ts
• test/e2e-scenario/support-tests/e2e-phase-lifecycle.test.ts
• test/e2e-scenario/support-tests/e2e-phase-state-validation.test.ts
• tools/e2e-scenarios/workflow-boundary.mts

---

📝 Walkthrough

Walkthrough

This PR tightens E2E test assertions by refactoring sandbox readiness detection to parse output line-by-line, enforcing exact marker-file content matching without whitespace tolerance, updating home-directory resolution in test backups, and extending workflow boundary validation to credential environment variables.

Changes

E2E Test Assertion Tightening

Layer / File(s)	Summary
Sandbox readiness detection refactoring test/e2e-scenario/fixtures/phases/lifecycle.ts, test/e2e-scenario/support-tests/e2e-phase-lifecycle.test.ts	outputContainsReadySandbox now parses output line-by-line, strips ANSI codes, and requires the first whitespace-delimited token to match the exact sandbox name before checking for a case-insensitive "Ready" word. Test case verifies the function ignores "Ready" lines from non-matching sandbox names and continues polling until the correct sandbox appears.
Marker file content validation tightening test/e2e-scenario/fixtures/phases/state-validation.ts, test/e2e-scenario/support-tests/e2e-phase-state-validation.test.ts	expectMarkerFileContent now enforces exact equality, removing prior trim-tolerance behavior. Test case verifies rejection when marker content contains unexpected leading/trailing whitespace.
Environment and workflow validation updates test/e2e-scenario/live/sandbox-rebuild.test.ts, tools/e2e-scenarios/workflow-boundary.mts	Test sandbox-rebuild backups now prefer process.env.HOME over os.homedir(). Workflow boundary validation extends to assert that DOCKERHUB_USERNAME and DOCKERHUB_TOKEN are unset in the Install OpenShell step run script.

Estimated code review effort

🎯 2 (Simple) | ⏱️ ~8 minutes

Possibly related PRs

• NVIDIA/NemoClaw#5333: Main PR that introduces the sandbox-rebuild-vitest scenario; this PR tightens assertions and validations on the same test infrastructure.
• NVIDIA/NemoClaw#5236: Also tightens E2E stdout parsing by stripping ANSI escape codes and matching exact sandbox names for readiness detection.

Suggested labels

area: e2e, area: sandbox, chore, v0.0.65

Suggested reviewers

• cv

Poem

🐇 Tightened assertions hop, hop, hop!
Sandbox names now match without a stop,
Whitespace trimmed from the fixture's chain,
Exact matches dance through the test terrain. ✨

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name	Status	Explanation	Resolution
Docstring Coverage	⚠️ Warning	Docstring coverage is 0.00% which is insufficient. The required threshold is 80.00%.	Write docstrings for the functions missing them to satisfy the coverage threshold.

✅ Passed checks (4 passed)

Check name	Status	Explanation
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Title check	✅ Passed	The pull request title clearly summarizes the main change: tightening sandbox rebuild helper assertions and workflow validation checks across multiple test files and configuration.
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.

_{✏️ Tip: You can configure your own custom pre-merge checks in the settings.}

✨ Finishing Touches

📝 Generate docstrings

• Create stacked PR
• Commit on current branch

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Commit unit tests in branch e2e-nit/sandbox-rebuild-review-cleanups

---

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

[github-actions]

E2E Advisor Recommendation

Required E2E: None
Optional E2E: sandbox-rebuild-vitest

Dispatch hint: sandbox-rebuild-vitest

Workflow run

Full advisor summary

E2E Recommendation Advisor

Base: origin/main
Head: HEAD
Confidence: high

Required E2E

• None. No merge-blocking E2E is required because the PR only changes E2E fixtures/tests and workflow-boundary validation tooling; it does not modify installer, onboarding, sandbox lifecycle implementation, credential handling implementation, network policy, inference routing, deployment, or shipped user-flow code.

Optional E2E

• sandbox-rebuild-vitest (high): Useful confidence check because the PR edits the sandbox-rebuild live test and its lifecycle/state-validation helpers. This job exercises real onboarding with Docker/OpenShell, nemoclaw <sandbox> rebuild --yes, exact sandbox Ready detection, marker preservation, registry metadata refresh, and rebuild-backup credential leak scanning.

New E2E recommendations

• None.

Dispatch hint

• Workflow: e2e-vitest-scenarios.yaml
• jobs input: sandbox-rebuild-vitest

[github-actions]

Vitest E2E Scenario Recommendation

Required Vitest E2E scenarios: sandbox-rebuild-vitest, e2e-scenarios-all
Optional Vitest E2E scenarios: None

Dispatch required Vitest E2E scenarios:

• gh workflow run e2e-vitest-scenarios.yaml --ref <pr-head-ref> --field jobs=sandbox-rebuild-vitest
• gh workflow run e2e-vitest-scenarios.yaml --ref <pr-head-ref>

Workflow run

Full Vitest E2E advisor summary

Vitest E2E Scenario Advisor

Base: origin/main
Head: HEAD
Confidence: high

Required Vitest E2E scenarios

• sandbox-rebuild-vitest: Focused free-standing Vitest job wired for changed live test test/e2e-scenario/live/sandbox-rebuild.test.ts.
  • Dispatch: gh workflow run e2e-vitest-scenarios.yaml --ref <pr-head-ref> --field jobs=sandbox-rebuild-vitest
• e2e-scenarios-all: The PR changes shared Vitest E2E phase fixtures and support coverage under test/e2e-scenario/fixtures plus scenario workflow-boundary validation. Per policy, shared fixture or workflow machinery changes require the full Vitest scenario fan-out; this also covers the wired sandbox-rebuild free-standing live test changed in this PR.
  • Dispatch: gh workflow run e2e-vitest-scenarios.yaml --ref <pr-head-ref>

Optional Vitest E2E scenarios

• None.

Relevant changed files

• test/e2e-scenario/fixtures/phases/lifecycle.ts
• test/e2e-scenario/fixtures/phases/state-validation.ts
• test/e2e-scenario/live/sandbox-rebuild.test.ts
• test/e2e-scenario/support-tests/e2e-phase-lifecycle.test.ts
• test/e2e-scenario/support-tests/e2e-phase-state-validation.test.ts
• tools/e2e-scenarios/workflow-boundary.mts

[github-actions]

PR Review Advisor

Findings: 0 needs attention, 0 worth checking, 0 nice ideas
Top item: No actionable findings

Consider writing more tests for

• **Runtime validation** — Validate sandbox-rebuild live readiness polling against real `openshell sandbox list` output to confirm the first-column exact-name parser matches the actual CLI format.. The helper-level changes are covered by support tests, and the workflow validator is exercised against the current workflow. Because one surface depends on real `openshell sandbox list` output and another protects workflow installer credential boundaries, targeted runtime/integration validation would further improve confidence.
• **Runtime validation** — Add a workflow-boundary mutation test that removes `-u DOCKERHUB_USERNAME` or `-u DOCKERHUB_TOKEN` from the sandbox-rebuild Install OpenShell step and expects `validateE2eVitestScenariosWorkflowBoundary` to report the missing scrub.. The helper-level changes are covered by support tests, and the workflow validator is exercised against the current workflow. Because one surface depends on real `openshell sandbox list` output and another protects workflow installer credential boundaries, targeted runtime/integration validation would further improve confidence.

Workflow run details

This is an automated advisory review. A human maintainer must make the final merge decision.