Skip to content

test(e2e): retire covered smoke/onboarding shell entrypoints#5055

Closed
cv wants to merge 3 commits into
codex/e2e-fanout-03-smoke-onboarding-scenariosfrom
codex/e2e-fanout-04-retire-smoke-onboarding-shell
Closed

test(e2e): retire covered smoke/onboarding shell entrypoints#5055
cv wants to merge 3 commits into
codex/e2e-fanout-03-smoke-onboarding-scenariosfrom
codex/e2e-fanout-04-retire-smoke-onboarding-shell

Conversation

@cv

@cv cv commented Jun 9, 2026

Copy link
Copy Markdown
Collaborator

Summary

Implements the conservative retirement guard for the smoke/onboarding fan-out slice.

#5054 moved useful smoke/onboarding behavior into the registry-backed Vitest path, but it did not prove whole-script parity for the broad direct legacy test/e2e/test-*.sh scripts. This PR records that boundary explicitly instead of deleting coverage too early.

Related Issue

Refs #4941
Refs #4990
Refs #4348
Refs #4357
Depends on #5054.
Stacked on branch codex/e2e-fanout-03-smoke-onboarding-scenarios.

Changes

  • Marks the narrow legacy-onboarding-assertion-workers internal surface as covered by test/e2e-scenario/live/registry-scenarios.test.ts, while keeping deletionReady: false because the YAML/bash bridge and assertion registry still reference those shell files.
  • Keeps every direct smoke/onboarding legacy test/e2e/test-*.sh entry as not-migrated, deletionReady: false, with notes describing what test(e2e): migrate smoke and onboarding scenarios #5054 partially covered and what whole-script parity is still missing.
  • Adds inventory tests that preserve the direct smoke/onboarding scripts until whole-script parity exists and prove the onboarding assertion worker scripts remain present while bridge metadata still consumes them.
  • Documents the partial-coverage rule: a domain PR may record narrower Vitest coverage without turning a broad legacy entrypoint into a deletion-ready row.

Type of Change

  • Code change (feature, bug fix, or refactor)
  • Code change with doc updates
  • Doc only (prose changes, no code sample modifications)
  • Doc only (includes code sample changes)

Verification

  • npx vitest run --project e2e-scenario-framework test/e2e-scenario/framework-tests/e2e-migration-inventory.test.ts test/e2e-scenario/framework-tests/e2e-migration-inventory-lock.test.ts --silent=false --reporter=default
  • npx vitest run --project e2e-scenario-framework --silent=false --reporter=default
  • npm run typecheck:cli
  • npx prek run --files test/e2e-scenario/migration/legacy-inventory.json test/e2e-scenario/framework-tests/e2e-migration-inventory.test.ts test/e2e-scenario/docs/MIGRATION.md --skip test-cli
  • git diff --check
  • Tests added or updated for new or changed behavior
  • No secrets, API keys, or credentials committed
  • Docs updated for user-facing behavior changes
  • npm run docs builds without warnings (doc changes only)
  • Doc pages follow the style guide (doc changes only)
  • New doc pages include SPDX header and frontmatter (new pages only)

Signed-off-by: Carlos Villela cvillela@nvidia.com

Signed-off-by: Carlos Villela <cvillela@nvidia.com>
@cv cv self-assigned this Jun 9, 2026
@copy-pr-bot

copy-pr-bot Bot commented Jun 9, 2026

Copy link
Copy Markdown

Auto-sync is disabled for draft pull requests in this repository. Workflows must be run manually.

Contributors can view more details about this message here.

@cv cv added area: e2e End-to-end tests, nightly failures, or validation infrastructure area: onboarding Onboarding FSM, provider setup, sandbox launch, or first-run flow chore Build, CI, dependency, or tooling maintenance labels Jun 9, 2026
@coderabbitai

coderabbitai Bot commented Jun 9, 2026

Copy link
Copy Markdown
Contributor

Important

Review skipped

Auto reviews are disabled on base/target branches other than the default branch.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: f6c04f92-fd2f-4127-8ab4-218254f5bc24

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

  • 🔍 Trigger review
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch codex/e2e-fanout-04-retire-smoke-onboarding-shell

Comment @coderabbitai help to get the list of available commands and usage tips.

@github-actions

github-actions Bot commented Jun 9, 2026

Copy link
Copy Markdown
Contributor

E2E Advisor Recommendation

Required E2E: None
Optional E2E: None

Workflow run

Full advisor summary

E2E Recommendation Advisor

Base: origin/codex/e2e-fanout-03-smoke-onboarding-scenarios
Head: HEAD
Confidence: high

Required E2E

  • None. No live E2E is needed. This PR is docs/test/audit-inventory only: it tightens migration inventory assertions and updates deletion-readiness notes, without changing runtime code, scenario execution, installer/onboarding behavior, sandbox lifecycle, credentials, security boundaries, network policy, inference routing, deployment, or real assistant flows. Run the normal e2e-scenario-framework Vitest project as a non-live validation, but do not dispatch live E2E jobs.

Optional E2E

  • None.

New E2E recommendations

  • None.

@github-actions

github-actions Bot commented Jun 9, 2026

Copy link
Copy Markdown
Contributor

E2E Scenario Advisor Recommendation

Required scenario E2E: None
Optional scenario E2E: None

Workflow run

Full scenario advisor summary

E2E Scenario Advisor

Base: origin/codex/e2e-fanout-03-smoke-onboarding-scenarios
Head: HEAD
Confidence: high

Required scenario E2E

  • None. Changes are limited to scenario migration documentation, framework-test coverage for the migration inventory, and the deletion-gate inventory metadata. They do not modify scenario runtime/runner code, dispatch workflows, scenario catalog metadata, expected-state contracts, validation suite definitions, suite scripts, onboarding helpers, or live scenario behavior.

Optional scenario E2E

  • None.

Relevant changed files

  • test/e2e-scenario/docs/MIGRATION.md
  • test/e2e-scenario/framework-tests/e2e-migration-inventory.test.ts
  • test/e2e-scenario/migration/legacy-inventory.json

@github-actions

github-actions Bot commented Jun 9, 2026

Copy link
Copy Markdown
Contributor

PR Review Advisor

Findings: 0 needs attention, 0 worth checking, 0 nice ideas
Since last review: 1 prior item resolved, 0 still apply, 0 new items found

Consider writing more tests for

Workflow run details

This is an automated advisory review. A human maintainer must make the final merge decision.

cv added 2 commits June 9, 2026 15:03
…ding-scenarios' into codex/e2e-fanout-04-retire-smoke-onboarding-shell
Signed-off-by: Carlos Villela <cvillela@nvidia.com>
@copy-pr-bot

copy-pr-bot Bot commented Jun 9, 2026

Copy link
Copy Markdown

This pull request requires additional validation before any workflows can run on NVIDIA's runners.

Pull request vetters can view their responsibilities here.

Contributors can view more details about this message here.

@cv

cv commented Jun 10, 2026

Copy link
Copy Markdown
Collaborator Author

Closing as superseded by #5106 and the post-#5098 one-E2E migration plan.

This branch belongs to the pre-cutover fanout stack. Any useful helper/scenario work should come back as a fresh, focused draft PR from current main: Vitest as the only E2E harness, GitHub Actions as the matrix, no revived runner path, no long-lived legacy-inventory.json roadmap expansion, and replacement/deletion evidence carried in the PR body plus linked issue.

@cv cv closed this Jun 10, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area: e2e End-to-end tests, nightly failures, or validation infrastructure area: onboarding Onboarding FSM, provider setup, sandbox launch, or first-run flow chore Build, CI, dependency, or tooling maintenance

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants