test(e2e): migrate channels add remove scenario#5355
Merged
Conversation
Signed-off-by: Carlos Villela <cvillela@nvidia.com>
|
Auto-sync is disabled for draft pull requests in this repository. Workflows must be run manually. Contributors can view more details about this message here. |
Contributor
E2E Advisor RecommendationRequired E2E: None Full advisor summaryE2E Recommendation AdvisorFailed: Could not parse JSON from advisor output; see /home/runner/work/NemoClaw/NemoClaw/artifacts/e2e-advisor/e2e-advisor-raw-output.txt |
Contributor
Vitest E2E Scenario RecommendationRequired Vitest E2E scenarios: None Full Vitest E2E advisor summaryVitest E2E Scenario AdvisorFailed: Could not parse JSON from advisor output; see /home/runner/work/NemoClaw/NemoClaw/artifacts/e2e-advisor/e2e-scenario-advisor-raw-output.txt |
Contributor
Vitest E2E Scenario Results — ❌ Some jobs failedRun: 27436257859
|
Contributor
PR Review AdvisorFindings: 0 needs attention, 1 worth checking, 0 nice ideas Review findings🛠️ Needs attention
🔎 Worth checking
🌱 Nice ideas
Consider writing more tests for
This is an automated advisory review. A human maintainer must make the final merge decision. |
Signed-off-by: Carlos Villela <cvillela@nvidia.com>
Contributor
|
Note Reviews pausedIt looks like this branch is under active development. To avoid overwhelming you with review comments due to an influx of new commits, CodeRabbit has automatically paused this review. You can configure this behavior by changing the Use the following commands to manage reviews:
Use the checkboxes below for quick actions:
📝 WalkthroughWalkthroughAdds a Vitest live E2E scenario that exercises Telegram channel add→rebuild→probe→remove cycles, a GitHub Actions job to run it (with isolated Docker auth and artifact upload), workflow-boundary validation for the job, selector test coverage, and a small onboarding regex update. ChangesChannels Add/Remove E2E Vitest Scenario
Sequence Diagram(s)sequenceDiagram
participant TestRunner as Test Runner
participant NemoClaw as nemoclaw CLI
participant OpenClaw as OpenClaw registry
participant Sandbox as Sandbox (OpenShell)
participant TelegramAPI as api.telegram.org
TestRunner->>NemoClaw: channels add telegram
NemoClaw->>OpenClaw: update messaging plan / policy preset
TestRunner->>Sandbox: wait for readiness / verify openclaw.json
Sandbox->>TelegramAPI: egress probe (getMe)
alt probe allowed
TestRunner->>TestRunner: assert success
else probe denied
TestRunner->>TestRunner: fail test
else probe inconclusive
TestRunner->>TestRunner: write soft-skip artifact
end
TestRunner->>NemoClaw: channels remove telegram
NemoClaw->>OpenClaw: remove messaging plan entries
TestRunner->>Sandbox: cleanup
Estimated code review effort🎯 4 (Complex) | ⏱️ ~60 minutes Possibly related PRs
Suggested labels
Suggested reviewers
Poem
🚥 Pre-merge checks | ✅ 4 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (4 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. ✨ Finishing Touches🧪 Generate unit tests (beta)
Comment |
Contributor
Vitest E2E Scenario Results — ❌ Some jobs failedRun: 27437481385
|
Signed-off-by: Carlos Villela <cvillela@nvidia.com>
Contributor
Vitest E2E Scenario Results — ✅ All jobs passedRun: 27438223361
|
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
Contributor
There was a problem hiding this comment.
Actionable comments posted: 1
🧹 Nitpick comments (2)
tools/e2e-scenarios/workflow-boundary.mts (1)
1052-1186: 🔒 Security & Privacy | ⚡ Quick winLock the Docker-auth isolation contract in the new validator.
The workflow job isolates Docker credentials with a job-level
DOCKER_CONFIGand an always-running cleanup step, but this validator never checks either. If either piece drifts later, the boundary tests will still pass while the job starts reusing or leaving behind Docker auth state on the runner. Add assertions for theDOCKER_CONFIGvalue and theClean up Docker authstep (if: always(),docker logout,rm -rf "${DOCKER_CONFIG}").🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@tools/e2e-scenarios/workflow-boundary.mts` around lines 1052 - 1186, In validateChannelsAddRemoveVitestJob ensure the job-level DOCKER_CONFIG isolation and the cleanup step are asserted: check jobEnv.DOCKER_CONFIG equals the expected job-local path (e.g. "${{ github.workspace }}/.docker-e2e" or the project convention) and add a requireJobStep/assertion for the step named "Clean up Docker auth" verifying it has if: "always()", that its run contains "docker logout" and that its run removes the DOCKER_CONFIG directory (e.g. contains 'rm -rf "${DOCKER_CONFIG}"'); update references in the function validateChannelsAddRemoveVitestJob and to the step name "Clean up Docker auth" and use existing helpers (requireJobStep, requireRunContains) to implement these checks.test/e2e-scenario/live/channels-add-remove.test.ts (1)
342-506: 📐 Maintainability & Code Quality | ⚡ Quick winSplit this scenario callback into phase helpers.
This block now owns destructive pre-cleanup, onboarding, add/rebuild assertions, egress classification, remove/rebuild assertions, and final state checks in one function. Extracting phase helpers would keep the JS/TS complexity in line with the repo rule and make failures much easier to localize. As per coding guidelines, "Keep function complexity low in JavaScript and TypeScript code."
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@test/e2e-scenario/live/channels-add-remove.test.ts` around lines 342 - 506, The test callback is too large and should be split into phase helper functions: extract the destructive pre-cleanup steps (the bestEffort cleanup and pre-cleanup calls) into a preCleanup helper, the onboarding and initial readiness/assertions (onboard.from, expectSandboxReady, expectProvider, expectOpenClawTelegram, expectPolicyPreset) into an onboardPhase helper, the add flow (host.nemoclaw channels add, expectHostTelegramConfig, expectHostTelegramPlan, rebuild via host.nemoclaw rebuild, lifecycle.assertSandboxReadyAfterRebuild, expectPolicyPreset/expectOpenClawTelegram/expectProvider) into an addPhase helper, the egress check (telegramEgressProbe and its denied/inconclusive handling) into an egressPhase helper, and the remove flow (host.nemoclaw channels remove, expectHostTelegramPlan, rebuild, lifecycle.assertSandboxReadyAfterRebuild, final expectOpenClawTelegram/expectProvider/expectPolicyPreset/expectHostTelegramPlan) into a removePhase helper; then have the liveTest callback sequentially call preCleanup, onboardPhase, addPhase, egressPhase, and removePhase, passing required values (instance, SANDBOX_NAME, apiKey, secretsToRedact, artifacts, sandbox, host, lifecycle) so each helper owns its assertions and artifact names (use existing symbols like host.nemoclaw, lifecycle.assertSandboxReadyAfterRebuild, telegramEgressProbe, expectHostTelegramConfig, expectHostTelegramPlan to locate code to extract).Source: Coding guidelines
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@test/e2e-scenario/live/channels-add-remove.test.ts`:
- Around line 239-257: The absence check in expectProvider is too loose—when
expected === "absent" only a non-zero exit code is asserted; instead verify the
specific "provider not found" response (or confirm via a provider list) to
ensure Telegram was actually removed: update expectProvider (which calls
host.command("openshell", ["provider", "get", PROVIDER_NAME], ...)) so that in
the "absent" branch you assert the command output (stdout/stderr via
resultText(result)) contains the known not-found message (or alternatively run
"openshell provider list" and assert PROVIDER_NAME is not present) rather than
accepting any non-zero exitCode. Ensure you reference PROVIDER_NAME and use the
same host.command/result/resultText utilities to locate and validate the correct
message.
---
Nitpick comments:
In `@test/e2e-scenario/live/channels-add-remove.test.ts`:
- Around line 342-506: The test callback is too large and should be split into
phase helper functions: extract the destructive pre-cleanup steps (the
bestEffort cleanup and pre-cleanup calls) into a preCleanup helper, the
onboarding and initial readiness/assertions (onboard.from, expectSandboxReady,
expectProvider, expectOpenClawTelegram, expectPolicyPreset) into an onboardPhase
helper, the add flow (host.nemoclaw channels add, expectHostTelegramConfig,
expectHostTelegramPlan, rebuild via host.nemoclaw rebuild,
lifecycle.assertSandboxReadyAfterRebuild,
expectPolicyPreset/expectOpenClawTelegram/expectProvider) into an addPhase
helper, the egress check (telegramEgressProbe and its denied/inconclusive
handling) into an egressPhase helper, and the remove flow (host.nemoclaw
channels remove, expectHostTelegramPlan, rebuild,
lifecycle.assertSandboxReadyAfterRebuild, final
expectOpenClawTelegram/expectProvider/expectPolicyPreset/expectHostTelegramPlan)
into a removePhase helper; then have the liveTest callback sequentially call
preCleanup, onboardPhase, addPhase, egressPhase, and removePhase, passing
required values (instance, SANDBOX_NAME, apiKey, secretsToRedact, artifacts,
sandbox, host, lifecycle) so each helper owns its assertions and artifact names
(use existing symbols like host.nemoclaw,
lifecycle.assertSandboxReadyAfterRebuild, telegramEgressProbe,
expectHostTelegramConfig, expectHostTelegramPlan to locate code to extract).
In `@tools/e2e-scenarios/workflow-boundary.mts`:
- Around line 1052-1186: In validateChannelsAddRemoveVitestJob ensure the
job-level DOCKER_CONFIG isolation and the cleanup step are asserted: check
jobEnv.DOCKER_CONFIG equals the expected job-local path (e.g. "${{
github.workspace }}/.docker-e2e" or the project convention) and add a
requireJobStep/assertion for the step named "Clean up Docker auth" verifying it
has if: "always()", that its run contains "docker logout" and that its run
removes the DOCKER_CONFIG directory (e.g. contains 'rm -rf "${DOCKER_CONFIG}"');
update references in the function validateChannelsAddRemoveVitestJob and to the
step name "Clean up Docker auth" and use existing helpers (requireJobStep,
requireRunContains) to implement these checks.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: CHILL
Plan: Enterprise
Run ID: 69aafb52-5928-4eea-bda0-25a466b9e13d
📒 Files selected for processing (6)
.github/workflows/e2e-vitest-scenarios.yamltest/e2e-scenario/fixtures/phases/onboarding.tstest/e2e-scenario/live/channels-add-remove.test.tstest/e2e-scenario/support-tests/e2e-scenarios-workflow.test.tstools/e2e-scenarios/free-standing-jobs.envtools/e2e-scenarios/workflow-boundary.mts
Signed-off-by: Carlos Villela <cvillela@nvidia.com>
Contributor
Vitest E2E Scenario Results — ❌ Some jobs failedRun: 27444253651
|
Signed-off-by: Carlos Villela <cvillela@nvidia.com>
Contributor
Vitest E2E Scenario Results — ✅ All jobs passedRun: 27444477479
|
…dd-remove # Conflicts: # .github/workflows/e2e-vitest-scenarios.yaml # test/e2e-scenario/support-tests/e2e-scenarios-workflow.test.ts # tools/e2e-scenarios/workflow-boundary.mts
…dd-remove # Conflicts: # .github/workflows/e2e-vitest-scenarios.yaml # test/e2e-scenario/support-tests/e2e-scenarios-workflow.test.ts # tools/e2e-scenarios/workflow-boundary.mts
…dd-remove # Conflicts: # .github/workflows/e2e-vitest-scenarios.yaml # test/e2e-scenario/support-tests/e2e-scenarios-workflow.test.ts # tools/e2e-scenarios/workflow-boundary.mts
![github-advanced-security[bot]](https://avatars.githubusercontent.com/in/57789?s=60&v=4){kind=small}
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Migrates the legacy
test-channels-add-remove.shscenario into a live Vitest test while preserving the Telegram add/remove lifecycle contract, sandbox rebuild boundaries, and soft egress-probe behavior. Adds a focused free-standing workflow job so the scenario can be dispatched independently from the E2E scenario matrix.Related Issue
Related to #5098
Changes
test/e2e-scenario/live/channels-add-remove.test.tsfor the OpenClaw Telegram channel add/remove flow, including registry, provider, policy, rebuild, and post-removal assertions.channels-add-remove-vitestfree-standing workflow job and dispatch selector mapping forchannels-add-remove.Type of Change
Verification
npx prek run --all-filespassesnpm testpassesnpm run docsbuilds without warnings (doc changes only)Signed-off-by: Carlos Villela cvillela@nvidia.com
Summary by CodeRabbit
Tests
Chores