Skip to content

Isolate browser-heavy tests in a sequential step#780

Open
mendral-app[bot] wants to merge 2 commits intomainfrom
mendral/isolate-browser-tests
Open

Isolate browser-heavy tests in a sequential step#780
mendral-app[bot] wants to merge 2 commits intomainfrom
mendral/isolate-browser-tests

Conversation

@mendral-app
Copy link
Copy Markdown

@mendral-app mendral-app Bot commented Apr 22, 2026

Summary

  • Exclude test_webvoyager_scripts.py and test_tools.py from the parallel pytest-xdist run (-n logical) and run them in a dedicated sequential step (-n 1) to prevent xdist worker crashes caused by multiple Chromium sessions exhausting runner memory.

Context

Recurring xdist worker crashes (node down: Not properly terminated) have been failing ~60% of main-branch CI pushes. The crashes occur when multiple browser-based tests run concurrently on a 4-vCPU runner, causing OOM kills of Chromium subprocesses. By isolating these browser-heavy tests into a sequential step, worker crashes no longer cascade to fail the entire suite.

Related insight: Recurring pytest-xdist worker crashes blocking main-branch CI

Changes

  • .github/workflows/test-cicd.yml: Added --ignore flags for tests/integration/test_webvoyager_scripts.py and tests/browser/test_tools.py to the parallel test step, and added a new "Run browser integration tests (sequential)" step that runs these files with -n 1.

Notes

This is a mitigation — the root cause (browser OOM under parallel load) should still be addressed by adding explicit session teardown in the tests or increasing runner resources.


Note

Created by Mendral. Tag @mendral-app with feedback or questions.

Greptile Summary

This PR isolates test_webvoyager_scripts.py and test_tools.py from the parallel pytest-xdist run and executes them in a dedicated sequential step (-n 1) to prevent OOM-induced worker crashes on CI. Coverage merging via --cov-append is correctly handled, and set -o pipefail is present in both steps.

Confidence Score: 5/5

Safe to merge — the change is a targeted CI fix with no production code impact.

The only finding is a P2 reporting gap (missing --junitxml in the sequential step) that doesn't affect test correctness or CI pass/fail behaviour. All other concerns from previous threads have been addressed.

No files require special attention.

Important Files Changed

Filename Overview
.github/workflows/test-cicd.yml Adds --ignore flags for browser-heavy tests to the parallel xdist step and introduces a new sequential step (-n 1) with --cov-append to run them without OOM risk; --junitxml is omitted from the sequential step so browser results won't appear in the JUnit XML report

Fix All in Claude Code

Prompt To Fix All With AI
This is a comment left during a code review.
Path: .github/workflows/test-cicd.yml
Line: 148-151

Comment:
**Missing `--junitxml` for browser test results**

The sequential step doesn't write to `pytest.xml`, so the "Pytest coverage comment" step (which reads `junitxml-path: ./pytest.xml`) won't include browser test pass/fail counts in the PR summary comment. CI will still fail correctly when these tests fail, but the reported test totals in the PR comment will be incomplete. Consider appending to a separate file or using the same path:

```suggestion
          uv run pytest -n 1 \
            tests/integration/test_webvoyager_scripts.py \
            tests/browser/test_tools.py \
            --durations=10 --cov-report=term-missing:skip-covered --cov=packages --cov-append --junitxml=pytest-browser.xml | tee -a pytest-coverage.txt
```

How can I resolve this? If you propose a fix, please make it concise.

Reviews (2): Last reviewed commit: "ci: add coverage flags to sequential bro..." | Re-trigger Greptile

Exclude test_webvoyager_scripts.py and test_tools.py from the parallel
pytest-xdist run and add a dedicated sequential step (-n 1) to prevent
worker crashes from OOM when multiple Chromium sessions run concurrently
on a 4-vCPU runner.
Comment thread .github/workflows/test-cicd.yml Outdated
Comment on lines +148 to +151
uv run pytest -n 1 \
tests/integration/test_webvoyager_scripts.py \
tests/browser/test_tools.py \
--durations=10 | tee -a pytest-coverage.txt
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 No coverage collected for browser tests

The sequential step omits --cov-report=term-missing:skip-covered --cov=packages, so it appends plain test-result text to pytest-coverage.txt but no actual coverage data. The "Pytest coverage comment" step won't reflect coverage from test_webvoyager_scripts.py or test_tools.py. If you want these tests included in the coverage report, add the same flags as the parallel step:

Suggested change
uv run pytest -n 1 \
tests/integration/test_webvoyager_scripts.py \
tests/browser/test_tools.py \
--durations=10 | tee -a pytest-coverage.txt
uv run pytest -n 1 \
tests/integration/test_webvoyager_scripts.py \
tests/browser/test_tools.py \
--durations=10 --cov-report=term-missing:skip-covered --cov=packages | tee -a pytest-coverage.txt

If intentionally skipping coverage for these slow browser tests, a comment in the workflow would clarify the intent.

Prompt To Fix With AI
This is a comment left during a code review.
Path: .github/workflows/test-cicd.yml
Line: 148-151

Comment:
**No coverage collected for browser tests**

The sequential step omits `--cov-report=term-missing:skip-covered --cov=packages`, so it appends plain test-result text to `pytest-coverage.txt` but no actual coverage data. The "Pytest coverage comment" step won't reflect coverage from `test_webvoyager_scripts.py` or `test_tools.py`. If you want these tests included in the coverage report, add the same flags as the parallel step:

```suggestion
          uv run pytest -n 1 \
            tests/integration/test_webvoyager_scripts.py \
            tests/browser/test_tools.py \
            --durations=10 --cov-report=term-missing:skip-covered --cov=packages | tee -a pytest-coverage.txt
```

If intentionally skipping coverage for these slow browser tests, a comment in the workflow would clarify the intent.

How can I resolve this? If you propose a fix, please make it concise.

Fix in Claude Code

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good catch — added --cov-report=term-missing:skip-covered --cov=packages --cov-append to the sequential step so browser test coverage is included in the report. The --cov-append flag ensures the data merges with the parallel step's coverage rather than overwriting it.

@mendral-app mendral-app Bot marked this pull request as ready for review April 22, 2026 22:17
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

0 participants