CI: run box/mcp/measurement/blufi unit suites (matrix); fix 3 CI-fragile box tests#103
CI: run box/mcp/measurement/blufi unit suites (matrix); fix 3 CI-fragile box tests#103danielrmerskine wants to merge 1 commit into
Conversation
There was a problem hiding this comment.
Code Review
This pull request introduces a unit test requirements file (test/requirements-unit.txt) and updates tests in test_box_authorize.py and test_monitor_state.py to mock shutil.which and additional protection-tripped methods. Feedback on these changes suggests pinning the newly added dependencies to specific versions to prevent CI failures and replacing the AGPL-licensed pymupdf with a permissively licensed alternative like pypdf to avoid licensing conflicts with the project's Apache-2.0 license.
Important
The consumer version of Gemini Code Assist on GitHub is being sunset. Starting June 18, 2026, new organization installations will be blocked, and all code review activity will officially cease on July 17, 2026.
For more details on the timeline and next steps, please review the Help Documentation.
| # Optional: unskips test/unit/test_pdf_pages.py (PDF text/page extraction). | ||
| pymupdf |
There was a problem hiding this comment.
PyMuPDF is licensed under the GNU AGPL v3, which is a strong copyleft license. Since this project is licensed under Apache-2.0, introducing an AGPL v3 dependency (even if optional or for testing) can create licensing compliance issues or copyleft concerns for downstream users and commercial deployments. Consider using a permissively licensed alternative like pypdf (BSD-3-Clause) or pdfminer.six (MIT) for PDF text and page extraction.
# Optional: unskips test/unit/test_pdf_pages.py (PDF text/page extraction).\npypdf
| numpy | ||
| simplejson | ||
| pyvisa | ||
| pyvisa-py | ||
| flask | ||
| pyserial | ||
| pyusb | ||
| cryptography | ||
| fastmcp | ||
| pexpect |
95ed8f1 to
bb0271d
Compare
…ile box tests
unit-tests.yml previously ran only test/unit/cli/ (pip install -e cli/), so
~1,190 already-written non-hardware tests — box-side debug/JTAG/RTT/gdbserver,
device locking, box config/provisioning, SSH, DUT, USB scanner, the MCP server,
measurement, and blufi — ran in no CI at all.
- unit-tests.yml: run each suite as its own ubuntu-latest matrix job
(cli, box, measurement, blufi, mcp, root). The suites cannot share a pytest
process (test/unit/measurement/conftest.py stubs `lager` in sys.modules,
which breaks box-suite collection), so each runs separately with
--import-mode=importlib -c /dev/null --timeout=60. PYTHONPATH is set to
repo-root + box/ so the `lager` package and the `test.*` package resolve
consistently in-process and in spawned subprocesses (without it, box
collection fails on Linux with `lager` resolving as a namespace package —
a Linux-only failure the never-run suite was hiding).
- test/requirements-unit.txt (new): the pure-Python deps needed to import the
box tree headless (numpy, simplejson, pyvisa[-py], flask, pyserial, pyusb,
cryptography, fastmcp, pexpect, pygdbmi, flask-socketio). pygdbmi and
flask-socketio are import-time deps of lager.debug / lager.http_handlers;
without them the box package half-imports and `lager` resolves as a namespace
on Linux, causing 7 collection errors (some box tests stub pygdbmi, but only
order-dependently). Hardware SDKs stay mocked. pymupdf is
intentionally excluded (AGPL-3.0 vs the project's Apache-2.0); test_pdf_pages
skips accordingly. Deps are unpinned to mirror the box's own runtime.
- Fix 3 CI-fragile box tests (test-only, no behavior change), all latent
because the suite never ran:
- test_box_authorize: the os.path.exists=False mock also defeated
shutil.which("ssh-keygen") (same os.path object), so the code raised
"ssh-keygen not found". Now also stubs shutil.which.
- test_monitor_state: get_monitor_state reads ocp/ovp_tripped from separate
*_is_tripped() methods; the test mocked only the *_value getters, so tripped
stayed False. Now also raises on the tripped getters, matching the
"falls back to None" intent.
- test_breakpoint_pause: the socket-console test SIGPIPE-killed the run
(exit 141) on headless Linux when the client disconnects mid-write. It now
ignores SIGPIPE for that test, exactly as the box does in production
(box_http_server sets SIG_IGN); the console already handles the resulting
BrokenPipeError.
Adds ~1,190 tests to PR gating.
bb0271d to
f876a35
Compare
Summary
Wire the ~1,190 already-written non-hardware unit tests into CI. Until now
unit-tests.ymlran onlytest/unit/cli/, so the box-side, MCP, measurement, and blufi unit suites — covering debug/JTAG/RTT/gdbserver, device locking, box config/provisioning, SSH, DUT, USB scanner, the MCP server, and more — executed in no CI at all.Changes
.github/workflows/unit-tests.yml: run each suite as its ownubuntu-latestmatrix job (cli,box,measurement,blufi,mcp,root). The suites can't share a pytest process —test/unit/measurement/conftest.pystubs thelagerpackage insys.modules(to avoid heavy box deps), which breaksbox/collection — so each runs separately with--import-mode=importlib -c /dev/null --timeout=60. The job also setsPYTHONPATH= repo-root +box/so thelagerpackage and thetest.*package resolve consistently in-process and in spawned subprocesses — without it,boxcollection fails on Linux withlagerresolving as a namespace package (a Linux-only failure the never-run suite hid; it passed on macOS).test/requirements-unit.txt(new): the pure-Python deps needed to import the box tree headless (numpy, simplejson, pyvisa[-py], flask, pyserial, pyusb, cryptography, fastmcp, pexpect, pygdbmi, flask-socketio).pygdbmiandflask-socketioare import-time deps oflager.debug/lager.http_handlers; without them the box package half-imports andlagerresolves as a namespace package on Linux → 7 collection errors (some box tests stubpygdbmi, but only order-dependently, so it was flaky). Hardware SDKs stay mocked by the conftests / importlib-by-path loading, so nothing here pulls a device driver.pymupdfis intentionally excluded (AGPL-3.0 vs this project's Apache-2.0);test_pdf_pages.pyskips without it. Deps are unpinned to mirror the box's own (apt/pip) runtime — happy to pin to known-good versions if preferred.test_box_authorize.py: theos.path.exists → Falsemock also defeatedshutil.which("ssh-keygen")(sameos.pathobject), so the code raised "ssh-keygen not found." Now also stubsshutil.which.test_monitor_state.py:get_monitor_statereadsocp/ovp_trippedfrom separate*_is_tripped()methods; the test only mocked the*_valuegetters, sotrippedstayedFalse. Now also raises on the tripped getters, matching the "falls back to None" intent.test_breakpoint_pause.py: the socket-console test SIGPIPE-killed the run (exit 141) on headless Linux when the client disconnects mid-write. It now ignores SIGPIPE for that test — exactly as the box does in production (box_http_serversetsSIG_IGN); the console already handles the resultingBrokenPipeError. (Passed on macOS locally, which ignores SIGPIPE by default — a Linux-only failure the never-run suite was hiding.)Coverage added
~1,190 tests now gate PRs (verified locally on a clean Python 3.11 env):
Testing
unit-testsrun is the on-CI proof.Notes / follow-ups
test/mcp/integration/still needs live hardware → stays on the self-hosted bench, not this ubuntu job.hardware_service._cleanup_device_cache("I/O operation on closed file") at interpreter shutdown — harmless (jobs pass); worth a small guard in a follow-up.test/requirements-unit.txtis a first cut of one.Checklist