test: add MCP stdio E2E test suite for vibration server#293
test: add MCP stdio E2E test suite for vibration server#293flamehaven01 wants to merge 2 commits into
Conversation
Six end-to-end scenarios over real MCP stdio transport. Key design decisions: - sys.executable: env-agnostic spawn (no PATH/venv dependency) - anyio.fail_after: propagates cancellation through anyio task groups, preventing zombie server processes on deadlock (FIND-010 regression guard) - _SENSITIVE_KEYS: strips LLM credentials from subprocess env before tests - OTEL_SDK_DISABLED=true: prevents concurrent JSONL write collisions under pytest-xdist parallel runs - vibration_session fixture: function-scoped for per-test process isolation Verified on commit a928284, Python 3.12.10: 6 passed in 18.54s. Signed-off-by: Flamehaven Initiative <flamehaven01@gmail.com>
0e1c192 to
29cdda4
Compare
Companion documentation for test_mcp_e2e.py explaining the five design decisions (sys.executable, anyio.fail_after, _SENSITIVE_KEYS, OTEL_SDK_DISABLED, vibration_session fixture) and extension patterns for adding E2E coverage to other MCP servers. Signed-off-by: Flamehaven Initiative <flamehaven01@gmail.com>
|
Thanks for the thorough write-up and the clear separation from the in-process tests in One location concern before merging: Could you move it to
Happy to take another look once it's moved. |
|
@flamehaven01 we provided feedback, appreciate your input. |
Summary
test_mcp_e2e.py: a pytest-native end-to-end test suite for the MCP stdio transport layer of the vibration serversrc/servers/vibration/tests/test_tools.pycallcall_tool(mcp, ...)in-process and never exercise the MCP wire protocolScenarios
list_known_bearingsreturns bearing DB without CouchDBassess_vibration_severityreturns correct ISO 10816 zones (A and D)diagnose_vibrationwith missingdata_id→Field required, no crashcompute_fft_spectrumwith unknowndata_id→ structured error dictanyio.fail_after(40s)— FIND-010 deadlock guardKey Engineering Decisions
sys.executableinstead ofcommand="python": resolves to the Python running pytest regardless of venv/conda/pyenv; avoidsModuleNotFoundErrorin subprocessanyio.fail_afterinstead ofasyncio.wait_for: propagates cancellation through anyio task groups (howstdio_clientmanages subprocess streams internally); terminates server subprocess on deadlock rather than leaving a zombie process_SENSITIVE_KEYSair-gap: strips LLM credentials (WATSONX_APIKEY,OPENAI_API_KEY, etc.) from subprocess environment before tests run; prevents accidental billable API calls if server logic changesOTEL_SDK_DISABLED=true: disables OTEL in subprocess; prevents concurrent JSONL write collisions underpytest -n auto(xdist parallel runs)vibration_sessionfixture: each test gets a fresh server subprocess, eliminating cross-test in-memory state leaksUsage
Verified On
a928284b06411ceb9e31663d936cf8b342b15ca2Related