fix(ci): isolate test files into separate processes to stop kuzu SIGSEGV

Running the full suite in one process accumulated native runtimes (cocoindex + lancedb Tokio, kuzu scheduler, torch) that corrupted the heap, crashing kuzu's NodeTableScanState::scanNext with a SIGSEGV at ~53%. pytest-xdist --dist loadfile gives each test file a fresh worker process so no cross-file native state accumulates. Verified on real x86 CI: 771 passed / 9 skipped, no segfault.

Co-Authored-By: Claude <noreply@anthropic.com>

Author: HumanBean17

.github/workflows/test.yml

@@ -67,7 +67,13 @@ jobs:
           HF_TOKEN: ${{ secrets.HF_TOKEN }}
           TRANSFORMERS_CACHE: ~/.cache/huggingface
           HF_HOME: ~/.cache/huggingface
-        run: pytest tests -v
+        # Process isolation: run each test file in its own fresh worker process.
+        # Running the full suite in a single process accumulates native runtimes
+        # (cocoindex + lancedb Tokio, plus kuzu's scheduler and torch) that corrupt
+        # the heap, crashing kuzu's NodeTableScanState::scanNext with a SIGSEGV ~53%.
+        # `--dist loadfile` keeps each file in one worker, so no cross-file state
+        # accumulates. Verified: 771 passed / 9 skipped, no segfault.
+        run: pytest tests -n auto --dist loadfile -v
       - name: Skip tests (docs-only)
         if: steps.changes.outputs.code != 'true'
         run: echo "Docs-only change; pytest skipped."

pyproject.toml

@@ -43,6 +43,7 @@ dependencies = [
 dev = [
     "pytest>=7",
     "pytest-asyncio>=0.21",
+    "pytest-xdist>=3",
     "ruff>=0.4",
 ]