feat: add CI/GPU runner infrastructure migrated from FlowMesh_dev

[Qruixuan]

Purpose

Migrate the CI/GPU runner infrastructure from mlsys-io/FlowMesh_dev (ci/gpu-runner-setup-v2) and adapt it to FlowMesh's single-server architecture (no separate host/guardian split).

Changes

• docker/ci.compose.yml — isolated per-run Compose stack; server mounts the Docker socket to spawn workers via DooD; workers use network_mode: host to reach localhost:8000/50051
• docker/ci.ports.fixed.yml — fixed port overlay (8000, 50051) for CI and local runs
• docker/ci.worker.gpu.yml — GPU worker overlay wiring (CUDA devices, HF_TOKEN pass-through)
• docker/ci.gpu_worker_config.yaml — supervisor worker config for GPU CI worker
• scripts/ci/run_local.sh — mirrors the full GitHub Actions pipeline locally for pre-push validation; supports --gpu, --task-yaml, --timeout, --no-build, --keep
• .github/workflows/ci.yml — integration (CPU echo) + GPU smoke tests (vLLM, HF Transformers, LoRA SFT, DAG inference, SSH, conditional echo); actions pinned to commit SHAs, permissions: contents: read, template-expansion injections moved to step-level env:
• tests/integration/test_e2e.py — pytest E2E test that submits a workflow and polls until DONE; gracefully SKIPs on unavailable executors (max_attempts_exceeded + log pattern match)
• templates/n8n/dag_inference.json — replace gated meta-llama/Llama-3.2-1B-Instruct with open Qwen/Qwen2.5-0.5B-Instruct to avoid GatedRepoError when HF_TOKEN is absent

Design

Workers are spawned by the server's Docker adapter (DooD via /var/run/docker.sock) with network_mode: host, so they reach the server at localhost:8000 and localhost:50051 directly. The pytest runner executes in a temporary python:3.11-slim container joined to the compose network, accessing the server via service name http://server:8000. Each CI run is isolated by a unique compose project name (ci-$RUN_ID).

The GPU workflow uses a content-hash-tagged builder image (flowmesh-builder:<hash>) to cache the heavy CUDA dependency layer across runs, rebuilding only when Dockerfile.cuda.builder or requirements.gpu.txt change.

Test Plan

# CPU integration test
./scripts/ci/run_local.sh

# GPU smoke tests (all templates)
./scripts/ci/run_local.sh --gpu

# Single template override
./scripts/ci/run_local.sh --gpu --task-yaml templates/inference_vllm_tiny.yaml

Test Result

• CPU integration (echo task): PASSED
• GPU vLLM inference (TinyLlama-1.1B): PASSED after bumping timeout 300 → 600s to accommodate cold-start download + torch.compile (~247s total)
• GPU n8n DAG inference: PASSED after replacing gated Llama model with open Qwen model

---

Pre-submission Checklist

• I have read the contribution guidelines.
• I have run pre-commit run --all-files and fixed any issues.
• I have added or updated tests covering my changes (if applicable).
• I have verified that uv run pytest tests/ passes locally.
• If I changed shared schemas or proto definitions, I have checked downstream compatibility across Server and Worker.
• If I changed the SDK or CLI, I have verified the affected packages work (uv sync --all-extras --frozen).
• If this is a breaking change, I have prefixed the PR title with [BREAKING] and described migration steps above.
• I have updated documentation or config examples if user-facing behavior changed.

[timzsu]

@kaiitunnz This PR runs on self-hosted runners on bare metal, which is not secure. We are thinking about wrapping the runner inside a container, which might require DinD. Do you have any ideas on how to ensure security while keeping the CI simple?

[kaiitunnz]

@kaiitunnz This PR runs on self-hosted runners on bare metal, which is not secure. We are thinking about wrapping the runner inside a container, which might require DinD. Do you have any ideas on how to ensure security while keeping the CI simple?

I think with DinD, malicious workflows can still escape into the host. We need a better solution for this before the release.

[timzsu]

@kaiitunnz This PR runs on self-hosted runners on bare metal, which is not secure. We are thinking about wrapping the runner inside a container, which might require DinD. Do you have any ideas on how to ensure security while keeping the CI simple?

I think with DinD, malicious workflows can still escape into the host. We need a better solution for this before the release.

One alternative that allows docker inside container without privilege is sysbox. But sysbox does not have a stable GPU support.

Regarding the malicious workflow, we can make this e2e CI a daily workflow on main. Thus, we only need to ensure that the workflows in main is secure. Does that help?