Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
26 commits
Select commit Hold shift + click to select a range
fc90b04
refactor: drop jsonl_export result echo and server rewriter
kaiitunnz May 18, 2026
69edc34
feat(shared): add typed executor result and artifact schemas
kaiitunnz May 18, 2026
def4085
refactor(shared): type ResultEnvelope.result and artifact helpers
kaiitunnz May 18, 2026
33785b6
refactor(worker): type non-inference, non-training executor results
kaiitunnz May 18, 2026
14b8aae
refactor(worker): type inference-family executor results
kaiitunnz May 18, 2026
08baf4d
refactor(worker): type training executor results
kaiitunnz May 18, 2026
4ad5153
test: cover BaseExecutorResult round-trip and fix subclass field seri…
kaiitunnz May 18, 2026
4fa6a43
docs: describe typed executor result schema
kaiitunnz May 18, 2026
9d03ecd
chore: satisfy pre-commit hooks (black, isort, mypy)
kaiitunnz May 18, 2026
cd5fd0c
refactor: tighten executor result typing end-to-end
kaiitunnz May 19, 2026
75083c9
refactor(worker): type upstream result and artifact resolver
kaiitunnz May 19, 2026
3b2fe17
refactor(worker): inline ArtifactRef constructor at call sites
kaiitunnz May 19, 2026
e541a62
refactor(server): return typed BaseExecutorResult from get_result
kaiitunnz May 19, 2026
62cfd96
refactor: unify sentinel attribute lookup in dispatcher and template …
kaiitunnz May 19, 2026
3c98358
refactor(sdk): port typed result and artifact schemas
kaiitunnz May 19, 2026
5762312
refactor(worker): construct executor results without post-hoc mutation
kaiitunnz May 19, 2026
f75aba8
docs: correct result schema references
kaiitunnz May 19, 2026
034a21a
fix(worker): retype omni summary fields as ArtifactRef
kaiitunnz May 19, 2026
210c6d2
refactor(worker): align lora/sft result schemas with peer training ex…
kaiitunnz May 19, 2026
909a45a
fix(worker): type upstream context in graph-template and data-retriev…
kaiitunnz May 20, 2026
0a42ae2
refactor: standardize ok across executor results and remove training_…
kaiitunnz May 20, 2026
9a24320
fix(shared): rename internal artifacts field to artifacts_ and lock a…
kaiitunnz May 20, 2026
bdbf74b
chore: tighten SDK pydantic pin to >=2.12.3
kaiitunnz May 20, 2026
48406cf
docs(worker): describe typed result contract in base_executor docstring
kaiitunnz May 20, 2026
2ef78e0
chore(security): ignore CVEs with no fix versions in pip-audit
kaiitunnz May 20, 2026
c99082e
chore(security): ignore vllm/gradio CVEs surfaced by GPU pip-audit step
kaiitunnz May 20, 2026
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
46 changes: 46 additions & 0 deletions .github/workflows/security.yml
Original file line number Diff line number Diff line change
Expand Up @@ -105,6 +105,28 @@ jobs:
--ignore-vuln GHSA-vfmq-68hx-4jfw \
--ignore-vuln GHSA-j7w6-vpvq-j3gm \
--ignore-vuln GHSA-98h9-4798-4q5v \
--ignore-vuln PYSEC-2025-189 \
--ignore-vuln PYSEC-2025-190 \
--ignore-vuln PYSEC-2025-191 \
--ignore-vuln PYSEC-2025-192 \
--ignore-vuln PYSEC-2025-193 \
--ignore-vuln PYSEC-2025-194 \
--ignore-vuln PYSEC-2025-195 \
--ignore-vuln PYSEC-2025-196 \
--ignore-vuln PYSEC-2025-197 \
--ignore-vuln PYSEC-2025-210 \
--ignore-vuln PYSEC-2026-139 \
--ignore-vuln PYSEC-2025-211 \
--ignore-vuln PYSEC-2025-212 \
--ignore-vuln PYSEC-2025-213 \
--ignore-vuln PYSEC-2025-214 \
--ignore-vuln PYSEC-2025-215 \
--ignore-vuln PYSEC-2025-216 \
--ignore-vuln PYSEC-2025-217 \
--ignore-vuln PYSEC-2025-218 \
--ignore-vuln PYSEC-2026-97 \
--ignore-vuln PYSEC-2025-183 \
--ignore-vuln PYSEC-2024-277 \
-r /tmp/requirements-worker-cpu-audit.txt
- name: Run pip-audit (worker GPU delta) # no --strict: flashinfer-jit-cache is unauditable on PyPI
run: |
Expand All @@ -130,4 +152,28 @@ jobs:
--ignore-vuln GHSA-83vm-p52w-f9pw \
--ignore-vuln GHSA-j7w6-vpvq-j3gm \
--ignore-vuln GHSA-98h9-4798-4q5v \
--ignore-vuln PYSEC-2025-189 \
--ignore-vuln PYSEC-2025-190 \
--ignore-vuln PYSEC-2025-191 \
--ignore-vuln PYSEC-2025-192 \
--ignore-vuln PYSEC-2025-193 \
--ignore-vuln PYSEC-2025-194 \
--ignore-vuln PYSEC-2025-195 \
--ignore-vuln PYSEC-2025-196 \
--ignore-vuln PYSEC-2025-197 \
--ignore-vuln PYSEC-2025-210 \
--ignore-vuln PYSEC-2026-139 \
--ignore-vuln PYSEC-2025-211 \
--ignore-vuln PYSEC-2025-212 \
--ignore-vuln PYSEC-2025-213 \
--ignore-vuln PYSEC-2025-214 \
--ignore-vuln PYSEC-2025-215 \
--ignore-vuln PYSEC-2025-216 \
--ignore-vuln PYSEC-2025-217 \
--ignore-vuln PYSEC-2025-218 \
--ignore-vuln PYSEC-2026-97 \
--ignore-vuln PYSEC-2025-183 \
--ignore-vuln PYSEC-2024-277 \
--ignore-vuln PYSEC-2025-222 \
--ignore-vuln PYSEC-2024-274 \
-r src/worker/requirements/requirements.gpu.txt
36 changes: 30 additions & 6 deletions docs/CODE_STYLE.md
Original file line number Diff line number Diff line change
Expand Up @@ -89,13 +89,13 @@ CI runs `pip-audit` against each generated requirements file
When pip-audit reports a new CVE, the only real fix is to bump the
offending dep in `pyproject.toml`, then `uv lock` and `uv run
scripts/dev/sync_requirements.py --write`. Silencing via `--ignore-vuln`
is a last resort; every silenced GHSA needs a written upgrade-blocker.
The currently-ignored advisories and the upgrade blocker that justifies
each are listed below; the same list is encoded as `--ignore-vuln`
flags in `.github/workflows/security.yml`.
is a last resort; every silenced advisory needs a written
upgrade-blocker. The currently-ignored advisories and the upgrade
blocker that justifies each are listed below; the same list is encoded
as `--ignore-vuln` flags in `.github/workflows/security.yml`.

| GHSA | Package | Fix version | Why ignored |
|------|---------|-------------|-------------|
| Advisory | Package | Fix version | Why ignored |
|----------|---------|-------------|-------------|
| GHSA-69w3-r845-3855 | transformers | 5.0.0rc3 | held by vllm/vllm-omni 0.18 compatibility |
| GHSA-pf3h-qjgv-vcpr | vllm | 0.19.0 | held by transformers 4.57 + adjacent inference deps |
| GHSA-pq5c-rjhq-qp7p | vllm | 0.19.0 | same |
Expand All @@ -117,6 +117,30 @@ flags in `.github/workflows/security.yml`.
| GHSA-w8v5-vhqr-4h9v | diskcache | (none) | upstream unmaintained, no fixed version published |
| GHSA-j7w6-vpvq-j3gm | diffusers | 0.38.0 | fix requires safetensors>=0.8.0rc0 pre-release; uv lock won't pick up pre-releases without explicit opt-in |
| GHSA-98h9-4798-4q5v | diffusers | 0.38.0 | same blocker as GHSA-j7w6-vpvq-j3gm — both fixed in 0.38.0 |
| PYSEC-2025-189 | torch | (none) | no fix version published |
| PYSEC-2025-190 | torch | (none) | same |
| PYSEC-2025-191 | torch | (none) | same |
| PYSEC-2025-192 | torch | (none) | same |
| PYSEC-2025-193 | torch | (none) | same |
| PYSEC-2025-194 | torch | (none) | same |
| PYSEC-2025-195 | torch | (none) | same |
| PYSEC-2025-196 | torch | (none) | same |
| PYSEC-2025-197 | torch | (none) | same |
| PYSEC-2025-210 | torch | (none) | same |
| PYSEC-2026-139 | torch | (none) | same |
| PYSEC-2025-211 | transformers | (none) | no fix version published; transformers also held by vllm-omni 0.18 |
| PYSEC-2025-212 | transformers | (none) | same |
| PYSEC-2025-213 | transformers | (none) | same |
| PYSEC-2025-214 | transformers | (none) | same |
| PYSEC-2025-215 | transformers | (none) | same |
| PYSEC-2025-216 | transformers | (none) | same |
| PYSEC-2025-217 | transformers | (none) | same |
| PYSEC-2025-218 | transformers | (none) | same |
| PYSEC-2026-97 | nltk | (none) | no fix version published |
| PYSEC-2025-183 | pyjwt | (none) | no fix version published |
| PYSEC-2024-277 | joblib | (none) | no fix version published |
| PYSEC-2025-222 | vllm | (none) | no fix version published; held by vllm-omni 0.18 pin |
| PYSEC-2024-274 | gradio | (none) | no fix version published; vllm-omni 0.18 pins gradio==5.50 |

When a blocker lifts (e.g. transformers 5 ↔ vllm 0.19 line stabilizes),
drop the corresponding `--ignore-vuln` flag from the workflow and the
Expand Down
20 changes: 20 additions & 0 deletions docs/EXECUTORS.md
Original file line number Diff line number Diff line change
Expand Up @@ -22,6 +22,26 @@ Helper utilities live in `src/worker/executors/utils/` (`artifacts`,
`src/worker/executors/mixins/` (`data`, `governance`, `inference`,
`training`).

## Result schema

Every executor's `run()` returns a subclass of `BaseExecutorResult`
(`src/shared/schemas/result.py`). The base class carries two
cross-cutting fields:

- `children: dict[str, BaseExecutorResult]` — per-child results when
merged tasks share a dispatch.
- `artifacts: ArtifactContext | None` (wire key `_artifacts`) —
resolution context for relative artifact refs.

Per-executor subclasses live next to the executor they describe — e.g.
`VLLMResult` in `src/worker/executors/vllm_executor.py`, `LoRAResult` in
`src/worker/executors/lora_sft_executor.py`. They add executor-specific
fields (`items`, `usage`, `final_lora`, `command`, …).

Artifact-bearing fields use `ArtifactRef` (`{"path": rel_path}`);
relative paths resolve against the producer's `_artifacts` context via
`artifact_to_source` / `_render_artifact_ref`.

## Agent executor (utu / youtu-agent)

`AgentExecutor` requires the following env vars to run; the executor
Expand Down
2 changes: 1 addition & 1 deletion sdk/pyproject.toml
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,7 @@ license-files = ["LICENSE"]
dependencies = [
"httpx>=0.27.0",
"pandas>=2.3.3",
"pydantic>=2.0.0",
"pydantic>=2.12.3",
"pyyaml>=6.0.0",
]

Expand Down
6 changes: 5 additions & 1 deletion sdk/src/flowmesh/models/__init__.py
Original file line number Diff line number Diff line change
@@ -1,5 +1,6 @@
"""FlowMesh SDK data models."""

from .artifacts import ArtifactContext, ArtifactRef
from .common import (
LogEntry,
LogEvent,
Expand All @@ -19,7 +20,7 @@
NodeWorkerInfo,
WorkerRegisterResponse,
)
from .results import PathResponse, ResultEnvelope
from .results import BaseExecutorResult, PathResponse, ResultEnvelope
from .tasks import HardwareUsage, TaskInfo, TaskUsage
from .traces import (
ActiveWaitBreakdown,
Expand Down Expand Up @@ -54,7 +55,10 @@

__all__ = [
"ActiveWaitBreakdown",
"ArtifactContext",
"ArtifactRef",
"AssetSummary",
"BaseExecutorResult",
"CPUInfo",
"CriticalPathSummary",
"E2EBreakdown",
Expand Down
10 changes: 10 additions & 0 deletions sdk/src/flowmesh/models/artifacts.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,10 @@
from pydantic import BaseModel, Field


class ArtifactRef(BaseModel):
path: str


class ArtifactContext(BaseModel):
base_dir: str
base_url: str | None = Field(default=None, exclude_if=lambda v: v is None)
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

exclude_if is a new feature since Pydantic v2.12.4 (https://pydantic.dev/docs/validation/latest/get-started/changelog/#v2124-2025-11-05). We need to bump the environment version pin.

Copy link
Copy Markdown
Collaborator Author

@kaiitunnz kaiitunnz May 20, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

exclude_if is actually available since Pydantic v2.12.0 (https://pydantic.dev/docs/validation/latest/get-started/changelog/#v2120b1-2025-10-03). The version you cited is a patch that refines exclude_if's behavior, so there is no need to bump.

In fact, we cannot bump because of the following dependency constraint: vllm-omni==0.18.0 → gradio==5.50 → pydantic≤2.12.3. .

Anyway, flowmesh-sdk's Pydantic pin still needs to be tightened.

29 changes: 27 additions & 2 deletions sdk/src/flowmesh/models/results.py
Original file line number Diff line number Diff line change
@@ -1,20 +1,45 @@
"""Result-related models."""

# This is necessary to allow for the recursive type hint of `children` in
# `BaseExecutorResult`.
from __future__ import annotations

from typing import Any

from pydantic import BaseModel, Field
from pydantic import BaseModel, ConfigDict, Field, SerializeAsAny

from .artifacts import ArtifactContext


class PathResponse(BaseModel):
ok: bool
path: str


class BaseExecutorResult(BaseModel):
model_config = ConfigDict(extra="allow", serialize_by_alias=True)

ok: bool = True
children: dict[str, SerializeAsAny[BaseExecutorResult]] = Field(
default_factory=dict, exclude_if=lambda v: not v
)
artifacts_: ArtifactContext | None = Field(default=None, alias="_artifacts")

@classmethod
def __pydantic_init_subclass__(cls, **kwargs: Any) -> None:
super().__pydantic_init_subclass__(**kwargs)
if "artifacts_" in cls.__annotations__:
raise TypeError(
f"{cls.__name__} may not redefine the internal "
"BaseExecutorResult.artifacts_ field"
)


class ResultEnvelope(BaseModel):
"""Canonical on-disk shape of ``results.json`` (mirrors the server)."""

task_id: str
result: dict[str, Any]
result: SerializeAsAny[BaseExecutorResult]
worker_id: str | None = None
metadata: dict[str, Any] | None = None
received_at: str | None = Field(default=None)
7 changes: 3 additions & 4 deletions sdk/src/flowmesh/resources/results.py
Original file line number Diff line number Diff line change
Expand Up @@ -198,10 +198,9 @@ def _finalize_materialize(
return {}, json_path, extracted

envelope = ResultEnvelope.model_validate_json(json_path.read_text())
if _wants_artifacts(sections):
ctx = envelope.result["_artifacts"]
ctx["base_dir"] = (output_dir / task_id).resolve().as_posix()
ctx.pop("base_url", None)
if _wants_artifacts(sections) and (ctx := envelope.result.artifacts_):
ctx.base_dir = (output_dir / task_id).resolve().as_posix()
ctx.base_url = None
payload = envelope.model_dump(mode="json")
json_path.write_text(json.dumps(payload, indent=2))
return payload, json_path, extracted
Expand Down
Loading
Loading