Skip to content
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
12 changes: 8 additions & 4 deletions cli/stack/src/flowmesh_cli_stack/assets/.env.example
Original file line number Diff line number Diff line change
Expand Up @@ -173,11 +173,15 @@ NEBULA_API_TOKEN=

# ==== External Plugins ====
# Plugins are Python packages dropped under FLOWMESH_PLUGIN_DIR
# (host-mounted to /app/plugins on the server) and selected by
# FLOWMESH_PLUGINS as a comma-separated list of top-level module
# names. Each named module must expose `install()` returning a
# `HookBindings`. Leave both empty unless you ship a plugin.
# (read-only at /app/plugins) and selected by FLOWMESH_PLUGINS as
# a comma-separated list of top-level module names. Each must
# expose `install()` returning a `HookBindings`.
# FLOWMESH_PLUGIN_DATA_DIR is writable at /app/plugin-data for
# plugin state. Leave all empty unless you ship a plugin.
FLOWMESH_PLUGIN_DIR=./plugins
# A path (`./x`, `/abs/x`) -> host bind-mount (auto-created).
# A bare name -> external Docker volume of that name.
FLOWMESH_PLUGIN_DATA_DIR=./plugin-data
FLOWMESH_PLUGINS=

# ==== Agent Executor (youtu-agent / utu) ====
Expand Down
4 changes: 4 additions & 0 deletions cli/stack/src/flowmesh_cli_stack/assets/compose.yml
Original file line number Diff line number Diff line change
Expand Up @@ -166,6 +166,7 @@ services:
- /var/run/docker.sock:/var/run/docker.sock
- ${SERVER_WORKER_CONFIG:-./configs/worker_config.yaml}:/etc/flowmesh/worker_config.yaml:ro
- ${FLOWMESH_PLUGIN_DIR:-./plugins}:/app/plugins:ro
- ${FLOWMESH_PLUGIN_DATA_DIR:-./plugin-data}:/app/plugin-data
- ${REDIS_TLS_DIR:-./secrets/tls/redis}:/etc/ssl/redis:ro
- ${SERVER_TLS_DIR:-./secrets/tls/server}:/etc/ssl/server:ro
restart: unless-stopped
Expand Down Expand Up @@ -199,3 +200,6 @@ volumes:
name: ${FLOWMESH_STACK_SLUG:-flowmesh_node}_metrics
flowmesh_server_logs:
name: ${FLOWMESH_STACK_SLUG:-flowmesh_node}_server_logs
flowmesh_plugin_data:
external: true
name: ${FLOWMESH_PLUGIN_DATA_VOLUME:-flowmesh_plugin_data}
18 changes: 14 additions & 4 deletions cli/stack/src/flowmesh_cli_stack/env_schema.py
Original file line number Diff line number Diff line change
Expand Up @@ -530,10 +530,11 @@
title="External Plugins",
description=[
"Plugins are Python packages dropped under FLOWMESH_PLUGIN_DIR ",
"(host-mounted to /app/plugins on the server) and selected by ",
"FLOWMESH_PLUGINS as a comma-separated list of top-level module ",
"names. Each named module must expose `install()` returning a ",
"`HookBindings`. Leave both empty unless you ship a plugin.",
"(read-only at /app/plugins) and selected by FLOWMESH_PLUGINS as ",
"a comma-separated list of top-level module names. Each must ",
"expose `install()` returning a `HookBindings`. ",
"FLOWMESH_PLUGIN_DATA_DIR is writable at /app/plugin-data for ",
"plugin state. Leave all empty unless you ship a plugin.",
],
vars=[
EnvVar(
Expand All @@ -543,6 +544,15 @@
use_default=True,
ensure_path="create",
),
EnvVar(
"FLOWMESH_PLUGIN_DATA_DIR",
"./plugin-data",
use_default=True,
description=[
"A path (`./x`, `/abs/x`) -> host bind-mount (auto-created).",
"A bare name -> external Docker volume of that name.",
],
),
EnvVar("FLOWMESH_PLUGINS", ""),
],
),
Expand Down
2 changes: 2 additions & 0 deletions cli/stack/src/flowmesh_cli_stack/stack.py
Original file line number Diff line number Diff line change
Expand Up @@ -35,6 +35,7 @@
from .utils import (
DEFAULT_ENV_FILE,
STACK_PATH_KEYS,
apply_plugin_data_env,
apply_stack_resource_env,
ensure_deploy_paths,
parse_node_role,
Expand All @@ -58,6 +59,7 @@ def _load(env_file: Path) -> None:
except ValueError as exc:
logging.error(str(exc))
raise typer.Exit(code=1)
apply_plugin_data_env(Path.cwd())

return DockerComposeStack(
compose_file=stack_compose_file(),
Expand Down
16 changes: 16 additions & 0 deletions cli/stack/src/flowmesh_cli_stack/utils.py
Original file line number Diff line number Diff line change
Expand Up @@ -57,6 +57,22 @@ def apply_stack_resource_env() -> None:
os.environ[WORKER_RESULTS_DIR_ENV] = results_volume


_PLUGIN_DATA_PATH_PREFIXES = ("/", "./", "../", "~/", "~")
_PLUGIN_DATA_ALIAS = "flowmesh_plugin_data"
_PLUGIN_DATA_DEFAULT = "./plugin-data"


def apply_plugin_data_env(base_dir: Path) -> None:
raw = os.environ.get("FLOWMESH_PLUGIN_DATA_DIR", "").strip()
if not raw or raw.startswith(_PLUGIN_DATA_PATH_PREFIXES):
resolved = resolve_path(raw, default=_PLUGIN_DATA_DEFAULT, base_dir=base_dir)
ensure_dir(resolved)
os.environ["FLOWMESH_PLUGIN_DATA_DIR"] = str(resolved)
else:
os.environ["FLOWMESH_PLUGIN_DATA_VOLUME"] = raw
os.environ["FLOWMESH_PLUGIN_DATA_DIR"] = _PLUGIN_DATA_ALIAS


def stack_compose_file() -> Path:
return asset_path("flowmesh_cli_stack.assets", "compose.yml")

Expand Down
1 change: 1 addition & 0 deletions docs/ENV.md
Original file line number Diff line number Diff line change
Expand Up @@ -35,6 +35,7 @@ listed here is in `.env.example`.
| `ENABLE_WORKER_WATCHDOG` | `true` | Worker death detection |
| `WORKER_DEATH_GRACE_SEC` | `60` | Grace period before marking dead |
| `FLOWMESH_PLUGINS` | – | Comma-separated plugin module names |
| `FLOWMESH_PLUGIN_DATA_DIR` | `./plugin-data` | Writable mount at `/app/plugin-data` for plugin state. A path -> host bind-mount (auto-created); a bare name -> external Docker volume of that name. |
| `SERVER_CUDA_PROBE_IMAGE` | `nvidia/cuda:12.9.1-base-ubuntu24.04` | CUDA image the server runs briefly to query local GPU names/indices |
| `DOCKER_GPU_RUNTIME` | nvidia | Optional Docker runtime name for GPU probe/worker containers; leave empty unless the host requires a named runtime such as `nvidia` |
| `FLOWMESH_API_KEY` | – | Forwarded to spawned workers as their server-callback bearer |
Expand Down
5 changes: 5 additions & 0 deletions docs/PLUGINS.md
Original file line number Diff line number Diff line change
Expand Up @@ -134,6 +134,11 @@ Each subdirectory of `FLOWMESH_PLUGIN_DIR` is importable as a
top-level module. The mount is read-only, so the plugin code is
treated as static deployment artifact.

For writable persistence, `FLOWMESH_PLUGIN_DATA_DIR` (default
`./plugin-data`) is mounted read-write at `/app/plugin-data`. A path
value is a host bind-mount (auto-created on `stack up`); a bare name
is an external Docker volume of that name.

This handles plugin **code** without rebuilding the server image.
When that isn't enough, build a thin overlay on top of the prebuilt
image. Two patterns, pick by need:
Expand Down
4 changes: 3 additions & 1 deletion hook/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,9 @@
FlowMesh-specific plugin extension surface. Carries the pieces FlowMesh adds
on top of [`lumid-hooks`](https://github.com/mlsys-io/lumid.hooks):

- `HookBindings` — concrete dataclass with FlowMesh's six fields (the five
- `HookBindings` — runtime-checkable Protocol extending the shared one with
`supplier_resolvers`, used by the server's plugin gate.
- `BaseBindings` — frozen dataclass with FlowMesh's six fields (the five
shared from `lumid-hooks` plus `supplier_resolvers`).
- `ResourceKind` / `ResourceAction` — FlowMesh resource and action enums.
- `SupplierResolver` / `WorkerView` — supplier attribution at dispatch time.
Expand Down
Loading