Skip to content

feat(stack): add FLOWMESH_PLUGIN_DATA_DIR for plugin writable storage#53

Closed
kaiitunnz wants to merge 2 commits into
mainfrom
kaiitunnz/fix/hook-docs
Closed

feat(stack): add FLOWMESH_PLUGIN_DATA_DIR for plugin writable storage#53
kaiitunnz wants to merge 2 commits into
mainfrom
kaiitunnz/fix/hook-docs

Conversation

@kaiitunnz
Copy link
Copy Markdown
Collaborator

@kaiitunnz kaiitunnz commented May 20, 2026

Purpose

FLOWMESH_PLUGIN_DIR is deliberately read-only — it's the code mount — so plugins that need writable persistence (a SQLite ACL, a cache file, an ownership table) currently have to ask each operator to add a custom bind via docker-compose.override.yml. That's brittle and easy to forget; the immediate use case is lumid-flowmesh-plugin's ResourceRegistrar/PermissionChecker SQLite ACL.

Add a second mount at /app/plugin-data driven by FLOWMESH_PLUGIN_DATA_DIR. Three modes, single env var:

  • Default / path (./plugin-data, /var/lib/foo) — host bind, auto-created on stack up. Relative paths resolve against the operator's cwd (via STACK_PATH_KEYS).
  • Bare name (my_external_vol) — external Docker volume of that name; operator precreates with docker volume create. CLI routes the value through an internal FLOWMESH_PLUGIN_DATA_VOLUME parameter consumed by a parameterized external: true volume declaration.

The discrimination (path vs bare name) lives in apply_plugin_data_env in cli/stack/.../utils.py — same rule docker compose uses for short-syntax mount sources.

Folds in the pre-existing one-line docs fix to hook/README.md from the original branch (HookBindings is the Protocol; BaseBindings is the dataclass) since it's already on the branch.

Changes

  • cli/stack/.../assets/compose.yml — add the /app/plugin-data mount on the server service and a matching external: true volume declaration parameterized by FLOWMESH_PLUGIN_DATA_VOLUME.
  • cli/stack/.../env_schema.py + cli/stack/.../assets/.env.example (regenerated) — declare FLOWMESH_PLUGIN_DATA_DIR in the "External Plugins" section.
  • cli/stack/.../utils.pyapply_plugin_data_env(base_dir) discriminates the env value into path vs volume-name mode and routes accordingly. Called from _stack()'s _load hook in stack.py.
  • docs/ENV.md + docs/PLUGINS.md — document the three modes and that plugin code stays read-only while plugin data is writable.
  • hook/README.md — pre-existing one-line HookBindings vs BaseBindings correction (from the original branch).

Test Plan

uv run scripts/dev/check_env_examples.py            # .env.example regen is stable
uv run pytest tests/sdk/test_env_schema.py          # structural tests still pass

Manual verification via docker compose -f cli/stack/.../assets/compose.yml config across all three modes:

  • Empty / path default → service mount is a type: bind to <cwd>/plugin-data.
  • Custom path → type: bind to the operator-supplied path.
  • Bare name → type: volume, top-level declaration external: true, name: set to the operator-supplied value.

Live end-to-end with flowmesh stack up: not run — left for the matching lumid-flowmesh-plugin PR's e2e test plan (tmp/e2e_tests/lumid_plugin/test_lumid_plugin.md), which exercises this feature against a real workload.

Test Result

scripts/dev/check_env_examples.py: clean (no diff after --write).
pytest tests/sdk/test_env_schema.py: 17 passed.
docker compose config across the three modes: produces the expected bind / volume shapes as enumerated above.


Pre-submission Checklist
  • I have read the contribution guidelines.
  • I have run pre-commit run --all-files and fixed any issues.
  • I have added or updated tests covering my changes (if applicable).
  • I have verified that uv run pytest tests/ passes locally.
  • If I changed shared schemas or proto definitions, I have checked downstream compatibility across Server and Worker.
  • If I changed the SDK or CLI, I have verified the affected packages work (uv sync --all-packages --group ci --frozen).
  • If this is a breaking change, I have prefixed the PR title with [BREAKING] and described migration steps above.
  • I have updated documentation or config examples if user-facing behavior changed.

Signed-off-by: Noppanat Wadlom <noppanat.wad@gmail.com>
@kaiitunnz kaiitunnz force-pushed the kaiitunnz/fix/hook-docs branch from 7a5781d to d27c0f4 Compare May 20, 2026 06:55
@kaiitunnz kaiitunnz marked this pull request as ready for review May 20, 2026 06:56
@kaiitunnz kaiitunnz marked this pull request as draft May 20, 2026 06:56
FLOWMESH_PLUGIN_DIR is deliberately read-only — it's the code mount —
so plugins that need persistence (a SQLite ACL, a cache file) currently
have to ask each operator to add a custom bind via
docker-compose.override.yml, which is brittle.

Add a second mount at /app/plugin-data driven by FLOWMESH_PLUGIN_DATA_DIR.
A path value (the default, `./plugin-data`) is a host bind, auto-created
on stack up; a bare name is an external Docker volume of that name,
which the operator precreates and owns the lifecycle of. The CLI
discriminates path vs bare name in apply_plugin_data_env and routes the
value to either the service mount directly (path mode) or an internal
FLOWMESH_PLUGIN_DATA_VOLUME parameter consumed by a parameterized
external volume declaration (volume mode).

Signed-off-by: Noppanat Wadlom <noppanat.wad@gmail.com>
@kaiitunnz kaiitunnz closed this May 20, 2026
@kaiitunnz kaiitunnz deleted the kaiitunnz/fix/hook-docs branch May 20, 2026 14:16
@kaiitunnz kaiitunnz changed the title docs: fix flowmesh-hook README HookBindings vs BaseBindings feat(stack): add FLOWMESH_PLUGIN_DATA_DIR for plugin writable storage May 20, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant