feat(stack): add FLOWMESH_PLUGIN_DATA_DIR for plugin writable storage#53
Closed
kaiitunnz wants to merge 2 commits into
Closed
feat(stack): add FLOWMESH_PLUGIN_DATA_DIR for plugin writable storage#53kaiitunnz wants to merge 2 commits into
kaiitunnz wants to merge 2 commits into
Conversation
Signed-off-by: Noppanat Wadlom <noppanat.wad@gmail.com>
7a5781d to
d27c0f4
Compare
FLOWMESH_PLUGIN_DIR is deliberately read-only — it's the code mount — so plugins that need persistence (a SQLite ACL, a cache file) currently have to ask each operator to add a custom bind via docker-compose.override.yml, which is brittle. Add a second mount at /app/plugin-data driven by FLOWMESH_PLUGIN_DATA_DIR. A path value (the default, `./plugin-data`) is a host bind, auto-created on stack up; a bare name is an external Docker volume of that name, which the operator precreates and owns the lifecycle of. The CLI discriminates path vs bare name in apply_plugin_data_env and routes the value to either the service mount directly (path mode) or an internal FLOWMESH_PLUGIN_DATA_VOLUME parameter consumed by a parameterized external volume declaration (volume mode). Signed-off-by: Noppanat Wadlom <noppanat.wad@gmail.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Purpose
FLOWMESH_PLUGIN_DIRis deliberately read-only — it's the code mount — so plugins that need writable persistence (a SQLite ACL, a cache file, an ownership table) currently have to ask each operator to add a custom bind viadocker-compose.override.yml. That's brittle and easy to forget; the immediate use case islumid-flowmesh-plugin'sResourceRegistrar/PermissionCheckerSQLite ACL.Add a second mount at
/app/plugin-datadriven byFLOWMESH_PLUGIN_DATA_DIR. Three modes, single env var:./plugin-data,/var/lib/foo) — host bind, auto-created onstack up. Relative paths resolve against the operator's cwd (viaSTACK_PATH_KEYS).my_external_vol) — external Docker volume of that name; operator precreates withdocker volume create. CLI routes the value through an internalFLOWMESH_PLUGIN_DATA_VOLUMEparameter consumed by a parameterizedexternal: truevolume declaration.The discrimination (path vs bare name) lives in
apply_plugin_data_envincli/stack/.../utils.py— same rule docker compose uses for short-syntax mount sources.Folds in the pre-existing one-line docs fix to
hook/README.mdfrom the original branch (HookBindingsis the Protocol;BaseBindingsis the dataclass) since it's already on the branch.Changes
cli/stack/.../assets/compose.yml— add the/app/plugin-datamount on the server service and a matchingexternal: truevolume declaration parameterized byFLOWMESH_PLUGIN_DATA_VOLUME.cli/stack/.../env_schema.py+cli/stack/.../assets/.env.example(regenerated) — declareFLOWMESH_PLUGIN_DATA_DIRin the "External Plugins" section.cli/stack/.../utils.py—apply_plugin_data_env(base_dir)discriminates the env value into path vs volume-name mode and routes accordingly. Called from_stack()'s_loadhook instack.py.docs/ENV.md+docs/PLUGINS.md— document the three modes and that plugin code stays read-only while plugin data is writable.hook/README.md— pre-existing one-lineHookBindingsvsBaseBindingscorrection (from the original branch).Test Plan
Manual verification via
docker compose -f cli/stack/.../assets/compose.yml configacross all three modes:type: bindto<cwd>/plugin-data.type: bindto the operator-supplied path.type: volume, top-level declarationexternal: true,name:set to the operator-supplied value.Live end-to-end with
flowmesh stack up: not run — left for the matchinglumid-flowmesh-pluginPR's e2e test plan (tmp/e2e_tests/lumid_plugin/test_lumid_plugin.md), which exercises this feature against a real workload.Test Result
scripts/dev/check_env_examples.py: clean (no diff after--write).pytest tests/sdk/test_env_schema.py: 17 passed.docker compose configacross the three modes: produces the expectedbind/volumeshapes as enumerated above.Pre-submission Checklist
pre-commit run --all-filesand fixed any issues.uv run pytest tests/passes locally.uv sync --all-packages --group ci --frozen).[BREAKING]and described migration steps above.