Skip to content

Add minion profiles#144

Merged
isbm merged 54 commits intomasterfrom
isbm-set-traits-remote
Mar 18, 2026
Merged

Add minion profiles#144
isbm merged 54 commits intomasterfrom
isbm-set-traits-remote

Conversation

@isbm
Copy link
Member

@isbm isbm commented Mar 13, 2026

It happened! Minion are now has artefact profiles, allowing each minion to have only those modules that actually needed on the particular host. The whole thing is here: sysinspect profiles --help.

But for that to happen, another feature emerged:

sysinspect traits --set "foo:bar,baz:Toto" '*'
sysinspect --sync

Now it is possible to set/unset/reset traits remotely for all minions. It probably had to be done long time ago though... 😉

This PR also drops console transport via unix socket FIFO which is limited to the localhost only. Now it is using encrypted TCP socket (defaulted to localhost) with the ability to use Master console remotely from another machine.

  • Implementation
  • Documentation
  • Unit tests
  • Examples

@isbm isbm self-assigned this Mar 13, 2026
@isbm isbm added documentation Improvements or additions to documentation feature New feature or request labels Mar 13, 2026
@isbm isbm requested a review from Copilot March 14, 2026 12:40
@isbm isbm changed the title Allow setting traits remotely Add minion profiles Mar 14, 2026
Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR adds a new encrypted local console channel between sysinspect and sysmaster (replacing FIFO), and introduces remote management for master-managed static traits plus deployment profiles that constrain what artefacts minions sync.

Changes:

  • Add an RSA-bootstrapped + libsodium-encrypted TCP “console” transport for sysinspectsysmaster.
  • Implement master-managed trait updates (set/unset/reset) pushed to minions, and ensure minions report refreshed traits.
  • Introduce deployment profiles (profiles.index + *.profile) and enforce them during minion sync.

Reviewed changes

Copilot reviewed 32 out of 33 changed files in this pull request and generated 5 comments.

Show a summary per file
File Description
sysminion/src/minion.rs Handles new internal command for trait updates; refreshes traits and profiles on startup/sync.
sysmaster/src/master.rs Replaces FIFO command path with encrypted TCP console; adds profile and trait-update dispatch.
sysmaster/src/dataserv/fls.rs Ensures fileserver publishes profiles/ and profiles.index.
src/main.rs Switches CLI → master transport from FIFO to console TCP; adds traits and profile subcommands.
src/clidef.rs Defines new traits and profile CLI subcommands and selectors.
man/sysinspect.8.md Documents remote model execution, traits updates, profiles, and console-driven commands.
libsysproto/src/query.rs Adds new cluster command URIs for traits update and profile management.
libsysinspect/src/util/mod.rs Adds pad_visible helper for aligned ANSI-colored table output.
libsysinspect/src/traits/traits_ut.rs Adds unit tests for master-managed traits file + update operations + effective profiles.
libsysinspect/src/traits/systraits.rs Loads *.cfg traits with master.cfg applied last; tolerates header-only master.cfg.
libsysinspect/src/traits/mod.rs Adds TraitUpdateRequest, master.cfg helpers, and effective_profiles.
libsysinspect/src/lib.rs Exposes new console module.
libsysinspect/src/context/mod.rs Adds shared helpers for parsing key lists and profile console request payloads.
libsysinspect/src/console/mod.rs Implements encrypted console envelope, bootstrapping, sealing/unsealing, and key authorization.
libsysinspect/src/console/console_ut.rs Adds unit tests for console bootstrap and sealed payload roundtrip.
libsysinspect/src/cfg/mod.rs Includes mmconf unit tests module.
libsysinspect/src/cfg/mmconf_ut.rs Adds tests for console bind address defaults and overrides.
libsysinspect/src/cfg/mmconf.rs Adds console bind config, console key paths, minion profiles dir, and setter for fileserver port.
libsysinspect/Cargo.toml Adds sodiumoxide dependency for console encryption.
libmodpak/tests/profile_sync.rs Adds integration tests for profile-enforced minion sync behavior.
libmodpak/src/mpk_ut.rs Adds unit tests for profile index/profile merge + repo filtering behavior.
libmodpak/src/mpk.rs Introduces profile index/profile types and filtering (retain_profiles) in repo index.
libmodpak/src/lib_ut.rs Adds profile CRUD tests and effective profiles behavior tests.
libmodpak/src/lib.rs Downloads and applies deployment profiles during minion sync; uses shared pad_visible.
examples/profiles/tiny-lua.profile Adds example narrow Lua-only deployment profile.
examples/profiles/runtime-full.profile Adds example “full runtime” deployment profile.
examples/profiles/README.md Documents example deployment profiles and their format.
docs/tutorial/profiles_tutor.rst Adds a full tutorial for profiles creation/tagging/sync/verification.
docs/index.rst Adds the profiles tutorial to documentation index.
docs/global_config.rst Documents new console bind configuration and key material layout.
docs/genusage/systraits.rst Updates trait docs for master-managed master.cfg and new CLI flows.
docs/genusage/cli.rst Updates CLI docs for traits/profiles and profile data model.
Cargo.lock Records new dependency (sodiumoxide) in lockfile.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR introduces deployment profiles and remote static-trait management for minions, and replaces the old local FIFO console transport with an encrypted TCP console shared between sysinspect and sysmaster.

Changes:

  • Add minion profiles (via minion.profile) and profile-aware sync that filters modules/libraries per host.
  • Add remote set/unset/reset of master-managed static traits via sysinspect traits ....
  • Replace FIFO console with an RSA-bootstrapped + libsodium secretbox TCP console, and update CLI/man/docs accordingly.

Reviewed changes

Copilot reviewed 32 out of 33 changed files in this pull request and generated 5 comments.

Show a summary per file
File Description
sysminion/src/minion.rs Ensures master-managed traits file + profiles dir exist; logs effective profiles; supports traits-update cluster command.
sysmaster/src/master.rs Replaces FIFO with encrypted TCP console; adds online summary formatting and profile console operations; routes trait updates.
sysmaster/src/dataserv/fls.rs Ensures fileserver layout includes profiles root + creates a default profiles.index.
src/main.rs Switches CLI master-communication from FIFO to encrypted console; adds traits and profile subcommands.
src/clidef.rs Defines new traits and profile CLI subcommands and flags.
man/sysinspect.8.md Documents new console behavior, traits updates, and profile workflow in the manpage.
libsysproto/src/query.rs Adds new cluster command constants for traits updates and profile management.
libsysinspect/src/util/mod.rs Adds ANSI-aware pad_visible helper for aligned console/table output.
libsysinspect/src/traits/traits_ut.rs Adds unit tests for master-managed traits file creation, updates, and effective profile resolution.
libsysinspect/src/traits/systraits.rs Adjusts OS naming traits; loads master.cfg last; accepts header-only master traits file.
libsysinspect/src/traits/mod.rs Adds master-managed traits overlay helpers, profile resolution (effective_profiles), and trait update request type.
libsysinspect/src/lib.rs Exposes new console module.
libsysinspect/src/context/mod.rs Adds shared helpers for parsing context keys plus ProfileConsoleRequest payload type.
libsysinspect/src/console/mod.rs Implements encrypted console envelope/bootstrap + sealing/opening + key management and authorization checks.
libsysinspect/src/console/console_ut.rs Unit tests for console bootstrap, sealing roundtrips, and keypair recovery behavior.
libsysinspect/src/cfg/mod.rs Adds config unit test module wiring.
libsysinspect/src/cfg/mmconf_ut.rs Tests default/override behavior for new console bind/connect addresses.
libsysinspect/src/cfg/mmconf.rs Adds console bind config + default port 4203; adds profiles dir helpers; updates telemetry os.type.
libsysinspect/Cargo.toml Adds sodiumoxide dependency for encrypted console transport.
libmodpak/tests/profile_sync.rs Integration tests for profile-filtered sync, multi-profile union behavior, and missing-profile failure.
libmodpak/src/mpk_ut.rs Unit tests for profiles index/profile roundtrip and profile-based repo filtering.
libmodpak/src/mpk.rs Adds profiles index/profile types; adds repo filtering (retain_profiles) and module matching helpers.
libmodpak/src/lib_ut.rs Unit tests for profile CRUD, filename/index behavior, rendering, and effective profile name parsing.
libmodpak/src/lib.rs Adds master-side profile CRUD/list/show; minion sync downloads profiles + filters repo by effective profiles.
examples/profiles/tiny-lua.profile Example narrow profile allowing only Lua runtime + related libs.
examples/profiles/runtime-full.profile Example profile allowing multiple runtimes and their libraries.
examples/profiles/README.md Explains profile file format, naming, and selector behavior.
docs/tutorial/profiles_tutor.rst New tutorial walkthrough for creating/tagging/syncing/verifying deployment profiles.
docs/index.rst Links the new profiles tutorial in documentation index.
docs/global_config.rst Documents new master console bind IP/port and key material layout.
docs/genusage/systraits.rst Updates traits docs to include master-managed overlay file + remote trait updates + profile assignment.
docs/genusage/cli.rst Updates CLI docs for remote model runs, traits management, and deployment profiles.
Cargo.lock Locks new sodiumoxide dependency.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR introduces deployment profiles for minions (so each minion syncs only the modules/libraries it needs) and adds remote master-managed trait updates, while also replacing the FIFO console transport with an encrypted TCP console shared by sysinspect and sysmaster.

Changes:

  • Add profile CRUD + profile index (profiles.index) on the master, and enforce profiles during minion sync via minion.profile.
  • Add sysinspect traits remote update flow for master-managed static traits (master.cfg) and a corresponding minion command handler.
  • Replace FIFO-based master console with an encrypted TCP console transport (RSA bootstrap + libsodium secretbox), and update CLI/manpages/docs accordingly.

Reviewed changes

Copilot reviewed 32 out of 33 changed files in this pull request and generated 7 comments.

Show a summary per file
File Description
sysminion/src/minion.rs Ensures master-managed traits file exists, adds profiles dir creation/logging, and handles CLUSTER_TRAITS_UPDATE.
sysmaster/src/master.rs Implements encrypted TCP console server, adds online summary response, and profile console operations.
sysmaster/src/dataserv/fls.rs Adds profiles directory + initializes profiles.index on the fileserver root.
src/main.rs Switches CLI to console TCP transport; adds traits and profile subcommands client-side request building.
src/clidef.rs Defines CLI flags/options for traits and profile subcommands.
man/sysinspect.8.md Documents new usage forms and adds sections for remote models, traits, and profiles.
libsysproto/src/query.rs Adds command constants for traits update and profile operations.
libsysinspect/src/util/mod.rs Adds pad_visible helper for ANSI-aware column padding.
libsysinspect/src/traits/traits_ut.rs Adds unit tests for master.cfg creation, trait updates, and profile resolution.
libsysinspect/src/traits/systraits.rs Ensures master.cfg loads last; updates OS trait population and custom trait load logic.
libsysinspect/src/traits/mod.rs Adds master-managed traits helpers (master.cfg), trait update request type, and effective_profiles.
libsysinspect/src/lib.rs Exposes new console module publicly.
libsysinspect/src/context/mod.rs Adds shared context helpers and ProfileConsoleRequest payload type.
libsysinspect/src/console/mod.rs Introduces encrypted console transport primitives and key management utilities.
libsysinspect/src/console/console_ut.rs Adds tests for console bootstrap/sealing and keypair recovery.
libsysinspect/src/cfg/mod.rs Enables config unit tests module.
libsysinspect/src/cfg/mmconf_ut.rs Adds tests for console bind/connect defaults and overrides.
libsysinspect/src/cfg/mmconf.rs Adds console bind config, profiles dir roots, and console key paths; adds DEFAULT_CONSOLE_PORT.
libsysinspect/Cargo.toml Adds sodiumoxide dependency for console encryption.
libmodpak/tests/profile_sync.rs Adds integration tests covering profile-filtered sync and traversal rejection.
libmodpak/src/mpk_ut.rs Adds unit tests for profiles index/profile YAML and profile filtering behavior.
libmodpak/src/mpk.rs Adds profile and profiles index types; adds repo filtering and module matching helpers.
libmodpak/src/lib_ut.rs Adds tests for profile CRUD, casing behavior, traversal rejection, and profile rendering.
libmodpak/src/lib.rs Implements profiles index/profile file persistence, profile rendering, and minion-side profile sync + filtering.
examples/profiles/tiny-lua.profile Adds example narrow Lua-only profile.
examples/profiles/runtime-full.profile Adds example full runtime profile (lua/py3/wasm).
examples/profiles/README.md Documents example profile files and key format notes.
docs/tutorial/profiles_tutor.rst Adds full deployment profiles tutorial.
docs/index.rst Links the new profiles tutorial into docs index.
docs/global_config.rst Documents new console TCP bind configuration and key material locations.
docs/genusage/systraits.rst Updates traits documentation to include master.cfg semantics and remote traits/profile usage.
docs/genusage/cli.rst Expands CLI documentation for remote models, traits updates, and profiles.
Cargo.lock Updates dependency lockfile for new/updated crates.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

@isbm isbm merged commit 3676d57 into master Mar 18, 2026
6 checks passed
@isbm isbm deleted the isbm-set-traits-remote branch March 18, 2026 13:37
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

documentation Improvements or additions to documentation feature New feature or request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants