feat(cli): embed plugin registry#8
Draft
QMalcolm wants to merge 25 commits into
Draft
Conversation
The OSI converters currently require manual environment setup and
per-converter invocation. This lays the foundation for a unified CLI
(osi) that will discover and invoke converters as plugins via a
stdin/stdout JSON protocol.
Creates cli/ at the repo root as a self-contained Go module
(github.com/open-semantic-interchange/osi/cli). All commands are
stubbed — no logic is wired in this commit.
Design decisions:
- Go chosen for static binary distribution; no runtime dependency
for end users (brew/apt installable)
- internal/osidir owns ~/.osi/plugins/ initialization and respects
$OSI_PLUGIN_DIR for override; uses os.UserHomeDir() rather than
$HOME for Windows portability
- PersistentPreRunE on the root command ensures dir init runs before
every subcommand; commented caveat that Cobra does not chain this
automatically if a subcommand defines its own
- MarkFlagsMutuallyExclusive("from", "to") handles the both-set case
on osi convert; the neither-set case is validated manually in RunE
since Cobra only guards against both being provided
Build pipeline (Makefile, .goreleaser.yaml) and CI follow in
separate commits.
Enables local builds and cross-platform release artifacts for the OSI CLI. Makefile provides standard targets for day-to-day development: build, test, lint, release-dry-run, and clean. All targets are designed to run from within cli/. GoReleaser config targets linux/amd64, linux/arm64, darwin/amd64, darwin/arm64, and windows/amd64. windows/arm64 is excluded — no widely available CI runner and negligible current demand. CGO_ENABLED=0 is set for fully static binaries, enabling cross-compilation from any host without a C toolchain. Version, commit, and date are injected at build time via ldflags from the vars declared in main.go.
Runs go build, go vet, and go test on every push and pull request that touches cli/ or the workflow file itself. The paths filter prevents CLI changes from triggering unrelated workflows in this polyglot repo and vice versa. Go version is derived from go-version-file: cli/go.mod so the workflow automatically picks up any future toolchain bumps without a separate workflow edit. defaults.run.working-directory avoids repeating cd cli/ on every step. Cross-platform build testing is not included — CGO_ENABLED=0 means the linux/amd64 build is representative of all targets. GoReleaser snapshot builds are deferred to a future release workflow.
Covers the only logic in F1 that warrants testing: $OSI_PLUGIN_DIR env var override, default path construction via os.UserHomeDir(), directory creation, and idempotent re-invocation of EnsurePluginDir. t.Setenv is used throughout so env var mutations are automatically restored after each test. t.TempDir is used for filesystem tests so no cleanup is needed and tests are safe to run in parallel.
Project branding has changed from OSI to OSSIE. Updates all user-facing and internal references in the CLI: - Binary name: osi → ossie - Go module path: .../osi/cli → .../ossie/cli - Default plugin directory: ~/.osi → ~/.ossie - Environment variable: OSI_PLUGIN_DIR → OSSIE_PLUGIN_DIR - GoReleaser project name and archive ids - All command descriptions and flag help text - Output directory default: ./osi-output → ./ossie-output
Completes the OSI → OSSIE rename by updating the internal package directory, package declaration, and import reference in cmd/root.go.
The osi → ossie rename missed this unexported constant. The value (.ossie) was already correct; only the identifier name was stale.
The osi → ossie rename missed the gitignore entry, so the local build artifact cli/ossie would no longer be ignored by git.
File names were inconsistent with the package directory name (ossiedir/). Pure rename — no code changes.
Downstream segments (convert, plugin list, plugin install) all need a way to find and parse installed plugins. This adds the shared foundation: a typed Plugin struct and a Discover() function that scans ~/.ossie/plugins/ for valid plugin.yaml files. Design decisions: - internal/plugin accepts pluginsDir as a parameter rather than calling ossiedir.PluginDir() internally — keeps the package testable without env var manipulation - plugin.yaml supports both ossie_* (preferred) and osi_* (deprecated fallback) key families to avoid breaking plugins written before the OSI → OSSIE rename; ossie_* wins when both are present - Malformed plugin dirs emit a warning to an io.Writer and are skipped; only an unreadable plugins directory is a hard error, matching the behavior specified in the architecture doc - Non-directory entries in the plugins dir are silently skipped (no warning); non-existent dir returns nil, nil Caveats: - Symlinked plugin directories are not yet handled — os.ReadDir reports symlinks with ModeSymlink, not IsDir(); a TODO comment marks the location for a future follow-up - cmd/plugin/list.go wiring is deferred to P1, which also requires the embedded registry before the full table output can be rendered
go.mod declares go 1.22 so slices.Equal (stdlib since 1.21) is available. Removes the hand-rolled utility in favour of the standard library equivalent.
os.IsNotExist does not handle wrapped errors correctly. Replace both occurrences with errors.Is(err, os.ErrNotExist), the modern idiom since Go 1.13.
Replace hardcoded /this/path/does/not/exist with a path derived from t.TempDir(). The parent is guaranteed to exist by the test harness but the child subdir does not, making the fixture deterministically nonexistent without relying on filesystem assumptions.
Adds two cases: setup path is correctly parsed when present, and setup is empty string when the field is absent from plugin.yaml. The setup path is consumed by the install segment (P2) when running the plugin's setup script.
Unknown fields in plugin.yaml are silently ignored by yaml.Unmarshal. Adding a comment to make clear this is a deliberate forward- compatibility choice, not an oversight.
…nd invoke args ConvertConfig.ToOSI/FromOSI and the plugin.yaml fields to_osi/from_osi were left using the old brand name. This sweeps them to ossie consistently with the rest of the CLI rename. to_ossie/from_ossie are now the canonical yaml keys. to_osi/from_osi are kept as deprecated fallbacks in rawPlugin, using the same ossie-wins merge pattern already established for ossie_plugin_spec/osi_plugin_spec. The invoke-arg convention (to-ossie, from-ossie) is updated in all canonical test fixtures; the two backward-compat tests (TestDiscover_ossieKeyPreferred, TestDiscover_osiKeyFallback) intentionally retain to_osi/from_osi since they exercise the deprecated-key path.
The preemptive osi_* fallback fields (osi_plugin_spec, osi_spec_version, to_osi, from_osi) were added in anticipation of backward compatibility, but no plugins exist yet using that format — the rename is happening before any external consumers. Carrying dead compatibility code with no real users adds noise and keeps osi in the codebase. Drops all deprecated fields and merge logic from rawPlugin, simplifying validate() and toPlugin() back to direct field access. TestDiscover_ossieKeyPreferred and TestDiscover_osiKeyFallback (which existed solely to exercise the now-removed paths) are replaced with TestDiscover_unknownFieldsIgnored, which covers the lenient-YAML behavior (future spec fields are silently tolerated) that discover.go explicitly documents.
Wires plugin list to the discovery infrastructure added in F2. Reads from the resolved plugin directory, prints a NAME/PLATFORM/SPEC table via tabwriter, and falls back to a "no plugins installed" message when the directory is empty or missing. This is a minimal first pass — the full P1 spec calls for cross-referencing against the embedded plugin registry (installed vs. available, latest version, update indicator), which depends on registry embedding (F4) not yet implemented.
go install . names the binary after the module path's last segment (cli) rather than the intended ossie. make install uses go build -o directly into $GOPATH/bin with the correct binary name.
C1 (osi convert base) needs a subprocess layer to pipe requests to plugins and receive responses. This adds the Request/Response/Issue envelope types and an Invoke() function that owns the full lifecycle: marshal request to stdin, capture stdout, forward stderr, enforce context timeout, and decode the JSON response. Design decisions: - Invoke takes pluginDir and invoke []string directly rather than *Plugin — decouples the invocation layer from the discovery types and makes the function independently testable - cmd.Stdout assigned as *bytes.Buffer rather than using cmd.Output() — cmd.Output() is incompatible with a pre-assigned cmd.Stderr writer and would silently discard the caller's stderr destination - ctx.Err() checked after cmd.Run() for timeout detection — exec kills the process with SIGKILL and returns "signal: killed", not context.DeadlineExceeded, so the context must be checked explicitly - Issue.Path uses omitempty — an absent key is semantically distinct from an empty string; plugins that have no path omit the field - A non-empty Issues slice is NOT a Go error — the caller (C1) is responsible for inspecting severities and determining exit code Tests use the TestMain subprocess self-invocation pattern: the test binary re-invokes itself with GO_TEST_PLUGIN=1, enters the fake plugin branch in TestMain, and exits without running any tests. This avoids compiling a separate helper binary and works portably. Caveats: - File collection, output writing, issue rendering, and wiring into cmd/convert.go are deferred to C1
A Request with nil Files marshals as {"files":null} rather than
{"files":{}}. Plugins that iterate over the files object may crash
on a null value. Guard added at the top of Invoke(); test added to
verify the wire format is always an object.
An empty invoke slice would panic with an index out of range on invoke[0]. While F2 validation prevents this in normal usage, an explicit check with a descriptive error is safer and makes the failure mode obvious if the invariant is ever violated.
Adds cli/internal/registry/ with the plugin registry embedded into the binary via //go:embed. The registry is read at runtime with no network call required. Initial registry contains placeholder 0.1.0 entries for the five known platforms (dbt, gooddata, polaris, salesforce, snowflake). Checksums are marked sha256:placeholder pending real plugin packaging. Entries will be updated with real tags and checksums when converters are packaged as distributable OSSIE plugins. Wiring into cmd/plugin/list.go and cmd/plugin/install.go is deferred to P1 and P2 respectively.
- Add comment on EntryType constants noting they are consumed by P2 - Clarify why knownPlatforms is package-level (slices cannot be const) - Guard TestRegistry_Platforms_sorted against vacuous pass on empty slice
Replaces the basic installed-only listing with a full P1 implementation that cross-references installed plugins against the embedded registry. Output now shows NAME/STATUS/INSTALLED/LATEST for all registry-known platforms, with status values of installed, update available, or not installed. Community plugins (installed but absent from the registry) are printed in a separate section below. The previous no-plugins-installed short-circuit is replaced by a no-plugins-available message that fires only when the registry is empty and nothing is installed. Matching installed plugins to registry entries is done by Platform.Name, not by directory name, since the registry is keyed by platform.
This was referenced Jul 8, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
The big part of this PR is defining the protocol for the "plugin registry", i.e. how first party plugins are tracked. In addition we've added stubs for known first party plugins. Though these plugins are now "visible" in
ossie plugin list, they are not actually installable yet as we haven't yetScreen.Recording.2026-06-24.at.14.02.40.mov
Migrated from apache/ossie#156 following the project's move to the Apache Software Foundation.