feat(cli): implement ossie convert base command#9
Draft
QMalcolm wants to merge 35 commits into
Draft
Conversation
The OSI converters currently require manual environment setup and
per-converter invocation. This lays the foundation for a unified CLI
(osi) that will discover and invoke converters as plugins via a
stdin/stdout JSON protocol.
Creates cli/ at the repo root as a self-contained Go module
(github.com/open-semantic-interchange/osi/cli). All commands are
stubbed — no logic is wired in this commit.
Design decisions:
- Go chosen for static binary distribution; no runtime dependency
for end users (brew/apt installable)
- internal/osidir owns ~/.osi/plugins/ initialization and respects
$OSI_PLUGIN_DIR for override; uses os.UserHomeDir() rather than
$HOME for Windows portability
- PersistentPreRunE on the root command ensures dir init runs before
every subcommand; commented caveat that Cobra does not chain this
automatically if a subcommand defines its own
- MarkFlagsMutuallyExclusive("from", "to") handles the both-set case
on osi convert; the neither-set case is validated manually in RunE
since Cobra only guards against both being provided
Build pipeline (Makefile, .goreleaser.yaml) and CI follow in
separate commits.
Enables local builds and cross-platform release artifacts for the OSI CLI. Makefile provides standard targets for day-to-day development: build, test, lint, release-dry-run, and clean. All targets are designed to run from within cli/. GoReleaser config targets linux/amd64, linux/arm64, darwin/amd64, darwin/arm64, and windows/amd64. windows/arm64 is excluded — no widely available CI runner and negligible current demand. CGO_ENABLED=0 is set for fully static binaries, enabling cross-compilation from any host without a C toolchain. Version, commit, and date are injected at build time via ldflags from the vars declared in main.go.
Runs go build, go vet, and go test on every push and pull request that touches cli/ or the workflow file itself. The paths filter prevents CLI changes from triggering unrelated workflows in this polyglot repo and vice versa. Go version is derived from go-version-file: cli/go.mod so the workflow automatically picks up any future toolchain bumps without a separate workflow edit. defaults.run.working-directory avoids repeating cd cli/ on every step. Cross-platform build testing is not included — CGO_ENABLED=0 means the linux/amd64 build is representative of all targets. GoReleaser snapshot builds are deferred to a future release workflow.
Covers the only logic in F1 that warrants testing: $OSI_PLUGIN_DIR env var override, default path construction via os.UserHomeDir(), directory creation, and idempotent re-invocation of EnsurePluginDir. t.Setenv is used throughout so env var mutations are automatically restored after each test. t.TempDir is used for filesystem tests so no cleanup is needed and tests are safe to run in parallel.
Project branding has changed from OSI to OSSIE. Updates all user-facing and internal references in the CLI: - Binary name: osi → ossie - Go module path: .../osi/cli → .../ossie/cli - Default plugin directory: ~/.osi → ~/.ossie - Environment variable: OSI_PLUGIN_DIR → OSSIE_PLUGIN_DIR - GoReleaser project name and archive ids - All command descriptions and flag help text - Output directory default: ./osi-output → ./ossie-output
Completes the OSI → OSSIE rename by updating the internal package directory, package declaration, and import reference in cmd/root.go.
The osi → ossie rename missed this unexported constant. The value (.ossie) was already correct; only the identifier name was stale.
The osi → ossie rename missed the gitignore entry, so the local build artifact cli/ossie would no longer be ignored by git.
File names were inconsistent with the package directory name (ossiedir/). Pure rename — no code changes.
Downstream segments (convert, plugin list, plugin install) all need a way to find and parse installed plugins. This adds the shared foundation: a typed Plugin struct and a Discover() function that scans ~/.ossie/plugins/ for valid plugin.yaml files. Design decisions: - internal/plugin accepts pluginsDir as a parameter rather than calling ossiedir.PluginDir() internally — keeps the package testable without env var manipulation - plugin.yaml supports both ossie_* (preferred) and osi_* (deprecated fallback) key families to avoid breaking plugins written before the OSI → OSSIE rename; ossie_* wins when both are present - Malformed plugin dirs emit a warning to an io.Writer and are skipped; only an unreadable plugins directory is a hard error, matching the behavior specified in the architecture doc - Non-directory entries in the plugins dir are silently skipped (no warning); non-existent dir returns nil, nil Caveats: - Symlinked plugin directories are not yet handled — os.ReadDir reports symlinks with ModeSymlink, not IsDir(); a TODO comment marks the location for a future follow-up - cmd/plugin/list.go wiring is deferred to P1, which also requires the embedded registry before the full table output can be rendered
go.mod declares go 1.22 so slices.Equal (stdlib since 1.21) is available. Removes the hand-rolled utility in favour of the standard library equivalent.
os.IsNotExist does not handle wrapped errors correctly. Replace both occurrences with errors.Is(err, os.ErrNotExist), the modern idiom since Go 1.13.
Replace hardcoded /this/path/does/not/exist with a path derived from t.TempDir(). The parent is guaranteed to exist by the test harness but the child subdir does not, making the fixture deterministically nonexistent without relying on filesystem assumptions.
Adds two cases: setup path is correctly parsed when present, and setup is empty string when the field is absent from plugin.yaml. The setup path is consumed by the install segment (P2) when running the plugin's setup script.
Unknown fields in plugin.yaml are silently ignored by yaml.Unmarshal. Adding a comment to make clear this is a deliberate forward- compatibility choice, not an oversight.
…nd invoke args ConvertConfig.ToOSI/FromOSI and the plugin.yaml fields to_osi/from_osi were left using the old brand name. This sweeps them to ossie consistently with the rest of the CLI rename. to_ossie/from_ossie are now the canonical yaml keys. to_osi/from_osi are kept as deprecated fallbacks in rawPlugin, using the same ossie-wins merge pattern already established for ossie_plugin_spec/osi_plugin_spec. The invoke-arg convention (to-ossie, from-ossie) is updated in all canonical test fixtures; the two backward-compat tests (TestDiscover_ossieKeyPreferred, TestDiscover_osiKeyFallback) intentionally retain to_osi/from_osi since they exercise the deprecated-key path.
The preemptive osi_* fallback fields (osi_plugin_spec, osi_spec_version, to_osi, from_osi) were added in anticipation of backward compatibility, but no plugins exist yet using that format — the rename is happening before any external consumers. Carrying dead compatibility code with no real users adds noise and keeps osi in the codebase. Drops all deprecated fields and merge logic from rawPlugin, simplifying validate() and toPlugin() back to direct field access. TestDiscover_ossieKeyPreferred and TestDiscover_osiKeyFallback (which existed solely to exercise the now-removed paths) are replaced with TestDiscover_unknownFieldsIgnored, which covers the lenient-YAML behavior (future spec fields are silently tolerated) that discover.go explicitly documents.
Wires plugin list to the discovery infrastructure added in F2. Reads from the resolved plugin directory, prints a NAME/PLATFORM/SPEC table via tabwriter, and falls back to a "no plugins installed" message when the directory is empty or missing. This is a minimal first pass — the full P1 spec calls for cross-referencing against the embedded plugin registry (installed vs. available, latest version, update indicator), which depends on registry embedding (F4) not yet implemented.
go install . names the binary after the module path's last segment (cli) rather than the intended ossie. make install uses go build -o directly into $GOPATH/bin with the correct binary name.
C1 (osi convert base) needs a subprocess layer to pipe requests to plugins and receive responses. This adds the Request/Response/Issue envelope types and an Invoke() function that owns the full lifecycle: marshal request to stdin, capture stdout, forward stderr, enforce context timeout, and decode the JSON response. Design decisions: - Invoke takes pluginDir and invoke []string directly rather than *Plugin — decouples the invocation layer from the discovery types and makes the function independently testable - cmd.Stdout assigned as *bytes.Buffer rather than using cmd.Output() — cmd.Output() is incompatible with a pre-assigned cmd.Stderr writer and would silently discard the caller's stderr destination - ctx.Err() checked after cmd.Run() for timeout detection — exec kills the process with SIGKILL and returns "signal: killed", not context.DeadlineExceeded, so the context must be checked explicitly - Issue.Path uses omitempty — an absent key is semantically distinct from an empty string; plugins that have no path omit the field - A non-empty Issues slice is NOT a Go error — the caller (C1) is responsible for inspecting severities and determining exit code Tests use the TestMain subprocess self-invocation pattern: the test binary re-invokes itself with GO_TEST_PLUGIN=1, enters the fake plugin branch in TestMain, and exits without running any tests. This avoids compiling a separate helper binary and works portably. Caveats: - File collection, output writing, issue rendering, and wiring into cmd/convert.go are deferred to C1
A Request with nil Files marshals as {"files":null} rather than
{"files":{}}. Plugins that iterate over the files object may crash
on a null value. Guard added at the top of Invoke(); test added to
verify the wire format is always an object.
An empty invoke slice would panic with an index out of range on invoke[0]. While F2 validation prevents this in normal usage, an explicit check with a descriptive error is safer and makes the failure mode obvious if the invariant is ever violated.
Adds cli/internal/registry/ with the plugin registry embedded into the binary via //go:embed. The registry is read at runtime with no network call required. Initial registry contains placeholder 0.1.0 entries for the five known platforms (dbt, gooddata, polaris, salesforce, snowflake). Checksums are marked sha256:placeholder pending real plugin packaging. Entries will be updated with real tags and checksums when converters are packaged as distributable OSSIE plugins. Wiring into cmd/plugin/list.go and cmd/plugin/install.go is deferred to P1 and P2 respectively.
- Add comment on EntryType constants noting they are consumed by P2 - Clarify why knownPlatforms is package-level (slices cannot be const) - Guard TestRegistry_Platforms_sorted against vacuous pass on empty slice
Replaces the basic installed-only listing with a full P1 implementation that cross-references installed plugins against the embedded registry. Output now shows NAME/STATUS/INSTALLED/LATEST for all registry-known platforms, with status values of installed, update available, or not installed. Community plugins (installed but absent from the registry) are printed in a separate section below. The previous no-plugins-installed short-circuit is replaced by a no-plugins-available message that fires only when the registry is empty and nothing is installed. Matching installed plugins to registry entries is done by Platform.Name, not by directory name, since the registry is keyed by platform.
CollectFiles walks an input file or directory, filters by accepted extensions, enforces a max-size limit, and returns a map of relative-path keys to file content (forward slashes on all platforms). Single-file inputs bypass the extension filter — the user explicitly chose the file, so the plugin's Accepts list is not applied. Empty accepts means all files are included (used by from_osi which has no Accepts field). LoadPlugin exports the existing unexported loadPlugin, used when --plugin bypasses name-based discovery.
Adds cli/internal/schema/ with the OSI JSON schema embedded via //go:embed. Validate() accepts YAML or JSON, round-trips through encoding/json to normalise yaml.v3 types to what jsonschema/v6 expects, then validates against the compiled schema. The schema is compiled once in init() — a panic here indicates a build defect (corrupted embed), not a runtime error. New dependency: github.com/santhosh-tekuri/jsonschema/v6 v6.0.2 (draft 2020-12 support, matching the schema's $schema declaration).
Wires together file collection (F2), plugin invocation (F3), schema validation, and output writing into a working end-to-end convert command. Flow: resolve plugin by name or --plugin path → collect input files → invoke plugin with timeout context → validate to_osi output against embedded OSI schema → write passing files to ossie-output/ → render issues to stderr → exit non-zero on any error-severity issue. Schema validation failures remove the offending file from output (rather than writing it with a warning) — the spec says validate before writing. Info-severity issues are suppressed unless --verbose. parseMaxInputSize supports MB/GB suffixes (case-insensitive) and raw byte counts. renderIssues formats each issue as: severity: message severity: path: message (when path is present)
cli/internal/schema/osi-schema.json was a committed copy of core-spec/osi-schema.json that could drift silently. The file is now gitignored and generated by `make prepare`, which copies from the canonical source before any build or test step. CI switches from bare go commands to make targets so the copy step runs automatically in all environments.
Empty accepts means no extension filter — all files in the directory are included. The godoc stated this but it was untested.
If all to_osi output files fail schema validation, resp.Files is empty. Creating the output directory in that case leaves an empty ossie-output/ on disk with no explanation. Guard os.MkdirAll behind len(resp.Files) > 0 so the directory only appears when there is something to write.
Consistent with Go error message conventions and the rest of the codebase. Single quotes were a shell quoting style.
runConvert was a single function with 11 numbered sections. Extract sections 2, 3, 9, and 10 into named functions so the orchestrator reads as a clean sequence of calls: resolvePlugin — discover or load plugin by path/name selectDirection — map --from/--to to direction + Direction struct validateOutput — schema-validate to_osi files, remove invalid ones writeOutput — create outDir and write response files to disk Sections 4 (parseMaxInputSize), 5 (CollectFiles), 8 (Invoke), and 11 (renderIssues) were already named functions. Sections 1, 6, and 7 are too small to warrant extraction.
selectDirection and validateOutput were written against the pre-rename field names (ToOSI/FromOSI, to_osi/from_osi). The direction string mismatch in validateOutput was a silent bug — output validation was never reached because "to_ossie" != "to_osi".
CollectFiles was doing two distinct things — single-file read and directory walk — in one flat function body. Extracting the walk into collectDir makes each path easier to follow and test independently.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Implement the convert base command. This means that convert collects the input files, invokes the conversion command for a specified plugin using utilizing the invocation protocol, writes out the return data to files, and logs an warnings/errors returned by the conversion script.
Screen.Recording.2026-06-25.at.16.24.40.mov
Migrated from apache/ossie#158 following the project's move to the Apache Software Foundation.