Unified reflow CLI: run, workspace, peer, discovery, trace, serve#2
Merged
Conversation
A single `reflow` binary (crates/reflow_cli) over the existing runtime APIs, so
graph execution, workspaces, distributed peers, and tracing share one entry
point instead of scattered binaries.
Commands:
- run <graph.json> load a graph, resolve actors (bundled reflow_components
catalog + --pack packs, mirroring rfl_template_actor_new),
build the Network (Network::with_graph), start, and run
until Ctrl-C. --trace / --trace-server / --trace-tail wire
tracing + stream live events via the local tap.
- graph validate|inspect parse a graph; report nodes/connections/components
and which are resolvable.
- workspace discover|list|namespaces|analyze|compose|run over
WorkspaceDiscovery + GraphComposer (multi_graph).
- peer spawn <toml> [--send] distributed peer; logic extracted into a new
reflow_distributed::peer::run_peer reused by both the
`reflow-peer` bin (now a thin shim) and this subcommand.
- discovery serve the registry server (reflow_distributed::serve).
- trace serve|tail|query|get collector (reflow_tracing::TraceServer) +
consumer (TracingClient subscribe/query/get).
- serve execs the reflow_server binary (--port; Zeal omitted). The
server crate carries an out-of-tree zeal-sdk path so it
isn't linked; the CLI spawns it instead.
clap 4 derive + tracing_subscriber, matching the house style. Integration tests
(graph validate/inspect + a run smoke that loads/resolves/starts) pass.
Follow-ups: script-actor resolution in `run` (needs runtime setup), distributed
graph composition across peers beyond `peer spawn`, and decoupling
reflow_server from zeal so `serve` can link it directly.
The CLI was the only thing left fragmented: `serve` shelled out to a separate
`reflow_server` binary because the server crate couldn't be linked. The blocker
was its out-of-tree `zeal-sdk` path dependency. This makes Zeal an opt-in
feature so the server core links cleanly into the unified `reflow` binary.
reflow_server:
- `zeal-sdk` is now `optional = true` behind a `zeal` feature, `default = ["zeal"]`
so the standalone `reflow_server` binary and existing Zeal users are unchanged.
- Feature-gate the Zeal-only modules (zeal_converter, zip_session, trace_collector,
template_adapter), `start_zeal_execution`, the `/zeal/*` routes + handlers, and
the Zeal branch of `start_server`. `EventBridge` stays in both configs: without
`zeal`, `attach` just drains the event channel so the engine never blocks.
- Self-contained manifest (concrete serde/serde_json/anyhow versions instead of
`workspace = true`): the crate is excluded from the workspace, so it must
resolve standalone when linked as a path dep.
- Fix a startup panic present in both configs: the `/webhook/{id}/{path:.*}`
route used actix/regex syntax axum 0.7 rejects ("only one parameter per path
segment"), so the server panicked on boot. matchit 0.7 also won't accept a
catch-all sibling of a param route, and the route never functioned, so it's
removed; `/webhook/{id}` is unaffected.
reflow_cli:
- Depend on `reflow_server` with `default-features = false` (Zeal omitted).
- `reflow serve` now calls `reflow_server::start_server` in-process, mapping
`--port/--bind/--redis` onto `ServerConfig`. No subprocess, no separate binary.
Verified: `reflow serve` answers GET /health 200; reflow_server builds with and
without `zeal`; reflow_cli tests pass; `cargo metadata` resolves.
Note: linking reflow_server pulls its `zeal-sdk` path into the resolve graph, so
`cargo metadata`/maturin now need the sibling Zeal checkout. Follow-up if the
Python-SDK CI must build without it: vendor zeal-sdk or make it a git dep.
Replace the out-of-tree `path = "../../../zeal/..."` with a git dependency (offbit-ai/zeal, pinned to a rev). Since `zeal-sdk` is optional and not enabled by any workspace member (the CLI links reflow_server with default-features = false), it never enters the workspace resolve graph: it's absent from Cargo.lock and `cargo metadata` only lists it as a declared optional dep. So workspace builds and maturin no longer need a sibling Zeal checkout — the git dep is fetched only when the `zeal` feature is explicitly enabled (standalone server).
Add `.github/workflows/release-cli.yml`. On a `cli-v*` tag it builds the unified
`reflow` binary for five targets and attaches per-target archives to a GitHub
Release; `workflow_dispatch` builds all targets for verification without
releasing.
Each target builds on its NATIVE runner (incl. an arm64 Linux runner) rather
than cross-compiling: the CLI links `reflow_server`, which pulls `native-tls`
(OpenSSL on Linux). Native runners avoid cross-compiling OpenSSL for aarch64.
Targets: {aarch64,x86_64}-apple-darwin, {x86_64,aarch64}-unknown-linux-gnu,
x86_64-pc-windows-msvc. Unix → .tar.gz, Windows → .zip. Matches the existing
publish-* workflows (dtolnay/rust-toolchain, sccache, action-gh-release).
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
A single
reflowbinary over the existing runtime APIs, so graph execution,multi-graph workspaces, distributed peers, and tracing share one entry point
instead of a scattered set of binaries (
reflow-peer,reflow-discovery,reflow_server). New cratecrates/reflow_cli([[bin]] reflow).Command tree
Highlights
runloads a graph, resolves each component against the bundledreflow_componentscatalog (+--packpacks, mirroring the C-ABIrfl_template_actor_new), builds theNetwork, starts it, and runs untilCtrl-C.
--trace*wires tracing and streams live events via the local tap.workspacereusesWorkspaceDiscovery+GraphComposer;workspace runcomposes then executes through the same shared
runtime::run_graph_export.peer/discoveryabsorb the old binaries. The peer logic moved intoreflow_distributed::peer::run_peer, soreflow-peeris now a thin shim andreflow peer spawndrives the exact same code.traceruns the collector (reflow_tracing::TraceServer) and consumesvia
TracingClient(tail/query/get).Whole CLI: Zeal made optional,
serveruns in-processservepreviously couldn't linkreflow_serverbecause of its out-of-treezeal-sdkpath dep, so it shelled out to a separate binary. This PR makes Zealopt-in so the server core links cleanly:
zeal-sdkis nowoptional = truebehind azealfeature,default = ["zeal"]— the standalonereflow_serverbinary and existing Zeal users areunchanged. The CLI links it with
default-features = false.start_zeal_execution,/zeal/*routes+handlers, and theZeal branch of
start_serverare feature-gated.EventBridgestays in bothconfigs (no-op
attachdrains the channel when Zeal is off).zeal-sdkis a pinned git dep (offbit-ai/zeal), not a path. Because it'soptional and unused by the workspace, it never enters
Cargo.lockor theresolve graph — workspace builds and maturin need no Zeal checkout; it's
fetched only when
reflow_serveris built with--features zeal.reflow servecallsreflow_server::start_serverin-process, mapping--port/--bind/--redisontoServerConfig./webhook/{id}/{path:.*}route used regex syntax axum 0.7 rejects, so theserver panicked on boot. The route never functioned and is removed;
/webhook/{id}is unaffected.CI
.github/workflows/release-cli.ymlbuilds thereflowbinary for five targetson native runners (incl. arm64 Linux — native-tls/OpenSSL doesn't cross-compile
cleanly) and attaches per-target archives to a Release on
cli-v*tags;workflow_dispatchbuilds all targets without releasing.Verification
reflow runloads/resolves/starts a fixture graph;graph validate/inspectand
workspace discover/listwork against the example workspace.reflow serveanswersGET /health200 in-process.reflow_serverbuilds with and withoutzeal; CLI integration testspass (3/3);
cargo metadataresolves with no Zeal checkout.reflow 0.2.0.Follow-ups: script-actor resolution in
run(needs async runtime setup),distributed-graph composition across peers beyond
peer spawn.