runtime: implement cow-api, chain, local-store host backends#8
runtime: implement cow-api, chain, local-store host backends#8brunota20 wants to merge 14 commits into
Conversation
Adds the dependencies the 0.2 host backends need: - cowprotocol (1.0.0-alpha) for the cow-api submission path (OrderBookApi, OrderCreation, OrderUid, Chain). - alloy-provider / -rpc-client / -transport-ws / -primitives (1.5) for the chain JSON-RPC dispatch. The reqwest feature on alloy-provider engages connect_http; the pubsub/ws features back eth_subscribe-class methods. - redb (2) for local-store. Same crate cowprotocol's own watch-tower picked, so the dep tree does not bifurcate when both are used in the same workspace. - reqwest (0.12, rustls-tls) — direct, so the import survives any future cowprotocol feature rearrangement. - tracing + tracing-subscriber (env-filter + fmt) — replaces the 0.1 eprintln! debug log so the engine can drop into a structured log pipeline without re-instrumenting every host call. - thiserror (2) — typed error enums in each backend. - tempfile + wiremock as dev-deps for the host backend tests. Adds engine.example.toml documenting the [engine] state_dir + per- chain RPC URLs the chain backend reads at boot; data/ is now ignored so a local run does not leave the redb file in tree.
Replaces the 0.2 Unsupported stubs with working backends. Each
capability lives in its own host submodule so the trait impls in
main.rs stay thin (dispatch + project the backend's typed error
onto HostError).
cow_api::submit_order
- Parses the guest's bytes as JSON cowprotocol::OrderCreation.
- Dispatches via cowprotocol::OrderBookApi::post_order.
- Returns the assigned OrderUid as a 0x-prefixed hex string.
cow_api::request
- REST passthrough. The base URL is whichever URL the pool's
OrderBookApi client carries — so OrderBookApi::new_with_base_url
overrides (staging, wiremock) flow through transparently.
- Method/path validated host-side; orderbook 4xx/5xx bodies are
surfaced verbatim so the guest can decode {errorType,description}.
chain::request
- Raw JSON-RPC dispatch over an alloy DynProvider opened from
engine.toml at boot. WebSocket URLs engage pubsub (eth_subscribe);
HTTP URLs use the HTTP transport. Params are passed as
serde_json::RawValue so alloy does not re-encode.
- request-batch falls back to per-call dispatch (same shape as the
earlier stub but now backed by real RPC).
local_store
- redb file under engine_config.engine.state_dir.
- Single shared table. Per-module namespacing is enforced
host-side via [len:u8][module_name][raw_key] prefix on every
key. list_keys strips the prefix before returning to the guest.
logging
- Routes through tracing::event! tagged with module=<namespace>.
- Engine boot installs an EnvFilter-based subscriber; RUST_LOG
overrides the engine.toml log_level.
identity / remote-store / messaging / http stay at Unsupported per
the 0.2 roadmap (keystore / Swarm / Waku land in 0.3).
Tests (14, all green):
- cow_orderbook: pool default chains, unknown-chain typing, REST
GET passthrough, relative-path resolution, unknown-method
rejection, submit_order round-trip — last three under wiremock
so the full HTTP path is exercised without hitting api.cow.fi.
- provider_pool: empty pool surfaces UnknownChain.
- local_store: roundtrip, namespace isolation, delete, list_keys
prefix-stripping, empty-namespace rejection.
End-to-end against modules/example: example.wasm loads under the
new wiring, logs init + on_event through the tracing pipeline.
…te hash (ADR-0003)
… imports (BLEU-816)
…ed_crate_dependencies)
- local_store_redb: table.range() instead of iter() for O(matching) keys - provider_pool: dedupe method clone on the success path - main: hex_encode writes into the pre-allocated buffer - cow_orderbook: drop blank line nit - manifest: collapse nested if into && chain (clippy) - alloy_rpc_client / alloy_transport(_ws) imports as _ to satisfy unused_crate_dependencies; they're transitive providers we depend on through alloy_provider's API but never name directly.
PR #9 specific: - main: warn + return when block/log streams end (WebSocket dropped) - supervisor: simplify dispatch_block by extracting chain_id before move - supervisor: temp_local_store returns (TempDir, LocalStore) instead of leaking - README: correct engine.toml chain syntax to [chains.<id>] with rpc_url Rebased from PR #8: - local_store_redb: table.range() instead of iter() for O(matching) keys - provider_pool: dedupe method clone on the success path - main: hex_encode writes into the pre-allocated buffer - cow_orderbook: drop blank line nit - manifest: collapse nested if and use ? operator (clippy) - alloy_rpc_client / alloy_transport(_ws) imports as _ to satisfy unused_crate_dependencies.
Move the 575-line manifest.rs into a directory module with four focused submodules. Behaviour unchanged - this is pure code motion plus a smaller, more navigable public surface.
main.rs went from 739 lines hosting eight Host trait impls + the WIT
bindings + bootstrap to ~200 lines of pure bootstrap. The 13 capability
files under host/impls/ each have one purpose; adding a new capability
is a new file rather than a diff in main.rs.
- bindings.rs: wasmtime::component::bindgen! moved out so other modules
can name the generated types.
- host/state.rs: HostState + WasiView impl.
- host/error.rs: unimplemented / internal_error / hex_encode helpers.
- host/impls/{chain,cow_api,identity,local_store,remote_store,messaging,
logging,clock,random,http,types}.rs: one Host trait impl per file.
local_store_redb.rs was 89% tests and cow_orderbook.rs was 60%, dwarfing the implementation. Promote each to a directory module with the test suite living in a sibling tests.rs, so impl-side diffs stop competing with test churn for attention.
KNOWN_CAPABILITIES guard so nexum:host/types (and any future type-only packages) return None from wit_import_to_cap. Matches the fix originally landed for BLEU-819 on feat/supervisor-event-loop and silences a false-positive capability violation reported there.
| impl shepherd::cow::cow_api::Host for HostState { | ||
| async fn request( | ||
| &mut self, | ||
| chain_id: u64, | ||
| method: String, | ||
| path: String, | ||
| body: Option<String>, | ||
| ) -> Result<String, HostError> { |
There was a problem hiding this comment.
Given the duplication of this request (essentially), it's basically the same thing as that for the chain. Semantically, it might be helpful to essentially take method and params as per chain, but do it for cow as well.
So, in which case, we could have a standard shepherd::jsonrpc::Host trait that does request as mentioned.
Now, this could be the "raw dog" version could be used, such as when bolting in another SDK like viem or alloy within WASM, in which case one can just pass that to it and it'll automatically connect to what it needs to connect to (it's basically wire compatible).
But, for cow where there are more advanced, ie. simpler apis like 'submit_order', that could be implemented on the struct, and not the trait.
So, tldr the cow and chain would both implement request (impl a trait), then for all intents and purposes, cow could also implement submit_order but on the struct as a local impl.
There was a problem hiding this comment.
On second thoughts, then when reading about local storage, I'm really not sure that the above that I've written is useful, as the domains start to favour different request / fetch / idiomatic structures. Maybe let's leave it for later 😅
There was a problem hiding this comment.
Agreed — leaving as-is until a third request-style domain shows up.
| # `OrderUid`, the orderbook base URL table per `Chain`, and the typed | ||
| # error surface the host re-projects into `HostError`. Pinned to the | ||
| # crates.io release Shepherd is shipping against. | ||
| cowprotocol = "1.0.0-alpha" |
There was a problem hiding this comment.
Be very careful about this as I'm not sure what is being pulled in just to make sure it pulls in the correct one as I know that another grantee published to crates as well.
There was a problem hiding this comment.
Verified: cowprotocol@1.0.0-alpha.3 on crates.io points at cowdao-grants/cow-rs (homepage + repository fields). Comment now cross-references ADR-0004. If you remember the conflicting package name, happy to add a deny entry. Changed on c83a042
| # `chain` backend. Each configured chain owns a `DynProvider` built | ||
| # from a `WsConnect`/`Http` transport so the host's `request` / | ||
| # `request-batch` impls can hand a raw `(method, params)` pair to | ||
| # alloy's JSON-RPC layer without reimplementing the codec. | ||
| alloy-provider = { version = "1.5", default-features = false, features = ["ws", "ipc", "pubsub", "reqwest"] } | ||
| alloy-rpc-client = { version = "1.5", default-features = false } | ||
| alloy-transport = { version = "1.5", default-features = false } | ||
| alloy-transport-ws = { version = "1.5", default-features = false } | ||
| alloy-primitives = { version = "1.5", default-features = false, features = ["std", "serde"] } |
There was a problem hiding this comment.
Bump to make sure very latest
There was a problem hiding this comment.
Bumped provider/rpc-client/transport/transport-ws to 1.8 (latest 1.x). Held off 2.x: alloy-provider@2 pulls alloy-signer@2, cowprotocol still pins 1.8.3 → cargo deny multiple-versions trip. Re-bumping when cow-rs cuts over. Changed on c83a042
| # Engine-side runtime configuration for `nexum-engine`. | ||
| # | ||
| # Distinct from `nexum.toml` (per-module manifest): this file | ||
| # describes the *engine*'s I/O wiring. Copy to `engine.toml` next to | ||
| # the binary, or pass the path as the third positional argument. | ||
|
|
||
| [engine] | ||
| # Directory the local-store redb file (and future engine artefacts) | ||
| # will be created under. Created automatically at boot. | ||
| state_dir = "./data" | ||
|
|
||
| # `tracing_subscriber::EnvFilter`-compatible directive. `RUST_LOG` | ||
| # overrides at process start. | ||
| log_level = "info" | ||
|
|
||
| # One [chains.<id>] table per chain the engine should be able to talk | ||
| # to. Chain ids are EVM decimal. `ws://` and `wss://` URLs engage | ||
| # alloy's pubsub transport (needed for `eth_subscribe`); `http://` and | ||
| # `https://` use the HTTP transport. | ||
|
|
||
| [chains.1] | ||
| rpc_url = "https://ethereum-rpc.publicnode.com" | ||
|
|
||
| [chains.100] | ||
| rpc_url = "https://rpc.gnosischain.com" | ||
|
|
||
| [chains.11155111] | ||
| rpc_url = "wss://ethereum-sepolia-rpc.publicnode.com" | ||
|
|
||
| [chains.42161] | ||
| rpc_url = "https://arb1.arbitrum.io/rpc" | ||
|
|
||
| [chains.8453] | ||
| rpc_url = "https://mainnet.base.org" |
There was a problem hiding this comment.
Engine configuration should definitely also include gas limits for the modules to make sure that they don't consume all resources (wasmtime supports gas metering)
There was a problem hiding this comment.
Added [engine.limits] with fuel_per_event + memory_bytes (both Option, defaults 1B / 64 MiB). main.rs sets config.consume_fuel(true), applies StoreLimits, seeds + refuels per event. Smoke log: applied module resource limits fuel=1000000000 memory_bytes=67108864. — 1adc58b
| //! Engine-side runtime configuration. | ||
| //! | ||
| //! Distinct from `module.toml` (module manifest): this file describes | ||
| //! the *engine*'s I/O wiring - chain RPC endpoints and the on-disk | ||
| //! location of the `local-store` database. Both are required for the | ||
| //! 0.2 reference engine to do anything other than print stubs. | ||
| //! | ||
| //! Lookup order: | ||
| //! | ||
| //! 1. `--engine-config <path>` CLI flag (future), or third positional | ||
| //! argument today; | ||
| //! 2. `engine.toml` in the current working directory; | ||
| //! 3. defaults - no chains configured, `state_dir = ./data`. | ||
| //! | ||
| //! A missing config is OK for the example module (it only logs); for | ||
| //! the cow-api / chain backends it surfaces as `HostError { | ||
| //! kind: unsupported }` so guests learn early. | ||
|
|
||
| use std::collections::BTreeMap; | ||
| use std::path::{Path, PathBuf}; | ||
|
|
||
| use serde::Deserialize; | ||
| use tracing::{info, warn}; | ||
|
|
||
| /// Engine-side configuration loaded from `engine.toml`. | ||
| #[derive(Debug, Default, Deserialize)] | ||
| pub struct EngineConfig { | ||
| #[serde(default)] | ||
| pub engine: EngineSection, | ||
| /// Per-chain RPC URLs keyed by EVM chain id (decimal in TOML). | ||
| /// Used by the `chain::request` host call and as the alloy provider |
There was a problem hiding this comment.
There needs to be a gas / fuel parameter so wasmtime can limit greedy wasm modules
There was a problem hiding this comment.
All errors, especially as library errors (non binary crate) should be thiserror and likely enums.
There was a problem hiding this comment.
Converted both ParseError and CapabilityViolation to thiserror::Error derives. ? replaces map_err in load.rs. — c83a042
| /// Empty pool - used by tests and as a default when no | ||
| /// `engine.toml` is found. Every `request` call returns | ||
| /// `UnknownChain`. | ||
| #[cfg_attr(not(test), allow(dead_code))] | ||
| pub fn empty() -> Self { | ||
| Self { | ||
| providers: Arc::new(BTreeMap::new()), | ||
| } | ||
| } |
There was a problem hiding this comment.
If this is used by a tests only, can't it be cfg gated to tests only?
There was a problem hiding this comment.
ProviderPool::empty is #[cfg(test)] now. Nothing outside the inline tests references it. — c83a042
There was a problem hiding this comment.
Given the general structure of this, it will work okay intially but as the data grows, the prefixing is going to destroy the machine running wasmtime. Every single function recalculates the prefix. This should likely be instead instantiated and progressively have modules added with some builder of sorts, like Localstore::builder().with_module("chain").with_module("local").with_module("cow") etc.
In this case, when the builder assembles the configuration before starting the local store backend, it can cache these respective prefixes so that it doesn't have to grind on every single interaction.
There was a problem hiding this comment.
LocalStore::module(name) returns a ModuleStore with the keccak prefix computed once. Per-call ops are prefix ++ key concat — no rehash, no Vec alloc. HostState.store: ModuleStore directly. Picked per-module handle over a one-shot builder because the supervisor adds modules dynamically; end-state is equivalent. 25 tests pass (+1 new). — 1adc58b
| /// Lowercase hex encoder. Kept in the engine binary rather than | ||
| /// pulling a `hex` crate just for one call site. Writes into the | ||
| /// pre-allocated buffer to avoid the per-byte `String` allocation | ||
| /// `format!("{b:02x}")` would do. | ||
| pub(crate) fn hex_encode(bytes: &[u8]) -> String { | ||
| use std::fmt::Write as _; | ||
| let mut s = String::with_capacity(bytes.len() * 2); | ||
| for b in bytes { | ||
| write!(s, "{b:02x}").expect("writing to String never fails"); | ||
| } | ||
| s | ||
| } |
There was a problem hiding this comment.
We already pull alloy, so pulling hex via there is really not much of a deal.
There was a problem hiding this comment.
hex_encode is now a thin wrapper over alloy_primitives::hex::encode. — c83a042
- Cargo.toml: clarify in comment that cowprotocol = "1.0.0-alpha" is the cowdao-grants/cow-rs crate (homepage / repository fields on crates.io point there) and cross-reference ADR-0004. Bump alloy declarations (provider / rpc-client / transport[-ws]) from 1.5 to 1.8 (latest 1.x at 1.8.3); 2.x is held off because alloy-provider@2 pulls alloy-signer@2 while cowprotocol still pins alloy-signer@1.8.3 — they'd collide via cargo-deny's multiple-versions check. alloy-primitives stays at 1.6 (its own release cadence; 1.6.0 is max stable today). - host/error.rs: drop the hand-rolled write!-loop hex encoder for alloy_primitives::hex::encode. alloy is already in the dep graph via the chain backend, so this is a net code-loss with no extra link cost. - host/provider_pool.rs: gate ProviderPool::empty() behind #[cfg(test)]. It is only referenced from the inline test module, so the non-test build now does not link an unused constructor. - manifest/error.rs: convert to thiserror::Error derives. ParseError becomes a clean enum with #[from] impls for std::io::Error / toml::de::Error, and UnknownCapability switches to a struct variant with a `name` field. Display strings preserved. manifest/load.rs adopts the `?` conversion now that From is in scope.
Two structural changes that share the HostState / main.rs hot path,
landed together to avoid an inconsistent intermediate revision.
## ModuleStore: cached prefix per module (review on local_store_redb.rs)
Per mfw78's main concern, the namespace prefix is no longer
recomputed on every get / set / delete / list-keys call. Introduces
`ModuleStore`: a per-module handle carrying the pre-computed
`[u8; 32]` keccak prefix plus an `Arc<Database>` ref. Hashing
happens once, in `LocalStore::module(name)`.
let store: LocalStore = LocalStore::open(path)?;
let twap: ModuleStore = store.module("twap-monitor")?; // hash once
twap.set("watch:...", &bytes)?; // concat-only
twap.list_keys("watch:")?; // concat-only
HostState now carries `store: ModuleStore` directly; the previous
`(LocalStore, module_namespace: String)` pair collapsed to just the
`ModuleStore`. The `module_namespace` field stays on HostState
purely for log tagging (the host's `logging::log` impl tags every
line with it) — the namespace identity is encoded in `ModuleStore`'s
prefix.
Tests adapt: the suite previously called `store.set("ns", "k",
"v")`; it now calls `store.module("ns")?.set("k", "v")`. Added a
new test (`module_handles_share_underlying_data`) confirming
cloning a `ModuleStore` is an Arc bump, not a fresh DB view.
## Fuel + memory limits (review on engine_config.rs + engine.example.toml)
Per mfw78's gas-metering concern, surfaces the wasmtime budget on
`engine.toml` so the operator can cap a runaway module. Both knobs
are `Option<u64>` with built-in defaults (1B fuel, 64 MiB memory) —
operators only write the keys they want to override.
[engine.limits]
fuel_per_event = 1_000_000_000
memory_bytes = 67_108_864
main.rs wires the budget into wasmtime:
config.consume_fuel(true);
let limits = StoreLimitsBuilder::new().memory_size(cap).build();
store.limiter(|s| &mut s.limits);
store.set_fuel(limits_cfg.fuel())?; // initial seed
// ... init ...
store.set_fuel(limits_cfg.fuel())?; // refuel for on_event
HostState gains a `limits: StoreLimits` field. `ModuleLimits` lives
in engine_config.rs with `fuel()` / `memory()` accessors that
resolve the override against the defaults. Smoke test logs the
applied budget:
applied module resource limits fuel=1000000000 memory_bytes=67108864
3 participants
Draft, opening for early visibility — happy to revise scope or split further.
Context: Bleu has been working on the same M2 deliverables (TWAP + EthFlow + cow-api host) in a parallel repo (
bleu/shepherd) on a raw-FFI ABI that predated this repo. Once we saw the Nexum/Component-Model architecture here, the right move was to port the CoW Protocol logic onto your contract rather than continue two parallel runtimes. This PR is that port — only the pieces that fit the WIT contract you've already shipped; the raw-FFI side stays in our repo as a reference implementation.What this lands
Replaces the 0.2
Unsupportedstubs forcow-api,chain, andlocal-storewith working backends. Each capability lives in its own host submodule (crates/nexum-engine/src/host/{cow_orderbook,provider_pool,local_store_redb}.rs) so the trait impls inmain.rsstay thin and the backends are unit-testable without a wasmtime store.cow_api
request(chain_id, method, path, body)OrderBookApiclient carries, soOrderBookApi::new_with_base_urloverrides (staging / wiremock) flow through. Orderbook 4xx/5xx bodies surface verbatim so guests can decode{errorType, description}.submit_order(chain_id, order_data)cowprotocol::OrderCreation, dispatches viaOrderBookApi::post_order, returns the assignedOrderUidas0x-prefixed hex.chain
request(chain_id, method, params)DynProvideropened fromengine.toml.ws:///wss://engage pubsub (needed foreth_subscribe);http:///https://use the HTTP transport. Params pass through asserde_json::RawValueso alloy does not re-encode.request_batchlocal-store
get/set/delete/list_keysagainst a redb file underengine_config.engine.state_dir. Single shared table; per-module namespacing is enforced host-side via[len:u8][module_name][raw_key]prefix on every key.list_keysstrips the prefix before returning to the guest. Two modules using the same key string see disjoint data.logging
Routes through
tracing::event!tagged withmodule=<namespace>. Engine boot installs anEnvFilter-based subscriber;RUST_LOGoverrides theengine.tomllog_level. The 0.1-styleeprintln!lines are gone — fmt subscriber renders the same info plus structured fields.Out of scope (deliberate)
identity,remote-store,messaging,httpstay atUnsupported. Each has the 0.3 roadmap (keystore / Swarm / Waku) cited inline inmain.rs.Engine config
New optional
engine.toml(sibling of the binary, or third positional arg):Missing file ⇒ defaults (no chain endpoints;
chain::requestandcow_api::submit_orderreturnUnsupporteduntil configured).engine.example.tomlships in tree.Tests
14 unit tests, all green:
cow_orderbook: pool default-chains, unknown-chain typing, REST GET passthrough, relative-path resolution, unknown-method rejection,submit_orderround-trip — last three underwiremockso the full HTTP path is exercised without hittingapi.cow.fi.provider_pool: empty pool surfacesUnknownChain.local_store: roundtrip, namespace isolation, delete,list_keysprefix-stripping, empty-namespace rejection.End-to-end against
modules/example:example.wasmloads under the new wiring, logsinit+on_eventthrough the tracing pipeline. `just run` works unchanged.```
cargo fmt --all --check # clean
RUSTFLAGS=-D warnings cargo clippy --workspace --all-targets --all-features # clean
cargo test -p nexum-engine --bins # 14 passed
```
Threat-model notes
cow-rs.cow_api::requestreturns the response body verbatim; we do not parse or validate beyond whatcowprotocol::OrderBookApialready does.submit_orderre-parses the guest's JSON viaserde_json::from_slice::<OrderCreation>and reuses cowprotocol'sOrderCreation::from_signed_order_datavalidation (from != ZERO, app-data digest check) before posting. A malicious guest cannot bypass that path because the host calls cowprotocol, not its own re-encoder.engine.toml.Open questions for the maintainer
engine.tomlshape. I put it sibling to the binary with[engine] state_dir+[chains.<id>] rpc_url. Open to moving knobs into env vars, splitting it differently, or merging withnexum.tomlif you prefer one config surface. Yelled at the missing-config path with atracing::warn!rather than failing boot.OrderBookPool::with_default_chainshard-codes the five chains cowprotocol exposes viaChain::try_from. If you'd rather it consultengine.toml.[cow_api]for URL overrides per chain (mirroring the chain RPC table), happy to add it — theOrderBookApi::new_with_base_urlplumbing is already there.Coupling to
cowprotocol. This PR pullscowprotocoldirectly intonexum-engine. If the long-term design is "engine stays nexum-only, cow extensions live behind a feature flag", I can gate everycow-apicode path behind--features cowso a pure-Nexum build does not see the cowprotocol tree.local_storequota / transactional rollback. docs/04 mentions "per-event all-or-nothing" semantics and per-module storage quotas. This PR ships neither — every host call commits its own redb txn. Both fit naturally on top (we have a working quota counter in our other repo); want to do them in a follow-up?Bleu happy to iterate on any of the above before this leaves draft.