-
Notifications
You must be signed in to change notification settings - Fork 1
runtime: implement cow-api, chain, local-store host backends #8
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Open
brunota20
wants to merge
14
commits into
nullislabs:main
Choose a base branch
from
bleu:feat/cow-api-impl
base: main
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
Open
Changes from all commits
Commits
Show all changes
14 commits
Select commit
Hold shift + click to select a range
2054584
chore(deps): pull cowprotocol, alloy, redb, reqwest, tracing
brunota20 f85d3d3
runtime: implement cow-api, chain, local-store host backends
brunota20 2994f21
feat(manifest): rename nexum.toml to module.toml with compat fallback…
brunota20 9d36adf
feat(local-store): redesign namespace prefix to keccak256(name) 32-by…
brunota20 0d8ce94
feat(cow-api): replace with_default_chains() with Default trait (ADR-…
brunota20 62849b7
feat(manifest): enforce capability declarations against component WIT…
brunota20 068e558
style: apply rust-idiomatic rules (em-dashes, #[from] Orderbook, unus…
brunota20 70ca51b
review: apply lgahdl feedback on PR #8
brunota20 b39cb64
refactor(manifest): split into types/load/capabilities/error submodules
brunota20 99ba1ce
refactor(main): extract host Host impls + HostState + error helpers
brunota20 95c794d
refactor(host): move large #[cfg(test)] modules to sibling files
brunota20 a19d2d4
fix(manifest): wit_import_to_cap skips type-only interfaces
brunota20 c83a042
review: apply mfw78 feedback on PR #8 (small items)
brunota20 1adc58b
review: per-module ModuleStore + fuel/memory limits (mfw78 PR #8)
brunota20 File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -21,3 +21,4 @@ Thumbs.db | |
| # Environment | ||
| .env | ||
| .env.* | ||
| data/ | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,16 @@ | ||
| //! WIT bindings generated by `wasmtime::component::bindgen!`. | ||
| //! | ||
| //! Both `wit/nexum-host` and `wit/shepherd-cow` packages are listed | ||
| //! explicitly so wit-parser can resolve the cross-package reference | ||
| //! natively - no vendored `deps/` tree needed. The world name is fully | ||
| //! qualified. | ||
| //! | ||
| //! Every `Host` trait impl in [`crate::host::impls`] consumes types | ||
| //! generated here. | ||
|
|
||
| wasmtime::component::bindgen!({ | ||
| path: ["../../wit/nexum-host", "../../wit/shepherd-cow"], | ||
| world: "shepherd:cow/shepherd", | ||
| imports: { default: async }, | ||
| exports: { default: async }, | ||
| }); |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,155 @@ | ||
| //! Engine-side runtime configuration. | ||
| //! | ||
| //! Distinct from `module.toml` (module manifest): this file describes | ||
| //! the *engine*'s I/O wiring - chain RPC endpoints and the on-disk | ||
| //! location of the `local-store` database. Both are required for the | ||
| //! 0.2 reference engine to do anything other than print stubs. | ||
| //! | ||
| //! Lookup order: | ||
| //! | ||
| //! 1. `--engine-config <path>` CLI flag (future), or third positional | ||
| //! argument today; | ||
| //! 2. `engine.toml` in the current working directory; | ||
| //! 3. defaults - no chains configured, `state_dir = ./data`. | ||
| //! | ||
| //! A missing config is OK for the example module (it only logs); for | ||
| //! the cow-api / chain backends it surfaces as `HostError { | ||
| //! kind: unsupported }` so guests learn early. | ||
|
|
||
| use std::collections::BTreeMap; | ||
| use std::path::{Path, PathBuf}; | ||
|
|
||
| use serde::Deserialize; | ||
| use tracing::{info, warn}; | ||
|
|
||
| /// Engine-side configuration loaded from `engine.toml`. | ||
| #[derive(Debug, Default, Deserialize)] | ||
| pub struct EngineConfig { | ||
| #[serde(default)] | ||
| pub engine: EngineSection, | ||
| /// Per-chain RPC URLs keyed by EVM chain id (decimal in TOML). | ||
| /// Used by the `chain::request` host call and as the alloy provider | ||
|
Comment on lines
+1
to
+31
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. There needs to be a gas / fuel parameter so wasmtime can limit greedy wasm modules
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Replied here: #8 (comment) |
||
| /// pool seed. | ||
| #[serde(default)] | ||
| pub chains: BTreeMap<u64, ChainConfig>, | ||
| } | ||
|
|
||
| #[derive(Debug, Deserialize)] | ||
| pub struct EngineSection { | ||
| #[serde(default = "default_state_dir")] | ||
| pub state_dir: PathBuf, | ||
| /// `tracing_subscriber::EnvFilter`-compatible directive. Defaults to | ||
| /// `info` when absent; `RUST_LOG` overrides at process start. | ||
| #[serde(default = "default_log_level")] | ||
| pub log_level: String, | ||
| /// Resource caps applied to every module store at instantiation. | ||
| /// `wasmtime` traps a module that overruns either; the operator | ||
| /// tunes the budget based on their hardware and the modules they | ||
| /// load. | ||
| #[serde(default)] | ||
| pub limits: ModuleLimits, | ||
| } | ||
|
|
||
| impl Default for EngineSection { | ||
| fn default() -> Self { | ||
| Self { | ||
| state_dir: default_state_dir(), | ||
| log_level: default_log_level(), | ||
| limits: ModuleLimits::default(), | ||
| } | ||
| } | ||
| } | ||
|
|
||
| /// Per-module resource caps the supervisor applies to each store. | ||
| /// | ||
| /// `wasmtime` exposes two complementary knobs: | ||
| /// | ||
| /// - **Fuel** is decremented per executed instruction (`Config::consume_fuel | ||
| /// (true)`). When the store runs out, the module traps with `OutOfFuel`. | ||
| /// The budget is reset before every `on_event` invocation so a single | ||
| /// greedy event cannot starve the next one. | ||
| /// - **Memory size** caps the module's linear memory growth, applied via | ||
| /// `StoreLimitsBuilder::memory_size`. A module that requests more linear | ||
| /// memory than this gets a `MemoryOutOfBounds`-class trap from | ||
| /// `memory.grow`. | ||
| /// | ||
| /// Both are `Option<u64>` so an unset value falls through to the engine's | ||
| /// built-in defaults; operators only need to write the keys they actually | ||
| /// want to override. `main.rs` reads these at instantiation; the multi- | ||
| /// module supervisor (BLEU-818) consumes the same accessors per module | ||
| /// store. | ||
| #[derive(Clone, Copy, Debug, Default, Deserialize)] | ||
| pub struct ModuleLimits { | ||
| /// Fuel granted before every `on_event` call. `None` -> engine default. | ||
| #[serde(default)] | ||
| pub fuel_per_event: Option<u64>, | ||
| /// Linear-memory ceiling per module (bytes). `None` -> engine default. | ||
| #[serde(default)] | ||
| pub memory_bytes: Option<u64>, | ||
| } | ||
|
|
||
| impl ModuleLimits { | ||
| /// Default fuel granted per `on_event` invocation (≈ 1 billion WASM | ||
| /// instructions). Modules that exceed this budget trap with | ||
| /// `OutOfFuel`. | ||
| pub const DEFAULT_FUEL_PER_EVENT: u64 = 1_000_000_000; | ||
|
|
||
| /// Default linear-memory cap per module (64 MiB). Prevents a single | ||
| /// runaway module from exhausting process memory. | ||
| pub const DEFAULT_MEMORY_BYTES: u64 = 64 * 1024 * 1024; | ||
|
|
||
| /// Resolved fuel budget (config override, falling back to the default). | ||
| pub fn fuel(&self) -> u64 { | ||
| self.fuel_per_event.unwrap_or(Self::DEFAULT_FUEL_PER_EVENT) | ||
| } | ||
|
|
||
| /// Resolved memory cap in bytes (config override, falling back to the | ||
| /// default). | ||
| pub fn memory(&self) -> u64 { | ||
| self.memory_bytes.unwrap_or(Self::DEFAULT_MEMORY_BYTES) | ||
| } | ||
| } | ||
|
|
||
| #[derive(Debug, Deserialize)] | ||
| pub struct ChainConfig { | ||
| /// JSON-RPC endpoint. `ws://` and `wss://` engage alloy's pubsub | ||
| /// transport (required for `eth_subscribe`); `http://` and `https://` | ||
| /// engage the HTTP transport (request/response only). | ||
| pub rpc_url: String, | ||
| } | ||
|
|
||
| fn default_state_dir() -> PathBuf { | ||
| PathBuf::from("./data") | ||
| } | ||
|
|
||
| fn default_log_level() -> String { | ||
| "info".to_owned() | ||
| } | ||
|
|
||
| /// Read an engine config from disk, returning defaults if the file is | ||
| /// missing. Parse errors propagate. | ||
| pub fn load_or_default(path: Option<&Path>) -> anyhow::Result<EngineConfig> { | ||
| let path = match path { | ||
| Some(p) => p.to_path_buf(), | ||
| None => PathBuf::from("engine.toml"), | ||
| }; | ||
|
|
||
| if !path.exists() { | ||
| warn!( | ||
| path = %path.display(), | ||
| "engine.toml not found - running with defaults (no chain RPC endpoints; \ | ||
| chain::request and cow_api::submit_order will return Unsupported)" | ||
| ); | ||
| return Ok(EngineConfig::default()); | ||
| } | ||
|
|
||
| let raw = std::fs::read_to_string(&path)?; | ||
| let cfg: EngineConfig = toml::from_str(&raw)?; | ||
| info!( | ||
| path = %path.display(), | ||
| chains = cfg.chains.len(), | ||
| state_dir = %cfg.engine.state_dir.display(), | ||
| "engine config loaded", | ||
| ); | ||
| Ok(cfg) | ||
| } | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,141 @@ | ||
| //! `shepherd:cow/cow-api` backend. | ||
| //! | ||
| //! Two responsibilities: | ||
| //! | ||
| //! 1. `request` - generic REST passthrough. Module gives the HTTP | ||
| //! method, path (relative to the chain's orderbook base URL), and | ||
| //! optional JSON body. We dispatch via `reqwest`, return the | ||
| //! response body verbatim. | ||
| //! 2. `submit_order` - typed submission. Module gives a JSON-encoded | ||
| //! `cowprotocol::OrderCreation`; we parse, dispatch via | ||
| //! `cowprotocol::OrderBookApi::post_order`, return the assigned | ||
| //! `OrderUid` as a `0x`-prefixed hex string. | ||
| //! | ||
| //! Per-chain `OrderBookApi` instances are constructed once at engine | ||
| //! boot from the discriminated chain set in `cowprotocol::Chain`. | ||
| //! Chains the SDK does not know about return `Unsupported` at the | ||
| //! host call boundary. | ||
|
|
||
| use std::collections::BTreeMap; | ||
|
|
||
| use cowprotocol::{Chain, OrderBookApi, OrderCreation, OrderUid}; | ||
| use thiserror::Error; | ||
|
|
||
| /// Process-wide pool of `OrderBookApi` clients keyed by EVM chain id. | ||
| #[derive(Debug, Clone)] | ||
| pub struct OrderBookPool { | ||
| clients: BTreeMap<u64, OrderBookApi>, | ||
| http: reqwest::Client, | ||
| } | ||
|
|
||
| impl Default for OrderBookPool { | ||
| /// Build a pool covering every `cowprotocol::Chain` variant. Each entry | ||
| /// uses the canonical `api.cow.fi/{slug}/api/v1` base URL from the SDK. | ||
| /// Override individual entries via `OrderBookApi::new_with_base_url` for | ||
| /// barn or staging targets. | ||
| fn default() -> Self { | ||
| let http = reqwest::Client::new(); | ||
| let chains = [ | ||
| Chain::Mainnet, | ||
| Chain::Gnosis, | ||
| Chain::Sepolia, | ||
| Chain::ArbitrumOne, | ||
| Chain::Base, | ||
| ]; | ||
| let clients = chains | ||
| .iter() | ||
| .map(|c| (c.id(), OrderBookApi::new(*c))) | ||
| .collect(); | ||
| Self { clients, http } | ||
| } | ||
| } | ||
|
|
||
| impl OrderBookPool { | ||
| /// Look up the client for a chain. | ||
| pub fn get(&self, chain_id: u64) -> Result<&OrderBookApi, CowApiError> { | ||
| self.clients | ||
| .get(&chain_id) | ||
| .ok_or(CowApiError::UnknownChain(chain_id)) | ||
| } | ||
|
|
||
| /// REST passthrough. The base URL is whichever URL the pool's | ||
| /// `OrderBookApi` client carries - overrides set via | ||
| /// `OrderBookApi::new_with_base_url` (staging, wiremock) flow | ||
| /// through here too, which keeps the passthrough and the typed | ||
| /// `submit_order_json` path aimed at the same orderbook. | ||
| pub async fn request( | ||
| &self, | ||
| chain_id: u64, | ||
| method: &str, | ||
| path: &str, | ||
| body: Option<&str>, | ||
| ) -> Result<String, CowApiError> { | ||
| let api = self.get(chain_id)?; | ||
| let base = api.base_url().clone(); | ||
| // `path` may or may not lead with a slash; `Url::join` handles | ||
| // both, but we strip a single leading `/` so consumers can | ||
| // write either `/orders/...` or `orders/...` interchangeably. | ||
| let trimmed = path.strip_prefix('/').unwrap_or(path); | ||
| let url = base | ||
| .join(trimmed) | ||
| .map_err(|e| CowApiError::BadPath(format!("{path:?}: {e}")))?; | ||
|
|
||
| let request = match method.to_ascii_uppercase().as_str() { | ||
| "GET" => self.http.get(url), | ||
| "POST" => self.http.post(url), | ||
| "PUT" => self.http.put(url), | ||
| "DELETE" => self.http.delete(url), | ||
| other => return Err(CowApiError::BadMethod(other.to_owned())), | ||
| }; | ||
| let request = if let Some(body) = body { | ||
| request | ||
| .header(reqwest::header::CONTENT_TYPE, "application/json") | ||
| .body(body.to_owned()) | ||
| } else { | ||
| request | ||
| }; | ||
|
|
||
| let response = request.send().await.map_err(CowApiError::Network)?; | ||
| // Surface the orderbook's structured 4xx / 5xx bodies verbatim | ||
| // so the guest can decode `{"errorType": "...", "description": | ||
| // "..."}` - projecting them into HostError here loses the | ||
| // detail the guest needs to recover. | ||
| let text = response.text().await.map_err(CowApiError::Network)?; | ||
| Ok(text) | ||
| } | ||
|
|
||
| /// Typed submission. `body` is the JSON encoding of | ||
| /// `cowprotocol::OrderCreation`. The chain's orderbook validates | ||
| /// `from`, the EIP-712 hash, and (if `Eip1271`) the contract | ||
| /// signature; we return whatever UID it assigns. | ||
| pub async fn submit_order_json( | ||
| &self, | ||
| chain_id: u64, | ||
| body: &[u8], | ||
| ) -> Result<OrderUid, CowApiError> { | ||
| let creation: OrderCreation = serde_json::from_slice(body).map_err(CowApiError::Decode)?; | ||
| let api = self.get(chain_id)?; | ||
| let uid = api.post_order(&creation).await?; | ||
| Ok(uid) | ||
| } | ||
| } | ||
|
|
||
| #[derive(Debug, Error)] | ||
| pub enum CowApiError { | ||
| #[error("unknown chain {0} (no cowprotocol::Chain variant)")] | ||
| UnknownChain(u64), | ||
| #[error("bad HTTP method `{0}` (expected GET/POST/PUT/DELETE)")] | ||
| BadMethod(String), | ||
| #[error("invalid path: {0}")] | ||
| BadPath(String), | ||
| #[error("network: {0}")] | ||
| Network(#[from] reqwest::Error), | ||
| #[error("decode OrderCreation JSON: {0}")] | ||
| Decode(#[from] serde_json::Error), | ||
| #[error("orderbook: {0}")] | ||
| Orderbook(#[from] cowprotocol::Error), | ||
| } | ||
|
|
||
|
|
||
| #[cfg(test)] | ||
| mod tests; |
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Be very careful about this as I'm not sure what is being pulled in just to make sure it pulls in the correct one as I know that another grantee published to crates as well.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Verified: cowprotocol@1.0.0-alpha.3 on crates.io points at cowdao-grants/cow-rs (homepage + repository fields). Comment now cross-references ADR-0004. If you remember the conflicting package name, happy to add a deny entry. Changed on c83a042