fix: use solver-determined fee for BYOS fulfillments#19
Closed
jean-neiverth wants to merge 470 commits into
Closed
fix: use solver-determined fee for BYOS fulfillments#19jean-neiverth wants to merge 470 commits into
jean-neiverth wants to merge 470 commits into
Conversation
# Description Migrates the trusted tokens to the main config. I've grouped to be easier to read/find in config files # Changes * Removes the trusted tokens from the CliArgs * Adds them to the new config file ## How to test Staging
# Description The majority of time fetching uni v3 liquidity is spent cloning pool states. This makes up 3.6% of the entire CPU time of the driver. <img width="1920" height="935" alt="Screenshot 2026-02-19 at 22 50 02" src="https://github.com/user-attachments/assets/a59efa24-60c0-4a4e-9c36-428521ef0bc5" /> # Changes This PR applies 2 optimizations: * reuse existing allocation with `into_iter().filter().collect()` * store `PoolInfo` in `Arc`s for cheap clones I'd like to highlight the use of `Arc::make_mut`. If there is no `Arc` pointing to the same allocation as our `Arc` we can simply get a mutable reference. If there are other `Arc`s pointing to our allocation we deep clone it and point our `Arc` to the new allocation (clone on write). This is quite cool because together with the fact that `append_events` takes an exclusive reference to the `pools` we know that there can be no new `Arc` pointing to a recently updated `PoolInfo` before the function is over. This ensures that every `PoolInfo` that needs to be updated will always be cloned at most once. ## How to test Briefly tested in prod This yielded surprising results. The amount of time spent inside the uni v3 liquidity fetching logic remained unchanged and even the time used for allocations was pretty much unchanged. But since the cloning of the data was moved to when it gets updated instead of every time it's needed for a request the time needed to fetch all the liquidity improved significantly. (still nothing to write home about, though) <img width="629" height="302" alt="Screenshot 2026-02-19 at 23 31 05" src="https://github.com/user-attachments/assets/6dc00c53-4afe-4761-af37-d5848f0da4a0" /> Also some other CPU bound pre-processing tasks got faster as well: <img width="630" height="300" alt="Screenshot 2026-02-19 at 23 31 14" src="https://github.com/user-attachments/assets/307e4ad1-de24-477d-8a4d-3f5f140233d8" /> <img width="625" height="294" alt="Screenshot 2026-02-19 at 23 31 31" src="https://github.com/user-attachments/assets/31060dcf-1a7b-4c84-8ee1-f3910ef65a8a" /> <img width="625" height="302" alt="Screenshot 2026-02-19 at 23 31 23" src="https://github.com/user-attachments/assets/6c8af19f-942c-44d2-82c2-8b0058d0cf45" />
# Description Serializing the auction and compressing the bytes is CPU bound work and takes quite long. That stuff should be spawned on a block_task to not hog one of the regular tokio runtime threads and cause spikes in tail latency. # Changes * spawn blocking task for serializing and compression s3 upload
# Description Currently the event indexing logic is dominated by 2 calls: * eth_getBlock * eth_getLogs (events of fetched block) This PR eliminates `eth_getLogs` calls. Since the rest of the indexing logic is not dominated by `eth_getLogs` I think the only way how we could meaningfully speed up the process further is by subscribing to log filters and streaming the results as soon as a new block was processed by the ethereum node. # Changes Since we already have a component that keeps track of the latest block I created a new type that implements `BlockRetrieving` that simply forwards `get_current_block()` calls to the block stream instead of sending an RPC request. Also adds a few tracing spans and replaces the awkward request batching logic of the event handler with something easier to read. ## How to test tested on staging mainnet (indexing overhead should be identical to prod) the time the autopilot waits for essential processing to be done decreased from ~65ms to ~45ms <img width="801" height="150" alt="Screenshot 2026-02-19 at 13 04 48" src="https://github.com/user-attachments/assets/18ce0eb8-5e8c-4412-a12b-efa4c6b1f313" /> <img width="657" height="300" alt="Screenshot 2026-02-19 at 13 03 12" src="https://github.com/user-attachments/assets/8f823a5c-a1d4-4bd0-ac10-c4c07fed068f" /> <img width="1262" height="302" alt="Screenshot 2026-02-19 at 13 05 44" src="https://github.com/user-attachments/assets/8f007f53-9efb-44a4-b426-cf91d25cb0c4" />
# Description Upgrades prometheus to 0.14 & the metrics macro to 0.6 Closes cowprotocol#3338 # Changes * Upgrades prometheus to 0.14 & the metrics macro to 0.6 * Changes some callsites to match the new with_label_values signatures ## How to test Staging
# Description When investigating CoinGecko usage I noticed our nginx cache is not being optimally used, it is in fact used very little. The cache can only trigger when the URL is the same. There are two issues: the batches have different compositions and the tokens are not in deterministic order. This PR fixes the second problem.
# Description Because the insert order_events query tries to avoid duplicating the last event entry it's quite awkward. That's probably why we didn't replace it with a bulk request earlier. However, flamegraphs showed that we spend a ridiculous 49% of our CPU time on inserting orders. # Changes Implemented awkward bulk insert query ## How to test single insert variant was already tested before 1 new unit test for multi insert variant Reduced CPU usage: **0.55 cores -> 0.2 cores** <img width="763" height="211" alt="Screenshot 2026-02-19 at 20 28 27" src="https://github.com/user-attachments/assets/f25e8c66-5480-4b21-a4c2-d4a1fdf654b7" /> Faster at handling other CPU bound work <img width="626" height="298" alt="Screenshot 2026-02-19 at 20 28 55" src="https://github.com/user-attachments/assets/e27730ca-31cc-4579-832b-d9ccaf2fe140" /> Total time spent on that query: **49% -> 3%** <img width="1920" height="767" alt="Screenshot 2026-02-19 at 20 30 08" src="https://github.com/user-attachments/assets/28b262ae-dd93-4a30-8122-d28543b7ac71" /> <img width="1920" height="636" alt="Screenshot 2026-02-19 at 20 30 17" src="https://github.com/user-attachments/assets/81a5894f-b2a5-43a7-83c0-33e38b28923a" />
# Description
To my surprise do `tracing` macros not lazily evaluate expressions. So
in following code:
```rust
tracing::trace!(
path = &url.path(),
body = %payload.body_to_string(),
"solver request",
);
```
`payload.body_to_string()` always gets evaluated but the `Display`
implementation of the resulting `String` gets called only when the log
is enabled.
This leads to the autopilot currently spending ~2% of the time in
`body_to_string()`.
To address this I introduced the `Lazy` type that delays the execution
of the passed in expression to when the `tracing` machinery actually
executed the `Display` or `Debug` functionality.
Additionally flamegraphs showed that the driver is currently burning
2.2% of its CPU time on logging auction deadlines. I downgraded that log
to `trace` since it's not really that useful.
# Changes
used `Lazy` in the autopilot
downgraded expensive log to `trace`
# Test
Manually tested that `body_to_string()` does not get called by injecting
a `panic!()` without causing tests to fail
added unit tests for `Lazy` in `observe` crate
…ile (cowprotocol#4192) # Description Move banned_users, order_events_cleanup_interval/threshold, and S3 upload settings from CLI arguments to the TOML config file. # Changes - Add config modules: banned_users, order_events_cleanup, s3 - Remove infra::persistence::cli (S3 CLI args) - Delete corresponding CLI arguments and Display impl entries - Wire config values through run.rs ## How to test Staging Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
# Description The task that writes the current auction to the DB has 2 performance issues: * unnecessary conversions: instead of serializing the DTO to a JSON string we convert it to `JsonValue` and let `sqlx` do the conversion to string * conversions on wrong task: serializing data is very slow and blocks an entire runtime thread if we don't do the sync work in a blocking task (the entire process often takes more than 1s) # Changes * skip 1 conversion by directly serializing the DTO to string and writing a string to the DB (same pattern already applied by some other JSON upload) * serialize the DTO on a blocking task ## How to test adjusted existing e2e tests, they show that the serialized data can be read back into the exact JSON value we serialized in the first place
# Description Follow up to this [comment](cowprotocol#4191 (comment)) suggesting a different structure for the `observe` crate. This PR goes a bit further than the comment suggested but I think this setup makes the most sense. # Changes old structure: ``` distributed_tracing - request_id.rs - trace_id_format.rs - tracing_axum.rs lazy.rs tracing.rs tracing_reload_handler.rs ``` new structure: ``` tracing - distributed - axum.rs - headers.rs - request_id.rs - trace_id_format.rs - lazy.rs - init.rs - reload_handler.rs ```
# Description The removed structs + impls were here because of incompatibility issues. We've since fixed them, it's time to remove them 🗑️ # Changes * Add opentelemetry-http dep * Remove the HeaderInjector + HeaderExtractor ## How to test Staging but this should be basically 1:1
# Description `estimate_prices_and_update_cache` iterates all tracked tokens, most of which are cache hits that resolve instantly. With .buffered() a pending cache miss at the front blocks yielding ready results behind it, keeping all remaining slots occupied and preventing new futures from entering the stream. This means expired tokens trickle into BufferedRequest too slowly producing batches of 2-3 tokens instead of the max 20. https://play.rust-lang.org/?version=stable&mode=debug&edition=2024&gist=63d75fe56901ecfe850ec03875615596 demonstrates this.
# Description This function gets called hundreds of times within one span leading to a tempo view that scrolls many pages which makes it hard to get the big picture of the call. The caller of the noisy call is also instrumented and measures the execution time of all sub calls together so we'll not lose much information. # Changes remove instrumentation of frequently called function
# Description Some solvers reported that `/solve` requests arrive with varying delays. It's quite hard to pin point where the issues lies. While some improvements on the autopilot definitely reduced the variance of the delay greatly there are still new reports about this. In order to get more data on this (especially from solvers NOT using the CoW hosted reference driver) this PR tries to approximate the data transfer times from the autopilot side. This is accomplished by writing a slim wrapper around the fully serialized payload `Bytes` that turns it into a stream. That way we can at least measure the time it takes `hyper` to write the bytes to the network stack. Since the consumer (external driver) will have back-pressure on the transfer this measurement should come reasonably close to the actual data transfer times. # Changes - added `BytesStream` that yields `Bytes` in 1MB chunks - also measures time to start the transfer and the entire transfer time - logs the result using the original tracing span (which now contains the auction_id, and driver) ## How to test added unit test for correctness of the yielded data ran in e2e test to confirm the logs contain the expected spans --------- Co-authored-by: José Duarte <duarte.gmj@gmail.com>
…protocol#4190) # Description Currently a lot of CPU time (~3.5%) is spent on fetching allowances for the liquidity sources we index. This is problematic for 2 reasons: * most liquidity sources are not used so fetching the settlement contract's allowance is wasted time * the driver seems to not use the fetched allowance and fetches them again when encoding the tx (see [1](https://github.com/cowprotocol/services/blob/main/crates/driver/src/domain/competition/solution/encoding.rs#L175), [2](https://github.com/cowprotocol/services/blob/main/crates/driver/src/domain/competition/solution/mod.rs#L294-L299), [3](https://github.com/cowprotocol/services/blob/main/crates/driver/src/domain/competition/solution/mod.rs#L427), [4](https://github.com/cowprotocol/services/blob/main/crates/driver/src/domain/competition/solution/interaction.rs#L33-L62)) # Changes deleted `solver::interactions::allowances` and it's usages ## How to test e2e tests still work
…fig file (cowprotocol#4194) # Description Move native price estimator settings (estimators, api_estimators, results_required, cache_refresh_interval, native_price_prefetch_time) from CLI arguments to the TOML config file. # Changes * Move the autopilot native price estimator settings * Add tests # Migration Guide Autopilot native price estimator CLI arguments have been moved to the TOML config file under the `[native-price-estimation]` table. ## Estimator format Previously, estimator variants were specified as `|`-separated strings. Now they are inline tables with a `type` field: | Old (CLI) | New (TOML) | |---|---| | `CoinGecko` | `{ type = "CoinGecko" }` | | `OneInchSpotPriceApi` | `{ type = "OneInchSpotPriceApi" }` | | `Driver\|solver1\|http://localhost:8080` | `{ type = "Driver", name = "solver1", url = "http://localhost:8080" }` | | `Forwarder\|http://localhost:12088` | `{ type = "Forwarder", url = "http://localhost:12088" }` | Stages (previously separated by `;`) become separate inner arrays. Estimators within a stage (previously separated by `,`) are elements of the same inner array. **Old (CLI):** ``` CoinGecko,OneInchSpotPriceApi;Driver|solver1|http://localhost:8080;Forwarder|http://localhost:12088 ``` **New (TOML):** ```toml [native-price-estimation] estimators = [ [{ type = "CoinGecko" }, { type = "OneInchSpotPriceApi" }], [{ type = "Driver", name = "solver1", url = "http://localhost:8080" }], [{ type = "Forwarder", url = "http://localhost:12088" }], ] ``` ## Arguments - `--native-price-estimators` (CLI) / `NATIVE_PRICE_ESTIMATORS` (env): ```toml [native-price-estimation] estimators = [] ``` - `--api-native-price-estimators` (CLI) / `API_NATIVE_PRICE_ESTIMATORS` (env): ```toml [native-price-estimation] api-estimators = [] ``` - `--native-price-estimation-results-required` (CLI) / `NATIVE_PRICE_ESTIMATION_RESULTS_REQUIRED` (env): ```toml [native-price-estimation] results-required = 2 ``` - `--native-price-cache-refresh` (CLI) / `NATIVE_PRICE_CACHE_REFRESH` (env): ```toml [native-price-estimation] cache-refresh-interval = "1s" ``` - `--native-price-prefetch-time` (CLI) / `NATIVE_PRICE_PREFETCH_TIME` (env): ```toml [native-price-estimation] prefetch-time = "80s" ``` ## How to test Staging --------- Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
# Description gzip is super heavy on the CPU and currently we are compressing data with the maximum level. However, gzip quickly reaches a level of diminishing returns so this PR picks a more reasonable gzip compression level based on a test. ``` Level 1: 2214921 bytes (62ms) Level 2: 2179485 bytes (62ms) Level 3: 2149025 bytes (65ms) Level 4: 1981237 bytes (116ms) Level 5: 1927338 bytes (133ms) Level 6: 1883464 bytes (145ms) Level 7: 1864462 bytes (159ms) Level 8: 1844328 bytes (225ms) Level 9: 1840352 bytes (301ms) ``` I picked a recently uploaded auction instance (~11MB) and compressed it once with each level. This showed that level 3 cuts the runtime by ~80% while degrading the compressed size by only 15% (from 1.84MB to 2.14MB). This caused other blocking operations in the autopilot to run faster (e.g. request serialization goes from ~90ms to ~70ms). <img width="1400" height="744" alt="Screenshot 2026-02-25 at 07 54 37" src="https://github.com/user-attachments/assets/59876cda-dd21-422c-830c-9e33850e2a0f" /> # Changes Reduced gzip compression level from max (i.e. 9) to 3.
…otocol#4195) # Description Migrate order validation, IPFS, volume fee, and misc orderbook settings (unsupported tokens, banned users, EIP-1271 validation, gas limits, same-tokens policy, etc.) from clap CLI arguments to a TOML configuration file. # Changes * Migrates the parameters above * Groups IPFS, order validation and volume fee * Refactors where applicable * Adds unit tests for (de)serialization # Orderbook Configuration Migration: CLI to TOML # Migration Guide The orderbook no longer accepts certain settings as CLI arguments. They must now be provided in a TOML config file, passed via the new **required** `--config` flag. ```bash orderbook --config=/path/to/orderbook.toml ... ``` ## Migrated settings | Old CLI Argument | TOML Key | |---|---| | `--min-order-validity-period` | `order-validation.min-order-validity-period` | | `--max-order-validity-period` | `order-validation.max-order-validity-period` | | `--max-limit-order-validity-period` | `order-validation.max-limit-order-validity-period` | | `--max-limit-orders-per-user` | `order-validation.max-limit-orders-per-user` | | `--max-gas-per-order` | `order-validation.max-gas-per-order` | | `--same-tokens-policy` | `order-validation.same-tokens-policy` | | `--unsupported-tokens` | `unsupported-tokens` | | `--banned-users` | `banned-users.addresses` | | `--banned-users-max-cache-size` | `banned-users.max-cache-size` | | `--eip1271-skip-creation-validation` | `eip1271-skip-creation-validation` | | `--ipfs-gateway` | `ipfs.gateway` | | `--ipfs-pinata-auth` | `ipfs.auth-token` | | `--app-data-size-limit` | `app-data-size-limit` | | `--active-order-competition-threshold` | `active-order-competition-threshold` | | `--volume-fee-factor` | `volume-fee.factor` | | `--volume-fee-effective-timestamp` | `volume-fee.effective-from-timestamp` | ## Example Before: ```bash orderbook \ --min-order-validity-period=2m \ --max-order-validity-period=6h \ --banned-users=0xdead000000000000000000000000000000000000 \ --ipfs-gateway=https://gateway.pinata.cloud/ipfs/ \ --ipfs-pinata-auth=my-secret-key \ --volume-fee-factor=0.0002 \ --volume-fee-effective-timestamp=2025-06-01T00:00:00Z \ --same-tokens-policy=allow-sell \ --bind-address=0.0.0.0:8080 ``` After — create `orderbook.toml`: ```toml same-tokens-policy = "allow-sell" [order-validation] min-order-validity-period = "2m" max-order-validity-period = "6h" [banned-users] addresses = ["0xdead000000000000000000000000000000000000"] [ipfs] gateway = "https://gateway.pinata.cloud/ipfs/" pinata-auth = "my-secret-key" [volume-fee] factor = 0.0002 effective-from-timestamp = "2025-06-01T00:00:00Z" ``` ## How to test Tests + Staging --------- Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
## Summary This PR completes **Milestone 3: Documentation** of the [CoW Grants Program RFP: CoW Protocol Playground Block Explorer Integration](https://forum.cow.fi/t/grant-application-cow-protocol-playground-block-explorer-integration/3284/1) proposal by CoBuilders. ### Documentation Strategy Most of the technical documentation for this integration was delivered inline with the implementation in M1 and M2: | Milestone | Documentation Delivered | |-----------|------------------------| | **M1** ([cowprotocol#4000](cowprotocol#4000)) | README sections for Sourcify configuration, contract verification, component table updates | | **M2** ([cowprotocol#4077](cowprotocol#4077), [cowswap#6774](cowprotocol/cowswap#6774)) | Environment variable documentation in `.env` files, JSDoc comments in source code | **This PR** adds the remaining user-facing documentation: a practical guide on how to use Otterscan for transaction inspection and debugging. ## Changes Adds a "Using Otterscan" section to the playground README covering: - How to access and use Otterscan (`http://localhost:8003`) - Inspecting transactions (overview, traces, logs, gas profiling) - Debugging failed transactions with trace analysis - Example workflow: Tracing a CoW Swap settlement ## Milestones | Milestone | Description | Status | |-----------|-------------|--------| | M1 | Otterscan Integration | [cowprotocol#4000](cowprotocol#4000) ✅ | | M2 | Frontend Integration | [cowprotocol#4077](cowprotocol#4077) + [cowswap#6774](cowprotocol/cowswap#6774) ✅ | | M3 | Documentation | **This PR** | --- *Submitted by [CoBuilders](https://cobuilders.xyz) as part of the CoW Grants Program* --------- Co-authored-by: Augusto Collerone <collerone.augusto@gmail.com> Co-authored-by: Ignacio <ignacio@cobuilders.xyz>
# Description Aave wants to track specific orders in bulk, knowing their ids. # Changes Adds POST handler for `v1/orders/lookup` endpoint that requires a list of order uids and responds with a vector of their data. Has a hardcoded limit of 128 orders per request (to fit the MAX_JSON_BODY_PAYLOAD size). ## How to test Test on staging, query multiple orders using this API. --------- Co-authored-by: José Duarte <duarte.gmj@gmail.com> Co-authored-by: Martin Magnus <martin.beckmann@protonmail.com> Co-authored-by: ilya <ilya@cow.fi>
# Description The playground was left behind amidst the config changes. This PR brings the playground back up to speed. # Changes * Remove unused variables * Migrate the orderbook migration * Migrate the autopilot migration ## How to test Run the playground and execute a trade
# Description The Dockerfile was compiling the whole workspace which is wasteful for deployment (it was compiling e2e for example). This PR only compiles the used packages. # Changes * Compiling workspace -> autopilot, driver, orderbook, refunder, solvers only ## How to test Build the dockerfile
# Description Based on @MartinquaXD's comment on Slack, I figured that there would be places where we can import the specific alloy crate, unlocking earlier and more parallel compilation. These changes alone seem to save ~5s on a clean build in my laptop; going down from ~1m to 55s. There's more gains to take from this, but for that we need to separate shared further and then split those dependencies. cowprotocol#4217 would also help a bunch. # Changes * app-data * replace alloy with alloy-primitives * since alloy uses tiny-keccak, replace the tiny-keccak with alloy's keccak * chain * manually implement thiserror (less 1 dep) * replace alloy with alloy-primitives * model * replace alloy with alloy-primives, alloy-signer, alloy-signer-local, alloy-sol-types * number * replace alloy with alloy-primitives * order-validation * replace alloy with alloy-primitives and alloy-contract * remove thiserror wrapper (less 1 dep) * serde-ext * replace alloy with alloy-primitives * testlib * replace alloy with alloy-primitives * winner-selection * replace alloy with alloy-primitives ## How to test Compiler + existing tests
…wprotocol#4213) # Description When users provide app data with wrong partner fees, we get less than good error reporting, for example: ``` {"errorType":"AppDataInvalid","description":"app data has the wrong format: data did not match any variant of untagged enum Helper at line 1 column 232"} ``` This is, not only a pain for us to debug but also for the user: * For us because the special deserializers are all named Helper * For the user because of the bad error message This PR changes the deserialization mechanism in a "non-obvious" way to provide proper error messages, as untagged enums stand very much against them. For example floating point bps are unsupported, they returned the previous error; after this PR they return: ``` {"errorType":"AppDataInvalid","description":"app data has the wrong format: invalid type: floating point `99.0`, expected u64 at line 1 column 128"} ``` # Changes * Rename the deserializers * Refactor the deserializers into weird structs (but better for error reporting) ## How to test Unit tests to ensure nothing broke + playground to test the error ``` curl 'http://localhost:8000/api/v1/app_data \ -X 'PUT' \ -H 'accept: application/json' \ -H 'content-type: application/json' \ --data-raw '{"fullAppData":"{\"appCode\":\"YOUR_APP_CODE\",\"metadata\":{\"partnerFee\":[{\"recipient\":\"0x28c716bC23ed77CAEc27f476A366318ad5F12d58\",\"volumeBps\":99.5},{\"priceImprovementBps\":9900,\"recipient\":\"0x28c716bC23ed77CAEc27f476A366318ad5F12d58\",\"volumeBps\":100}]},\"version\":\"1.14.0\"}"}' ```
# Description I have run a bunch of experiment goal of which was to make CoinGecko batches bigger (fetch price for more tokens at once) to save on costs. In a previous experiment I increased `concurrent_requests` on the `CachingNativePriceEstimator` to 500 and that totally worked and CoinGecko debounces all these requests nicely, but also means we would be spamming solvers with a lot of requests at once when we start the service. The first issue at hand is that before this PR we would create futures that would hit the cache and return immediately, which was bad, because they take up a slot in the queue when using buffered() so fewer useful futures that actually issue a native price request would get to run. I changed this to buffered_unordered() and that helped a little bit, but then the queue gets full and after that new futures get in only as previous futures finishe, so the execution is spread out in time and our CoinGecko debouncing can't gather enough tokens to form a full batch. This PR introduces a solution where we omit non-expired tokens altogether which solves the issue of futures that hit the cache taking up a spot in the queue _and_ we just run batches of 19 sequentially. We wait for each batch to finish (max ~3s, limited by QUERY_TIMEOUT) and only then issues a new batch. As a tradeoff we get a slower cache warm up/refresh, but we get to save a lot of money on CoinGecko. In addition to this change I set the refresh rate to 30s from 1s, so we can gather more expired tokens at once. <img width="2103" height="480" alt="Screenshot 2026-03-02 at 11 30 33" src="https://github.com/user-attachments/assets/190bbefe-ac26-4837-b2c2-05a7c63ebbcb" /> # Changes * [x] Don't try to fetch non-expired prices * [x] Run batches sequentially * [x] NATIVE_PRICE_CACHE_REFRESH set to 30s instead of 1s in service config --------- Co-authored-by: Martin Magnus <martin.beckmann@protonmail.com>
# Description Our current `/solve` request can be heavy on certain chains, and given the fact that it is expected to eventually switch to a cross-chain auction, the data will be growing significantly. While it is expected to implement a delta-sync approach with event-based auction updates, this is quite involved, and as a first step, we can look into compressing the request to reduce its size and reduce network latency/delays. The tests showed that `brotli` with compression level 1 outperforms `gzip` cowprotocol#4212 (comment) # Changes ##Autopilot - New `--compress-solve-request` boolean CLI flag threaded through `run_loop::Config` - `solve::Request` gains a `compressed()` method that br-compresses the body on a blocking task (CPU-intensive, like the existing serialization) - On compression failure, falls back to sending the uncompressed body with an error log - Compression time is tracked via the existing `auction_overhead` metric - Sets the `Content-Encoding: br` header only when the body is actually compressed - Add a `runloop_solve_request_body_size` histogram metric that records the size (in bytes) of the JSON body sent in each `/solve` request to drivers. ## Driver - Adds `RequestDecompressionLayer` to the axum middleware stack, which automatically decompresses br-encoded request bodies based on the `Content-Encoding` header ## How to test New unit and e2e tests. Deploy manually with disabled compression to collect some metrics and then with compression enabled. ## Related issues Fixes cowprotocol#4206
# Description
`shared` is a big ball of spaghetti where dependencies are extremely
unclear.
This PR untangles this somewhat
# Changes
- moves some code out of shared to the only location where it's used
- `trace_many` into `bad_token::trace_call`
- `CodeFetching` into `trade_verifier`
- `BlockRetrieving` into `event-indexing`
- `tenderly_api` into `trade_verifier`
- `subgraph` into `sources`
- `RecentBlockCache` into `sources`
- create separate crates for `balance-overrides`, `event-indexing`,
`account-balances`, `request-sharing`, `token-info`, `liquidity-sources`
- moved `BaseTokens` and `BaselineSolvable` into `liquidity-sources` as
that made the most sense to me
Some of the new crates are super tiny but are necessary to later move
`price-estimation` into a separate crate as well.
This already cut the compilation time of `shard` in half which are gains
we'll often see in incremental builds
<img width="296" height="240" alt="Screenshot 2026-03-03 at 10 31 50"
src="https://github.com/user-attachments/assets/6e69a751-5e7a-435b-b74b-4d51fb5dc1ee"
/>
<img width="277" height="353" alt="Screenshot 2026-03-03 at 10 31 55"
src="https://github.com/user-attachments/assets/2026624c-d3aa-464c-af71-408f2e51c3f1"
/>
## How to test
compiler
# Description Resolves cowprotocol#3966. Enable parallel settlement submission from a single solver by using EIP-7702 delegation. When a solver wins multiple solutions in overlapping auctions, each settlement can now be submitted concurrently through independent submission EOAs instead of waiting in a sequential queue. The solver EOA delegates its code to a CowSettlementForwarder contract via EIP-7702. Approved submission EOAs call forward(target, data) on the solver EOA, which forwards to the actual target (settlement contract, wrapper, or flashloan router) with msg.sender = solver EOA — preserving the existing whitelisting. # Changes - [x] `CowSettlementForwarder.sol` — New forwarder contract with caller whitelist. Supports forwarding to any target (settlement contract, wrappers, flashloan router) via `forward(address target, bytes data)` - [x] `SubmissionAccountPool` type: a channel-based pool that lends submission accounts for the duration of a settlement and reclaims them afterward. In EIP-7702 mode, settle requests are spawned as concurrent tasks instead of processed sequentially - [x] `submission_accounts` config that allows specifying accounts that are whitelisted to submit settlements. If omitted we fallback to non-delegated submissions directly from the solver EOA # Requirements before deployment * Deploy & verify the CowSettlementForwarder contract * Set up EIP7022 delegation to whitelisted solver EOAs to the newly deployed contract * Create new submissions accounts and fund them with eth (or other base token) & monitor their eth balance * Whitelist the submissions accounts on the solver EOA (CowSettlementForwarder code) Only then can the submissions accounts be configured and deployed. ## How to test cargo nextest run -p e2e local_node_parallel_settlement --test-threads 1 --run-ignored ignored-only <!-- ## Related Issues Fixes # -->
# Description Further untangles the `shared` crate. # Changes * deletes `bytes.rs` which was forgotten in an old PR * moved `trade_finding` into `price-estimation` * move `tenderly_api` arguments from `shared::Arguments` to `price_estimation::Arguments` * removed a few dependencies * downgraded a few dependencies to `dev-dependencies` new crates split off of `shared`: * `gas-price-estimation` * `price-estimation` * `bad-tokens` Unfortunately `price-estimation` is relying on a few utils which didn't make much sense to move into a new crate so I duplicated a few things in `price_estimation::utils` but I think this does not outweigh the structure and compile time gained by this PR. This PR does again not migrate the catch all `alloy` imports to the more specific sub-crates (e.g. `alloy_primitives`) ## How to test compiler
# Description Removes the `AlloyU256` alias. # Changes * Replace `U256 as AlloyU256` import alias with plain `U256` in `price-estimation/src/sanitized.rs` and `shared/src/order_quoting.rs` Fixes cowprotocol#4502
# Description Adds a barebones `solana-indexer` crate to the repository. # Changes Nothing worthy of note, this is just an empty crate. Other PRs will follow.
# Description Currently, claude review only runs when explicitly mentioning claude in a comment (e.g. https://github.com/cowprotocol/services/actions/runs/27002342510) , not on PR creation (e.g. https://github.com/cowprotocol/services/actions/runs/26947333940/attempts/1) The reason seems to be that `github.event.pull_request.author_association` doesn't resolve to Member (reproed here: https://github.com/cowprotocol/claude-gate-repro/actions/runs/27007356481/job/79702173516#step:2:16). Claude says this is due to an inconsistency in github event data (comment authors contain the correct association). We cannot rely on the `Contributor` association because people that committed code in the past but are not part of the organisation would also resolve to that. This PR changes the check to use repository permissions for the given author (and automatically reviews for authors that can write) # Changes * Add a new step that checks repository permissions for PR or comment auther * Remove old gate ## How to test Verified that both comments as well as new PRs pass the authentication in the test repo - Comment: https://github.com/cowprotocol/claude-gate-repro/actions/runs/27008892569 - PR creation: https://github.com/cowprotocol/claude-gate-repro/actions/runs/27008147177
…4370) # Description The autopilot no longer needs per-solution uniform clearing prices: scoring uses auction-level native prices, and on-chain settlement verification reads UCPs straight from the calldata. This PR removes UCPs from the autopilot's domain model and persistence path while keeping the wire format intact, so the change is safe under a rolling k8s deploy. # Changes * Drop `prices` from the autopilot's domain `Solution` (field, constructor arg, and `prices()` getter). * Remove `SolutionError::InvalidPrice` and its `invalid_price` metric label — price validation is no longer performed when ingesting a `/solve` response. * Persist empty arrays into `proposed_solutions.price_tokens` / `price_values`. The `NOT NULL` columns are retained for now (an empty array still satisfies the constraint); they can be dropped in a follow-up. * Make the autopilot tolerant of drivers that omit `clearingPrices`: add `#[serde(default)]` to the deserializer and emit a `debug!` log when a driver still sends a non-empty map, so we can chase down stragglers before fully removing the field. * Leave the driver emitting `clearingPrices` on `/solve` responses (with a deprecation comment) so autopilots running the previous code can still deserialize during the rolling deploy. * Mark `clearingPrices` as `deprecated: true` in the driver `/solve` response schema and in the orderbook `solver_competition_v2` schema, with a note explaining that recent autopilots will return it empty. A follow-up PR will remove the driver-side field, the deprecation log, and (optionally) drop the now-unused `proposed_solutions.price_tokens` / `price_values` columns. ## How to test 1. `cargo nextest run -p autopilot -p driver` — unit tests, including `winner_selection` tests updated to construct solutions without prices. 2. Hit `solver_competition_v2` for an auction produced by the new autopilot and confirm `clearingPrices` is `{}`; for an auction produced before the change it remains populated. ### Staging Ran in staging with these orders: * https://dev.explorer.cow.fi/orders/0x34ce7c3c6b4c762663d087b2c6ffba30fc966d9bf98039eee98ad1722e79805309fbad1ea29c36dfe4f8f7baa87c5edf85e0d9f369fc62fe * https://dev.explorer.cow.fi/orders/0xaf624d224a01e7ecaff3d0be2a04a1e073090f3e9cfdad220cea741748f3e83109fbad1ea29c36dfe4f8f7baa87c5edf85e0d9f369fc66f9 Logs: * https://victorialogs.dev.cow.fi/goto/eflbzd4yhhy4ga?orgId=1 * https://victorialogs.dev.cow.fi/goto/bflc0jc6wa874b?orgId=1 Context: https://nomevlabs.slack.com/archives/C036JAGRQ04/p1780923984399839?thread_ts=1775741386.025859&cid=C036JAGRQ04
…tion to 593d7a5 (cowprotocol#4507) This PR contains the following updates: | Package | Type | Update | Change | |---|---|---|---| | [anthropics/claude-code-action](https://redirect.github.com/anthropics/claude-code-action) | action | digest | `fbda2eb` → `593d7a5` | --- > [!WARNING] > Some dependencies could not be looked up. Check the warning logs for more information. --- ### Configuration 📅 **Schedule**: (UTC) - Branch creation - "on monday" - Automerge - At any time (no schedule defined) 🚦 **Automerge**: Disabled by config. Please merge this manually once you are satisfied. ♻ **Rebasing**: Whenever PR is behind base branch, or you tick the rebase/retry checkbox. 🔕 **Ignore**: Close this PR and you won't be reminded about this update again. --- - [ ] <!-- rebase-check -->If you want to rebase/retry this PR, check this box --- This PR was generated by [Mend Renovate](https://mend.io/renovate/). View the [repository job log](https://developer.mend.io/github/cowprotocol/services). <!--renovate-debug:eyJjcmVhdGVkSW5WZXIiOiI0My4yMDkuNCIsInVwZGF0ZWRJblZlciI6IjQzLjIwOS40IiwidGFyZ2V0QnJhbmNoIjoibWFpbiIsImxhYmVscyI6W119--> Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
# Description 1. Updated responses of `/api/v1/quote` corresponding to the actual implementation 2. Updated `PriceEstimationError` enum 3. Added new error codes to `PriceEstimationError` response corresponding to the changes from cowprotocol#4268. The changes were generated via the script: [sync_quote_errors.py](https://github.com/user-attachments/files/28601528/sync_quote_errors.py) # Changes New codes: ``` - TradingOutsideAllowedWindow - TokenTemporarilySuspended - InsufficientLiquidity - CustomSolverError ``` ## How to test 1. Open Swagger 3. Expand `PriceEstimationError` - [ ] it should display the new error codes <img width="698" height="632" alt="image" src="https://github.com/user-attachments/assets/63c1d26c-bcac-42d4-9832-a64e53e8fe3c" />
…cowprotocol#4510) # Description On some chains (Avalanche especially) the node reports a new block before its by-hash receipt lookup catches up. Right after our settlement mines, the driver asks "did my tx land?" by hash, gets "not found", and re-simulates the tx against the pending state to check it still works. That state already includes our just-mined (or still-queued) settlement, so the re-run reverts (`GPv2: order filled` on a full fill, `transfer amount exceeds balance` on a partial). The driver reads that as a real revert, cancels (the cancel then fails with `nonce too low`), and reports the settlement as failed even though it landed. This is the main source of the Avalanche "too many failing settlements" alert, and the settlements actually succeed on-chain. Instead of trusting the lagging by-hash lookup, the driver now leans on two signals that do not lag: - Our `Settlement` event in the latest block. The contract emits it only on a successful settle, and `eth_getLogs` reads block logs directly, so it sees the event while the receipt-by-hash lookup is still catching up. If it is there, the tx mined, so report success. - The signer's pending nonce (`eth_getTransactionCount` with the `pending` tag). While our tx is still in the mempool the pending nonce sits above the nonce we submitted with, so the re-simulation revert is just our own queued tx re-applied, a false positive, and we keep waiting. Only when the pending nonce shows our tx has left the mempool, and the re-simulation still reverts, do we cancel. An earlier version of this PR used `eth_getLogs` on the `pending` block for the second signal, but measuring our nodes showed `pending` log queries are rejected on 4 of 11 chains (polygon, bnb, arbitrum, linea, with "pending logs are not supported"). `eth_getTransactionCount` on `pending` is accepted on all 11, and since the re-simulation already runs against the `pending` state, the two signals read the same state. Old code cancelled whenever the re-simulation reverted, minus a brittle `GPv2: order filled` string match that only caught full fills. New code cancels only when the tx is provably gone from the mempool, so a settlement that is mined-but-receipt-lagging or still queued is never cancelled. # Changes - Added `Ethereum::contains_successful_tx`, which checks for our `Settlement` event in the latest block via `eth_getLogs`. - Added `Ethereum::pending_transaction_count`, the signer's nonce including the mempool. - Reworked the submission loop: report success when the settlement event is already in the latest block, and cancel only when the re-simulation reverts and the pending nonce shows the tx has left the mempool. Anything else keeps waiting. - Dropped the old `GPv2: order filled` revert-byte special case. # How to test New unit tests cover the cancel decision (`requires_cancellation`): a revert with the tx gone from the mempool cancels, a revert while the tx is still queued waits, and an unknown pending nonce never cancels. The log and nonce lookups and the receipt-lag race itself have no automated coverage. The e2e harness runs anvil, which returns receipts immediately, so it cannot reproduce the lag. Verify on Avalanche: settlements that land on-chain should stop being counted as `driver_settlements{result="SubmissionError"}` and should stop logging a cancel right after a re-simulation revert, while still showing up as on-chain settlements.
…owprotocol#4511) # Description I'd like to see how a non-winning, non-colocated solver would have settled a transaction in case they had won. I believe this is currently not possible (we only log the transaction when a solver was selected for winning). This information shouldn't be publicly available (solvers only want to commit to a solution if they win), but logging it for debugging purposes should be fine. I only log the internalized representation, because this is the one that would make it on-chain. Since we simplified buffer accounting (solver takes the risk), we probably no longer need to look at uninternalized calldata. # Changes * add a tx field to the simulation log
…owprotocol#4516) # Description Before submitting a winning settlement, the driver checks that the solver has enough ETH to cover gas. It sizes that check against its own gas price estimate. Since cowprotocol#4299, a solver can override the gas price for its solution with a higher `maxFeePerGas`, and the settlement is then submitted at that override. An Ethereum node reserves `gas_limit * maxFeePerGas` of the sender's balance when it accepts a transaction, so a solver funded for the driver's cheap estimate but not for its own higher override passes the check, wins the auction, then gets rejected at submission with `insufficient funds`. This happened on base on 2026-06-11. Solver `zurui-solve` set `maxFeePerGas = 10 gwei`, roughly 1900x the driver's ~0.005 gwei estimate on base, while holding ~0.0046 ETH. The balance check passed, the solver won two auctions, and both settlements were rejected by every mempool (`have 4618974035905571 want 8438180000000000`, where `want = gas_limit * 10 gwei`). Before cowprotocol#4299 the check price and the submitted price were the same, so insufficient funds only surfaced at simulation time on nearly empty wallets. # Changes - Size the pre-submission balance check to the `maxFeePerGas` the tx will actually be submitted with: the solver's gas fee override when set, otherwise the driver's estimate doubled. This mirrors `apply_gas_fee_override`, which submits at the override directly when one is present. Behavior is unchanged when no override is set. # How to test New unit test covering the fee selection (override when set, else driver estimate doubled, including an override below the doubled estimate). An underfunded solver that sets a high `maxFeePerGas` override is now rejected at settlement encoding with `SolverAccountInsufficientBalance` instead of winning and failing to broadcast. The existing `settle_with_gas_fee_override` integration test still covers the funded happy path.
cowprotocol#4518) This PR contains the following updates: | Package | Type | Update | Change | Pending | |---|---|---|---|---| | [taiki-e/install-action](https://redirect.github.com/taiki-e/install-action) | action | minor | `2.79.7` → `v2.81.1` | `v2.81.10` (+8) | # Warnings (1) Please correct - or verify that you can safely ignore - these warnings before you merge this PR. - `taiki-e/install-action`: Could not determine new digest for update (github-tags package taiki-e/install-action) --- --- > [!WARNING] > Some dependencies could not be looked up. Check the warning logs for more information. --- ### Release Notes <details> <summary>taiki-e/install-action (taiki-e/install-action)</summary> ### [`v2.81.1`](https://redirect.github.com/taiki-e/install-action/releases/tag/v2.81.1): 2.81.1 [Compare Source](https://redirect.github.com/taiki-e/install-action/compare/v2.81.0...v2.81.1) - Update `cargo-no-dev-deps@latest` to 0.2.24. - Update `cargo-hack@latest` to 0.6.45. ### [`v2.81.0`](https://redirect.github.com/taiki-e/install-action/releases/tag/v2.81.0): 2.81.0 [Compare Source](https://redirect.github.com/taiki-e/install-action/compare/v2.80.0...v2.81.0) - Support `convco`. ([#​1831](https://redirect.github.com/taiki-e/install-action/pull/1831), thanks [@​graelo](https://redirect.github.com/graelo)) - Support `docgarden` ([#​1830](https://redirect.github.com/taiki-e/install-action/pull/1830), thanks [@​jesse-black](https://redirect.github.com/jesse-black)) - Update `vacuum@latest` to 0.28.0. - Update `cargo-binstall@latest` to 1.19.1. ### [`v2.80.0`](https://redirect.github.com/taiki-e/install-action/releases/tag/v2.80.0): 2.80.0 [Compare Source](https://redirect.github.com/taiki-e/install-action/compare/v2.79.15...v2.80.0) - Support `kingfisher`. ([#​1874](https://redirect.github.com/taiki-e/install-action/pull/1874), thanks [@​SAY-5](https://redirect.github.com/SAY-5)) ### [`v2.79.15`](https://redirect.github.com/taiki-e/install-action/releases/tag/v2.79.15): 2.79.15 [Compare Source](https://redirect.github.com/taiki-e/install-action/compare/v2.79.14...v2.79.15) - Update `typos@latest` to 1.47.0. - Update `wasm-tools@latest` to 1.251.0. - Update `vacuum@latest` to 0.27.2. - Update `uv@latest` to 0.11.17. - Update `tombi@latest` to 1.1.1. - Update `mise@latest` to 2026.5.16. ### [`v2.79.14`](https://redirect.github.com/taiki-e/install-action/releases/tag/v2.79.14): 2.79.14 [Compare Source](https://redirect.github.com/taiki-e/install-action/compare/v2.79.13...v2.79.14) - Update `vacuum@latest` to 0.27.0. - Update `cargo-deny@latest` to 0.19.8. ### [`v2.79.13`](https://redirect.github.com/taiki-e/install-action/releases/tag/v2.79.13): 2.79.13 [Compare Source](https://redirect.github.com/taiki-e/install-action/compare/v2.79.12...v2.79.13) - Update `gungraun-runner@latest` to 0.19.1. - Update `biome@latest` to 2.4.16. ### [`v2.79.12`](https://redirect.github.com/taiki-e/install-action/releases/tag/v2.79.12): 2.79.12 [Compare Source](https://redirect.github.com/taiki-e/install-action/compare/v2.79.11...v2.79.12) - Update `prek@latest` to 0.4.3. - Remove uses of crates.io API, which potentially cases 403 error. ### [`v2.79.11`](https://redirect.github.com/taiki-e/install-action/releases/tag/v2.79.11): 2.79.11 [Compare Source](https://redirect.github.com/taiki-e/install-action/compare/v2.79.10...v2.79.11) - Update `vacuum@latest` to 0.26.8. - Update `cargo-nextest@latest` to 0.9.137. ### [`v2.79.10`](https://redirect.github.com/taiki-e/install-action/releases/tag/v2.79.10): 2.79.10 [Compare Source](https://redirect.github.com/taiki-e/install-action/compare/v2.79.9...v2.79.10) - Update `tombi@latest` to 1.1.0. - Update `prek@latest` to 0.4.2. - Update `editorconfig-checker@latest` to 3.7.0. ### [`v2.79.9`](https://redirect.github.com/taiki-e/install-action/releases/tag/v2.79.9): 2.79.9 [Compare Source](https://redirect.github.com/taiki-e/install-action/compare/v2.79.8...v2.79.9) - Update `vacuum@latest` to 0.26.7. - Update `tombi@latest` to 1.0.0. ### [`v2.79.8`](https://redirect.github.com/taiki-e/install-action/releases/tag/v2.79.8): 2.79.8 [Compare Source](https://redirect.github.com/taiki-e/install-action/compare/v2.79.7...v2.79.8) - Update `parse-dockerfile@latest` to 0.1.6. - Update `knope@latest` to 0.23.0. </details> --- ### Configuration 📅 **Schedule**: (UTC) - Branch creation - "on monday" - Automerge - At any time (no schedule defined) 🚦 **Automerge**: Disabled by config. Please merge this manually once you are satisfied. ♻ **Rebasing**: Whenever PR is behind base branch, or you tick the rebase/retry checkbox. 🔕 **Ignore**: Close this PR and you won't be reminded about this update again. --- - [ ] <!-- rebase-check -->If you want to rebase/retry this PR, check this box --- This PR was generated by [Mend Renovate](https://mend.io/renovate/). View the [repository job log](https://developer.mend.io/github/cowprotocol/services). <!--renovate-debug:eyJjcmVhdGVkSW5WZXIiOiI0My4yMTkuMCIsInVwZGF0ZWRJblZlciI6IjQzLjIxOS4wIiwidGFyZ2V0QnJhbmNoIjoibWFpbiIsImxhYmVscyI6W119--> Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
…cowprotocol#4519) This PR contains the following updates: | Package | Type | Update | Change | Pending | |---|---|---|---|---| | [tombi-toml/setup-tombi](https://redirect.github.com/tombi-toml/setup-tombi) | action | minor | `v1.0.11` → `v1.1.1` | `v1.1.3` (+1) | --- > [!WARNING] > Some dependencies could not be looked up. Check the warning logs for more information. --- ### Release Notes <details> <summary>tombi-toml/setup-tombi (tombi-toml/setup-tombi)</summary> ### [`v1.1.1`](https://redirect.github.com/tombi-toml/setup-tombi/releases/tag/v1.1.1) [Compare Source](https://redirect.github.com/tombi-toml/setup-tombi/compare/v1.1...v1.1.1) #### What's Changed - \[codex] Automate release note generation on version tags by [@​ya7010](https://redirect.github.com/ya7010) in [#​38](https://redirect.github.com/tombi-toml/setup-tombi/pull/38) **Full Changelog**: <tombi-toml/setup-tombi@v1...v1.1.1> ### [`v1.1.0`](https://redirect.github.com/tombi-toml/setup-tombi/releases/tag/v1.1.0) [Compare Source](https://redirect.github.com/tombi-toml/setup-tombi/compare/v1.1...v1.1) #### What's Changed Starting with this version, we will synchronize [tombi](https://redirect.github.com/tombi-toml/tombi) with version control. - update: version by [@​ya7010](https://redirect.github.com/ya7010) in [#​33](https://redirect.github.com/tombi-toml/setup-tombi/pull/33) - Clarify enable-cache behavior in README by [@​ya7010](https://redirect.github.com/ya7010) in [#​35](https://redirect.github.com/tombi-toml/setup-tombi/pull/35) - Sync package version before retagging by [@​ya7010](https://redirect.github.com/ya7010) in [#​34](https://redirect.github.com/tombi-toml/setup-tombi/pull/34) - \[codex] Align setup-tombi versioning by [@​ya7010](https://redirect.github.com/ya7010) in [#​36](https://redirect.github.com/tombi-toml/setup-tombi/pull/36) - Unify release tag and version sync workflows by [@​ya7010](https://redirect.github.com/ya7010) in [#​37](https://redirect.github.com/tombi-toml/setup-tombi/pull/37) **Full Changelog**: <tombi-toml/setup-tombi@v1.0...v1.1.0> ### [`v1.1`](https://redirect.github.com/tombi-toml/setup-tombi/compare/v1.0.11...v1.1) [Compare Source](https://redirect.github.com/tombi-toml/setup-tombi/compare/v1.0.11...v1.1) </details> --- ### Configuration 📅 **Schedule**: (UTC) - Branch creation - "on monday" - Automerge - At any time (no schedule defined) 🚦 **Automerge**: Disabled by config. Please merge this manually once you are satisfied. ♻ **Rebasing**: Whenever PR is behind base branch, or you tick the rebase/retry checkbox. 🔕 **Ignore**: Close this PR and you won't be reminded about this update again. --- - [ ] <!-- rebase-check -->If you want to rebase/retry this PR, check this box --- This PR was generated by [Mend Renovate](https://mend.io/renovate/). View the [repository job log](https://developer.mend.io/github/cowprotocol/services). <!--renovate-debug:eyJjcmVhdGVkSW5WZXIiOiI0My4yMTkuMCIsInVwZGF0ZWRJblZlciI6IjQzLjIxOS4wIiwidGFyZ2V0QnJhbmNoIjoibWFpbiIsImxhYmVscyI6W119--> Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
…col#4411) # Description Remove `Clone` implementation for `RequestSharing`, allow the GC task to exit once `Arc` is dropped. # Changes * Remove clone implementation. * `spawn_gc` modified to check `Arc` and exit if dropped. ## How to test cargo nextest run -p request-sharing --------- Co-authored-by: Martin Magnus <martin.beckmann@protonmail.com>
# Description In some of the captured heap memory profiles a surprising amount of memory is allocated for logging the generated cow amm orders. This is because we currently log all of their fields. This was somewhat useful when the feature was just introduced and still needed to be debugged but by now nobody actually looks at all those order details. This PR changes the code to only log the order_uids. To still allow debugging the order details when it's necessary the original log was not deleted but only downgraded to the `trace` level. <img width="1124" height="818" alt="Screenshot 2026-06-15 at 07 18 42" src="https://github.com/user-attachments/assets/b37f8ea3-48c1-4f5c-9b37-c27aab017908" />
…l#4522) # Description The current score handling logic is quite confusing as it first creates intermediate collections which are unnecessary and splits logic across multiple iterator variables instead of chaining. This PR merges all score related handling into 1 (IMO more readable) iterator chain which avoids allocations as much as possible (no intermediate `collect_vec`)
…otocol#4521) # Description This PR applies 3 optimizations. First it moves the creation of the auction into a separate function which cuts down the size of the original function and makes the code easier to read. Secondly it avoids `Arc::unwrap_or_clone(auction)` and instead "manually" constructs the auction. That way we can avoid 1 expensive re-allocation when we push the cow amm orders into the fully cloned auction. Since the majority of the memory used by the auctions are used by orders and `Vec` grows by 2x this should significantly reduce the total memory footprint since we only have a few cow amm orders anyway. Lastly it wraps `tokens` and `surplus_capturing_jit_order_owners` into `Arc`s as this data does not need to be modified by solvers individually so they can just share the same allocation.
# Description This PR applies a few optimizations for memory usage while the driver sends the `/solve` request. # Changes First I adjusted the `s3` machinery to allow you to upload already serialized `Bytes` instead of having it re-serialize the data again. Since `Bytes` is internally ref-counted and compatible with `reqwest` we can serialize the request once and then use the same data for the HTTP request and for the S3 upload. That also allows us to move the creation of the `DTO` into the scope of the request serialization so the `DTO` already gets deallocated before we even send the request (instead of sticking around until we processed the results).
…#4524) # Description The driver can service multiple solvers at once. Therefore when multiple solvers are connected we somehow need to manage all the duplicate work the driver is doing for each solver. This is already happening for the most part but 1 aspect could not be de-duplicated well yet: finalizing the list of orders for each solver. This has 2 issues which don't allow us to straight up ref-count the order array and be done with it: 1. solvers can have different order prioritization strategies, also some strategies depend on the address of the solver (e.g. prioritize orders my solver provided the best quote for). That means the list of orders can have a different order for different solvers. 2. if a user has only funds for a subset of their open orders the driver allocates funds based on order priority. There is also the issue that our pipelining was designed that we can already start processing orders when we haven't already fetched all appdata JSON documents for each order. Orders where the doc is missing will have it filled in at a later stage. Those details means the driver will have to deep clone the orders (which make up the majority of the auction size) for EACH SOLVER! That obviously requires a lot more memory than reusing the memory smartly. # Changes This PR addresses those issues by splitting the orders into 2 parts: 1. a reference counted struct that contains the immutable data - which is the majority of the order 2. mutable fields where each solver gets their own instance to modify To hide those implementation details from the caller the `Order` implements `Deref`. That way the caller doesn't have to know in which part of the order the data it needs lives when it's only reading.
…tion to 4d7e1f0 (cowprotocol#4517) This PR contains the following updates: | Package | Type | Update | Change | |---|---|---|---| | [anthropics/claude-code-action](https://redirect.github.com/anthropics/claude-code-action) | action | digest | `593d7a5` → `4d7e1f0` | --- ### Configuration 📅 **Schedule**: (UTC) - Branch creation - "on monday" - Automerge - At any time (no schedule defined) 🚦 **Automerge**: Disabled by config. Please merge this manually once you are satisfied. ♻ **Rebasing**: Whenever PR is behind base branch, or you tick the rebase/retry checkbox. 🔕 **Ignore**: Close this PR and you won't be reminded about this update again. --- - [ ] <!-- rebase-check -->If you want to rebase/retry this PR, check this box --- This PR was generated by [Mend Renovate](https://mend.io/renovate/). View the [repository job log](https://developer.mend.io/github/cowprotocol/services). <!--renovate-debug:eyJjcmVhdGVkSW5WZXIiOiI0My4yMTkuMCIsInVwZGF0ZWRJblZlciI6IjQzLjIxOS4wIiwidGFyZ2V0QnJhbmNoIjoibWFpbiIsImxhYmVscyI6W119--> Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
…ol#4422) # Description Replaces our driver's third-party Uniswap V3 subgraph dependency with our own pool-indexer service. This is the **subgraph-bootstrap slice** of the larger cowprotocol#4349, scoped down to keep review focused. Includes a few fixes surfaced during local testing. Out of scope (deferred to follow-up PRs): - Cold-seed-from-genesis (chains without a subgraph) - Multi-factory-per-network # Changes **New service: `crates/pool-indexer/`** - Bootstraps pool / tick state from an existing Uniswap V3 subgraph - Follows head via `eth_getLogs` and persists incremental state to Postgres - Serves four HTTP routes consumed by the driver: `/pools`, `/pools/by-ids`, `/pools/{addr}/ticks`, `/pools/ticks` **Driver-side abstraction** (same as original PR, already reviewed) - New `V3PoolDataSource` trait in `liquidity-sources/src/uniswap_v3/mod.rs` - Two impls: existing `UniV3SubgraphClient` (no behavior change) + new `PoolIndexerClient` - `build_pool_data_source` selects the impl based on the optional `pool-indexer-url` driver config; defaults to subgraph **Database migration: `V110__pool_indexer_uniswap_v3.sql`** - 4 tables: `pool_indexer_checkpoints`, `uniswap_v3_pools`, `uniswap_v3_pool_states`, `uniswap_v3_ticks` - Partial indexes on `IS NULL` columns for the backfill hot path **E2E tests** (`crates/e2e/tests/e2e/pool_indexer.rs`) - `driver_integration` — driver → pool-indexer wiring, asserted via per-route request counters on all four endpoints - `checkpoint_resume` — restart idempotency: pool count, per-pool state, and checkpoint advance all survive a stop+start - `api_errors` — input validation: 400 on unparseable address, 200 + empty ticks on valid-but-unknown address - `pagination` — cursor traversal with `limit=1` across multiple pools, no duplicates, terminates on `next_cursor = null` - Mock V3 factory and pool inlined via `alloy::sol!` with embedded compiled bytecode — no additions to the `contracts` crate ## How to test ### 1. e2e tests Expect 4 passes in ~10s after the build settles. These cover the driver↔indexer wiring, restart idempotency, the input-validation surface, and cursor pagination. (Same from original) ### 2. Manual (against a real network with a Uniswap V3 subgraph) ```bash # tears down + recreates local stack, applies migrations, # runs the indexer in release mode. Wipes the local DB volume each run. ./crates/pool-indexer/run-local.sh ``` Before running, create `crates/pool-indexer/config.local.toml` (schema = `Configuration` struct in `src/config.rs`). String fields accept `%ENV_VAR`, so RPC URLs and subgraph bearer tokens can come from the environment. Once the indexer is live, point a local driver at it by setting `pool-indexer-url = "http://localhost:8080"` in the driver's Uniswap V3 liquidity config (replacing the usual. `graph-url`). Then submit a quote environment. Once the indexer is live, point a local driver at it by setting `pool-indexer-url = "http://localhost:8080"` in the driver's Uniswap V3 liquidity config (replacing the usual `graph-url`). Then submit a quote and confirm the log line `uniswap v3: using pool-indexer as data source url=...` appears — that's the driver picking the indexer path. **Verified manually**: Ink mainnet (chain 57073), Uniswap V3 subgraph (via bearer auth), USDT0 ↔ WETH quotes in both directions — prices internally consistent and matched the live market.
# Description Removes a bunch of "dead code" — follow up to cowprotocol#4370 (comment) # Changes * Removes error paths that are no longer triggered ## How to test Existing tests
# Description Our uni v3 liquidity indexing currently works like this: 1. fetch all known pools and most recent state from subgraph 2. spawn background task that continue to update the pool states Since this approach does not learn about new pools that were created AFTER fetching the known pools at the process start this logic was effectively wrapped in an interval which periodically re-runs this logic (see cowprotocol#1759). That logic assumed that the cache will get dropped when we reinitialize the liquidity source. However, because each initialization also spawns a maintenance background task that has a STRONG reference to the cache this cache will actually never be freed on re-init. The result is that we get a full copy of the uni v3 cached states every 12h. This can be seen nicely by the logs which shows that every 12h we get more logs with `running maintenance` which results in a visible step function. <img width="1855" height="414" alt="Screenshot 2026-06-16 at 10 07 13" src="https://github.com/user-attachments/assets/6adaf315-82bd-49a8-88fd-04ce74b0153a" /> # Changes I adjusted the `ServiceMaintenance` task to take weak references and terminate gracefully if none of the original sub-tasks need to run anymore. ## How to test added a unit test
# Description In the heap memory profiles of the driver 10-13% belong to the cache storing all known uni v3 pools. # Changes This PR applies 2 simple optimizations: 1. turn cache from `HashMap<K, HashSet<Address>>` to `HashMap<K, Vec<Address>>`. HashSets require significantly more memory than vectors because they have more bookkeeping and also always leave some slots empty to manage hash collisions. And since we never need random access of those sets vector will do just fine. 2. call `.shrink_to_fit()` on the cache and all it's values to minimize the used memory. This cache will never be updated so any memory that is there to accommodate future growth is just wasted. <img width="1728" height="507" alt="Screenshot 2026-06-15 at 19 51 08" src="https://github.com/user-attachments/assets/1dbca7f9-1702-4565-914a-0760690b7970" />
Limit orders require the solver to declare a fee (Fee::Dynamic). Setting fee: None was only valid for market orders (Fee::Static). Use fee: Some(U256::ZERO) so the solver declares a 0 fee, which is valid for all order types. The protocol still collects its own fee via fee policies.
|
I have read the CLA Document and I hereby sign the CLA 0 out of 9 committers have signed the CLA. |
![github-advanced-security[bot]](https://avatars.githubusercontent.com/in/57789?s=60&v=4){kind=small}
Comment on lines
+1333
to
+1343
| [[package]] | ||
| name = "rustls-webpki" | ||
| version = "0.103.10" | ||
| source = "registry+https://github.com/rust-lang/crates.io-index" | ||
| checksum = "df33b2b81ac578cabaf06b89b0631153a3f416b0a886e8a7a1707fb51abbd1ef" | ||
| dependencies = [ | ||
| "aws-lc-rs", | ||
| "ring", | ||
| "rustls-pki-types", | ||
| "untrusted", | ||
| ] |
Comment on lines
+1333
to
+1343
| [[package]] | ||
| name = "rustls-webpki" | ||
| version = "0.103.10" | ||
| source = "registry+https://github.com/rust-lang/crates.io-index" | ||
| checksum = "df33b2b81ac578cabaf06b89b0631153a3f416b0a886e8a7a1707fb51abbd1ef" | ||
| dependencies = [ | ||
| "aws-lc-rs", | ||
| "ring", | ||
| "rustls-pki-types", | ||
| "untrusted", | ||
| ] |
Comment on lines
+1333
to
+1343
| [[package]] | ||
| name = "rustls-webpki" | ||
| version = "0.103.10" | ||
| source = "registry+https://github.com/rust-lang/crates.io-index" | ||
| checksum = "df33b2b81ac578cabaf06b89b0631153a3f416b0a886e8a7a1707fb51abbd1ef" | ||
| dependencies = [ | ||
| "aws-lc-rs", | ||
| "ring", | ||
| "rustls-pki-types", | ||
| "untrusted", | ||
| ] |
Comment on lines
+1112
to
+1121
| [[package]] | ||
| name = "rand" | ||
| version = "0.9.2" | ||
| source = "registry+https://github.com/rust-lang/crates.io-index" | ||
| checksum = "6db2770f06117d490610c7488547d543617b21bfa07796d7a12f6f1bd53850d1" | ||
| dependencies = [ | ||
| "rand_chacha", | ||
| "rand_core 0.9.5", | ||
| "serde", | ||
| ] |
Comment on lines
+1103
to
+1110
| [[package]] | ||
| name = "rand" | ||
| version = "0.8.5" | ||
| source = "registry+https://github.com/rust-lang/crates.io-index" | ||
| checksum = "34af8d1a0e25924bc5b7c43c079c942339d8f0a8b57c39049bef581b46327404" | ||
| dependencies = [ | ||
| "rand_core 0.6.4", | ||
| ] |
|
Reminder: Please consider backward compatibility when modifying the API specification.
Caused by: |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to subscribe to this conversation on GitHub.
Already have an account?
Sign in.
20 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
/solvehandler to usefee: Some(U256::ZERO)instead offee: Nonewhen constructing fulfillment tradesfee: Nonemeans protocol-determined fee (only valid for market orders), but BYOS orders are limit orders which require the solver to declare a feefee: Some(U256::ZERO)tells the driver the solver charges 0 fee — the protocol still collects its own fee via fee policiesTest plan
cargo check -p byoscompilescargo test -p byospasses🤖 Generated with Claude Code