feat(engine): integrate contradiction strategy and implement brain state sync#13
Merged
Merged
Conversation
- Introduced a new `/state` endpoint in `main.py` to return the synchronized graph state for use by the Rust Engine. - Includes metadata for asset end timestamps, implications, contradictions, and partition mappings. - Added `BrainStatePayload`, `ImplicationMapping`, `PartitionMapping`, and `ContradictionMapping` models in `models.py` to structure graph state data. - Integrated logic to fetch: - Conditional token mappings and asset end timestamps based on active markets. - Implications and contradictions using relationship graph data. - Partition mappings for mutually exclusive market groups, with deterministic condition IDs based on asset hashes. - Enhanced consistency and compatibility between the Brain Service and the Rust Engine by providing a complete state snapshot in a structured payload. This addition facilitates efficient polling and synchronization with the Rust Engine while maintaining flexibility for future extensions.
- Added `ContradictionMapping` to `BrainStatePayload` for state synchronization. - Defined `ContradictionMapping` struct and implemented parsing logic (`parse_contradictions`). - Extended `EngineActor` with `contradiction_edges` for storing mutually exclusive asset relationships. - Implemented `evaluate_contradictions` in `EngineActor` to identify and execute profitable contradiction opportunities. - Logic includes validation of weights, gas fees, and profit margins. - Updated book handlers to trigger contradiction evaluations alongside implications and partitions. - Adjusted asset frequency tracking and expiry validation logic to account for contradiction mapping (`compute_tracked_assets`). - Added comprehensive unit test for contradiction evaluation flow (`test_engine_actor_triggers_contradiction_arb`). - Updated documentation in `strategies/mod.rs` to mark `ContradictionStrategy` as integrated. This implementation connects the Tier-3 contradiction strategy with the runtime, enabling evaluation and execution of mutually exclusive market inconsistencies.
There was a problem hiding this comment.
Pull request overview
Integrates Tier-2 Brain “contradiction” discovery with the Tier-3 Rust engine by adding a /state graph-sync payload and wiring contradiction evaluation into the engine’s execution loop.
Changes:
- Added Brain
/stateendpoint and payload models for implications, partitions, contradictions, and asset end timestamps. - Wired contradiction mappings into the Rust
EngineActorsync loop and added runtime evaluation for contradiction arbs. - Updated strategy module wiring/comments and added a Rust unit test for contradiction triggering.
Reviewed changes
Copilot reviewed 4 out of 4 changed files in this pull request and generated 10 comments.
| File | Description |
|---|---|
| services/brain/models.py | Adds Pydantic models for the new /state payload contract. |
| services/brain/main.py | Implements /state endpoint to emit synchronized graph state + token/timestamp mapping. |
| crates/engine/src/strategies/mod.rs | Updates strategy module comment to reflect contradiction strategy wiring. |
| crates/engine/src/actor.rs | Extends payload parsing for contradictions, tracks contradiction edges, evaluates contradiction arbs, and adds a unit test. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
…diction evaluation - Renamed `condition_id` to `asset_id` across the contradiction evaluation flow in `EngineActor` to ensure consistent terminology and logic. - Fixed incorrect usage of `vwap_ask` for bid-related calculations; replaced with `vwap_bid` to align with strategy logic. - Updated sizing calculation to use `sum_bids` instead of the highest price, ensuring a more precise allocation of liquidity. - Adjusted asset frequency computation to add directional constraints, ensuring order and correctness when processing parent-child pairs. - Updated test order book parameters to match bid/ask logic corrections and reflect valid cases for evaluation. These changes improve the accuracy and readability of the contradiction evaluation and support the intended strategy behavior while addressing inconsistencies in asset handling and computational logic.
…ence-based partitioning - Enhanced error handling in `_graph_manager` by adding detailed warnings during end_date parsing failures, including `condition_id`, `end_date`, and error details. - Introduced confidence-based logic in partition group mapping: - Built a lookup table for mutually exclusive relationship confidences. - Integrated minimum confidence calculation for grouped partitions. - Updated deterministic hashing to include a `v1_` prefix for future migrations. - Improved confidence tracking for partitions, applying calculated values instead of a hardcoded `1.0`. These changes increase the clarity of logged issues, improve partition grouping accuracy, and lay the foundation for future versioned hashing enhancements.
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 4 out of 4 changed files in this pull request and generated 6 comments.
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
…tion handling - Added `asset_to_partition` mapping in `EngineActor` to enable O(1) partition lookups for assets. - Refactored `market_outcomes` and `market_expected_outcome_counts` to use `String` keys instead of `B256` for consistent data handling across the actor. - Updated partition-related methods (`get_partition_bids`, `is_partition_exhaustively_tradable`, `check_and_send_partition`) to adopt the new `asset_to_partition` logic. - Enhanced `parse_partitions` to generate `asset_to_partition` mapping alongside market outcomes and expected outcomes. - Deferred order book ejection during contradiction evaluation for correct processing of peer execution. - Improved code clarity and reduced redundancy in partition and contradiction evaluation logic. These changes improve the engine's partition-handling performance, simplify the data structure interface, and ensure more robust execution in complex asset scenarios.
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 4 out of 4 changed files in this pull request and generated 5 comments.
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
… improve partition bid handling
- **Engine Updates:**
- Updated `get_partition_bids` and `is_partition_exhaustively_tradable` methods to use `&str` instead of `String` for `partition_id` parameters, enhancing performance with borrowed references.
- Added `BRAIN_API_KEY` retrieval and included it as a header in API client requests, securing communication with the Brain service.
- **Brain Service Enhancements:**
- Secured the `/state` endpoint with admin-level authentication via `Depends(_verify_admin)`.
- Added environment variable check for `ADMIN_API_KEY` in test setups.
- Expanded testing for `/state`:
- Verified missing authentication response (`401 Unauthorized`).
- Verified proper response with valid API key and mocked graph data.
- **Model Enhancements:**
- Introduced and tested new Brain state models:
- `ImplicationMapping`, `PartitionMapping`, `ContradictionMapping`, and `BrainStatePayload`.
- Confirmed compatibility and correctness of structured state payload serialization.
These changes improve the Rust engine's partition handling and ensure secure, robust collaboration between the Brain service and the Rust Engine.
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 6 out of 6 changed files in this pull request and generated 3 comments.
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
- Updated `brain_api_key` retrieval to require `ADMIN_API_KEY` instead of `BRAIN_API_KEY`. - Added error handling to ensure `ADMIN_API_KEY` is present and properly configured to avoid runtime issues during state synchronization. This change improves security and robustness for engine-to-brain communication.
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 6 out of 6 changed files in this pull request and generated 3 comments.
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
…stency - Renamed `market_outcomes` to `partition_outcomes` for alignment with partition-centric terminology. - Updated `market_expected_outcome_counts` to `partition_expected_outcome_counts`, ensuring consistent field naming across the codebase. - Refactored relevant methods and logic to match the updated terminology (`get_partition_bids`, `parse_partitions`, etc.). - Improved environment variable handling for `ADMIN_API_KEY` during API requests to avoid errors when the variable is missing or empty. - Updated the Brain service to use `len(group_tokens)` for outcome count calculation in `PartitionMapping`, ensuring correctness. This refactor standardizes field names, improves clarity, and ensures accuracy in partition-based handling within the Engine and related services.
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 6 out of 6 changed files in this pull request and generated 3 comments.
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Comment on lines
+549
to
+551
| # 3. Partitions | ||
| partitions_lists = await _graph_manager.get_partition_groups() | ||
| partitions = [] |
There was a problem hiding this comment.
/state builds partitions from _graph_manager.get_partition_groups() (connected components of MUTUALLY_EXCLUSIVE edges). Connected components don’t guarantee a fully mutually-exclusive (clique) set, and they also don’t imply the set is exhaustive, but downstream PartitionStrategy assumes exhaustiveness when shorting the full basket. Consider only emitting partitions when Tier 2 can assert exhaustiveness (or at least when the ME subgraph is a clique), otherwise keep these as contradictions-only.
Comment on lines
+494
to
+498
| # 1. Fetch metadata (tokens and timestamps) | ||
| result = await session.execute( | ||
| select(Market.condition_id, Market.end_date, Market.clob_token_ids) | ||
| .where(Market.active.is_(True), Market.closed.is_(False)) | ||
| ) |
There was a problem hiding this comment.
This query pulls all active, open markets on every /state poll, then materializes the full result set with result.all(). Since the engine syncs every 60s, this can become a steady DB load as the market table grows. Consider caching condition_id -> yes_token/end_ts in-memory with an expiry, or restricting the query to only condition_ids that appear in the current graph edges/partition groups.
Comment on lines
+525
to
+531
| for rel in implications_dicts: | ||
| parent_token = condition_to_yes_token.get(rel["parent_condition_id"]) | ||
| child_token = condition_to_yes_token.get(rel["child_condition_id"]) | ||
|
|
||
| if not parent_token or not child_token: | ||
| continue | ||
|
|
There was a problem hiding this comment.
When a relationship references a condition_id that isn’t present in condition_to_yes_token (e.g., inactive/closed market, stale relationship), it’s silently dropped. That makes graph/engine mismatches hard to diagnose. Consider logging (at least debug/warn with counts) when relationships are skipped due to missing token mappings.
… refined partition processing - **Performance Enhancements:** - Added condition-based filtering to database queries, ensuring only relevant `condition_ids` are processed to reduce data overhead. - Optimized implication and partition logic by pre-collecting `condition_ids` to streamline metadata queries. - **Partition Handling Improvements:** - Excluded partitions with incomplete token mappings or invalid configurations (e.g., non-cliques). - Refined confidence calculations to consider minimum confidence across relationships while ensuring clique validation. - **Logging Enhancements:** - Added detailed debug-level logs for skipped relationships and partitions, highlighting reasons such as missing tokens or invalid configurations. These updates enhance the `/state` endpoint with better performance, more reliable partition processing, and improved visibility into processing behaviors.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
This PR implements the end-to-end integration of the Contradiction Strategy into the ArbOS project. It bridges the gap between the Tier 2 Brain Service (logic discovery) and the Tier 3 Rust Engine (high-frequency execution).
Key Changes:
/stateendpoint inmain.pywhich serves the current synchronized graph state (implications, partitions, and contradictions).condition_ids into actualclob_token_ids (YES tokens). This ensures the Rust Engine can correctly track assets usingU256identifiers instead of failing on string hashes.BrainStatePayloadand its constituent mapping models inmodels.py.EngineActorinactor.rsto fetch and parse contradictions during thesync_brain_stateloop.evaluate_contradictions, which leverages theContradictionStrategyto detect arbitrage opportunities when mutually exclusive markets violate thecontradictionmodule instrategies/mod.rsand removed the "unwired" legacy comments.test_engine_actor_triggers_contradiction_arbtoactor.rsto verify that the engine correctly generates anArbSignalwhen a contradiction is detected.Type of change
How Has This Been Tested?
uv run pytest, verifying that the new models and/stateendpoint logic do not regression existing graph or API tests (50 passed).cargo test, specifically validating the new engine loop logic and existing strategy checks (14 passed)./stateis correctly deserialized by the Rustserdeimplementation and that the Token ID mapping allows the engine to successfully match orderbook updates to graph edges.Checklist: