feat(bot): implement main orchestrator with multi-tier system and WS/REST server#14
Merged
Conversation
- Removed Redis service from `docker-compose.yml`, including its configuration, health checks, and associated volumes. - Eliminated `redis-cli` target from `Makefile` as Redis is no longer part of the stack. - Updated the Brain service configuration to remove references to `REDIS_URL` and its dependency check on Redis. This change simplifies the stack by removing unused Redis functionality.
…REST server - **Orchestrator Implementation:** - Designed the ArbOS orchestrator (`main.rs`) to manage multi-tier tasks: Ingestor, Engine, Executor, and Ledger. - Integrated Axum-based WebSocket + REST server for real-time updates and API interaction. - Set up communication channels (MPSC & broadcast) for tier collaboration and state flow. - Included a demo execution mode. Flag `ARBOS_LIVE_MODE=true` added as a placeholder for future live trading implementation. - **Config Module:** - Added centralized configuration for the orchestrator via `BotConfig`, parsing environment variables (`ARBOS_LIVE_MODE`, `ARBOS_WS_PORT`, etc.). - **Engine Enhancements:** - Added `strategy` field to `ArbSignal` and incorporated `StrategyType::Implication`, `Partition`, and `Contradiction`. - Extended signal generation logic to record strategy type for improved tracking and debugging. - **Executor Enhancements:** - Built `ExecutorActor` to process `ArbSignal` via rate-limited and deduplicated execution flow. - Added demo-mode execution simulation and placeholder logic for live trading. - **Server Module:** - Introduced a WebSocket handler to stream `ExecutionReport` to connected clients. - Added `/api/status` and `/api/health` endpoints for system status updates. - **Testing:** - Added unit tests in `executor.rs` for rate-limiting, deduplication, and execution logic verification. - Implemented config parsing tests in `config.rs`. - Verified overall tiers through `cargo test`. This implementation establishes the orchestrator foundation, enabling real-time multi-tier task execution and future extensibility.
There was a problem hiding this comment.
Pull request overview
This PR introduces a unified Bot-tier orchestrator that wires together the Ingestor → Engine → Executor pipeline and exposes runtime status/execution reporting via REST + WebSocket, while simplifying local infrastructure by removing Redis from docker-compose.
Changes:
- Added a Tokio-based orchestrator (bot tier) that spawns actors, connects them via MPSC channels, and runs an Axum REST/WS server.
- Extended core domain types to include strategy attribution (
StrategyType) and execution reporting (ExecutionReport,FillDetail,ExecutionMode). - Removed Redis from
docker-compose.ymland related Makefile tooling.
Reviewed changes
Copilot reviewed 13 out of 13 changed files in this pull request and generated 9 comments.
Show a summary per file
| File | Description |
|---|---|
| docker-compose.yml | Removes Redis service and related wiring/volume. |
| Makefile | Removes redis-cli target now that Redis is gone. |
| crates/ingestor/src/lib.rs | Exposes ingestor modules as a library for the bot orchestrator to depend on. |
| crates/engine/src/actor.rs | Adds strategy field population on emitted ArbSignals. |
| crates/core/src/domain.rs | Introduces StrategyType, ExecutionMode, ExecutionReport, FillDetail, and adds strategy to ArbSignal. |
| crates/core/src/constants.rs | Adds executor/report channel sizing and executor rate limit constants. |
| crates/bot/Cargo.toml | Adds deps for Axum server, JSON, tracing, etc. |
| crates/bot/src/main.rs | Implements the main orchestrator spawning tiers and server. |
| crates/bot/src/server.rs | Adds REST (/api/health, /api/status) and WS (/ws) endpoints. |
| crates/bot/src/executor.rs | Implements executor actor with rate limiting, dedup, ledger integration, and report broadcast. |
| crates/bot/src/ledger.rs | Adds in-memory ledger/history/positions tracking with tests. |
| crates/bot/src/dedup.rs | Adds signal deduplication helper with tests. |
| crates/bot/src/config.rs | Adds env-based configuration parsing with defaults and tests. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
…ion with canonical hashing - **Server Updates:** - Enhanced CORS initialization by dynamically parsing `VITE_ALLOWED_ORIGINS` from environment variables. - Fallback to permissive CORS when no origins are specified, ensuring flexible but secure defaults. - **Deduplication Enhancements:** - Replaced tuple-based canonical key for deduplication with a deterministic hash-based approach. - Supports arbitrage signals with multi-leg asset baskets by hashing sorted asset IDs. - Improved deduplication logic to prevent re-execution of permutations of the same asset combinations. - Added a garbage collection process to clean up stale deduplication entries, ensuring memory efficiency. - **Testing:** - Added unit tests for multi-leg deduplication and validated canonical hashing for consistent results. - Updated deduplication test scenarios for both pair-based and multi-leg asset handling. These enhancements improve CORS setup, extend deduplication capabilities, and ensure compliance with multi-leg arbitrage strategies while maintaining test coverage.
…hance config tests - **Actor Shutdown:** - Added `tokio_util::sync::CancellationToken` for orchestrating graceful shutdown of actors (Ingestor, Engine, Executor). - Updated actor loops to handle cancellation tokens, ensuring they terminate cleanly without abrupt drops. - **Error Handling Improvements:** - Enhanced WebSocket server logic to properly handle deserialization failures, logging errors and gracefully closing connections. - **Config Module Testing:** - Updated `test_config_defaults` to isolate environment variable handling, preventing flakiness in CI environments with pre-set variables. - **Dependencies:** - Introduced `tokio-util` to support cancellation tokens. These updates improve the resiliency of the orchestrator by adding cancellation support, enhance test reliability, and refine error handling in server logic.
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 13 out of 13 changed files in this pull request and generated 6 comments.
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
…nd add endpoint tests
- **CORS Updates:**
- Improved CORS initialization to dynamically parse `VITE_ALLOWED_ORIGINS`.
- Added fallback to `http://localhost:5173` for development safety when parsing fails or the environment variable is missing.
- **Performance Improvements:**
- Optimized status JSON generation by minimizing shared ledger lock contention.
- Replaced inline locking with quick data snapshots to allow concurrent reads.
- **Response Refinements:**
- Standardized `mode` field in `/api/status` response to return stable string values (`LIVE`, `DEMO`) instead of `Debug` representations.
- **Testing:**
- Introduced tests for `/api/health` and `/api/status` endpoints:
- Verified correctness of response payloads and defaults.
- Ensured stable handling of uninitialized states.
- **Ledger Logic:**
- Added safeguard to `Ledger::record_trade` to prevent handling history updates when `max_history` is zero.
- **Dependencies:**
- Updated `Cargo.toml` to include `tower` v0.5.3.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
This PR introduces the main orchestrator for the bot execution tier. It implements a multi-tier system architecture and adds WebSocket and REST interfaces for seamless communication. Additionally, it removes the Redis service from the docker-compose setup to streamline our dependencies.
Fixes # (issue)
Type of change
How Has This Been Tested?
Please describe the tests that you ran to verify your changes.
Checklist: