feat(brain): implement APScheduler for background market syncing and scanning#15
Merged
Conversation
… automation - Added `apscheduler` integration to enable automated background tasks within the Brain service. - Implemented a scheduler that periodically: - **Syncs markets** from the Gamma API every 15 minutes. - **Scans market pairs** for analysis using pre-defined logic. - Refactored market sync (`_internal_sync_markets`) and scan logic (`_internal_scan_markets`) into reusable functions for both API endpoints and background jobs. - Enhanced error handling and logging: - Added structured logs for sync and scan tasks via API and background jobs. - Improved exception handling to retain service stability. - Introduced shutdown hooks for graceful cleanup of the scheduler during service exit. - Updated dependencies: - Included `apscheduler>=3.11.2` in `pyproject.toml` and `uv.lock`. - Added `auto_discover_and_analyze` method for integrated sync and scan in the background. This update automates critical market update and analysis workflows, reducing manual overhead and improving operational efficiency.
There was a problem hiding this comment.
Pull request overview
Adds an in-process background scheduler to the Brain FastAPI service so it can periodically sync markets from Gamma and scan/analyze market pairs without external triggers.
Changes:
- Added
apschedulerdependency (and lockfile updates) for scheduled background execution. - Introduced an
AsyncIOSchedulerstarted/stopped from the FastAPI lifespan, running a 15-minute sync+scan job. - Refactored
/markets/syncand/scanto delegate to internal functions so both API routes and background jobs can reuse the same logic.
Reviewed changes
Copilot reviewed 2 out of 3 changed files in this pull request and generated 7 comments.
| File | Description |
|---|---|
| services/brain/uv.lock | Locks apscheduler and its dependency tzlocal. |
| services/brain/pyproject.toml | Adds apscheduler to Brain service dependencies. |
| services/brain/main.py | Adds scheduler lifecycle + background job; refactors sync/scan logic for internal reuse. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
… analysis response messaging - Enhanced `auto_discover_job` scheduler: - Added `max_instances=1` to prevent overlapping job executions. - Enabled `coalesce` to merge missed runs during downtime. - Set `misfire_grace_time=300` to allow a 5-minute buffer for delayed jobs. - Improved error handling for API endpoints: - Updated generic exceptions to return a standardized "Internal server error" message for better consistency and user experience. - Explicitly re-raised `HTTPException` without modification. - Refined response structure for market analysis: - Added more descriptive messages for scenarios with insufficient markets or analyzed pairs. - Included `skipped` field to clarify skipped market counts in analysis results. These updates improve scheduler reliability, API robustness, and response clarity in the Brain service.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
This PR introduces an in-process background task scheduler to the Python Brain service using
apscheduler. Previously, market syncing and pair scanning against the Gamma API were only available via manual/external API triggers.This change allows the Brain to become an autonomous entity that automatically runs _internal_sync_markets and _internal_scan_markets on a 15-minute interval. This is handled gracefully within the FastAPI llifespan without needing external cron jobs, heavy distributed message queues (like Celery), or blocking the main HTTP event loop.
Additionally, a bug regarding
AsyncSessioninstantiation during Pytest execution has been resolved, allowing the get_session() generator to be properly accessed by background tasks and fixtures.Type of change
How Has This Been Tested?
I ran the full
pytestregression suite to ensure the refactored FastAPI/markets/syncand/scanroutes still behave identically when accessed via HTTP, and that the database session lifecycle properly opens and closes during background execution.Checklist: