feat: Add comprehensive job scheduling and management features#29
Open
imahmedismail wants to merge 1 commit intosheharyarn:masterfrom
Conversation
This commit implements five major enhancements to the Que job processing library:
• Delayed Jobs: Schedule jobs to run at specific times or after delays
- Add Que.add_scheduled/3 for specific datetime scheduling
- Add Que.add_in/3 convenience function for delay-based scheduling
- Jobs with :scheduled status are automatically promoted when ready
• Job Cancellation: Cancel pending jobs before execution
- Add Que.cancel/1 to cancel jobs by ID
- Add Que.cancel_all/1 to cancel all jobs for a worker
- Only :scheduled and :queued jobs can be cancelled
• Retry Strategies: Automatic retry with exponential backoff
- Configurable max_retries per job (defaults to 3)
- Exponential backoff with jitter to prevent thundering herd
- Workers can define custom retry limits via callbacks
• Job Timeouts: Prevent infinite running jobs
- Configurable timeout per job (defaults to 60 seconds)
- Jobs exceeding timeout are killed and marked as :timeout
- Workers can define custom timeouts via callbacks
• Job Priority: Priority-based job processing
- Four priority levels: :low, :normal, :high, :urgent
- Priority queue processing ensures high-priority jobs run first
- Convenience functions: Que.add_high_priority/2, Que.add_urgent/2
Breaking Changes: None - all changes are backward compatible
Database Schema: Updated Mnesia table to include new fields:
scheduled_at, retry_count, max_retries, last_error, timeout, timeout_ref, priority
Author
|
cc: @sheharyarn |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Roadmap Implementation
This PR implements several key items from the project roadmap:
The following roadmap items can now be marked as completed:
Future Roadmap Suggestions
Based on this implementation, I suggest adding these new roadmap items:
Documentation Updates Needed