Claim jobs by MATERIALIZED CTE instead of locked sub-select#42
Draft
stephanschuler wants to merge 4 commits into
Draft
Claim jobs by MATERIALIZED CTE instead of locked sub-select#42stephanschuler wants to merge 4 commits into
stephanschuler wants to merge 4 commits into
Conversation
Doctrine RetryableExceptions are either LockWaitTimeoutException or DeadlockException. As the name suggests, they should be safe to retry. This change adds them to the Flow throwable storage anyway, because having those in abundance, at least when it comes to scheduled jobs, might be an indicator of bad database layout or network issues.
The candidate row MUST be selected in a `MATERIALIZED` CTE, never in an inline `FROM (SELECT ... LIMIT 1 FOR UPDATE SKIP LOCKED)` subquery. PostgreSQL is free to place such an inline subquery on the inner side of a nested-loop join and re-evaluate it once per outer row. Combined with `FOR UPDATE SKIP LOCKED`, every re-evaluation skips the rows already locked by previous iterations and returns the *next* candidate, so the join ends up matching - and claiming - more than the single intended row (all with the same claim value). Whether this happens depends on the query plan, i.e. on table size and statistics, which is why it only surfaces on large production tables and not on small dev databases. `AS MATERIALIZED` forces the candidate selection to be evaluated exactly once, so `LIMIT 1` reliably bounds the update to a single row.
stephanschuler
changed the title
stephanschuler
marked this pull request as ready for review
July 3, 2026 16:07
The scheduler previously nested two retry mechanisms: an outer Retry (exponential backoff + logging) in AbstractScheduler around the inner Connection::withAutoReconnectAndRetry, which caught ConnectionLost and RetryableException with only a single, non-logged retry attempt. Retry, exponential backoff and throwable logging now live entirely in Connection::withAutoReconnectAndRetry. Both ConnectionLost (reconnect before the next attempt) and RetryableException are retried through the same path. Callers pass an optional $logContext callable to enrich the logged data (step, claim, group name).
stephanschuler
force-pushed
the
feat/log-retryable-errors
branch
from
July 3, 2026 17:49
93845e7 to
93abc38
Compare
stephanschuler
marked this pull request as draft
July 4, 2026 06:17
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The candidate row MUST be selected in a
MATERIALIZEDCTE, never in aninline
FROM (SELECT ... LIMIT 1 FOR UPDATE SKIP LOCKED)subquery.PostgreSQL is free to place such an inline subquery on the inner side of
a nested-loop join and re-evaluate it once per outer row. Combined with
FOR UPDATE SKIP LOCKED, every re-evaluation skips the rows alreadylocked by previous iterations and returns the next candidate, so the
join ends up matching - and claiming - more than the single intended row
(all with the same claim value). Whether this happens depends on the
query plan, i.e. on table size and statistics, which is why it only
surfaces on large production tables and not on small dev databases.
AS MATERIALIZEDforces the candidate selection to be evaluated exactlyonce, so
LIMIT 1reliably bounds the update to a single row.