docs: draft delayed-delivery rotation blueprint // v0.3 scope#237
Draft
NikolayS wants to merge 2 commits into
Draft
docs: draft delayed-delivery rotation blueprint // v0.3 scope#237NikolayS wants to merge 2 commits into
NikolayS wants to merge 2 commits into
Conversation
Two-table TRUNCATE rotation for pgque.delayed_events modeled on PgQ's event_N pattern. Removes DELETE/UPDATE/VACUUM from the delayed-delivery hot path so send_at() can serve as a primary write path without dead-tuple churn or catalog growth.
cancel_scheduled() is the single exception to the no-DELETE invariant. Dead tuples it produces are bounded above by one rotation interval and TRUNCATEd away on the next rotation that targets the affected table. The bloat-free property of the design is therefore conditional on a low cancellation rate. That caveat is required to appear: - in the SQL comment on cancel_scheduled() - in docs/reference.md next to the function entry - in tutorials / examples that introduce cancellation - in the send_at() function comment For very-high-cancel workloads, docs must steer users to the application-level "skip on consume" pattern instead.
NikolayS
changed the title
NikolayS
pushed a commit
that referenced
this pull request
May 30, 2026
Earlier draft concluded the zero-bloat differentiator does not transfer to a workflow layer, assuming a mutable workflow_status row updated per step (the DBOS/absurd strategy). That was wrong. Model workflow state transitions as appended events over the rotating log (continuation-passing): each step enqueues its successor instead of mutating a row. Transitions become appends, not UPDATEs, so zero-bloat carries through. Exactly-once handoff falls out of insert_event + finish_batch in one transaction; sleep/timers use the rotating send_at from PR #237; exclusivity is structural via cooperative consumers; the only mutable state is a current-state projection bounded by concurrency. Verdict flips from 'do not compete' to 'compete on a substrate SKIP-LOCKED systems cannot match for high-throughput durable workflows'. Remaining real risk: awaitEvent/join semantics.
NikolayS
pushed a commit
that referenced
this pull request
Jun 2, 2026
Map the durable-workflow design to pgque's real primitives and verify the keystone against sql/pgque.sql: insert_event + finish_batch compose atomically in the caller transaction (exactly-once handoff), finish_batch is one subscription UPDATE per batch (amortization), ev_extra1..4 are settable+indexable (workflow_id lookup). Flags the retry_queue DELETE-bloat constraint (route sleeps through rotating send_at, PR #237), gives the new coordination DDL, concrete awaitEvent/emit + join SQL, a bloat audit, and the pgque gaps to close (promote send_at, ev_extra1 index, durable.sql).
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Draft blueprint for replacing the current single-heap
pgque.delayed_eventsdesign with a two-table TRUNCATE rotation, modeled on PgQ's
event_Npattern. The goal is to make
pgque.send_at()viable as a primary writepath without the dead-tuple churn, index bloat, and
VACUUMdependencethe current experimental design would incur at scale.
Key properties of the proposed design:
DELETEand noUPDATEon delayed-event rows. Reclamation isTRUNCATE-only.VACUUMdependence on the hot path.CREATE/DROPchurn.floor.
1 + rotations_crossedper row) forlong-horizon scheduled messages.
Includes a comparison against the alternative (declarative range
partition + drop), explicit invariants, a TDD-shaped implementation
plan, and acceptance criteria phrased against
pg_stat_user_tablesandpg_classso the no-bloat property is testable.Open questions in the draft
These are flagged in the blueprint and want a call before code lands:
default trades catalog stability for write amplification on
long-horizon rows.
pgque.delayed_eventsexposed as a view over the two physical tables,vs. renaming the public surface.
pgque.cancel_scheduled) is deferred to a follow-upblueprint; locking in a tombstone-set design now would be cheap insurance
if cancel is likely-yes for 0.3.
drop-and-recreate (current draft assumes the latter).
Test plan
Blueprint-only change; no code or test diffs. Once approved, the
implementation slice will:
Add a failing
pg_stat_user_tables.n_dead_tup == 0acceptance check totests/acceptance/us4_delayed_delivery.sql.Make it green by replacing the single heap with the rotation scheme.
Capture before/after dead-tuple and catalog-row counts in the
implementation PR description.
review blueprint design and tradeoffs
decide open questions (rotation interval, view vs. rename, cancellation scope)
convert to implementation PR with failing test + rotation code
https://claude.ai/code/session_01STJ9TCTr7ykbwG23t9ZxJv
Generated by Claude Code