[improve][broker] Reduce persistent topic backlog check overhead with ManagedCursor.hasBacklog#26058
Open
void-ptr974 wants to merge 1 commit into
Open
Conversation
void-ptr974
force-pushed
the
issue-26036-managed-cursor-has-backlog
branch
5 times, most recently
from
093665c to
da0836b
Compare
… ManagedCursor.hasBacklog Motivation: Issue apache#26036 points out that several broker paths only need to know whether a cursor has any backlog, but they currently call getNumberOfEntriesInBacklog(true), which computes an exact backlog count. For boolean decisions in persistent topic paths, this can add unnecessary overhead. Modifications: - Add backward-compatible hasBacklog methods to ManagedCursor, Subscription, and Replicator. - Implement an optimized precise ManagedCursorImpl.hasBacklog() that skips individually deleted ranges and stops as soon as it finds one readable backlog entry. - Use hasBacklog in persistent subscription, replicator, topic migration, and backlog checks instead of computing precise counts for yes/no decisions. - Add managed-ledger tests that compare hasBacklog with the original getNumberOfEntriesInBacklog count semantics across empty, caught-up, mixed ack, cross-ledger, batch-index, reset, recovery, deleted-range-to-tail, negative-counter fallback, and uninitialized mark-delete states. - Add broker tests for subscription delegation, replicator usage, and migration cleanup behavior. Performance impact: This adds a fast existence check for persistent topic backlog paths, avoiding exact backlog counting when callers only need a boolean result. Fixes apache#26036
void-ptr974
force-pushed
the
issue-26036-managed-cursor-has-backlog
branch
from
da0836b to
d203c4e
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Motivation
Fixes #26036.
Some persistent topic paths only need to know whether backlog exists, but currently call
getNumberOfEntriesInBacklog(true), which computes an exact backlog count. This can add unnecessaryoverhead for yes/no decisions such as migration cleanup, replicator backlog checks, and subscription
backlog checks.
Modifications
hasBacklogmethods toManagedCursor,Subscription, andReplicator.ManagedCursorImpl.hasBacklog()to short-circuit once it finds one readable backlog entry.hasBacklogin persistent topic, subscription, and replicator paths.hasBacklog()/hasBacklog(true)with the originalgetNumberOfEntriesInBacklog(true) > 0semantics across empty, caught-up, mixed ack, cross-ledger,batch-index, reset, recovery, deleted-range-to-tail, negative-counter fallback, and uninitialized
mark-delete states.
Performance impact
This reduces persistent topic backlog check overhead by avoiding exact backlog counting when callers only
need a boolean result. In the common case, the check can stop at the first valid backlog entry.
Verifications
:managed-ledger:test --tests 'org.apache.bookkeeper.mledger.impl.ManagedCursorTest.hasBacklog*':pulsar-broker:testtargetedPersistentTopicTestandPersistentSubscriptionTestbacklog tests:pulsar-broker:checkstyleTest:managed-ledger:checkstyleTestgit diff --check