Skip to content

feat(client): make consumer checkpoint lookup bounded and observable#199

Merged
Migorithm merged 11 commits into
mainfrom
opt/c-lookup
Jul 20, 2026
Merged

feat(client): make consumer checkpoint lookup bounded and observable#199
Migorithm merged 11 commits into
mainfrom
opt/c-lookup

Conversation

@Migorithm

Copy link
Copy Markdown
Collaborator

Changes

  • Limit rebalance checkpoint lookups to ranges being started and their transitive split/merge predecessors.
  • Use the consumer-group generation as a shared snapshot boundary across independently routed range lookups.
  • Require each range leader to observe the durable generation seal before returning a checkpoint.
  • Serialize checkpoint lookup with commits from the recovering consumer.
  • Keep range actors stopped when checkpoint lookup fails.
  • Surface background lookup failures through Consumer::next_record().
  • Return lookup failures directly from explicit stale-commit recovery.
  • Document the rebalance checkpoint boundary and failure behavior.

Why

Rebalances previously requested checkpoints for every known topic range. Individual range requests also lacked a common observed boundary, and lookup failures were only logged before an internal retry.

The new flow makes lookup cost proportional to the ranges being provisioned, gives the result set one logical generation boundary, and makes failures visible without treating unavailable state as a missing checkpoint.

@Migorithm Migorithm self-assigned this Jul 20, 2026
@Migorithm Migorithm added bug Something isn't working documentation Improvements or additions to documentation enhancement New feature or request feat labels Jul 20, 2026
@Migorithm
Migorithm merged commit 1cdd76f into main Jul 20, 2026
1 check passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug Something isn't working documentation Improvements or additions to documentation enhancement New feature or request feat

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant