Skip to content

⚡ Bolt: Zero-allocation refill optimization#3

Draft
google-labs-jules[bot] wants to merge 1 commit into
mainfrom
bolt-refill-optimization-11331851479498510601
Draft

⚡ Bolt: Zero-allocation refill optimization#3
google-labs-jules[bot] wants to merge 1 commit into
mainfrom
bolt-refill-optimization-11331851479498510601

Conversation

@google-labs-jules

Copy link
Copy Markdown

💡 What:

  • Replaced AtomicInteger.updateAndGet() with a manual compareAndSet loop in TokenBucketLimiter.refill().
  • Added a check to clamp tokensToAdd to capacity immediately if periodsElapsed is large enough.

🎯 Why:

  • GC Pressure: The original code used a capturing lambda current -> Math.min(capacity, current + tokensToAdd) which allocates a new object every time refill() successfully updates tokens. In a high-throughput rate limiter, this creates significant garbage.
  • CPU Efficiency: If the limiter is idle for a long time, calculating periodsElapsed * refillTokens is unnecessary if we know it exceeds capacity. The optimization skips this multiplication.
  • Safety: Prevents potential long overflow if periodsElapsed becomes extremely large (though rare).

📊 Impact:

  • Reduces object allocation rate in the critical path (zero-allocation refill).
  • Reduces CPU cycles for refill calculations after long idle periods.

🔬 Measurement:

  • Verify with pnpm test (or ./gradlew test).
  • Functional behavior remains identical, but micro-benchmarks would show reduced GC activity.

PR created automatically by Jules for task 11331851479498510601 started by @SpaceLeam

- Replaced `AtomicInteger.updateAndGet` with a manual CAS loop in `TokenBucketLimiter.refill` to eliminate lambda object allocation on the hot path.
- Added an early clamp check (`periodsElapsed >= capacity`) to skip unnecessary multiplication and avoid potential long overflow during large time gaps.
- Maintained thread safety and functional correctness.
@google-labs-jules

Copy link
Copy Markdown
Author

👋 Jules, reporting for duty! I'm here to lend a hand with this pull request.

When you start a review, I'll add a 👀 emoji to each comment to let you know I've read it. I'll focus on feedback directed at me and will do my best to stay out of conversations between you and other bots or reviewers to keep the noise down.

I'll push a commit with your requested changes shortly after. Please note there might be a delay between these steps, but rest assured I'm on the job!

For more direct control, you can switch me to Reactive Mode. When this mode is on, I will only act on comments where you specifically mention me with @jules. You can find this option in the Pull Request section of your global Jules UI settings. You can always switch back!

New to Jules? Learn more at jules.google/docs.


For security, I will only act on instructions from the user who triggered this task.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

0 participants