Replace bits-and-blooms/bitset with roaring bitmaps by ValentaTomas · Pull Request #2386 · e2b-dev/infra

ValentaTomas · 2026-04-14T00:12:30Z

Replace all direct usage of bits-and-blooms/bitset with roaring/v2 (v2.17.0). Uses the new Ranges() iterator (#522) for range iteration in CreateMapping and BitmapRanges.

cursor · 2026-04-14T00:12:37Z

PR Summary

Medium Risk
Touches snapshot diff metadata/mapping generation and dirty-page tracking used to export rootfs/memory diffs, so a subtle bitmap semantic change could produce incorrect snapshot contents or sizes. The rest is mostly mechanical API migration plus a test timeout bump.

Overview
This PR migrates snapshot/memory/rootfs diff tracking from bits-and-blooms/bitset to roaring/v2 bitmaps, updating dirty/empty metadata types, range iteration to use Bitmap.Ranges(), and related callers/telemetry so diff export and mapping generation operate on roaring structures. It also updates module dependencies to pin roaring/v2 (via an e2b-dev/roaring replace) and increases the cgroup round-trip test timeout to reduce flakiness.

^{Reviewed by Cursor Bugbot for commit fa35957. Bugbot is set up for automated code reviews on this repo. Configure here.}

packages/orchestrator/go.mod

Use roaring v2.17.0 with the new Ranges() iterator throughout. Remove bitset as a direct dependency.

Point roaring/v2 at e2b-dev/roaring@efficient-range-iteration until the Ranges() iterator fix is merged upstream. This resolves CI failures from the efficient range iteration implementation.

The process_dies_with_cgroups test gives 5s for the OOM killer to fire after `sleep 1 && tail /dev/zero` with a 1MB memory limit. On ARM64 runners the OOM kill can take longer, causing the context to time out before the process is killed. Increase to 15s.

cursor

Cursor Bugbot has reviewed your changes and found 1 potential issue.

^{❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.}

^{Reviewed by Cursor Bugbot for commit 405f3d0. Configure here.}

packages/shared/pkg/atomicbitset/bitset.go

chatgpt-codex-connector

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: fa35957fa3

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

packages/shared/pkg/storage/header/metadata.go

jakubno · 2026-04-14T07:15:02Z

packages/orchestrator/go.mod


 tool github.com/vektra/mockery/v3

+replace github.com/RoaringBitmap/roaring/v2 => github.com/e2b-dev/roaring/v2 v2.16.1-0.20260414062913-60ea6a744802


is this needed?

Yes, waiting for them to merge my bugfix upstream.

RoaringBitmap/roaring#524

packages/shared/go.mod

jakubno · 2026-04-14T11:26:25Z

packages/orchestrator/pkg/sandbox/block/tracker.go


-	return bitsetOffsets(t.b.Clone(), t.BlockSize())
-}
+	snapshot := t.b.Clone()


why do we need to clone here?

jakubno · 2026-04-14T12:06:04Z

packages/orchestrator/pkg/sandbox/fc/client.go

 	return &header.DiffMetadata{
-		Dirty:     bitset.From(res.Payload.Resident),
-		Empty:     bitset.From(res.Payload.Empty),
+		Dirty:     roaring.FromDense(res.Payload.Resident, false),


could you explain why with doCopy=false and why you don't call RunOptimize

jakubno · 2026-04-14T12:08:30Z

packages/orchestrator/pkg/sandbox/uffd/noop.go

+	allPages := roaring.New()
+	allPages.AddRange(0, uint64(numberOfPages))

-	empty = empty.Difference(dirty)
+	empty := roaring.AndNot(allPages, dirty)


wouldn't this work better?

Suggested change

allPages := roaring.New()

allPages.AddRange(0, uint64(numberOfPages))

empty = empty.Difference(dirty)

empty := roaring.AndNot(allPages, dirty)

empty := roaring.FlipInt(dirty, 0, uint64(numberOfPages))

jakubno · 2026-04-14T12:16:26Z

packages/orchestrator/pkg/sandbox/uffd/noop.go

 	}

-	dirty := diffInfo.Dirty.Difference(diffInfo.Empty)
+	dirty := roaring.AndNot(diffInfo.Dirty, diffInfo.Empty)


use

Suggested change

dirty := roaring.AndNot(diffInfo.Dirty, diffInfo.Empty)

diffInfo.Dirty.AndNot(diffInfo.Empty)

to prevent allocating an extra bitmap

then in

header.DiffMetadata{ Dirty: diffInfo.Dirty,

levb

a few nits, otherwise LGTM

levb · 2026-04-14T11:59:08Z

packages/envd/internal/services/cgroups/cgroup2_test.go

 	}

-	maxTimeout := time.Second * 5
+	maxTimeout := time.Second * 15


unrelated to the change, right?

levb · 2026-04-14T12:13:27Z

packages/orchestrator/pkg/sandbox/block/cache.go

 	}

-	diffMetadata := header.NewDiffMetadata(c.blockSize, c.dirty.BitSet())
+	diffMetadata := header.NewDiffMetadata(c.blockSize, c.dirty.Bitmap())


0/5 nit: Do we need to clone under the RLock here, or (since we are holding c.mu - maybe we can operate on dirty's data directly here?

levb · 2026-04-14T12:23:09Z

packages/orchestrator/pkg/sandbox/uffd/userfaultfd/helpers_test.go

 	}

-	return slices.Collect(b.EachSet())
+	result := make([]uint, 0, b.GetCardinality())


0/5 nit: This function is called only in tests, should it be moved there? I flagged the allocation, but it's fine for the use case.

levb · 2026-04-14T12:25:46Z

packages/shared/pkg/atomicbitset/bitset.go


-	roaring "github.com/RoaringBitmap/roaring/v2"
-	"github.com/bits-and-blooms/bitset"
+	"github.com/RoaringBitmap/roaring/v2"


0/5 now that it is always roaring and we intermix it with explicit roaring sets in the code, rename the package to syncroaring? the generic atomicbitset is no longer intention-revealing?

levb · 2026-04-14T12:26:20Z

packages/shared/pkg/atomicbitset/bitset.go

 }

-func (b *Bitset) BitSet() *bitset.BitSet {
+func (b *Bitset) Bitmap() *roaring.Bitmap {


See my comment in ExportToDiff - I question the need to do this.

e2b-request-same-site-reviewers bot assigned levb Apr 14, 2026

ValentaTomas force-pushed the replace-bitset-with-roaring branch 6 times, most recently from fc7f1b4 to 0205899 Compare April 14, 2026 00:38

ValentaTomas marked this pull request as ready for review April 14, 2026 00:40

ValentaTomas requested review from dobrac and jakubno as code owners April 14, 2026 00:40

ValentaTomas force-pushed the replace-bitset-with-roaring branch from 6ec4c90 to b81ec2a Compare April 14, 2026 00:49

claude bot reviewed Apr 14, 2026

View reviewed changes

packages/orchestrator/go.mod Outdated Show resolved Hide resolved

ValentaTomas force-pushed the replace-bitset-with-roaring branch from b81ec2a to b6fa8a5 Compare April 14, 2026 00:53

refactor: replace bits-and-blooms/bitset with roaring bitmaps

b0717ba

Use roaring v2.17.0 with the new Ranges() iterator throughout. Remove bitset as a direct dependency.

ValentaTomas force-pushed the replace-bitset-with-roaring branch from b6fa8a5 to b0717ba Compare April 14, 2026 00:56

ValentaTomas and others added 3 commits April 13, 2026 22:37

fix: use e2b-dev/roaring fork with Ranges() fix

b699cc1

Point roaring/v2 at e2b-dev/roaring@efficient-range-iteration until the Ranges() iterator fix is merged upstream. This resolves CI failures from the efficient range iteration implementation.

chore: auto-commit generated changes

feaff58

ValentaTomas marked this pull request as draft April 14, 2026 06:10

Merge branch 'main' into replace-bitset-with-roaring

405f3d0

cursor bot reviewed Apr 14, 2026

View reviewed changes

packages/shared/pkg/atomicbitset/bitset.go Outdated Show resolved Hide resolved

ValentaTomas added 2 commits April 13, 2026 23:22

fix: remove COW from atomicbitset to prevent data race on Clone()

84c5813

fix: update roaring fork to include CardinalityInRange fix

fa35957

ValentaTomas marked this pull request as ready for review April 14, 2026 07:13

chatgpt-codex-connector bot reviewed Apr 14, 2026

View reviewed changes

packages/shared/pkg/storage/header/metadata.go Show resolved Hide resolved

jakubno reviewed Apr 14, 2026

View reviewed changes

jakubno requested changes Apr 14, 2026

View reviewed changes

levb requested changes Apr 14, 2026

View reviewed changes


		tool github.com/vektra/mockery/v3

		replace github.com/RoaringBitmap/roaring/v2 => github.com/e2b-dev/roaring/v2 v2.16.1-0.20260414062913-60ea6a744802

	dirty := roaring.AndNot(diffInfo.Dirty, diffInfo.Empty)
	diffInfo.Dirty.AndNot(diffInfo.Empty)

Conversation

ValentaTomas commented Apr 14, 2026

Uh oh!

cursor bot commented Apr 14, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

PR Summary

Uh oh!

Uh oh!

cursor bot left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

chatgpt-codex-connector bot left a comment

Choose a reason for hiding this comment

💡 Codex Review

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

levb left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

cursor bot commented Apr 14, 2026 •

edited

Loading