perf(pebble): −68% on 1 M-grant paginated read (autoresearch) by c1-squire-dev[bot] · Pull Request #877 · ConductorOne/baton-sdk

autoresearch: scaffold Pebble read-perf loop (UnpackReadGrants)

pquerna

committed

Baseline for the read-perf session, starting from the autoresearch/pebble-perf-20260525 branch (WritePack session end state). pebble_readpaginated_1m_ms = 1229.6 ms. Allocs at 1M scale = 17M (versu…

pquerna

committed

Outer-only grantReadArena for PaginateGrantsBySync. Collapses per-iter v3.GrantRecord allocations from O(records) to O(pages) \u2014 for the 1M read bench, 1M outer allocs become 100 slice allocs (…

pquerna

committed

grantV2ReadArena: arena-allocate the 6 v2.Grant nested stubs (Grant + Entitlement + 2 \u00d7 (Resource + ResourceId)) in adapter.ListGrants. Each translateV3Grant call now allocates O(1) into the p…

pquerna

committed

autoresearch.ideas.md: log read-perf session results (-20.7% in 5 iterations)

pquerna

committed

BATCHED parallel proto.Unmarshal for PaginateGrantsBySync, fixing the #54 design that dispatched per-record. Main goroutine batches 64 records into a single concatenated value buffer + offsets arra…

pquerna

committed

Parallel file writes in ExtractZstdTar: 4-worker pool consumes (target, mode, data) jobs from the serial tar+zstd reader. tar reading remains serial (tar framing is sequential), but the per-file op…

pquerna

committed

sync.Pool for per-batch unmarshal buffers in PaginateGrantsBySync. Previously each batch allocated a fresh ~32 KB valueBuf via `make([]byte, 0, batchSize*512)` plus a fresh []int for offsets \u2014…

pquerna

committed

Parallel v3\u2192v2 translation in a SEPARATE worker pool that runs AFTER PaginateGrantsBySync's decode workers finish (vs #57 which folded translate INTO the decode workers and stole memory bandwi…

pquerna

committed

Bumped decode worker batchSize from 64\u2192256, matching the translate pool's batch size (#61). 4\u00d7 fewer channel dispatches per page (156\u219239 batches at 10k page size). Pool's initial val…

pquerna

committed

Bumped pageUnmarshalWorkers from 4\u21926. Profile showed workers doing ~125 ms wallclock in 4-way parallel vs main's ~137 ms iter loop \u2014 slight under-saturation. 6 workers drop parallel decod…

pquerna

committed

Bisected the 6\u21928 worker-count regression by trying 7. Primary 405.9\u2192402.85 ms (-0.7 %, within noise). Mixed scales: 100k +1.1 %, 10k -1 %, 1k -5 %, 100 +5 % \u2014 all within noise. The o…

pquerna

committed

Custom hand-rolled wire-format decoder for v3.GrantRecord (unmarshalGrantRecordFast). Hand-decodes fields 1-4 (sync_id, external_id, entitlement, principal) using google.golang.org/protobuf/encodin…

pquerna

committed

Skip the SetSyncId call in the fast decoder \u2014 no read-path consumer of PaginateGrantsBySync reads r.GetSyncId() (callers know syncID separately, it's passed into PaginateGrantsBySync as a para…

pquerna

committed

Single-byte tag comparison in the fast decoder \u2014 skip protowire.ConsumeTag's varint-decode + bit-split for the common case of fields 1-15 (which have single-byte wire tags). Pre-computed 9 byt…

pquerna

committed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

perf(pebble): −68% on 1 M-grant paginated read (autoresearch)#877

perf(pebble): −68% on 1 M-grant paginated read (autoresearch)#877
c1-squire-dev[bot] wants to merge 15 commits into
autoresearch/pebble-perf-20260525from
autoresearch/pebble-read-perf-20260525

Commits on May 25, 2026

autoresearch: scaffold Pebble read-perf loop (UnpackReadGrants)

Baseline for the read-perf session, starting from the autoresearch/pebble-perf-20260525 branch (WritePack session end state). pebble_readpaginated_1m_ms = 1229.6 ms. Allocs at 1M scale = 17M (versu…

Outer-only grantReadArena for PaginateGrantsBySync. Collapses per-iter v3.GrantRecord allocations from O(records) to O(pages) \u2014 for the 1M read bench, 1M outer allocs become 100 slice allocs (…

grantV2ReadArena: arena-allocate the 6 v2.Grant nested stubs (Grant + Entitlement + 2 \u00d7 (Resource + ResourceId)) in adapter.ListGrants. Each translateV3Grant call now allocates O(1) into the p…

autoresearch.ideas.md: log read-perf session results (-20.7% in 5 iterations)

BATCHED parallel proto.Unmarshal for PaginateGrantsBySync, fixing the #54 design that dispatched per-record. Main goroutine batches 64 records into a single concatenated value buffer + offsets arra…

Parallel file writes in ExtractZstdTar: 4-worker pool consumes (target, mode, data) jobs from the serial tar+zstd reader. tar reading remains serial (tar framing is sequential), but the per-file op…

sync.Pool for per-batch unmarshal buffers in PaginateGrantsBySync. Previously each batch allocated a fresh ~32 KB valueBuf via `make([]byte, 0, batchSize*512)` plus a fresh []int for offsets \u2014…

Parallel v3\u2192v2 translation in a SEPARATE worker pool that runs AFTER PaginateGrantsBySync's decode workers finish (vs #57 which folded translate INTO the decode workers and stole memory bandwi…

Bumped decode worker batchSize from 64\u2192256, matching the translate pool's batch size (#61). 4\u00d7 fewer channel dispatches per page (156\u219239 batches at 10k page size). Pool's initial val…

Bumped pageUnmarshalWorkers from 4\u21926. Profile showed workers doing ~125 ms wallclock in 4-way parallel vs main's ~137 ms iter loop \u2014 slight under-saturation. 6 workers drop parallel decod…

Bisected the 6\u21928 worker-count regression by trying 7. Primary 405.9\u2192402.85 ms (-0.7 %, within noise). Mixed scales: 100k +1.1 %, 10k -1 %, 1k -5 %, 100 +5 % \u2014 all within noise. The o…

Custom hand-rolled wire-format decoder for v3.GrantRecord (unmarshalGrantRecordFast). Hand-decodes fields 1-4 (sync_id, external_id, entitlement, principal) using google.golang.org/protobuf/encodin…

Skip the SetSyncId call in the fast decoder \u2014 no read-path consumer of PaginateGrantsBySync reads r.GetSyncId() (callers know syncID separately, it's passed into PaginateGrantsBySync as a para…

Single-byte tag comparison in the fast decoder \u2014 skip protowire.ConsumeTag's varint-decode + bit-split for the common case of fields 1-15 (which have single-byte wire tags). Pre-computed 9 byt…