Skip to content

Add memory usage benchmarks for pack generation #11

@ggueret

Description

@ggueret

Summary

The streaming pack refactoring (#6, PR #10) reduced memory from ~3x repo size to ~1 object at a time, but this gain is not measured by existing benchmarks (which only track time).

We need benchmarks that measure peak memory allocation during pack generation to:

  • Quantify the improvement from the streaming refactoring
  • Detect memory regressions in future changes
  • Establish a baseline for v0.2 further optimizations

Solution

Add a memory-tracking benchmark using jemalloc stats or a custom allocator wrapper that records peak allocation. Measure peak RSS for small/medium/large fixtures during generate_pack.

Optionally integrate with the CI bench workflow to track memory alongside time on gh-pages.

Sessions

  1. Implement memory benchmark -- custom allocator or jemalloc stats to measure peak allocation during pack generation
  2. Compare before/after streaming -- checkout pre-streaming commit, measure, compare with current
  3. CI integration -- if feasible, add memory data points to the gh-pages dashboard

Metadata

Metadata

Assignees

Labels

No labels
No labels

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions