Skip to content

Benchmark: optimize by parallel builds and caching#76

Open
liorsve wants to merge 4 commits intomainfrom
opt-bench-1-runner-clean
Open

Benchmark: optimize by parallel builds and caching#76
liorsve wants to merge 4 commits intomainfrom
opt-bench-1-runner-clean

Conversation

@liorsve
Copy link
Copy Markdown
Collaborator

@liorsve liorsve commented Mar 23, 2026

Added optimizations for the benchmark workflow.

benchmark.yml changes

  • Parallel background builds
    Replaced 5 sequential steps (Build SDV, Install glide deps, Build glide, Build resp-bench, Install deps) with a single step that runs all builds concurrently as background processes:

    • Valkey server compilation
    • SDV build from source (if needed and not cached)
    • Glide build from source (if needed and not cached)
    • pip install
    • async-profiler download
  • Build caching (keyed by commit ID / version, persists across runs)

    • SDV build artifacts (sdv-build-{commit})
    • Glide build artifacts (glide-build-{commit})
    • Valkey server binary (valkey-server-{version})
    • Maven local repository (maven-bench-{resp-bench-commit})
    • async-profiler (async-profiler-3.0)
  • Consolidated apt-get
    Merged 3 separate apt-get update + install calls into a single step that installs all system packages upfront
    (Java, Maven, pip, sysstat, numactl, gcc, cmake, etc.)

  • Unified Glide build
    Merged duplicate "Build valkey-glide from source (primary/secondary)" steps into a single step with a unified build_glide flag that handles both cases.

Timing Results

Tested on opt-bench-1-runner-clean branch.

# Scenario Overall Parallel Builds step Parallel Build step details resp-bench build Benchmark Job exec Queue Total (Queue+Job exec)
1 SDV + Glide defaults (no builds) 0m04s cached 0m12s 1m08s 1m58s 1m53s 3m51s
2 SDV commit (cache miss) 1m54s sdv 1m54s 0m11s 0m39s 3m18s 1m53s 5m11s
3 Glide commit (cache miss) 7m47s glide 7m47s 0m12s 0m42s 9m14s 1m53s 11m07s
4 SDV + Glide commits (cache misses) 7m48s sdv ~2m, glide 7m48s 0m12s 0m41s 9m17s 1m53s 11m10s
5 Jedis version override 0m04s cached 0m12s 0m32s 1m22s 1m53s 3m15s
6 SDV + Glide release versions (no builds) 0m04s cached 0m12s 1m07s 2m03s 1m53s 3m56s
7 SDV commit (cached) 0m04s all cached 0m12s 0m39s 1m26s 1m53s 3m19s
8 SDV + Glide commits (cached) 0m05s all cached 0m12s 0m40s 1m27s 1m53s 3m20s

liorsve added 4 commits March 23, 2026 12:33
Signed-off-by: Lior Sventitzky <liorsve@amazon.com>
Signed-off-by: Lior Sventitzky <liorsve@amazon.com>
Signed-off-by: Lior Sventitzky <liorsve@amazon.com>
Signed-off-by: Lior Sventitzky <liorsve@amazon.com>
run: |
sudo apt-get update
sudo apt-get install -y openjdk-21-jdk-headless maven
sudo apt-get install -y openjdk-21-jdk-headless maven python3-pip \
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Note that GLIDE requires JDK 11 (otherwise it will fallback to using Foojay, which is down a lot, causing the benchmark run in the CI to fail). We may want to install JDK 21 and JDK 11 in this step, then in GLIDE-specific steps set JAVA_HOME to use JDK 11. See #75 for an example.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants