[WIP] Enhance benchmark_rounds.sh with config file, warm-up rounds, statistics, and plotting

[chenshengxin2026]

Summary

• Externalize benchmark configuration: Replace the hardcoded EXAMPLES array
  and default values in benchmark_rounds.sh with a benchmark_config.json file,
  supporting --config/-c to load alternate configs. CLI arguments override config
  values.
• Add warm-up rounds and richer statistics: Introduce --warmup/-w to discard
  initial rounds before measuring. Timing output now includes median, trimmed mean
  (excluding min & max), range, MAD, standard deviation, and fluctuation rate (CV%)
  in addition to arithmetic mean.
• Optional scatter plots and log persistence: Add --plot to generate per-example
  scatter plot PNGs via matplotlib, and --log to save per-example statistics to
  benchmark_logs/. Both are configurable from the JSON config file.
• Verbose mode: Add --verbose/-v to show full run_example.py output instead of
  suppressing it.
• Translate documentation to English: Convert tools/README.md from Chinese to
  English, update content to reflect version 2 perf data support, and add entries for
  the new benchmark_config.json file.
• Add dedicated benchmark README: Create tools/README_benchmark.md with
  comprehensive documentation covering quick start, CLI options, config fields,
  execution logic, timing analysis methodology, and output examples.

Changed Files

File	Change
tools/benchmark_rounds.sh	Config loading, warm-up, stats, plot, log, verbose
tools/benchmark_config.json	New — default benchmark configuration
tools/README.md	Translated to English, updated for v2 and new features
tools/README_benchmark.md	New — dedicated benchmark tool documentation

[gemini-code-assist]

Summary of Changes

Hello, I'm Gemini Code Assist¹! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!

This pull request significantly upgrades the benchmarking capabilities by centralizing configuration, enhancing statistical analysis, and providing better output options. It moves hardcoded settings into a flexible JSON file, introduces warm-up rounds for more accurate measurements, and expands the statistical metrics reported. Additionally, it adds features for visualizing performance data with scatter plots and persisting detailed logs, alongside comprehensive documentation updates to ensure clarity and ease of use for these new functionalities.

Highlights

• Configuration Externalization: Externalized benchmark configuration from benchmark_rounds.sh into a new benchmark_config.json file, allowing for flexible settings and command-line overrides.
• Enhanced Benchmarking Statistics: Introduced warm-up rounds to discard initial runs and expanded timing output to include median, trimmed mean, range, Mean Absolute Deviation (MAD), standard deviation, and fluctuation rate (CV%).
• Optional Visualization and Logging: Added --plot option to generate per-example scatter plot PNGs using matplotlib and --log to save detailed per-example statistics to benchmark_logs/.
• Verbose Mode: Implemented a --verbose/-v option to display the full output of run_example.py during benchmarking.
• Documentation Updates: Translated tools/README.md from Chinese to English, updated its content to reflect version 2 performance data support, and added entries for the new benchmark_config.json file. A new dedicated tools/README_benchmark.md was also created for comprehensive documentation of the benchmarking tool.

🧠 New Feature in Public Preview: You can now enable Memory to help Gemini Code Assist learn from your team's feedback. This makes future code reviews more consistent and personalized to your project's style. Click here to enable Memory in your admin console.

Changelog

• tools/README.md
  • Translated the entire document from Chinese to English.
  • Updated descriptions for swimlane_converter.py and sched_overhead_analysis.py to reflect support for version 2 performance data and enhanced output details.
  • Added a new entry for benchmark_config.json in the tool list.
  • Updated the 'Output File Reference' section to include new benchmark-related output files.
• tools/README_benchmark.md
  • Added a new, dedicated documentation file for benchmark_rounds.sh.
  • Provided quick start instructions, detailed CLI options, and descriptions of configuration file fields.
  • Explained the execution logic, timing analysis methodology, and output examples.
  • Listed prerequisites for using the benchmarking tool.
• tools/benchmark_config.json
  • Added a new JSON configuration file to define default settings for benchmark_rounds.sh.
  • Included fields for project root, example subdirectory, example list, device ID, rounds, warm-up rounds, platform, verbose mode, logging, and plotting.
• tools/benchmark_rounds.sh
  • Refactored to load configuration from benchmark_config.json.
  • Implemented warm-up rounds to exclude initial runs from statistics.
  • Added advanced statistical calculations including median, trimmed mean, range, MAD, standard deviation, and coefficient of variation (CV%).
  • Introduced verbose output mode for run_example.py.
  • Integrated optional plotting functionality to generate scatter plots of performance data.
  • Added optional logging to save detailed statistics for each example.

Activity

• No human activity has been recorded for this pull request yet.

Using Gemini Code Assist

The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.

Invoking Gemini

You can request assistance from Gemini at any point by creating a comment using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands on the current page.

Feature	Command	Description
Code Review	/gemini review	Performs a code review for the current pull request in its current state.
Pull Request Summary	/gemini summary	Provides a summary of the current pull request in its current state.
Comment	@gemini-code-assist	Responds in comments when explicitly tagged, both in pull request comments and review comments.
Help	/gemini help	Displays a list of available commands.

Customization

To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.

Limitations & Feedback

Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for GitHub and other Google products, sign up here.

You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension.

Footnotes

1. Review the Privacy Notices, Generative AI Prohibited Use Policy, Terms of Service, and learn how to configure Gemini Code Assist in GitHub here. Gemini can make mistakes, so double check it and use code with caution. ↩

[gemini-code-assist]

Code Review

The pull request primarily translates existing documentation to English and significantly enhances the benchmark_rounds.sh script and its documentation. The benchmark_rounds.sh script is refactored to use a new benchmark_config.json file for configuration, introduces warm-up rounds, and provides detailed statistical analysis, optional logging, and scatter plot generation for performance benchmarking. The README.md is updated to reflect these changes and is fully translated, and a new README_benchmark.md provides dedicated, in-depth documentation for the benchmarking tool. Review comments indicate inconsistencies in the documentation regarding the default value of the --log option and suggest refactoring duplicated configuration loading logic in benchmark_rounds.sh for improved maintainability.