Add a script to plot benchmark results#314
Open
bernhardmgruber wants to merge 5 commits intoNVIDIA:mainfrom
Open
Add a script to plot benchmark results#314bernhardmgruber wants to merge 5 commits intoNVIDIA:mainfrom
bernhardmgruber wants to merge 5 commits intoNVIDIA:mainfrom
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This script is 100% vibe-coded. I only looked at the result, since I don't understand Python. Here is a generated summary of what this PR proposes:
Examples filtering by axis and benchmark name:
Examples with two files, filtering by axis, dark mode:
Here are the input files for the plots:
pytorch_ublkcpy_B200.json
babelstream_ublkcp_B200.json
Here are the prompts that I used:
python/scripts. Then look at the benchmark resultbabelstream_ublkcp_B200.json. Create a new scriptnvbench_plot.pyin the above directory that can read the benchmark result and plot a horizontal bar chart, where each benchmark state is plotted as a separate box. The box length is taken fromnv/cold/bw/global/utilization”python ./python/scripts/nvbench_plot.py ./babelstream_ublkcp_B200.jsonit errors, please fix the error”-awhere I can specify an axis name and value like-a T{ct}=I8. When specified only benchmark states are shown where theaxis_valuenode in the JSON has an axis namedT{ct}with avalueofI8”-aargument handling where when I enter an like-a Elements{io}[pow2]=28the[pow2]is parsed before the=and it converts the value after the=(28in this case) to 2 to the power of that value (2^28in this case)”-bwhere i can select the benchmark name”Device=Npart from the bar label”The get_cmap function was deprecated in Matplotlib 3.7 and will be removed in 3.11. Use
matplotlib.colormaps[name]ormatplotlib.colormaps.get_cmap()orpyplot.get_cmap()instead.Please fix it”
14. “Please reduce the empty space above the first bar and below the last bar”
15. “Please write a pull request description about what this new script can do”
16. "Add --dark mode with black background and white text."
17. "Ensure title is visible in dark mode."
18. "Remove the x‑axis description label."
19. "If -a is used, remove those axis tokens from labels and add them to the title."
20. “Please read through the nvbench code here and figure out whether it supports -a paramters where we can also specify multiple values”
21. “Please implement this”
22. “If multiple values are specified for an axis, retain the axis value in the box label”