feat: extra_info label columns in compare (--columns extra:NAME)#177
Merged
Conversation
--columns now accepts extra:NAME entries alongside the metric names. Each one renders the numeric extra_info value NAME as a plain, stat-less label column: one readout per (benchmark x series), count- formatted, with no (N.NN) multiplier and no ranking colour — a label describes the benchmark, it doesn't compete. Values load per series, so a label that legitimately differs between runs (a model size after a formulation change) shows both values, and the --diff view diffs it like a metric. Non-numeric values are skipped; like a metric, a label absent from every run is dropped. Motivating use: benchmarks that record model-size labels in extra_info (variables, constraints, component counts) can now surface them in the compare table and markdown PR comments next to time/peak. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Documentation build overview
4 files changed± compare-runs/index.html± reference/index.html± visualize/index.html± gallery/linopy-memory/index.html |
Review fixes for the extra_info label columns: - Keep labels (and the run's benchmarks) when every requested metric is absent from every run — the label-only fallback now triggers on no labels loaded, not on no metrics requested, so 'compare --columns rss,extra:variables' on non-isolated runs renders the labels instead of dying with 'no benchmarks found'. - Format non-integral labels with a new fmt_label (4 sig figs) instead of fmt_count, which silently truncated 0.85 to 0. - Reject extra:NAME for the long frame's reserved columns (snapshot/id/value) — extra:value used to silently render metric values as a label. - DRY the renderers: _Col.is_label carries the 'a label doesn't compete' rule, and a shared _col_header builds the header text for both the rich and markdown views. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Note
The following content was generated by AI.
--columnsnow acceptsextra:NAMEalongside the metric names: it renders the numericextra_infovalueNAMEas a plain, stat-less label column — one readout per(benchmark × series), count-formatted, with no(N.NN)multiplier and no ranking colour (a label describes the benchmark, it doesn't compete).Behavior details:
--diffview diffs it like a metric (Δ% per comparison run).extra_infovalues are skipped; like a metric, a label absent from every run is dropped, and one absent from a single series shows—.--format md), with label-only--columns extra:...supported, and flows into--csv.--pivot(params +extra_info);extra:disambiguates from metric names.Implementation:
_resolve_columnsadmitsextra:NAME,_load_columnscollects the labels from the long df it already walks (a stat-lessextra:NAMEcolumn id via the new_col_idhelper), and the table/diff renderers suppress multiplier/colour for label columns. 7 new tests intests/test_compare.py; docs updated indocs/compare-runs.md. Full suite: 347 passed; ruff + mypy clean.Motivating use: fluxopt's reference benchmarks record model-size labels (
variables,flows, component counts) inextra_info— this lets its benchmark-hint PR comment show them next to time/peak (fluxopt/fluxopt#268).🤖 Generated with Claude Code