Greatly reduce GCC build logs#122496
Conversation
|
rustbot has assigned @Mark-Simulacrum. Use r? to explicitly pick a reviewer |
rustbot
added
A-testsuite
GuillaumeGomez
force-pushed
the
reduce-gcc-build-logs
branch
from
7fe087f to
e948e9f
Compare
GuillaumeGomez
force-pushed
the
reduce-gcc-build-logs
branch
from
e948e9f to
2d74388
Compare
|
The |
GuillaumeGomez
force-pushed
the
reduce-gcc-build-logs
branch
from
2d74388 to
7bc73a3
Compare
|
That will be annoying if it ever fails though...
Is there an upstream issue tracking "please provide a build option that outputs less than 100 lines on success"?
|
The errors are printed on stderr, so it's not an issue imo. |
|
In my experience the output often becomes hard to follow when throwing away just stdout, as nobody tests that.
But sure, we can give it a shot.
|
GuillaumeGomez
force-pushed
the
reduce-gcc-build-logs
branch
from
7bc73a3 to
bf4d079
Compare
|
I don't know if it's easily installable in whatever OS the CI Docker container uses, but there's a tool called
Which seems like it would be useful here. https://manpages.ubuntu.com/manpages/noble/man1/chronic.1.html edit: Looks like |
GuillaumeGomez
force-pushed
the
reduce-gcc-build-logs
branch
2 times, most recently
from
0028fb7 to
6d312a2
Compare
|
Well, you can tune log analyzer to skip these build logs instead turning off logs. |
|
Sometimes one has to look at the logs though. I don't think it is reasonable to have 80% of them filled with the GCC build when that's a tiny part of what we are testing. |
|
It wasn't an issue before, so there something changed in pipeline? |
|
It's not been that long since GCC was added to the build. I think it was a problem ever since, or do you have evidence to the contrary? |
| # of output. Unfortunately, even when using `--quiet` option for all commands, | ||
| # the output still thousands of lines of output, forcing us to use this solution. | ||
| make > /dev/null | ||
| make install > /dev/null |
There was a problem hiding this comment.
Please use hide_output from src/ci/docker/scripts/shared.sh (feel free to grep for other usages). That makes sure there's some progress indication and we do the right thing on failure.
|
Btw is it expected that the GCC build takes about 30 minutes? (If I understand this log correctly.) That's 30% of the entire CI job! And in particular this happens before the rustc test suite is run, greatly increasing the latency until regular test failures are discovered. |
|
Also why do both the gnu-llvm-16 and the gnu-tools runner build GCC...? |
It is run it 3 CIs: gnu-tools, llvm-16 and llvm-17. When I made the last sync of If people agree on reducing the number of CIs building GCC, I can remove some of them. |
GuillaumeGomez
force-pushed
the
reduce-gcc-build-logs
branch
from
6d312a2 to
664ca6f
Compare
For now it's built using only one thread. I don't know if I can get the number of CPUs available in the CI and pass it to the |
This comment has been minimized.
This comment has been minimized.
GuillaumeGomez
force-pushed
the
reduce-gcc-build-logs
branch
from
ede2082 to
cbe81e6
Compare
This comment has been minimized.
This comment has been minimized.
|
https://github.com/rust-lang/rust/blob/master/src/ci/docker/host-x86_64/dist-x86_64-linux/build-clang.sh#L42 is an example. It also demonstrates the use of hide_output. |
GuillaumeGomez
force-pushed
the
reduce-gcc-build-logs
branch
from
cbe81e6 to
0a1b64b
Compare
This comment has been minimized.
This comment has been minimized.
GuillaumeGomez
force-pushed
the
reduce-gcc-build-logs
branch
from
0a1b64b to
1dce674
Compare
| make install | ||
| --prefix=$(pwd)/../gcc-install \ | ||
|
|
||
| hide_output make -j$(nproc) |
There was a problem hiding this comment.
This seems to speed up the build by a factor of 4.^^ It was 30min before, now it's...
2024-03-15T12:47:03.8776557Z #14 DONE 446.8s
|
|
||
| set -ex | ||
|
|
||
| source /tmp/shared.sh |
There was a problem hiding this comment.
Why didn't you put this in the same folder as the build-gccjit script?
There was a problem hiding this comment.
Mostly because I was first trying to make it work. Cleanup time!
GuillaumeGomez
force-pushed
the
reduce-gcc-build-logs
branch
from
1dce674 to
86a0c67
Compare
This comment has been minimized.
This comment has been minimized.
GuillaumeGomez
force-pushed
the
reduce-gcc-build-logs
branch
from
86a0c67 to
c4ece1f
Compare
|
Clean up done. Nice speed up in any case. :) |
|
@bors r+ |
bors
added
S-waiting-on-bors
…llaumeGomez Rollup of 6 pull requests Successful merges: - rust-lang#114651 (rustdoc: add `--test-builder-wrapper` arg to support wrappers such as RUSTC_WRAPPER when building doctests) - rust-lang#122468 (Cleanup `MirBorrowckCtxt::prefixes`) - rust-lang#122496 (Greatly reduce GCC build logs) - rust-lang#122512 (Cursor.rs documentation fix) - rust-lang#122513 (hir: Remove `opt_local_def_id_to_hir_id` and `opt_hir_node_by_def_id`) - rust-lang#122530 (less symbol interner locks) r? `@ghost` `@rustbot` modify labels: rollup
Rollup merge of rust-lang#122496 - GuillaumeGomez:reduce-gcc-build-logs, r=Mark-Simulacrum Greatly reduce GCC build logs Fixes rust-lang/rust-log-analyzer#80. Based on [makefile documentation](https://www.gnu.org/software/make/manual/html_node/Options-Summary.html#index-_002d_002dquiet-1) and [configure documentation](https://www.gnu.org/software/autoconf/manual/autoconf-2.68/html_node/configure-Invocation.html). cc `@RalfJung` `@antoyo`
9 participants
Fixes rust-lang/rust-log-analyzer#80.
Based on makefile documentation and configure documentation.
cc @RalfJung @antoyo