Conversation
Separating test results from other output **avoids contamination**. If `libtest` only writes output to stdout, any non-test output (log messages, debug prints, etc.) may corrupt the stream and break parsers. Rust’s `println`’s are wrapped by libtest, but anything can (and does, in real world) use `libc`, or have C code using `libc` that corrupts stdout. Also, in practice, projects often resort to external post-processing to filter test output. As one [tracking discussion notes](rust-lang#85563), *“due to limitations of Rust libtest formatters, Rust developers often use a separate tool to postprocess the test results output”*. By writing test results directly to a file, we can guarantee the test results are isolated and parseable, without third-party noise.
I had to introduce `test_helepr.rs` basically re-copying it yet another time from `std` as it was done in other places. It is better than keeping new functionality untested.
rustbot
added
S-waiting-on-review
|
These commits modify the If this was unintentional then you should revert the changes before this PR is merged. |
|
r? testing-devex |
rustbot
added
the
T-testing-devex
|
Triage note: I can't speak for T-testing-devex, but AFAIK libtest is currently in a soft feature freeze, and the proposed options here AFAIK are insta-stable CLI and behavior? |
|
They also raised this in rust-lang/testing-devex-team#11. |
epage
left a comment
epage
left a comment
There was a problem hiding this comment.
Not looking over the code changes until rust-lang/testing-devex-team#11 is resolved
| .optflag("", "list", "List all tests and benchmarks") | ||
| .optflag("h", "help", "Display this message") | ||
| .optopt("", "logfile", "Write logs to the specified file (deprecated)", "PATH") | ||
| .optopt("", "test-results-file", "Write test results to the specified file", "PATH") |
|
@rustbot blocked (rust-lang/testing-devex-team#11) |
rustbot
added
S-blocked
|
☔ The latest upstream changes (presumably #145077) made this pull request unmergeable. Please resolve the merge conflicts. |
7 participants
Separating test results from other output avoids contamination. If
libtestonly writes output to stdout, any non-test output (log messages, debug prints, etc.) may corrupt the stream and break parsers. Rust’sprintln’s are wrapped by libtest, but anything can (and does, in real world) uselibc, or have C code usinglibcthat corrupts stdout. Also, in practice, projects often resort to external post-processing to filter test output. As one tracking discussion notes, “due to limitations of Rust libtest formatters, Rust developers often use a separate tool to postprocess the test results output”. By writing test results directly to a file, we can guarantee the test results are isolated and parseable, without third-party noise.Here's the full proposal: rust-lang/testing-devex-team#11