Add --filter-mode=run-pass,compile-pass to compiletest

[Centril]

Add a --filter-mode option to compiletest and expose it through ./x.py.

The --filter-mode flag will take a comma separated list of modes:

• compile-pass
• compile-fail
• run-pass
• ui
• and more:
  Details

  • run-fail
  • run-pass-valgrind
  • pretty
  • debuginfo-cdb
  • debuginfo-gdb+lldb
  • debuginfo-lldb
  • debuginfo-gdb
  • codegen
  • rustdoc
  • codegen-units
  • incremental
  • run-make
  • js-doc-test
  • mir-opt
  • assembly

When the mode is ui (e.g. src/test/ui) then:

• // compile-pass is interpreted as provided mode compile-pass.
• // run-pass is interpreted as provided mode run-pass.
• Absence of those two is interpreted as mode compile-fail.

When --filter-mode is used, only tests fitting the modes included in the flag are retained. Other tests are ignored.

As an example run, you can use ./x.py -i test --stage 1 --filter-mode run-pass to execute all run-pass tests whether they are in src/test/ui or inside src/test/run-pass.

Another example is ./x.py -i test src/test/ui --stage 1 --filter-mode run-pass,compile-pass which will cover both run-pass and compile-pass tests in src/test/ui specifically.

This PR moves us towards a more systematic testing scheme in the spirit of #61712 and may enable moving src/test/run-pass altogether to src/test/ui.

r? @petrochenkov

[petrochenkov]

This is a bit different from what I expected.

I'll write some more about the common use scenarios in #61712 once I get to it.

[petrochenkov]

As implemented, --filter-mode will mostly duplicate the "primary" nameless argument of x.py test when compile-fail and run-pass directories are merged into ui (which is the desired end state).

Forcing all the ui tests to check mode (run fast, but still produce all the .stderr files) may be a more useful operation.

[Centril]

As implemented, --filter-mode will mostly duplicate the "primary" nameless argument of x.py test when compile-fail and run-pass directories are merged into ui (which is the desired end state).

I'm not sure I follow... what is the nameless argument?

My desired end state is to not have compile-fail and run-pass directories at all but rather that tests be organized by their semantic area (what the rfc is, what parts of the language they affect, e.g. ctfe, borrowck, typeck, parser...). This is aligned with @nikomatsakis's vision in #61526 (comment).

Forcing all the ui tests to check mode (run fast, but still produce all the .stderr files) may be a more useful operation.

I don't know which one is most useful, but I feel these are orthogonal operations that both can be supported. At least, --filter-mode can be useful in the interim to allow us to move the tests.

[petrochenkov]

Let's close this for now and reconsider when #61755 is done and run-pass tests are merged into UI.