WIP: enhancing x test tidy with lint checks

[onur-ozkan]

As an initial step aiming to include only simple crates, likely tools.

r? ghost

[rust-log-analyzer]

The job mingw-check-tidy failed! Check out the build log: (web) (plain)

Click to see the possible cause of the failure (guessed by this bot)

Getting action download info
Download action repository 'msys2/setup-msys2@v2.22.0' (SHA:cc11e9188b693c2b100158c3322424c4cc1dadea)
Download action repository 'actions/checkout@v4' (SHA:b4ffde65f46336ab88eb53be808477a3936bae11)
Download action repository 'actions/upload-artifact@v3' (SHA:a8a3f3ad30e3422c9c7b888a15615d19a852ae32)
Complete job name: PR - mingw-check-tidy
git config --global core.autocrlf false
shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0}
---
GITHUB_ACTION=__run_7
GITHUB_ACTIONS=true
GITHUB_ACTION_REF=
GITHUB_ACTION_REPOSITORY=
GITHUB_ACTOR=onur-ozkan
GITHUB_API_URL=https://api.github.com
GITHUB_BASE_REF=master
GITHUB_ENV=/home/runner/work/_temp/_runner_file_commands/set_env_db958071-bb95-42ed-b067-c12f4f99cd99
GITHUB_EVENT_NAME=pull_request
GITHUB_EVENT_NAME=pull_request
GITHUB_EVENT_PATH=/home/runner/work/_temp/_github_workflow/event.json
GITHUB_GRAPHQL_URL=https://api.github.com/graphql
GITHUB_HEAD_REF=lint-check-in-tidy
GITHUB_JOB=pr
GITHUB_PATH=/home/runner/work/_temp/_runner_file_commands/add_path_db958071-bb95-42ed-b067-c12f4f99cd99
GITHUB_REF=refs/pull/121696/merge
GITHUB_REF_NAME=121696/merge
GITHUB_REF_PROTECTED=false
---
GITHUB_SERVER_URL=https://github.com
GITHUB_SHA=73fb29f6d8791b12421196ac043692fdfba66dd3
GITHUB_STATE=/home/runner/work/_temp/_runner_file_commands/save_state_db958071-bb95-42ed-b067-c12f4f99cd99
GITHUB_STEP_SUMMARY=/home/runner/work/_temp/_runner_file_commands/step_summary_db958071-bb95-42ed-b067-c12f4f99cd99
GITHUB_TRIGGERING_ACTOR=onur-ozkan
GITHUB_WORKFLOW_REF=rust-lang/rust/.github/workflows/ci.yml@refs/pull/121696/merge
GITHUB_WORKFLOW_SHA=73fb29f6d8791b12421196ac043692fdfba66dd3
GITHUB_WORKSPACE=/home/runner/work/rust/rust
GOROOT_1_19_X64=/opt/hostedtoolcache/go/1.19.13/x64
---
COPY scripts/sccache.sh /scripts/
RUN sh /scripts/sccache.sh

COPY host-x86_64/mingw-check/reuse-requirements.txt /tmp/
RUN pip3 install --no-deps --no-cache-dir --require-hashes -r /tmp/reuse-requirements.txt \
    && pip3 install virtualenv
COPY host-x86_64/mingw-check/validate-toolstate.sh /scripts/
COPY host-x86_64/mingw-check/validate-error-codes.sh /scripts/

# NOTE: intentionally uses python2 for x.py so we can test it still works.
# NOTE: intentionally uses python2 for x.py so we can test it still works.
# validate-toolstate only runs in our CI, so it's ok for it to only support python3.
ENV SCRIPT TIDY_PRINT_DIFF=1 python2.7 ../x.py test \
           --stage 0 src/tools/tidy tidyselftest --extra-checks=py:lint
# This file is autogenerated by pip-compile with Python 3.10
# by the following command:
#
#    pip-compile --allow-unsafe --generate-hashes reuse-requirements.in
---

#10 [5/8] COPY host-x86_64/mingw-check/reuse-requirements.txt /tmp/
#10 DONE 0.0s

#11 [6/8] RUN pip3 install --no-deps --no-cache-dir --require-hashes -r /tmp/reuse-requirements.txt     && pip3 install virtualenv
#11 0.678   Downloading binaryornot-0.4.4-py2.py3-none-any.whl (9.0 kB)
#11 0.701 Collecting boolean-py==4.0
#11 0.708   Downloading boolean.py-4.0-py3-none-any.whl (25 kB)
#11 0.727 Collecting chardet==5.1.0
---
#11 4.259 Building wheels for collected packages: reuse
#11 4.260   Building wheel for reuse (pyproject.toml): started
#11 4.618   Building wheel for reuse (pyproject.toml): finished with status 'done'
#11 4.619   Created wheel for reuse: filename=reuse-1.1.0-cp310-cp310-manylinux_2_35_x86_64.whl size=181117 sha256=f5f58750481f69515c2c0d1d503daf565e2565c370d07fc6aeb95fe3498b4269
#11 4.619   Stored in directory: /tmp/pip-ephem-wheel-cache-5vwlpafa/wheels/c2/3c/b9/1120c2ab4bd82694f7e6f0537dc5b9a085c13e2c69a8d0c76d
#11 4.621 Installing collected packages: boolean-py, binaryornot, setuptools, reuse, python-debian, markupsafe, license-expression, jinja2, chardet
#11 4.645   Attempting uninstall: setuptools
#11 4.646     Found existing installation: setuptools 59.6.0
#11 4.647     Not uninstalling setuptools at /usr/lib/python3/dist-packages, outside environment /usr
---
#11 6.049   Downloading virtualenv-20.25.1-py3-none-any.whl (3.8 MB)
#11 6.121      ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 3.8/3.8 MB 54.3 MB/s eta 0:00:00
#11 6.180 Collecting filelock<4,>=3.12.2
#11 6.187   Downloading filelock-3.13.1-py3-none-any.whl (11 kB)
#11 6.208 Collecting distlib<1,>=0.3.7
#11 6.230   Downloading distlib-0.3.8-py2.py3-none-any.whl (468 kB)
#11 6.236      ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 468.9/468.9 KB 114.4 MB/s eta 0:00:00
#11 6.274 Collecting platformdirs<5,>=3.9.1
#11 6.281   Downloading platformdirs-4.2.0-py3-none-any.whl (17 kB)
#11 6.379 Installing collected packages: distlib, platformdirs, filelock, virtualenv
#11 6.556 Successfully installed distlib-0.3.8 filelock-3.13.1 platformdirs-4.2.0 virtualenv-20.25.1
#11 DONE 6.6s

#12 [7/8] COPY host-x86_64/mingw-check/validate-toolstate.sh /scripts/
#12 DONE 0.0s
---
DirectMap4k:      274368 kB
DirectMap2M:     7065600 kB
DirectMap1G:    11534336 kB
##[endgroup]
Executing TIDY_PRINT_DIFF=1 python2.7 ../x.py test            --stage 0 src/tools/tidy tidyselftest --extra-checks=py:lint
+ TIDY_PRINT_DIFF=1 python2.7 ../x.py test --stage 0 src/tools/tidy tidyselftest --extra-checks=py:lint
    Finished dev [unoptimized] target(s) in 0.03s
##[endgroup]
downloading https://ci-artifacts.rust-lang.org/rustc-builds-alt/8790c3cc7c23057957ba7e9248961b01ea4521a1/rust-dev-nightly-x86_64-unknown-linux-gnu.tar.xz
extracting /checkout/obj/build/cache/llvm-8790c3cc7c23057957ba7e9248961b01ea4521a1-true/rust-dev-nightly-x86_64-unknown-linux-gnu.tar.xz to /checkout/obj/build/x86_64-unknown-linux-gnu/ci-llvm
---
fmt check
lint check
downloading https://static.rust-lang.org/dist/2024-02-04/clippy-beta-x86_64-unknown-linux-gnu.tar.xz
extracting /checkout/obj/build/cache/2024-02-04/clippy-beta-x86_64-unknown-linux-gnu.tar.xz to /checkout/obj/build/x86_64-unknown-linux-gnu/stage0
error: Read-only file system (os error 30) at path "/checkout/target1PQLJP"
  local time: Tue Feb 27 19:46:20 UTC 2024
  network time: Tue, 27 Feb 2024 19:46:20 GMT
##[error]Process completed with exit code 1.
Post job cleanup.

[Noratrieb]

I think this should be a separate step from x test tidy which is supposed to be really fast and used as a pre-push hook to validate formatting etc.

[onur-ozkan]

I think this should be a separate step from x test tidy which is supposed to be really fast and used as a pre-push hook to validate formatting etc.

I was thinking to perform this check only on the modified crates for non-CI environments, so the cost wouldn't be high for developers.

[onur-ozkan]

I was thinking to perform this check only on the modified crates for non-CI environments, so the cost wouldn't be high for developers.

Or, maybe I could include x clippy {tool} invocations into CI, similar to this:

rust/src/ci/docker/host-x86_64/mingw-check/Dockerfile

Line 45 in ef32456

python3 ../x.py clippy compiler -Aclippy::all -Dclippy::correctness && \

But to do that, we'll need to merge #121543 first and that is currently blocked by rust-lang/backtrace-rs#586.

[onur-ozkan]

Or, maybe I could include x clippy {tool} invocations into CI, similar to this:

rust/src/ci/docker/host-x86_64/mingw-check/Dockerfile

Line 45 in ef32456

python3 ../x.py clippy compiler -Aclippy::all -Dclippy::correctness && \

I will follow that approach in #122883, closing this.