-
Notifications
You must be signed in to change notification settings - Fork 1.6k
Modernising Build Process #9010
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: dev
Are you sure you want to change the base?
Changes from all commits
ad1122c
f2b5460
1a7be99
4e410ee
43da203
7725ded
2103022
c0ee0c5
ce9b4ab
1d619ad
24d4b80
bbdb0bc
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -43,7 +43,7 @@ jobs: | |
| python -m pip install --upgrade pip wheel | ||
| python -m pip uninstall -y torch torchvision | ||
| python -m pip install ${{ matrix.pytorch }} | ||
| python -m pip install -r requirements-dev.txt | ||
| python -m pip install .[all,testing] | ||
| python -m pip list | ||
| - name: Run tests report coverage | ||
| env: | ||
|
|
@@ -91,7 +91,7 @@ jobs: | |
| run: | | ||
| which python | ||
| python -m pip install --upgrade pip wheel | ||
| python -m pip install -r requirements-dev.txt | ||
| python -m pip install .[all,testing] | ||
| python -m pip list | ||
| - name: Run tests report coverage | ||
| env: | ||
|
|
@@ -186,7 +186,7 @@ jobs: | |
| python -c $'import torch\na,b=torch.zeros(1,device="cuda:0"),torch.zeros(1,device="cuda:1");\nwhile True:print(a,b)' > /dev/null & | ||
| python -c "import torch; print(torch.__version__); print('{} of GPUs available'.format(torch.cuda.device_count()))" | ||
|
|
||
| python -m pip install -r requirements-dev.txt | ||
| python -m pip install .[all,testing] | ||
|
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. 🩺 Stability & Availability | 🟠 Major | ⚡ Quick win Do not install At this point 🤖 Prompt for AI Agents |
||
| PYTHONPATH="$tmp_dir":$PYTHONPATH BUILD_MONAI=1 python ./tests/runner.py -p 'test_((?!integration).)' # unit tests | ||
| if pgrep python; then pkill python; fi | ||
|
|
||
|
|
@@ -239,7 +239,7 @@ jobs: | |
| run: | | ||
| which python | ||
| python -m pip install --upgrade pip wheel | ||
| python -m pip install -r requirements-dev.txt | ||
| python -m pip install .[all,testing] | ||
|
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. 📐 Maintainability & Code Quality | 🟠 Major | ⚡ Quick win 🧩 Analysis chain🏁 Script executed: #!/bin/bash
set -euo pipefail
echo "== workflow matching files =="
fd -a 'cron\.yml$|cron\.yaml$' .github/workflows 2>/dev/null || true
echo "== target lines =="
if [ -f .github/workflows/cron.yml ]; then
nl -ba .github/workflows/cron.yml | sed -n '228,250p'
fi
echo "== pip install / setup.py occurrences =="
rg -n "pip install|setup\.py develop|\[all,testing\]|BUILD_MONAI" .github/workflows/cron.yml .github/workflows 2>/dev/null || true
echo "== repository references to install commands =="
rg -n "pip install.*\[all,testing\]|setup\.py develop|BUILD_MONAI|editable|develop" .github workflows setup.py pyproject.toml 2>/dev/null | head -120 || trueRepository: Project-MONAI/MONAI Length of output: 292 🏁 Script executed: #!/bin/bash
set -euo pipefail
echo "== target lines =="
awk 'NR>=228 && NR<=252 { printf "%s\t%s\n", NR, $0 }' .github/workflows/cron.yml
echo "== install-related occurrences in cron workflow =="
awk '
/pip install|setup\.py develop|\[all,testing\]|BUILD_MONAI|develop|editable|editable_install/ {
printf "%s\t%s\n", NR, $0
}
' .github/workflows/cron.yml
echo "== repository installs mentioning all/testing/build_monai/develop =="
git grep -n "pip install.*\[all,testing\]\|setup\.py develop\|BUILD_MONAI\|\[all,testing\]\|\[testing\]" .github setup.py pyproject.toml README.md 2>/dev/null | head -200 || trueRepository: Project-MONAI/MONAI Length of output: 7270 Finish the pip-only migration. Line 243 still runs 🤖 Prompt for AI Agents |
||
| BUILD_MONAI=1 python setup.py develop # install monai | ||
| nvidia-smi | ||
| export CUDA_VISIBLE_DEVICES=$(python -m tests.utils | tail -n 1) | ||
|
|
||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -37,8 +37,7 @@ jobs: | |
| pip uninstall -y monai | ||
| pip uninstall -y monai-weekly | ||
| pip uninstall -y monai-weekly | ||
| python -m pip install --upgrade torch torchvision torchaudio torchtext | ||
| python -m pip install -r requirements-dev.txt | ||
| python -m pip install .[all,testing] | ||
|
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. 🩺 Stability & Availability | 🟠 Major | ⚡ Quick win Use a Python 3.10+ integration image. Both jobs run
📍 Affects 1 file
🤖 Prompt for AI Agents |
||
| rm -rf /github/home/.cache/torch/hub/mmars/ | ||
| - name: Clean directory | ||
| run: | | ||
|
|
@@ -113,8 +112,7 @@ jobs: | |
| pip uninstall -y monai | ||
| pip uninstall -y monai-weekly | ||
| pip uninstall -y monai-weekly | ||
| python -m pip install --upgrade torch torchvision torchaudio torchtext | ||
| python -m pip install -r requirements-dev.txt | ||
| python -m pip install .[all,testing] | ||
| rm -rf /github/home/.cache/torch/hub/mmars/ | ||
| - name: Clean directory | ||
| run: | | ||
|
|
||
| Original file line number | Diff line number | Diff line change | ||||||||||||||||||||||||||||||||||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
|
|
@@ -28,9 +28,9 @@ RUN apt update && apt upgrade -y && \ | |||||||||||||||||||||||||||||||||||||||||||||||
| wget https://developer.download.nvidia.com/compute/cuda/repos/debian12/x86_64/cuda-keyring_1.1-1_all.deb && \ | ||||||||||||||||||||||||||||||||||||||||||||||||
| dpkg -i cuda-keyring_1.1-1_all.deb && \ | ||||||||||||||||||||||||||||||||||||||||||||||||
| apt update && \ | ||||||||||||||||||||||||||||||||||||||||||||||||
| ${APT_INSTALL} cuda-toolkit-12-9 && \ | ||||||||||||||||||||||||||||||||||||||||||||||||
| ${APT_INSTALL} cuda-toolkit-13-3 && \ | ||||||||||||||||||||||||||||||||||||||||||||||||
| rm -rf /usr/lib/python*/EXTERNALLY-MANAGED /var/lib/apt/lists/* && \ | ||||||||||||||||||||||||||||||||||||||||||||||||
| python -m pip install --upgrade --no-cache-dir --no-build-isolation pip | ||||||||||||||||||||||||||||||||||||||||||||||||
| python -m pip install --upgrade --no-cache-dir pip | ||||||||||||||||||||||||||||||||||||||||||||||||
|
|
||||||||||||||||||||||||||||||||||||||||||||||||
| # TODO: remark for issue [revise the dockerfile](https://github.com/zarr-developers/numcodecs/issues/431) | ||||||||||||||||||||||||||||||||||||||||||||||||
| RUN if [[ $(uname -m) =~ "aarch64" ]]; then \ | ||||||||||||||||||||||||||||||||||||||||||||||||
|
|
@@ -46,18 +46,14 @@ RUN wget -q ${NGC_CLI_URI} && unzip ngccli_linux.zip && chmod u+x ngc-cli/ngc && | |||||||||||||||||||||||||||||||||||||||||||||||
|
|
||||||||||||||||||||||||||||||||||||||||||||||||
| WORKDIR /opt/monai | ||||||||||||||||||||||||||||||||||||||||||||||||
|
|
||||||||||||||||||||||||||||||||||||||||||||||||
| # copy relevant parts of repo | ||||||||||||||||||||||||||||||||||||||||||||||||
| COPY requirements.txt requirements-min.txt requirements-dev.txt versioneer.py setup.py setup.cfg pyproject.toml ./ | ||||||||||||||||||||||||||||||||||||||||||||||||
| COPY LICENSE CHANGELOG.md CODE_OF_CONDUCT.md CONTRIBUTING.md README.md MANIFEST.in runtests.sh ./ | ||||||||||||||||||||||||||||||||||||||||||||||||
| # TODO: remark for issue [revise the dockerfile #1276](https://github.com/Project-MONAI/MONAI/issues/1276) | ||||||||||||||||||||||||||||||||||||||||||||||||
| # please specify exact files and folders to be copied -- else, basically always, the Docker build process cannot cache | ||||||||||||||||||||||||||||||||||||||||||||||||
| # this or anything below it and always will build from at most here; one file change leads to no caching from here on... | ||||||||||||||||||||||||||||||||||||||||||||||||
| COPY LICENSE CHANGELOG.md CODE_OF_CONDUCT.md CONTRIBUTING.md README.md versioneer.py setup.py pyproject.toml runtests.sh MANIFEST.in ./ | ||||||||||||||||||||||||||||||||||||||||||||||||
| COPY tests ./tests | ||||||||||||||||||||||||||||||||||||||||||||||||
| COPY monai ./monai | ||||||||||||||||||||||||||||||||||||||||||||||||
|
|
||||||||||||||||||||||||||||||||||||||||||||||||
| # install full deps | ||||||||||||||||||||||||||||||||||||||||||||||||
| RUN python -m pip install --no-cache-dir --no-build-isolation -U wheel wheel-stub | ||||||||||||||||||||||||||||||||||||||||||||||||
| RUN python -m pip install --no-cache-dir --no-build-isolation "torch>=2.8.0,<2.11" -r requirements-dev.txt | ||||||||||||||||||||||||||||||||||||||||||||||||
|
|
||||||||||||||||||||||||||||||||||||||||||||||||
| # compile ext | ||||||||||||||||||||||||||||||||||||||||||||||||
| RUN CUDA_HOME=/usr/local/cuda FORCE_CUDA=1 USE_COMPILED=1 BUILD_MONAI=1 python setup.py develop | ||||||||||||||||||||||||||||||||||||||||||||||||
| RUN BUILD_MONAI=1 FORCE_CUDA=1 pip install --no-cache-dir -e .[all,testing] | ||||||||||||||||||||||||||||||||||||||||||||||||
|
Comment on lines
+49
to
+56
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. 🩺 Stability & Availability | 🟠 Major | ⚡ Quick win Missing Unlike 🔧 Proposed fix mirroring Dockerfile's approach-RUN BUILD_MONAI=1 FORCE_CUDA=1 pip install --no-cache-dir -e .[all,testing]
+RUN python monai/config/print_dependencies.py build-system | xargs pip install --no-cache-dir --no-build-isolation \
+ && BUILD_MONAI=1 FORCE_CUDA=1 pip install --no-cache-dir --no-build-isolation -e .[all,testing]📝 Committable suggestion
Suggested change
🤖 Prompt for AI Agents
Member
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. @coderabbitai This should use the same PyTorch version for the isolated environment as the install environment as setuptools is in the "dependencies" explicitly. This was the library that causes some of the PyTorch version mismatches. Is there any way of forcing pip to use the same versions of dependencies in the isolated environment as it would install into the target environment without discarding build isolation?
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. 🧩 Analysis chain🏁 Script executed: #!/bin/bash
set -euo pipefail
printf '%s\n' '=== pyproject build-system and dependency-related sections ==='
awk '
/^\[build-system\]$/ {show=1}
/^\[/ && $0 != "[build-system]" && show {exit}
show {print}
' pyproject.toml
printf '%s\n' '=== dependency/constraint file names ==='
fd -HI -t f '(^|/)(requirements.*|constraints.*|.*\.lock)$' . | sort
printf '%s\n' '=== PyTorch and build-isolation references in packaging files ==='
rg -n -i -C2 '(torch|build-system|build-isolation|setuptools)' \
pyproject.toml setup.py setup.cfg requirements*.txt 2>/dev/null || trueLength of output: 7706 🤖 web_search executed:
Length of output: 5187
RUN BUILD_MONAI=1 FORCE_CUDA=1 pip install --no-cache-dir \
--constraint constraints.txt \
--build-constraint constraints.txt \
-e .[all,testing]
|
||||||||||||||||||||||||||||||||||||||||||||||||
|
|
||||||||||||||||||||||||||||||||||||||||||||||||
| # recreate the image without the installed CUDA packages then copy the installed MONAI and Python directories | ||||||||||||||||||||||||||||||||||||||||||||||||
| FROM ${IMAGE} AS build2 | ||||||||||||||||||||||||||||||||||||||||||||||||
|
|
||||||||||||||||||||||||||||||||||||||||||||||||
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
🩺 Stability & Availability | 🟠 Major | ⚡ Quick win
Keep the Hyena dependency install
--no-deps..[hyena]resolvesnvsubquadraticand its Torch constraint, contradicting the job’s stated isolation strategy. Install.[all,testing], then explicitly installomegaconfandnvsubquadraticwith--no-deps; remove the second editable install.🤖 Prompt for AI Agents