GH-50090: [Packaging] Avoid building CUDA for Debian forky due to missing nvidia-cuda-toolkit#50279
Conversation
|
|
github-actions
Bot
added
CI: Extra: Package: Linux
raulcd
added
CI: Extra: Package: Linux
|
@kou what do you think about an approach like this? I will clean up the PR and description if this sounds reasonable to you. I've used the suggestion from claude because it seems reasonable to me. Let me know if you think a different approach would be better. I've validated debian-forky is not pushing the debian packages but other debian amd64 are building them. |
kou
left a comment
kou
left a comment
There was a problem hiding this comment.
Wow! I didn't know the build profile feature!
| if ! dpkg -l nvidia-cuda-toolkit > /dev/null 2>&1; then | ||
| DEB_BUILD_PROFILES="${DEB_BUILD_PROFILES:-} pkg.apache-arrow.nocuda" |
There was a problem hiding this comment.
Could you do this check in Dockerfile (ENV DEB_BUILD_PROFILES=...) instead of here? build.sh is reused in all packages.
diff --git a/dev/tasks/linux-packages/apache-arrow/apt/debian-forky/Dockerfile b/dev/tasks/linux-packages/apache-arrow/apt/debian-forky/Dockerfile
index 0f679f0388..ed24e124eb 100644
--- a/dev/tasks/linux-packages/apache-arrow/apt/debian-forky/Dockerfile
+++ b/dev/tasks/linux-packages/apache-arrow/apt/debian-forky/Dockerfile
@@ -88,3 +88,5 @@ RUN \
apt install -y -V ${quiet} nvidia-cuda-toolkit; \
fi && \
apt clean
+
+ENV DEB_BUILD_PROFILES=pkg.apache-arrow.nocudaWe don't need export DEB_BUILD_PROFILES too. So we don't need to change build.sh for this.
There was a problem hiding this comment.
ok, I guess the only thing is that this won't auto-detect whether nvidia-cuda-toolkit is available or not so we will have to manually remove this line once available. I've added a comment.
github-actions
Bot
added
awaiting changes
… it into debian-forky
github-actions
Bot
added
awaiting change review
raulcd
changed the title
There was a problem hiding this comment.
Pull request overview
This PR addresses Debian forky amd64 packaging failures caused by nvidia-cuda-toolkit being temporarily unavailable by introducing a Debian build profile to disable CUDA-related build dependencies and binary packages for that distribution.
Changes:
- Add a new Debian build profile
pkg.apache-arrow.nocudato conditionally skipnvidia-cuda-toolkitand all CUDA binary packages indebian/control.in. - Set
DEB_BUILD_PROFILES=pkg.apache-arrow.nocudain thedebian-forkypackaging Dockerfile to disable CUDA packaging on forky.
Reviewed changes
Copilot reviewed 2 out of 2 changed files in this pull request and generated 2 comments.
| File | Description |
|---|---|
| dev/tasks/linux-packages/apache-arrow/debian/control.in | Adds pkg.apache-arrow.nocuda build-profile guards for the CUDA build-dependency and CUDA binary packages. |
| dev/tasks/linux-packages/apache-arrow/apt/debian-forky/Dockerfile | Sets DEB_BUILD_PROFILES for Debian forky to avoid CUDA packaging while the toolkit is missing upstream. |
| # nvidia-cuda-toolkit may be unavailable (e.g. removed from Debian forky). | ||
| # The pkg.apache-arrow.nocuda build profile (set via DEB_BUILD_PROFILES, e.g. | ||
| # in the debian-forky Dockerfile) skips this build dependency and the CUDA | ||
| # packages below. | ||
| nvidia-cuda-toolkit [!arm64] <!pkg.apache-arrow.nocuda>, |
There was a problem hiding this comment.
I think that's fine ... as we are controlling whether nvidia-cuda-toolkit is installed or not. Also if it starts breaking because nvidia-cuda-toolkit is now installed on debian-forky might be a good sign to remove the build profile so might not be bad to have a "notification"
| # nvidia-cuda-toolkit was removed from Debian forky amd64, so disable the | ||
| # CUDA packages via the pkg.apache-arrow.nocuda build profile.See GH-50090. | ||
| # Once nvidia-cuda-toolkit is re-added to Debian forky, this can be removed. | ||
| ENV DEB_BUILD_PROFILES=pkg.apache-arrow.nocuda |
There was a problem hiding this comment.
Same as my previous comment. I think that's going to be the notification I was looking for on when to re-enable. I see this as a feature more than as a bug :)
|
After merging your PR, Conbench analyzed the 4 benchmarking runs that have been run so far on merge-commit 649a667. There were no benchmark performance regressions. 🎉 The full Conbench report has more details. |
Rationale for this change
debian-forky amd64 is currently failing because nvidia-cuda-toolkit as been removed from the testing dependencies due to a bug.
What changes are included in this PR?
Create new Build Profile spec to conditionally disable cuda packages for debian:
pkg.apache-arrow.nocudaSet
pkg.apache-arrow.nocudafor debian-forky so cuda isn't built andnvidia-cuda-toolkitis conditionally checked.Are these changes tested?
Yes on CI via the Linux Package jobs.
Are there any user-facing changes?
No but debian-forky (testing) won't have CUDA packages until re-enabled