Conversation
…ce strings Cherry-picks the shared-library-by-default and pybind-simplification work from copilot/separate-pybind-and-libopen3d without the host/device (libOpen3D vs libOpen3D_cuda/xpu) library split. - BUILD_SHARED_LIBS now defaults to ON. - Single open3d.pybind module; CUDA/SYCL detected at runtime via core.cuda.is_available()/core.sycl.is_available() instead of shipping separate cpu/cuda package variants. - Device strings accept a bare type name (e.g. "cuda"), defaulting id to 0. - New utility::filesystem::GetSelfBinaryDirectory() helper; Logging.cpp print function is now a function-local static. - Renamed pybind/core/tensor_type_caster -> type_caster and added an implicit Device<->str caster. - Flattened ML custom-op output layout (no cpu/cuda subfolder) and fixed the torch/tf ops loaders to match. - Fixed a static-link ordering bug (curl/BoringSSL) exposed by building Open3D as a shared library, using CMake's LINK_GROUP:RESCAN genex. - Ported unrelated CI/Docker/WebRTC improvements from the source branch. - Updated docs/compilation.rst to describe the single combined library. Verified: BUILD_SHARED_LIBS=ON by default, cpp tests (Device, MemoryManager) and python/test/core/test_core.py device tests pass, and python import/ smoke test shows no cpu/cuda subpackages. Co-authored-by: Cursor <cursoragent@cursor.com>
|
Thanks for submitting this pull request! The maintainers of this repository would appreciate if you could update the CHANGELOG.md based on your changes. |
Several bugs slipped through when cherry-picking CI/Docker changes and the
simplified single-pybind-module python packaging from
copilot/separate-pybind-and-libopen3d:
- python/open3d/__init__.py: _insert_pybind_names() stripped the "open3d."
prefix (not just "pybind."), so e.g. "open3d.pybind.t" was registered as
bare "t" instead of "open3d.t", breaking `import open3d.t` and any
`from open3d.t... import ...`. This failed pytest collection on the
Ubuntu and OpenBLAS CI jobs.
- .github/workflows/windows.yml: BUILD_WEBRTC was hardcoded to 'OFF', but
main enables it for the STATIC_RUNTIME=ON + BUILD_SHARED_LIBS=OFF matrix
cells, since our own prebuilt BoringSSL is built with the dynamic CRT and
is only ABI-compatible when WebRTC (built with matching static CRT)
supplies crypto/ssl symbols instead. Restored the conditional; this fixes
LNK2001 "unresolved external symbol __imp_bsearch" failures in the three
static-runtime build-lib jobs.
- .github/workflows/ubuntu-cuda.yml: GCE image family was bumped to
"ubuntu-os-docker-gpu-2204-lts", which does not exist in the GCP project
("was not found" at instance-create time). Reverted to main's working
"ubuntu-os-docker-gpu-2004-lts" (the GCE host image is independent of the
Ubuntu version used inside the CI Docker container).
- .github/workflows/ubuntu-wheel.yml: the new split-out "build-lib" job
calls docker_build.sh, which requires BUILD_TENSORFLOW_OPS/
BUILD_PYTORCH_OPS to be set (bash `set -u`); these were set on the
build-wheel job but missing on build-lib, causing "unbound variable".
Not fixed (pre-existing/flaky, reproduced identically on unrelated
branches, unrelated to this port): macOS gfortran symlink lookup failing
on macos-14 runners, and transient pyenv-installer network fetch failures
on ARM64/py313+ wheel builds.
Verified: reproduced the "open3d.t" ModuleNotFoundError from the CI logs
using a local minimal CPU-only build (BUILD_PYTHON_MODULE=ON, no GUI/tests/
ISPC/WebRTC) and confirmed `import open3d.t`, `from open3d.t.geometry
import Image`, and pytest collection of python/test/t/geometry/test_image.py
all work after the fix; python/test/core/test_core.py (279 tests) still
passes.
Co-authored-by: Cursor <cursoragent@cursor.com>
… artifacts
The new split "build-lib" + "build-wheel"/"build-wheel-per-python" CI job
architecture (ported from copilot/separate-pybind-and-libopen3d) reuses a
previously-built lib image/artifact as the starting point for wheel jobs
(Docker BASE_IMAGE, or an extracted build.tar.xz/zip). Two steps in the
common build scripts were not idempotent against that reuse:
- docker/Dockerfile.{ci,wheel,openblas}: pyenv-installer refuses to run if
$PYENV_ROOT already exists, which is the case when BASE_IMAGE is a
previously-built lib image (SYCL/CUDA/ARM64 wheel jobs) that already
bootstrapped pyenv for its own (dummy) Python version. Only run the
installer bootstrap if pyenv isn't already present; always pass
`pyenv install -s` (skip-if-installed) and `ln -sf` so a second Python
version can still be added on top of an existing pyenv installation.
- util/ci_utils.sh build_pip_package(): skipped `cmake` configure entirely
if CMakeCache.txt already existed, to support being invoked more than
once. In practice it is only ever called once per job, and inheriting a
build dir from a build-lib step (BUILD_PYTHON_MODULE=OFF) meant the
Makefile never got a pip-package target, failing with
"No rule to make target 'pip-package'" (macOS wheel job). Always run
cmake with the current BUILD_PYTHON_MODULE value instead.
- util/ci_utils.sh build_pip_package(): the extra "open3d-cpu" companion
wheel was built when BUILD_CUDA_MODULE==OFF instead of ==ON. A CPU-only
build already *is* the CPU wheel; the companion wheel should only be
built in addition to a CUDA-enabled build (matching main's original
behavior). The inverted condition caused every CPU-only build (e.g. all
macOS wheel jobs) to redundantly rebuild the whole project a second time
for no reason.
Co-authored-by: Cursor <cursoragent@cursor.com>
… artifacts
The split "build-lib" + "build-wheel" jobs (ARM64, macOS, SYCL, CUDA wheel)
reuse a previously-configured build directory/image whose CMakeCache.txt was
generated for a different Python interpreter (e.g. build-lib's own "dummy"
Python version) and/or BUILD_PYTHON_MODULE=OFF. Simply re-invoking cmake
with new -D flags is not enough: find_package(Python3)/find_package(Torch)
still resolve to the stale cached Python3_EXECUTABLE, so the wheel-build's
freshly pip-installed torch for the *new* Python version is invisible
("ModuleNotFoundError: No module named 'torch'" from FindPytorch.cmake).
Use `cmake --fresh` (already used by windows.yml) everywhere a build
directory may be inherited from a build-lib artifact, so CMakeCache.txt is
discarded and all interpreter/library paths are rediscovered from scratch.
ccache (already configured in these jobs) keeps the ensuing rebuild fast
since object code for unchanged sources is still reused.
- util/ci_utils.sh build_pip_package(): `cmake --fresh` for the main build
directory (used by macOS/ARM64/CUDA wheel jobs).
- docker/Dockerfile.openblas: `mkdir -p build` (was `mkdir build`, which
failed outright when build/ already existed) and `cmake --fresh`.
- docker/Dockerfile.ci (SYCL/CUDA "Build all" step): `cmake --fresh`.
Co-authored-by: Cursor <cursoragent@cursor.com>
build-wheel-arm64 (Ubuntu OpenBLAS workflow) calls actions/attest@v4 to sign the built wheel, which requires "id-token: write" and "attestations: write" permissions. Every other build-wheel job (macOS, Ubuntu Wheel, Ubuntu SYCL, Windows) already grants these; this one only had "contents: read", causing it to fail with "missing id-token permission" after a successful build. Co-authored-by: Cursor <cursoragent@cursor.com>
…TC=ON) The Windows wheel build (shared libs) explicitly sets BUILD_WEBRTC=OFF because no WebRTC DLL is available for Windows shared-lib builds yet, but still requested BUILD_JUPYTER_EXTENSION=ON. CMakeLists.txt enforces BUILD_JUPYTER_EXTENSION=ON requires BUILD_WEBRTC=ON, so configure failed. main built this wheel as a static lib (BUILD_WEBRTC=ON is possible there); our shared-lib-by-default wheel can't enable WebRTC yet, so disable the Jupyter extension for it until WebRTC gains Windows shared-lib support. Co-authored-by: Cursor <cursoragent@cursor.com>
Same class of bug as the pyenv/build-dir issues: when BASE_IMAGE is a previously-built lib image (build-lib mode / CI's split lib+wheel jobs), /root/Open3D-ML was already cloned there, so cloning again into the same non-empty directory fails with "destination path already exists". Skip the clone if the repo is already present. Co-authored-by: Cursor <cursoragent@cursor.com>
Same class of bug as the other reused-build-lib-artifact issues: BASE_IMAGE may already have /etc/apt/keyrings/nodesource.gpg from build-lib's own run of this step. Without --yes, gpg --dearmor tries to interactively prompt whether to overwrite the existing file, which fails with "cannot open '/dev/tty'" in a non-interactive Docker build step. Co-authored-by: Cursor <cursoragent@cursor.com>
- visualization/CMakeLists.txt: add_dependencies(visualization_impl ext_vkmemalloc ext_vmahpp) so the Vulkan gaussian-splat sources always build after the vk_mem_alloc headers are available. OBJECT library target_link_libraries doesn't reliably enforce build order for ExternalProject targets on the VS generator, which caused vk_mem_alloc.hpp not found when building the narrower "pip-package" target (Windows wheel build). - util/ci_utils.sh build_pip_package(): replace `cmake --fresh` with a normal reconfigure that force-overrides Python3_EXECUTABLE (and unsets stale Python3/Torch cache vars) to fix the stale interpreter left over from a previous BUILD_PYTHON_MODULE=OFF "build-lib" stage. `--fresh` wipes CMakeFiles/ which, on Docker union-filesystem layers stacked on an already-built base image, doesn't reclaim space and was exhausting runner disk (Ubuntu CUDA wheel job). Co-authored-by: Cursor <cursoragent@cursor.com>
…regex
- util/ci_utils.sh build_pip_package(): the companion open3d-cpu wheel's
cmake call put -DBUILD_CUDA_MODULE=OFF before "${cmakeOptions[@]}",
which already contains -DBUILD_CUDA_MODULE=ON from the main CUDA build
above. CMake resolves repeated -D flags left-to-right with the last one
winning, so the "CPU-only" wheel was actually being built with CUDA
enabled. Reorder so the OFF override comes last. Verified locally that
cmake's last-flag-wins behavior reproduces/fixes this with a minimal
project.
- docker/docker_build.sh sycl-shared_export_env(): the python-version
regex expected undotted "py310" but the SYCL wheel workflow passes
dotted "py3.10" (py${{ matrix.python_version }}), so the branch setting
PYTHON_VERSION/BUILD_PYTHON_MODULE was dead code (masked by the
PYTHON_VERSION:-3.12 fallback picking up the workflow's own env var).
Fix the regex to match the dotted form.
Co-authored-by: Cursor <cursoragent@cursor.com>
`docker/Dockerfile.wheel` had `rm -rf build_cpu build_cuda` but `build_cuda`
doesn't exist under the new design (the main build directory is just `build`),
and more importantly, the quotes in the condition `if [ ${CI:-}a != a ]`
were unquoted, which shellcheck/bash could evaluate incorrectly. Cleaned up
and formatted the cleanup block.
Co-authored-by: Cursor <cursoragent@cursor.com>
When compiling Open3D, CMake produced several "dev" warnings: The feature 'RESCAN', specified as part of a generator-expression '$<LINK_GROUP:RESCAN>', will not be applied to the INTERFACE library '3rdparty_curl'. This is because CMake's modern `<LINK_GROUP:feature, ...>` is only officially supported on targets with a linkable binary file (such as static or shared library targets, or imported targets with IMPORTED_LOCATION), and explicit warnings are emitted when applied to logical INTERFACE targets. Replace the generator expression with explicit linker flags `-Wl,--start-group` and `-Wl,--end-group` for UNIX (non-Apple) systems. This resolves the developer warnings while achieving the identical linking behavior to resolve mutual dependencies between curl and openssl static archives during library linkage. Co-authored-by: Cursor <cursoragent@cursor.com>
Reusing a stripped Docker build tree caused overlay ENOSPC and forced 3rdparty recompiles in wheel jobs. Package CPU/CUDA devel (+ viewer) once, run C++ tests in parallel with per-Python wheels on stock nvidia/cuda, and compile Torch/TF ops only in the wheel stage where their ABI belongs. Co-authored-by: Cursor <cursoragent@cursor.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Cherry-picks the shared-library-by-default and pybind-simplification work from copilot/separate-pybind-and-libopen3d without the host/device (libOpen3D vs libOpen3D_cuda/xpu) library split.
Type
Motivation and Context
Checklist:
python util/check_style.py --applyto apply Open3D code styleto my code.
updated accordingly.
results (e.g. screenshots or numbers) here.
Description