A few issues around path discovery of externally built extension modules:
1) Runtime nanobind module discovery repeatedly appends to path
There are two locations where this happens for the runtime registry module:
|
sys.path.append(get_lib_path("runtime", "RUNTIME_LIB_DIR")) |
and
|
sys.path.append(rt_lib_path) |
Because these lines are run every-time functionality from the module is used, the system path would get polluted with more and more duplicate entries. This doesn't break anything but is not ideal.
A solution could perform this task once upon import for example, similar to how we handle the mlir_quantum module:
|
if not INSTALLED: |
|
import os |
|
|
|
default_bindings_path = os.path.join( |
|
os.path.dirname(__file__), "../../mlir/build/python_packages/quantum" |
|
) |
|
if os.path.exists(default_bindings_path): # pragma: no cover |
|
sys.path.insert(0, default_bindings_path) |
The thing to keep in mind is that these modules may not exist in some contexts, in particular for a CI docs build which uses only the Python code. So code importing such modules would fail in this context. Typically, we resolve this with mocking the modules in the docs config:
|
MOCK_MODULES = [ |
|
"mlir_quantum", |
|
"mlir_quantum.runtime", |
2) The fix should consider a similar module that will be merged soon from the MLIR component to make sure all such modules are handled uniformly
See #2259 which introduced the new module.
3) The path mlir_quantum should consider custom build directories when not in INSTALLED mode.
See also #2713 (comment) for a discussion of this.
A few issues around path discovery of externally built extension modules:
1) Runtime nanobind module discovery repeatedly appends to path
There are two locations where this happens for the runtime registry module:
catalyst/frontend/catalyst/jax_primitives.py
Line 485 in c6f3180
and
catalyst/frontend/catalyst/compiler.py
Line 92 in c6f3180
Because these lines are run every-time functionality from the module is used, the system path would get polluted with more and more duplicate entries. This doesn't break anything but is not ideal.
A solution could perform this task once upon import for example, similar to how we handle the
mlir_quantummodule:catalyst/frontend/catalyst/__init__.py
Lines 55 to 62 in c6f3180
The thing to keep in mind is that these modules may not exist in some contexts, in particular for a CI docs build which uses only the Python code. So code importing such modules would fail in this context. Typically, we resolve this with mocking the modules in the docs config:
catalyst/doc/conf.py
Lines 88 to 90 in c6f3180
2) The fix should consider a similar module that will be merged soon from the MLIR component to make sure all such modules are handled uniformly
See #2259 which introduced the new module.
3) The path
mlir_quantumshould consider custom build directories when not in INSTALLED mode.See also #2713 (comment) for a discussion of this.