Skip to content

Fix issues around frontend path discovery of externally built extension modules #2730

Description

@dime10

A few issues around path discovery of externally built extension modules:

1) Runtime nanobind module discovery repeatedly appends to path

There are two locations where this happens for the runtime registry module:

sys.path.append(get_lib_path("runtime", "RUNTIME_LIB_DIR"))

and

sys.path.append(rt_lib_path)

Because these lines are run every-time functionality from the module is used, the system path would get polluted with more and more duplicate entries. This doesn't break anything but is not ideal.

A solution could perform this task once upon import for example, similar to how we handle the mlir_quantum module:

if not INSTALLED:
import os
default_bindings_path = os.path.join(
os.path.dirname(__file__), "../../mlir/build/python_packages/quantum"
)
if os.path.exists(default_bindings_path): # pragma: no cover
sys.path.insert(0, default_bindings_path)

The thing to keep in mind is that these modules may not exist in some contexts, in particular for a CI docs build which uses only the Python code. So code importing such modules would fail in this context. Typically, we resolve this with mocking the modules in the docs config:

catalyst/doc/conf.py

Lines 88 to 90 in c6f3180

MOCK_MODULES = [
"mlir_quantum",
"mlir_quantum.runtime",

2) The fix should consider a similar module that will be merged soon from the MLIR component to make sure all such modules are handled uniformly

See #2259 which introduced the new module.

3) The path mlir_quantum should consider custom build directories when not in INSTALLED mode.

See also #2713 (comment) for a discussion of this.

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions