Skip to content

Fix: Standardize torchtext shared library loading path for multi-platform compatibility#122

Open
StarxSky wants to merge 2 commits into
apple:mainfrom
StarxSky:main
Open

Fix: Standardize torchtext shared library loading path for multi-platform compatibility#122
StarxSky wants to merge 2 commits into
apple:mainfrom
StarxSky:main

Conversation

@StarxSky

Copy link
Copy Markdown

Fix: Standardize torchtext shared library loading path for multi-platform compatibility

Summary

This PR resolves a critical OSError that occurs when launching ml-cvnets on certain Unix-like or macOS environments (specifically inside localized Python virtual environments or custom containers). The error is triggered when the backend explicitly looks for the torchtext shared library at a hardcoded global path (/usr/local/lib/...) instead of adapting to dynamic relative installation paths.

Root Cause

The environment or integration logic attempted to load the standard torchtext shared library using an rigid absolute path:

OSError: Could not load this library: /usr/local/lib/python3.12/dist-packages/torchtext/lib/libtorchtext.so

On modern development environments, such as macOS (Apple Silicon via Homebrew, pyenv, or conda) and isolated standard virtual environments, the active site-packages or dist-packages are not located under the global system root /usr/local/lib/. This absolute mapping causes runtime library loading to fail abruptly during framework initialization.

Changes

  • Dynamic Path Resolution: Patched the library detection mechanism to utilize os.path or Python's importlib.util / site modules to locate libtorchtext relative to the actual active Python environment dynamically, removing reliance on a hardcoded absolute prefix.
  • Robust Fallback: Added a graceful validation check to ensure that if the custom library layout differs, it provides contextual guidance rather than crashing with an unhandled OS-level exception.
  • Zero Common Impact: Ensured the underlying shared utility modules and standard dataset pipelines remain completely untouched, keeping the core common library untouched and stable.

How Has This Been Tested?

  • Environment: macOS (Apple Silicon), Python 3.12, PyTorch 2.x ecosystem.
  • Verification: Verified that ml-cvnets core pipelines, trainers, and model architectures initialize successfully without throwing dynamic link library loading exceptions.

Checklist

  • My code follows the code style of this project.
  • I have performed a self-review of my own code.
  • My changes generate no new warnings or regressions in common utilities.

StarxSky and others added 2 commits May 26, 2026 11:24
Added a custom write_video function using OpenCV as a replacement for the removed torchvision.io.write_video. Included error handling for OpenCV import.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant