Skip to content

Move the import of langchain_text_splitters into chunking.default_chunk_fn#37

Open
jun76 wants to merge 1 commit intovoyage-ai:mainfrom
jun76:delayed_import_of_langchain_text_splitters
Open

Move the import of langchain_text_splitters into chunking.default_chunk_fn#37
jun76 wants to merge 1 commit intovoyage-ai:mainfrom
jun76:delayed_import_of_langchain_text_splitters

Conversation

@jun76
Copy link

@jun76 jun76 commented Nov 30, 2025

Description

  • Prevent VoyageAI from pulling in the LangChain stack at import time by moving the langchain_text_splitters import inside the chunking helper.
  • Surface a error message if chunking is used without the optional dependency.
  • No changes to functional behavior; just defers the heavy dependency until the chunking utilities are actually called.

Background

Thank you for your continued support.

I noticed that simply import voyageai in my project causes python -X importtime to take about 5 seconds.

Upon investigating the cause, I discovered that langchain_text_splitters, loaded in the global scope of voyageai/chunking.py, finds and imports heavy dependencies within my project, such as sentence-transformers, torch, transformers, and scipy.

Therefore, I would greatly appreciate it if langchain_text_splitters could be deferred. I have submitted this PR accordingly.

Thank you.

Pytest

The following has been verified to pass.

pytest tests/test_client.py -k 'with_chunking_fn'

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant