Skip to content

Close gRPC channels and HTTP clients in batch uploaders#1182

Open
joaquinhuigomez wants to merge 1 commit intoqdrant:masterfrom
joaquinhuigomez:fix/batch-uploader-resource-leak
Open

Close gRPC channels and HTTP clients in batch uploaders#1182
joaquinhuigomez wants to merge 1 commit intoqdrant:masterfrom
joaquinhuigomez:fix/batch-uploader-resource-leak

Conversation

@joaquinhuigomez
Copy link
Copy Markdown

Summary

  • Close gRPC channels and HTTP clients in batch uploaders after upload completes
  • Prevents file descriptor leaks when using parallel > 1

Each parallel worker in GrpcBatchUploader and RestBatchUploader creates its own gRPC channel or HTTP client. These were never closed after the upload loop finished, leaking file descriptors. Wrapping the upload loops in try/finally ensures cleanup happens both on normal completion and on error.

Fixes #1172

When using parallel > 1, each worker creates its own gRPC channel
or HTTP client that was never cleaned up after the upload loop
completed. Wrap the upload loops in try/finally to ensure channels
and clients are closed, preventing file descriptor leaks.

Fixes qdrant#1172
@netlify
Copy link
Copy Markdown

netlify bot commented Mar 28, 2026

Deploy Preview for poetic-froyo-8baba7 failed.

Name Link
🔨 Latest commit 4887efa
🔍 Latest deploy log https://app.netlify.com/projects/poetic-froyo-8baba7/deploys/69c7c44a8f65c600086fcc1e

@coderabbitai
Copy link
Copy Markdown

coderabbitai bot commented Mar 28, 2026

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro

Run ID: a5740763-b895-42d8-8b3e-3f1bd248ba72

📥 Commits

Reviewing files that changed from the base of the PR and between cd5eb25 and 4887efa.

📒 Files selected for processing (2)
  • qdrant_client/uploader/grpc_uploader.py
  • qdrant_client/uploader/rest_uploader.py

📝 Walkthrough

Walkthrough

The pull request adds resource cleanup to two batch uploader classes to prevent connection leaks. In grpc_uploader.py, the gRPC channel obtained from get_channel() is now explicitly closed in a finally block after batch processing completes. Similarly, in rest_uploader.py, the SyncApis HTTP client is now unconditionally closed in a finally block during item iteration. Both changes wrap previously unmanaged resource lifecycles with try/finally blocks to ensure deterministic cleanup.

Estimated code review effort

🎯 2 (Simple) | ⏱️ ~10 minutes

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 0.00% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (4 passed)
Check name Status Explanation
Title check ✅ Passed The PR title accurately and clearly summarizes the main change: closing gRPC channels and HTTP clients in batch uploaders to prevent resource leaks.
Description check ✅ Passed The PR description is well-related to the changeset, explaining the resource leak problem, the fix applied (try/finally blocks), and the issue being fixed.
Linked Issues check ✅ Passed The changes successfully address all requirements from #1172: gRPC channels are closed in GrpcBatchUploader.process_upload() and HTTP clients are closed in RestBatchUploader.process() using try/finally blocks.
Out of Scope Changes check ✅ Passed All changes are in-scope, focused exclusively on adding resource cleanup (try/finally blocks) to GrpcBatchUploader and RestBatchUploader as specified in #1172.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Resource leak: gRPC channels and HTTP clients not closed in batch uploaders

1 participant