⚡ Bolt: Parallelize folder deletion by abhimehro · Pull Request #164 · abhimehro/ctrld-sync

abhimehro · 2026-02-05T14:49:10Z

⚡ Bolt: Parallelize folder deletion

💡 What:
Replaced sequential folder deletion loop with a parallel implementation using ThreadPoolExecutor (5 workers).

🎯 Why:
Deleting folders is a blocking I/O operation. Doing it sequentially for many folders (e.g. 20+) adds significant latency. Parallelizing it reduces this phase's duration from sum(latency) to max(latency) / workers.

📊 Impact:

Reduces folder deletion time by up to ~5x (with 5 workers).
Verified with tests/test_parallel_delete.py showing significant speedup over sequential mock.

🔬 Measurement:
Run pytest tests/test_parallel_delete.py.

PR created automatically by Jules for task 7500484463374288460 started by @abhimehro

- Refactored folder deletion logic into `delete_existing_folders_parallel` - Uses `ThreadPoolExecutor` (max 5 workers) to delete folders concurrently - Updates local `existing_folders` state in-place to ensure consistency - Added `tests/test_parallel_delete.py` to verify speedup and correctness This optimization reduces the time taken to clean up existing folders before re-syncing, especially when many folders need to be deleted. Co-authored-by: abhimehro <84992105+abhimehro@users.noreply.github.com>

google-labs-jules · 2026-02-05T14:49:11Z

👋 Jules, reporting for duty! I'm here to lend a hand with this pull request.

When you start a review, I'll add a 👀 emoji to each comment to let you know I've read it. I'll focus on feedback directed at me and will do my best to stay out of conversations between you and other bots or reviewers to keep the noise down.

I'll push a commit with your requested changes shortly after. Please note there might be a delay between these steps, but rest assured I'm on the job!

For more direct control, you can switch me to Reactive Mode. When this mode is on, I will only act on comments where you specifically mention me with @jules. You can find this option in the Pull Request section of your global Jules UI settings. You can always switch back!

New to Jules? Learn more at jules.google/docs.

For security, I will only act on instructions from the user who triggered this task.

trunk-io · 2026-02-05T14:49:14Z

Merging to main in this repository is managed by Trunk.

To merge this pull request, check the box to the left or comment /trunk merge below.

tests/test_parallel_delete.py

+    duration = end_time - start_time
+
+    # Assertions
+    assert result is True


tests/test_parallel_delete.py

+
+    # Check that the 10 folders were deleted
+    for i in range(10):
+        assert f"Folder {i}" not in existing_folders


tests/test_parallel_delete.py

+        assert f"Folder {i}" not in existing_folders
+
+    # Check that "Keep Me" is still there
+    assert "Keep Me" in existing_folders


tests/test_parallel_delete.py

+
+    # Check that "Keep Me" is still there
+    assert "Keep Me" in existing_folders
+    assert existing_folders["Keep Me"] == "id_keep"


tests/test_parallel_delete.py

+    assert existing_folders["Keep Me"] == "id_keep"
+
+    # Check that "New Folder" is not in existing_folders (it wasn't there to begin with)
+    assert "New Folder" not in existing_folders


tests/test_parallel_delete.py

+        client, profile_id, folder_data_list, existing_folders
+    )
+
+    assert result is False


tests/test_parallel_delete.py

+    )
+
+    assert result is False
+    assert "Folder B" in existing_folders


tests/test_parallel_delete.py

+        client, profile_id, folder_data_list, existing_folders
+    )
+
+    assert result is True # At least one succeeded


tests/test_parallel_delete.py

+        name = f"Folder {i}"
+        if i % 2 != 0:
+            # Odd: Succeeded, should be removed
+            assert name not in existing_folders


tests/test_parallel_delete.py

+            assert name not in existing_folders
+        else:
+            # Even: Failed, should remain
+            assert name in existing_folders


gemini-code-assist · 2026-02-05T14:49:37Z

Warning

Gemini encountered an error creating the summary. You can try again by commenting /gemini summary.

tests/test_parallel_delete.py

+    duration = end_time - start_time
+
+    # Assertions
+    assert result is True


tests/test_parallel_delete.py

+
+    # Check that the 10 folders were deleted
+    for i in range(10):
+        assert f"Folder {i}" not in existing_folders


tests/test_parallel_delete.py

+        assert f"Folder {i}" not in existing_folders
+
+    # Check that "Keep Me" is still there
+    assert "Keep Me" in existing_folders


tests/test_parallel_delete.py

+
+    # Check that "Keep Me" is still there
+    assert "Keep Me" in existing_folders
+    assert existing_folders["Keep Me"] == "id_keep"


tests/test_parallel_delete.py

+    assert existing_folders["Keep Me"] == "id_keep"
+
+    # Check that "New Folder" is not in existing_folders (it wasn't there to begin with)
+    assert "New Folder" not in existing_folders


tests/test_parallel_delete.py

+        client, profile_id, folder_data_list, existing_folders
+    )
+
+    assert result is False


tests/test_parallel_delete.py

+    )
+
+    assert result is False
+    assert "Folder B" in existing_folders


tests/test_parallel_delete.py

+        client, profile_id, folder_data_list, existing_folders
+    )
+
+    assert result is True # At least one succeeded


tests/test_parallel_delete.py

+        name = f"Folder {i}"
+        if i % 2 != 0:
+            # Odd: Succeeded, should be removed
+            assert name not in existing_folders


tests/test_parallel_delete.py

+            assert name not in existing_folders
+        else:
+            # Even: Failed, should remain
+            assert name in existing_folders


tests/test_parallel_delete.py

@@ -0,0 +1,124 @@
+


main.py

+
+
 def sync_profile(
    profile_id: str,


main.py

-                            del existing_folders[name]
-                            deletion_occurred = True
+                deletion_occurred = delete_existing_folders_parallel(
+                    client, profile_id, folder_data_list, existing_folders


tests/test_parallel_delete.py

@@ -0,0 +1,124 @@
+
+import concurrent.futures


tests/test_parallel_delete.py

+import concurrent.futures
+import time
+from unittest.mock import MagicMock
+import pytest


tests/test_parallel_delete.py

+
+    # All these folders exist in existing_folders
+    existing_folders = {f"Folder {i}": f"id_{i}" for i in range(10)}
+    original_len = len(existing_folders)


tests/test_parallel_delete.py

+    # Verify mock calls
+    assert mock_delete.call_count == 10
+
+def test_delete_existing_folders_parallel_no_deletions(monkeypatch):


tests/test_parallel_delete.py

+
+    # Mock delete_folder: Fail for even numbers
+    mock_delete = MagicMock()
+    def conditional_delete(client, profile_id, name, folder_id):


tests/test_parallel_delete.py

+
+    # Mock delete_folder: Fail for even numbers
+    mock_delete = MagicMock()
+    def conditional_delete(client, profile_id, name, folder_id):


tests/test_parallel_delete.py

+
+    # Mock delete_folder: Fail for even numbers
+    mock_delete = MagicMock()
+    def conditional_delete(client, profile_id, name, folder_id):


main.py

+    with concurrent.futures.ThreadPoolExecutor(max_workers=5) as executor:
+        future_to_name = {
+            executor.submit(delete_folder, client, profile_id, name, fid): name
+            for name, fid in folders_to_delete


main.py

+            executor.submit(delete_folder, client, profile_id, name, fid): name
+            for name, fid in folders_to_delete
+        }
+


main.py

+            for name, fid in folders_to_delete
+        }
+
+        for future in concurrent.futures.as_completed(future_to_name):


tests/test_parallel_delete.py

@@ -0,0 +1,124 @@
+


tests/test_parallel_delete.py

+
+    # Mock delete_folder: Fail for even numbers
+    mock_delete = MagicMock()
+    def conditional_delete(client, profile_id, name, folder_id):


tests/test_parallel_delete.py

+    # Mock delete_folder to simulate network delay
+    # Each deletion takes 0.1s
+    mock_delete = MagicMock()
+    def slow_delete(client, profile_id, name, folder_id):


tests/test_parallel_delete.py

+    # Mock delete_folder to simulate network delay
+    # Each deletion takes 0.1s
+    mock_delete = MagicMock()
+    def slow_delete(client, profile_id, name, folder_id):


tests/test_parallel_delete.py

+    # Mock delete_folder to simulate network delay
+    # Each deletion takes 0.1s
+    mock_delete = MagicMock()
+    def slow_delete(client, profile_id, name, folder_id):


tests/test_parallel_delete.py

+import concurrent.futures
+import time
+from unittest.mock import MagicMock
+import pytest


tests/test_parallel_delete.py

@@ -0,0 +1,124 @@
+
+import concurrent.futures


main.py

            if not no_delete:
-                deletion_occurred = False
-                for folder_data in folder_data_list:
-                    name = folder_data["group"]["group"].strip()


main.py

tests/test_parallel_delete.py

@@ -0,0 +1,124 @@
+
+import concurrent.futures


tests/test_parallel_delete.py

+import concurrent.futures
+import time
+from unittest.mock import MagicMock
+import pytest


tests/test_parallel_delete.py

+import concurrent.futures
+import time
+from unittest.mock import MagicMock
+import pytest


tests/test_parallel_delete.py

+    # Verify mock calls
+    assert mock_delete.call_count == 10
+
+def test_delete_existing_folders_parallel_no_deletions(monkeypatch):


tests/test_parallel_delete.py

+
+    # Mock delete_folder: Fail for even numbers
+    mock_delete = MagicMock()
+    def conditional_delete(client, profile_id, name, folder_id):


tests/test_parallel_delete.py

+
+    # Mock delete_folder: Fail for even numbers
+    mock_delete = MagicMock()
+    def conditional_delete(client, profile_id, name, folder_id):


tests/test_parallel_delete.py

+
+    # Mock delete_folder: Fail for even numbers
+    mock_delete = MagicMock()
+    def conditional_delete(client, profile_id, name, folder_id):


tests/test_parallel_delete.py

+
+    # Mock delete_folder: Fail for even numbers
+    mock_delete = MagicMock()
+    def conditional_delete(client, profile_id, name, folder_id):


Copilot

Pull request overview

This PR parallelizes folder deletion to improve performance by replacing sequential deletion with a ThreadPoolExecutor-based implementation using 5 workers. The change reduces folder deletion time from O(sum of latencies) to O(max latency / workers), potentially achieving up to 5x speedup for scenarios with many folders.

Changes:

Added delete_existing_folders_parallel() function that uses ThreadPoolExecutor with 5 workers to delete folders concurrently
Replaced sequential folder deletion loop in sync_profile() with a call to the new parallel implementation
Added comprehensive test suite with 3 test cases covering happy path, no-match scenario, and partial failure handling

Reviewed changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated 5 comments.

File	Description
main.py	Implements parallel folder deletion function and integrates it into sync_profile workflow
tests/test_parallel_delete.py	Adds comprehensive tests validating parallel execution, correctness, and partial failure handling

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Copilot · 2026-02-05T14:54:43Z

tests/test_parallel_delete.py

@@ -0,0 +1,124 @@
+
+import concurrent.futures


The import of concurrent.futures is unused in this test file. It should be removed to keep the imports clean.

Suggested change

import concurrent.futures

Copilot · 2026-02-05T14:54:44Z

tests/test_parallel_delete.py

+
+    # All these folders exist in existing_folders
+    existing_folders = {f"Folder {i}": f"id_{i}" for i in range(10)}
+    original_len = len(existing_folders)


The variable original_len is assigned but never used. Consider removing it or using it in an assertion to verify the test setup.

Copilot · 2026-02-05T14:54:44Z

tests/test_parallel_delete.py

+    # 10 tasks * 0.1s = 1.0s sequential time.
+    # With 5 workers, it should take roughly 0.2s + overhead.
+    # We assert it takes < 0.6s to be safe but prove parallelism.
+    assert duration < 0.6, f"Execution took {duration}s, expected < 0.6s (parallel)"


The timing assertion could be flaky on slower systems or under heavy load. Consider either increasing the threshold (e.g., to 0.8s or 1.0s) or using a more flexible assertion that checks if the parallel version is faster than sequential without relying on absolute time, such as comparing the duration to the theoretical sequential time with a tolerance factor.

Copilot · 2026-02-05T14:54:44Z

main.py

+    # Using 5 workers to be safe with rate limits for write operations
+    with concurrent.futures.ThreadPoolExecutor(max_workers=5) as executor:
+        future_to_name = {
+            executor.submit(delete_folder, client, profile_id, name, fid): name


The httpx.Client instance is being shared across multiple threads (5 workers). According to httpx documentation, Client instances are not thread-safe and should not be shared across threads without synchronization. While this pattern exists elsewhere in the codebase (e.g., in get_all_existing_rules), the concurrent modifications to the client's connection pool could lead to race conditions. Consider using a lock around client operations, creating a separate client instance in each thread, or using a thread-safe connection pool pattern.

Suggested change

# Using 5 workers to be safe with rate limits for write operations

with concurrent.futures.ThreadPoolExecutor(max_workers=5) as executor:

future_to_name = {

executor.submit(delete_folder, client, profile_id, name, fid): name

# Protect shared httpx.Client instance when used across multiple threads.

client_lock = threading.Lock()

def _threadsafe_delete_folder(profile_id_arg: str, name_arg: str, fid_arg: str) -> bool:

"""

Wrapper around delete_folder that serializes access to the shared client.

"""

with client_lock:

return delete_folder(client, profile_id_arg, name_arg, fid_arg)

# Using 5 workers to be safe with rate limits for write operations

with concurrent.futures.ThreadPoolExecutor(max_workers=5) as executor:

future_to_name = {

executor.submit(_threadsafe_delete_folder, profile_id, name, fid): name

Copilot · 2026-02-05T14:54:44Z

tests/test_parallel_delete.py

+import concurrent.futures
+import time
+from unittest.mock import MagicMock
+import pytest


Import of 'pytest' is not used.

Suggested change

import pytest

…itization, add dry-run plan details Incorporates the best changes from 36 Jules PRs, addressing review feedback: Bolt (Performance) - from PR #173: - Pre-compile PROFILE_ID_PATTERN and RULE_PATTERN at module level - Use compiled patterns in is_valid_profile_id_format, validate_profile_id, and is_valid_rule - Supersedes PRs: #140, #143, #152, #155, #158, #161, #167, #170, #173 Sentinel (Security) - from PR #172 with review feedback: - Enhance sanitize_for_log to redact Basic Auth credentials in URLs - Redact sensitive query parameters (token, key, secret, password, etc.) - Handle fragment separators (#) per Gemini Code Assist review - Use [^&#\s]* pattern per Copilot reviewer suggestion - Update docstring per reviewer suggestion - Supersedes PRs: #142, #145, #148, #151, #154, #157, #160, #169, #172 Palette (UX) - from PR #174 with lint fixes: - Add print_plan_details function for dry-run visibility - Fix duplicate render_progress_bar definition bug - Supersedes PRs: #139, #141, #144, #147, #150, #153, #156, #159, #162, #165, #168, #171, #174 Also: #146, #149, #164 (parallel folder deletion) and #166 (auto-fix .env perms) are independent features not consolidated here. Co-authored-by: abhimehro <84992105+abhimehro@users.noreply.github.com>

Copilot AI review requested due to automatic review settings February 5, 2026 14:49

github-actions bot added the python label Feb 5, 2026

github-advanced-security bot found potential problems Feb 5, 2026

View reviewed changes

Copilot started reviewing on behalf of abhimehro February 5, 2026 14:49 View session

github-advanced-security bot found potential problems Feb 5, 2026

View reviewed changes

Copilot AI reviewed Feb 5, 2026

View reviewed changes

Copilot AI mentioned this pull request Feb 8, 2026

Consolidate 36 Jules PRs: pre-compile regex, harden log sanitization, add dry-run plan details #175

Merged

This was referenced Feb 9, 2026

Consolidate parallel deletion, RTLO prevention, and .env auto-fix from PRs #146, #149, #163, #164, #166 #176

Merged

Combine API access check and folder listing into single request; close stale Jules PRs #178

Merged

abhimehro closed this Feb 9, 2026

abhimehro deleted the bolt-parallel-delete-folders-7500484463374288460 branch February 9, 2026 02:18

Conversation

abhimehro commented Feb 5, 2026

Uh oh!

google-labs-jules bot commented Feb 5, 2026

Uh oh!

trunk-io bot commented Feb 5, 2026

Uh oh!

Check notice

Check notice

Check notice

Check notice

Check notice

Check notice

Check notice

Check notice

Check notice

Check notice

gemini-code-assist bot commented Feb 5, 2026

Uh oh!

Check notice

Check notice

Check notice

Check notice

Check notice

Check notice

Check notice

Check notice

Check notice

Check notice

Check warning

Check notice

Check notice

Check notice

Check notice

Check notice

Check notice

Check notice

Check notice

Check notice

Check warning

Check warning

Check warning

Check warning

Check warning

Check notice

Check notice

Check notice

Check notice

Check notice

Check warning

Check warning

Check warning

Check warning

Check warning

Check warning

Check warning

Check warning

Check warning

Check warning

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Copilot AI Feb 5, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Feb 5, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Feb 5, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Feb 5, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Feb 5, 2026

Choose a reason for hiding this comment

Uh oh!

Reviewers