fix(mcp): add use_isolated_event_loop to McpToolset for Vertex AI Agent Engine compatibility

[vipin-v-nair]

Problem

McpToolset with StreamableHTTPConnectionParams fails on Vertex AI Agent
Engine with:

  Attempted to exit cancel scope in a different task than it was entered in                                                                                                     

Root cause: anyio's CancelScope binds to the asyncio.Task that enters
it. Agent Engine's scheduler context-switches tasks between entering and
exiting the scope inside streamablehttp_client's anyio.create_task_group().

Fix

Add use_isolated_event_loop=True to McpToolset (and the underlying
McpTool). When set, each MCP operation runs via asyncio.to_thread() in a
dedicated thread with asyncio.new_event_loop(). The cancel scope is created
and destroyed entirely within that isolated loop.

Changes

• mcp_thread_utils.py (new): list_tools_in_thread and call_tool_in_thread
  helpers that open a fresh connection per call inside an isolated loop.
• mcp_tool.py: use_isolated_event_loop param; branches to thread path in
  _run_async_impl.
• mcp_toolset.py: use_isolated_event_loop param; branches to thread path
  in get_tools; passes the flag through to each McpTool.

Usage

McpToolset(                                               
    connection_params=StreamableHTTPConnectionParams(
        url="https://my-mcp-server.run.app/mcp"      
    ),                                         
    use_isolated_event_loop=True,  # required on Vertex AI Agent Engine                                                                                                           
)
                                                                                                                                                                                  
Notes                                                     
                                                                                                                                                                                  
- Opt-in, defaults to False — no behaviour change for existing users.                                                                                                             
- Only supported with StreamableHTTPConnectionParams; raises ValueError otherwise.
- auth_scheme, auth_credential, and header_provider are fully supported.                                                                                                          
- progress_callback and MCP sampling are not invoked in this mode (documented).                                                                                                   
- Verified against Vertex AI Agent Engine with three Cloud Run MCP servers.                                                                                                       

---                                                                                                                                                                                                                    
Unit Tests:
                                                                                                                                                                                                                       
- I have added or updated unit tests for my change.
- All unit tests pass locally.                                                                                                                                                                                         
 
Unit tests for mcp_thread_utils.py (list_tools_in_thread, call_tool_in_thread) were not added in this PR — the helpers wrap streamablehttp_client which requires a live MCP server. Integration coverage is provided by the E2E test below.
                                                                                                                                                                                                                       
---             
Manual End-to-End (E2E) Tests:
                                                                                                                                                                                                                       
Setup:
                                                                                                                                                                                                                       
1. Deploy a Streamable HTTP MCP server (any Cloud Run MCP endpoint works).                                                                                                                                             
2. Deploy an ADK agent to Vertex AI Agent Engine (Reasoning Engine) that connects to that MCP server using McpToolset with use_isolated_event_loop=True:
                                                                                                                                                                                                                       
McpToolset(                                                                                                                                                                                                            
    connection_params=StreamableHTTPConnectionParams(url="https://<mcp-server-url>/mcp"),                                                                                                                              
    use_isolated_event_loop=True,                                                                                                                                                                                      
)               
                                                                                                                                                                                                                       
Test 1 — Tools are discovered correctly:                                                                                                                                                                               
- Invoke get_tools() on the deployed agent engine.
- Verify the tool list from the MCP server is returned without error.                                                                                                                                                  
                
Test 2 — Tool call succeeds end-to-end:                                                                                                                                                                                
- Send a prompt to the deployed agent that triggers an MCP tool call.
- Verify the agent receives the tool result and returns a coherent response.                                                                                                                                           
                                                                            
Test 3 — Regression: default path is unchanged:                                                                                                                                                                        
- Deploy the same agent with use_isolated_event_loop=False (default).                                                                                                                                                  
- Confirm existing behavior is unaffected.                                                                                                                                                                             
                                                                                                                                                                                                                       
Observed failure without the fix (baseline):                                                                                                                                                                           
anyio._backends._asyncio.ExceptionGroup: ...                                                                                                                                                                           
  RuntimeError: Attempted to exit cancel scope in a different task than it was entered in
                                                                                                                                                                                                                       
Observed result with the fix:
- Agent Engine successfully calls MCP tools across multiple turns with no cancel scope errors.                                                                                                                         
- Tested on Vertex AI Agent Engine (us-central1) with three Cloud Run MCP servers.                                                                                  
                                                                                                                                     
       

[rohityan]

Hi @vipin-v-nair , Thank you for your contribution! We appreciate you taking the time to submit this pull request. Please fix the failing mypy-diff and precommit tests before we can proceed with a review.

[vipin-v-nair]

Hope this addresses the failures. Please let me know if i need to do anything else