Add MCP server support to live agent with Firecrawl web search#245
Add MCP server support to live agent with Firecrawl web search#245alexkroman wants to merge 2 commits into
Conversation
Thread Model Context Protocol servers into the deepagents brain behind `assembly live`, so a spoken conversation can reach real tools mid-turn — bringing it toward Gemini-Live / ChatGPT-voice parity. - `--mcp-config FILE` (repeatable) loads tools from standard `mcpServers` JSON, the same shape Claude Desktop / Claude Code use. - `--demo-tools` loads a curated, no-auth set (time, fetch, memory, filesystem, weather) for a reliable on-stage demo. Each server launches best-effort and independently, so one that won't start (missing npx/uvx, offline host) drops only its own tools and never sinks the session. The system prompt advertises MCP tools generically by name so the agent reaches for them without promising capabilities it lacks. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01S2tt5oaMXD2RCVoAqGbqyR
Drop the --demo-tools opt-in: `assembly live` now loads the curated MCP toolset (time, fetch, memory, filesystem, weather) by default, alongside its built-in URL fetch and the AssemblyAI docs MCP. --mcp-config still layers your own servers on top (and can override a default by name). Switch the default web search from Tavily to Firecrawl, reusing Firecrawl's official LangChain integration (langchain-firecrawl's FirecrawlSearch). It loads when FIRECRAWL_API_KEY is set; when it isn't, the session prints a one-line notice and runs without web search (every other default tool is no-auth). `assembly code` keeps its Tavily search unchanged. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01S2tt5oaMXD2RCVoAqGbqyR
![aikido-pr-checks[bot]](https://avatars.githubusercontent.com/in/898896?s=60&v=4){kind=small}
| answer. With no tools at all the model is told to answer from its own knowledge. | ||
| The guidance is tailored to the bound tools so the model is only told about | ||
| capabilities it actually has — advertising a missing tool (web search without a | ||
| ``TAVILY_API_KEY``) made the agent announce an action it then couldn't take, leaving |
There was a problem hiding this comment.
Doc text still says web search depends on TAVILY_API_KEY, but this module now uses Firecrawl. This contradicts actual runtime gating and misstates when search is available.
Details
✨ AI Reasoning
The surrounding implementation now uses Firecrawl for web search gating, yet the guidance text still says missing web search is tied to a different API key name. That mismatch makes the documented condition impossible to satisfy as written and can mislead future maintenance and debugging.
🔧 How do I fix it?
Trace execution paths carefully. Ensure precondition checks happen before using values, validate ranges before checking impossible conditions, and don't check for states that the code has already ruled out.
Reply @AikidoSec feedback: [FEEDBACK] to get better review comments in the future.
Reply @AikidoSec ignore: [REASON] to ignore this issue.
More info
| # Extra streaming-TTS query params (the --tts-config escape hatch). | ||
| tts_extra: Mapping[str, str] = field(default_factory=dict[str, str]) | ||
| # MCP servers (name -> launch spec) whose tools the deepagents brain can call. Empty | ||
| # by default; populated from --mcp-config files and/or the --demo-tools curated set. |
There was a problem hiding this comment.
Comment says mcp_servers is populated via --demo-tools, but that flag is not part of the current live command flow. The documented population path is no longer true.
| # by default; populated from --mcp-config files and/or the --demo-tools curated set. | |
| # by default; populated from --mcp-config files. |
Details
✨ AI Reasoning
The configuration field description claims population via a flag that is no longer part of the command flow. This creates an impossible source path per the current code and can cause incorrect assumptions about how mcp_servers gets populated.
Reply @AikidoSec feedback: [FEEDBACK] to get better review comments in the future.
Reply @AikidoSec ignore: [REASON] to ignore this issue.
More info
2 participants
Adds Model Context Protocol (MCP) server integration to the
assembly livevoice agent, enabling it to use external tools mid-conversation. This includes a curated default toolset and support for user-configured servers via--mcp-config.Summary
The live agent can now load tools from MCP servers, both a curated default set (time, fetch, memory, filesystem, weather) and user-provided servers from standard
mcpServersJSON config files. Web search is now powered by Firecrawl (whenFIRECRAWL_API_KEYis set) instead of Tavily, and MCP tools are advertised in the system prompt alongside built-in capabilities.Key Changes
New MCP tools module (
aai_cli/agent_cascade/mcp_tools.py):default_servers()provides five curated, no-auth MCP serversparse_mcp_config()reads standard Claude Desktop/CodemcpServersJSON filesload_mcp_tools()launches servers and collects their tools, with best-effort per-server error handling (one broken server doesn't sink the session)Firecrawl web search (
aai_cli/code_agent/firecrawl_search.py):FIRECRAWL_API_KEYenvironment variableNonewhen key is absent (graceful degradation)Brain system prompt updates (
aai_cli/agent_cascade/brain.py):_extra_capability()generates generic tool-listing guidance for MCP toolsbuild_system_prompt()now acceptsextra_toolsparameter for MCP toolsbuild_graph()acceptsmcp_toolsparameter and binds them alongside built-in toolsCommand integration (
aai_cli/commands/agent_cascade/):--mcp-configflag (repeatable) for user-provided server configs_resolve_mcp_servers()merges defaults with config files (later files win)_warn_without_web_search()emits a notice whenFIRECRAWL_API_KEYis absentCascadeConfig.mcp_serversConfig updates (
aai_cli/agent_cascade/config.py):CascadeConfig.mcp_serversfield holds the resolved server specsComprehensive test coverage (
tests/test_agent_cascade_mcp.py):Documentation (REFERENCE.md):
--mcp-configusageImplementation Details
_safe_load(), which catches exceptions and returns an empty tool list on failure. This ensures a missingnpx/uvxor offline host doesn't abort the session.langchain-mcp-adaptersMultiServerMCPClientto fetch tools from each server, driven byasyncio.run().FIRECRAWL_API_KEY); other tools loadhttps://claude.ai/code/session_01S2tt5oaMXD2RCVoAqGbqyR