Skip to content

desktop tools silently return empty results on Win11 26200 — in-process UIA calls on threads without proper COM init; errors swallowed as empty vec #3

Description

Summary

On Windows 11 (build 26200), the automata-agent MCP server's desktop tools silently return empty results — desktop list_windows[], element_tree / find_elements[] — while vision/screenshot tools work fine. No error is surfaced. The bundled standalone helpers (list-windows.exe, element-tree.exe) work perfectly on the same host, same session.

Environment

  • Windows 11 Pro, build 10.0.26200
  • automata-agent.exe dated 2026-04-05 (15,014,400 bytes), run via --stdio as a Claude Code MCP server
  • Same user session (session 1), no elevation mismatch

Empirical evidence chain

  1. desktop list_windows via MCP → [] (no error). element_tree by hwnd[]. Title lookup → proper error (No window found with title=...), so the agent is alive and responding.
  2. UIA from .NET in the same session (PowerShell, System.Windows.Automation) → enumerates 6 root children correctly, including the target window. The host UIA stack is healthy.
  3. The bundled list-windows.exe helper run standalone → works, lists all windows.
  4. A process watcher confirmed the agent spawns no helper processes during desktop calls — the enumeration runs in-process.
  5. A fresh agent instance driven directly over JSON-RPC stdio (initialize → tools/call desktop list_windows) → also [] (agent log: automata_client: ← list_windows, result []). Not state corruption — systemic.
  6. --self-test only validates ui-workflow.exe and ui-sight-windows.exe; the in-process UIA path is never covered.

Root-cause analysis (from the open automata-windows source)

The agent binary is closed-source, but the automata-windows crate shows the pattern:

  • desktop.rs:73Desktop::application_windows() uses UIAutomation::new_direct(), which skips COM initialization and assumes the calling thread already initialized COM correctly.
  • lib.rs:187-193init_com() calls CoInitializeEx(None, COINIT_MULTITHREADED) but discards the result (let _ =), and only affects the thread it happens to run on.
  • desktop.rs:446-470 (tooltip_windows) — UIA failures are swallowed into empty results: let Ok(auto) = UIAutomation::new_direct() else { return vec![]; } plus find_all(...).unwrap_or_default(). If the agent wraps application_windows() errors the same way, a COM/threading failure surfaces as a clean [] — exactly what we observe.
  • desktop.rs:290-293 — other code paths already anticipate RPC_E_CHANGED_MODE (0x80010106), i.e. threads in the process can disagree about their COM apartment.

If the MCP server dispatches tool calls on async runtime worker threads (tokio), those threads may never have run init_com() — or may have been initialized STA by clipboard/dialog code. Either way, in-process UIA calls then violate Microsoft's documented UIA threading contract, whose stated failure modes include empty/unresponsive results:

This also explains why the standalone helpers work: their main() initializes COM on the same thread that makes the UIA calls.

Suggested fix

  1. Route all in-process UIA client calls through a dedicated, window-less MTA worker thread (initialize COM once on that thread; marshal requests to it), per the MS threading guidance.
  2. Alternatively, call CoInitializeEx + check RPC_E_CHANGED_MODE at every UIA entry point (the pattern resolve_lnk() already uses at desktop.rs:206-209) before UIAutomation::new_direct().
  3. Propagate errors instead of returning vec![] — a silent empty list is indistinguishable from "no windows", which made this very hard to diagnose. Even a log line at warn level would have saved hours.
  4. Consider extending --self-test to cover the in-process UIA path (list_windows returning ≥1 window on a desktop session would have caught this).

Happy to provide more diagnostics from this host if useful.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions