feat(mcp): add optics mcp server exposing keywords over MCP#308
Open
chinmayajha wants to merge 5 commits into
Open
feat(mcp): add optics mcp server exposing keywords over MCP#308chinmayajha wants to merge 5 commits into
optics mcp server exposing keywords over MCP#308chinmayajha wants to merge 5 commits into
Conversation
Add a Model Context Protocol server (fastmcp) and an `optics mcp` command so MCP clients (Claude, Cursor) can drive a live device/browser through optics: start a session, run keywords as tools, and read device state via resources. Built as a thin wrapper over the existing keyword execution path.
Add docs/usage/mcp_usage.md (install, usage, the user journey, and troubleshooting), wire it into the docs nav, and record the MCP server in CLAUDE.md.
Tests for tool/resource registration, the string-typed parameter schemas, keyword dispatch and stringification, error translation, and the optional-dependency guard.
fastmcp needs starlette>=1.0, which FastAPI supports from 0.119, so the pin moves from <0.119 to <0.138; fastmcp is pinned >=3.4.2 (earlier 3.4.x omit the starlette pin). Lock regenerated.
Collaborator
Author
Follow-up issuesThe two follow-ups from the description, tracked separately:
|
malto101
requested changes
Jun 17, 2026
… agreement The recent action_keyword.py refactor and MCP-server commits shifted many file:line anchors in CLAUDE.md. Re-verify every cited anchor against source and correct the drifted ones (cli, action_keyword, strategies, live, nl_agent, session_manager, optics_builder, base_factory, config_handler, error, factories, and minor nudges elsewhere). Also fix a factual error: LLM backends do not set a NAME attribute (GeminiLLM has none) - engines are resolved by module filename matching the config key, with NAME only used for driver<->element-source matching via REQUIRED_DRIVER_TYPE. Reword the driver and LLM 'where to put things' bullets accordingly. Add a 'Working agreement' section: no AI co-author trailers, assume Codex reviews every PR, split large pushes into reviewable commits, and run a post-PR gap analysis that turns approved findings into tracked GitHub issues referenced from the PR.
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What this adds
A new
optics mcpcommand that runs a Model Context Protocol server. It lets an MCP client like Claude or Cursor drive a live device or browser through optics — start a session, run keywords, and look at the screen.Usage
The server exposes:
start_session,terminate_session,screenshot, and every optics keyword (press_element,enter_text,swipe,assert_presence, …). Each keyword tool takes asession_id.optics://keywords, and live device state per session (screenshot, page source, interactive elements).Full guide:
docs/usage/mcp_usage.md.Dependency change
fastmcpneedsstarlette >= 1.0, which FastAPI only supports from0.119. So the FastAPI pin moves from<0.119to<0.138. This affects every install, not just themcpextra — the lock now resolves FastAPI0.137/ starlette1.3. The existing test suite passes on the new versions.Testing
Follow-ups
A couple of things left for later, tracked as separate issues (linked in a comment):
application/octet-streamrather thanimage/png— a fastmcp quirk with templated resources. Thescreenshottool returns a proper image in the meantime.optics mcpandoptics serveeach keep their own sessions and can't share them.