Skip to content

Add Skills-over-MCP SEP support to host#1

Draft
olaservo wants to merge 1 commit intomainfrom
experimental/skills-over-mcp
Draft

Add Skills-over-MCP SEP support to host#1
olaservo wants to merge 1 commit intomainfrom
experimental/skills-over-mcp

Conversation

@olaservo
Copy link
Copy Markdown
Owner

@olaservo olaservo commented Apr 21, 2026

Staging PR against the fork main for iteration and demo — not yet ready to forward upstream. Implements the host side of the Skills-over-MCP SEP (extension ID io.modelcontextprotocol/skills). Single squashed commit; this description is the implementation narrative.

What this does

Skills served by connected MCP servers are discovered at agent init via skill://index.json, surfaced into the model's context alongside filesystem-based skills, and loaded on demand through the existing read_skill tool (extended to accept resource URIs). One <available_skills> block in the system prompt unifies both sources — same XML shape, same loading tool, same relative-path resolution. The model never types a server name; the URI alone is enough for the host to route the read.

<available_skills>
  <skill>
    <name>pull-requests</name>
    <description>Workflows for creating, reviewing, and templating GitHub pull requests</description>
    <location>skill://pull-requests/SKILL.md</location>
    <directory>skill://pull-requests</directory>
  </skill>
</available_skills>

This is the lazy in-memory cell of the host-guide 2×2: fetch the index + each SKILL.md frontmatter at startup; defer body / supporting-file reads to the model.

How it's organized

Discovery lives in a new mcp/mcp_skills_loader.py. load_mcp_skill_manifests walks each connected server, reads skill://index.json (size-bounded, missing-index is silent), then fetches each skill-md entry's SKILL.md and parses frontmatter via the existing SkillRegistry.parse_manifest_text. merge_filesystem_and_mcp_manifests folds the discovered manifests into the filesystem set; filesystem wins on name collision.

Trust boundary. The reader (tools/skill_reader.py) admits a URI only if it descends from a discovered skill root, with raw and percent-encoded ../. rejected and query/fragment suffixes refused. file:// is rejected at both the loader and the reader as defense in depth — the SEP trust model treats the MCP server as the authority for content under its published roots, and admitting file:// collapses that to "whatever the server's process can read from disk." Each URI-backed manifest must name the server that published it (enforced as a SkillManifest invariant); without that, the aggregator would fall through every connected server until one answered, collapsing the per-server boundary.

Agent wiring. McpAgent.initialize() calls the loader after server connect and before _apply_instruction_templates() so the merged manifest set populates {{agentSkills}}. format_skills_for_prompt branches on uri vs path: URI-backed skills render <location>/<directory> as URIs and emit a one-paragraph preamble note; the <scripts>/<references>/<assets> subdirectory tags are suppressed (the host can't stat an MCP resource tree up front). skill_read_tool_name is forced to read_skill whenever any manifest is URI-backed — read_text_file would mangle a URI by treating it as a relative path under cwd. (Caught by the first live e2e run.)

Config. MCPServerSettings.mcp_skills: bool = True opts a server out of skill discovery while leaving its tools available. Independent of include_instructions.

Helpers. mcp/skill_uri.py holds two shared functions (strip_skill_md, skill_name_from_uri) used by the loader, registry, and reader. Both tolerate trailing slashes and reject degenerate shapes (e.g. skill://SKILL.md returns None) so a malformed entry can't seed skill:/ into the reader's allow-list.

Scheme-agnostic. Per the SEP, skill:// is SHOULD not MUST: the reader detects URIs by <scheme>:// shape and routes any URI through the aggregator if it descends from a discovered manifest root.

Out of scope

  • type: "mcp-resource-template" index entries + completion-API discovery UI
  • resources/subscribe for skill://index.json change notifications
  • Eager-to-filesystem materialization (other cell of the host-guide 2×2)

Verification

  • Skills unit suite: 95 passing (uv run pytest tests/unit/fast_agent/skills/)
  • E2e validation against olaservo/github-mcp-server @ add-agent-skills lives in a sibling harness repo (kept outside this fork to keep it pure reference-host material)
  • WG notes (fast-agent skills-over-mcp host - implementation recap.md) carry the activation-contamination audit and the conformance checklist against SEP host requirements

Related

@olaservo olaservo marked this pull request as draft April 21, 2026 12:41
olaservo added a commit that referenced this pull request Apr 22, 2026
The previous script was a canned-prompt activation heuristic (did the
model mention `submit_pending`?). Replace it with a real Scenario #1
runner: the agent reviews an actual PR on olaservo/code-review-subject
using the pull-requests skill's three-step pending-review workflow.

Pass criteria (all five must hold):
1. `read_skill` called with `skill://pull-requests/SKILL.md` before any
   mutating PR tool.
2. `pull_request_review_write` called with method=create and no event.
3. At least one `add_comment_to_pending_review` after step 2.
4. `pull_request_review_write` called with method=submit_pending and
   event in {APPROVE, REQUEST_CHANGES, COMMENT}.
5. No single-shot bypass (no create call that also sets event).

Auto-detects PR_NUMBER from the canonical head branch via `gh pr list`
so the script is one-step when run against a freshly-scaffolded PR.
olaservo added a commit that referenced this pull request Apr 22, 2026
The e2e harness now reads prompt template, subject repo, head branch,
skill URI, valid verdicts, and mutating-tool names from a scenario
YAML shipped by the WG repo (modelcontextprotocol/experimental-ext-skills)
at experiments/scenarios/<id>.yaml. SOM_SCENARIO env var overrides;
default resolves any sibling experimental-ext-skills* worktree via glob
so the runner works across branches without path edits.

Pass criteria and behavior are unchanged — this just stops hardcoding
Scenario #1's metadata in Python so the other client runners (Gemini
CLI, Goose, Codex) can consume the same file and substitute {pr_number}
/ {repo} into the prompt.
olaservo added a commit that referenced this pull request Apr 22, 2026
- Index through agent.pr_reviewer.message_history; AgentApp.__getattr__
  now routes plain attribute access to an agent-name lookup, so the old
  agent.message_history raised AttributeError before the banner printed.
- Strip <server>__ prefix from MCP tool names so they match the bare
  names in the scenario YAML's mutating_tools list. Without this, every
  pass/fail criterion beyond skill-read-before-write falsely failed even
  when the review posted cleanly.
- Docstring: cd into the script dir (fast-agent walks upward from CWD
  for fastagent.config.yaml); add Windows PYTHONIOENCODING/PYTHONUTF8
  note since Rich's block-drawing characters can't encode to cp1252.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
olaservo added a commit that referenced this pull request Apr 22, 2026
Two related contamination-control changes for the Scenario #1 harness:

- Remove the custom `instruction=` kwarg. Fast-agent now falls through
  to DEFAULT_AGENT_INSTRUCTION so the activation test sees only this
  CLI's native system prompt plus the scenario's user prompt. The old
  string named "GitHub pull request workflows", priming the model
  toward the skill's domain ahead of how Gemini CLI and Codex are
  configured (neither injects a custom system prompt).
- Read the model from the scenario's per-client `models:` map instead
  of letting fastagent.config.yaml's `default_model` drive the choice.
  Model selection belongs in the shared scenario so every client (and
  every run) is explicit and reproducible.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
olaservo added a commit that referenced this pull request Apr 22, 2026
…e2e

Force read_skill when any manifest is URI-backed so activation routes through the SEP rather than inlined server text. Consolidate SKILL.md URI helpers, make SkillReader scheme-agnostic, migrate logging to structured data= kwargs, fix a misleading MCP-vs-MCP collision warning, pin include_instructions:false in the activation test, add a smart-agent variant documenting the tool-redundancy finding, and rewrite the e2e harness to run Scenario #1 against a real PR. Squashes 1ffdef6, 0193369, e4955b4, 10d08e4, 8766815, 2ea41dd, cfaeeee, c404a3a, 48f951d, d8946ec.
olaservo added a commit that referenced this pull request Apr 22, 2026
Load scenario config from the shared experiments YAML, take the scenario path as a positional argument, read the model from the YAML, and drop the custom instruction. Also fixes the harness evaluator after fast-agent API drift. Squashes 888e6c3, c1d1df4, 1378687, d956280.
@olaservo olaservo force-pushed the experimental/skills-over-mcp branch from d956280 to 80a5363 Compare April 22, 2026 15:58
olaservo added a commit that referenced this pull request Apr 22, 2026
Force read_skill when any manifest is URI-backed so activation routes through the SEP rather than inlined server text. Consolidate SKILL.md URI helpers, make SkillReader scheme-agnostic, migrate logging to structured data= kwargs, and fix a misleading MCP-vs-MCP collision warning. The Scenario #1 e2e harness lives alongside the fork at clients/fast-agent-harness/.
@olaservo olaservo force-pushed the experimental/skills-over-mcp branch from 80a5363 to 293215f Compare April 22, 2026 16:44
Implements the io.modelcontextprotocol/skills extension: skills served
by connected MCP servers are discovered via the well-known
skill://index.json resource and surfaced through the existing skills
pipeline so they behave identically to filesystem skills.

- SkillManifest gains optional uri/server_name; path becomes optional.
  Construction-time invariants reject manifests with neither backing,
  and URI-backed manifests with no server_name (without it the
  aggregator falls through every connected server, collapsing the
  per-server trust boundary).
- format_skills_for_prompt renders resource URIs as <location>/
  <directory> for MCP-backed skills (any scheme -- skill:// is the SEP
  default, but github://, repo://, etc. are valid) and adds an
  MCP-specific preamble note when any URI-backed skill is present.
- New mcp_skills_loader module: fetches skill://index.json with a
  size bound, parses concrete skill-md entries, rejects file://
  entries (the MCP-server-is-authority trust model breaks down if
  local-disk paths enter the allow list), and rejects degenerate URIs
  with no skill-path segment.
- SkillReader accepts URIs of any scheme, dispatches via
  aggregator.get_resource, and enforces a URI-root trust boundary
  mirroring the filesystem allowed-dirs check (rejects raw and
  percent-encoded ../. traversal markers, query/fragment suffixes,
  and file:// URIs as defense in depth).
- McpAgent.initialize fetches MCP skills post-connect and merges with
  filesystem manifests; filesystem wins on name collision. Discovery
  failures are logged but never abort agent startup.
- MCPServerSettings gains mcp_skills bool for per-server opt-out
  (independent of include_instructions).

Includes unit tests for the loader, reader, URI helpers, prompt
formatting, and SkillManifest invariants.

SEP: https://github.com/modelcontextprotocol/experimental-ext-skills

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@olaservo olaservo force-pushed the experimental/skills-over-mcp branch from 293215f to a6dbdae Compare April 23, 2026 04:22
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant