Skip to content
Open
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
72 changes: 72 additions & 0 deletions docs/experimental-findings.md
Original file line number Diff line number Diff line change
Expand Up @@ -90,3 +90,75 @@ Multiple community members have independently reported that models do not reliab
> "I've seen lazy load skills with various degrees of success, actually looks like it might be model specific… [best pattern is] putting them in with a subagent that similarly named or mentions the topic in their description." — Kryspin (qcompute), via Discord

**See also:** [#37](https://github.com/modelcontextprotocol/experimental-ext-skills/issues/37) — Compare skill delivery mechanisms: file-based vs MCP-based

## PHP MCP SDK + Symfony AI Mate: Skills as `skill://` resources

**Server / SDK:** [modelcontextprotocol/php-sdk#372](https://github.com/modelcontextprotocol/php-sdk/pull/372) — adds `io.modelcontextprotocol/skills` support to the official PHP MCP SDK
**Consumer:** [symfony/ai#2132](https://github.com/symfony/ai/pull/2132) — ships Agent Skills in the Symfony AI "Mate" MCP server
**Contributor:** Johannes Wachter ([@wachterjohannes](https://github.com/wachterjohannes))

First PHP-ecosystem implementation of SEP-2640 (prior documented implementations are
Python/TS). The SDK PR adds a one-line server affordance — `addSkillsFromDirectory()` —
that walks a directory, registers each `SKILL.md` and its supporting files as `skill://`
resources, derives `name`/`description` from YAML frontmatter, enforces the spec's
final-path-segment ↔ frontmatter-`name` rule, guards against path traversal, and serves
a `skill://index.json` discovery index. The Mate PR ships two real skills colocated with
the tools they orchestrate, including a multi-file skill with a `references/` subdirectory.

**Tested (works):**

- Serving is covered by MCP Inspector **stdio snapshot tests**: `resources/list`,
`resources/read` of a `SKILL.md`, of a supporting file, and of `skill://index.json`,
plus `resource_templates/list`. Unit tests cover frontmatter parsing (BOM/CRLF,
non-mapping rejection), the name↔segment rule, and resource-name sanitization. PHPStan
level 6 and the full suite (792 tests) green.
- The **directory model + relative supporting-file URIs** resolve correctly in a
non-Python implementation — e.g. `skill://code-review/references/SECURITY.md` is a
sibling resource of `skill://code-review/SKILL.md`. Positive evidence the directory
model travels across ecosystems.

**`_meta` prefix — independent convergence (not a gap).** Our SDK independently chose
`io.modelcontextprotocol.skills/` to namespace extra frontmatter fields on the resource
descriptor — which matches the prefix SEP-2640 recommends ("When `_meta` keys are used for
skill resources, implementations SHOULD use the `io.modelcontextprotocol.skills/`
reverse-domain prefix"). Useful corroboration of the recommended prefix. Note: the
working-group repo-local draft ([`docs/sep-draft-skills-extension.md`](sep-draft-skills-extension.md))
does not yet include that sentence — its `_meta` paragraph ends at "…via the resource's
`_meta` object." — so the SEP PR and this repo's copy have drifted and could be synced.
(Discussed on [SEP-2640](https://github.com/modelcontextprotocol/modelcontextprotocol/pull/2640#issuecomment-4622668503).)

**SDK implementation notes (not spec gaps):**

- **Resource-name uniqueness, not charset.** The skill `name` charset (`[a-z0-9-]`, ≤64, no
leading/trailing hyphen) is a strict subset of the MCP resource-name charset, so the
SKILL.md resource `name` can carry the frontmatter `name` directly — "resource `name`
SHOULD equal frontmatter `name`" is satisfiable. The wrinkle is uniqueness: our SDK
registers every resource under a unique name key, including a skill's **supporting files**
(`references/SECURITY.md`) and skills that **share a frontmatter `name` under different
prefixes** (`acme/billing/refunds` vs `acme/support/refunds`). So we derive a unique name
from the URI path and keep the frontmatter `name` in `title`. Identity is the URI
regardless — an SDK registration detail, not a SEP issue.
- **Empty-payload capability serialization trap.** An extension advertising an empty `{}`
payload (as Skills does) serialized to `[]` rather than `{}` and had to be coerced. A
likely footgun for any SDK implementing an empty-payload extension.
- **`symfony/yaml` required** for frontmatter parsing — the feature is non-functional
without a YAML parser; frontmatter handling is a real dependency, not free.

**Client consumption (observed from docs, not yet eval'd):**

- Per current **Claude Code** documentation (June 2026), Claude Code loads skills from the
filesystem and plugins only; it does not discover or load MCP-served `skill://` resources
as skills, and its MCP resource support is **user-`@`-mention attachments, not
model-driven `resources/read`**. So end-to-end, model-driven consumption of MCP-served
skills is not exercisable in Claude Code today — a data point for
[#38](https://github.com/modelcontextprotocol/experimental-ext-skills/issues/38).
- **FastMCP 3.0** (per this repo's existing findings) is the consumer best positioned to
validate the serving half against; not yet done.

**Remaining / untested:**

- No model-adherence eval yet comparing filesystem vs. `skill://` delivery.
- `mcp-resource-template` skill type (parameterized namespaces) is deferred in the SDK PR —
the `SkillType` enum carries the value for forward-compat, but only `skill-md` entries are
emitted; the template path is unimplemented and untested.
- Not yet tested against any client that implements model-driven `skill://` loading.