Design: @gemstack/mcp — standalone MCP server framework (agent-agnostic peer)

[suleimansh]

Design: @gemstack/mcp — graduate the standalone MCP server framework

Tracking: epic #1 (the last unchecked family box). This is the server-authoring axis of the MCP taxonomy, the peer of @gemstack/ai-mcp (the agent bridge). It is agent-agnostic, NOT ai-* — nothing here depends on @gemstack/ai-sdk, and ai-sdk must never depend on it. See Architecture.md "MCP taxonomy (two axes, do not conflate)".

This is a design proposal, no code. Gated on family alignment before anything lands.

What graduates

@rudderjs/mcp (mature, v6.2.1) is a full framework for authoring MCP servers: McpServer / McpTool / McpResource / McpPrompt / McpResponse / Mcp, the metadata + MCP-spec-annotation decorators, OAuth2 middleware, a streamable-HTTP transport, an observer registry, and a McpTestClient. The framework itself is already mostly agent-agnostic and standalone-testable. The work is to lift the framework-agnostic core out from under the Rudder framework wiring.

The carve-out differs from the ai-mcp one in one important way

ai-mcp was a hard-move-no-shim because it was days old with zero installed base. @rudderjs/mcp is the opposite: published, mature, real consumers. Architecture.md reserved exactly this case:

Keep the deprecated-shim-for-one-minor play in reserve for the first carve-out after 1.0, where a real installed base exists.

So the shape is the @rudderjs/ai -> @gemstack/ai-sdk shim pattern, not the hard-move:

• @gemstack/mcp = the standalone, framework-agnostic core, fresh 0.1.0.
• @rudderjs/mcp stays a (now-thin) package: it re-exports the core from @gemstack/mcp and keeps only the Rudder binding (the McpProvider, the @Handle -> Rudder-container wiring, the CLI commands, the doctor check). Existing @rudderjs/mcp importers keep working unchanged.

Difference from the @rudderjs/ai shim: that one was a pure re-export (zero Rudder-specific surface left). @rudderjs/mcp legitimately owns Rudder-specific surface (provider, DI decorator semantics, CLI), so it remains a real thin binding package on top of @gemstack/mcp, not an empty re-export.

Coupling map (from a full source audit of packages/mcp)

@rudderjs/* dep	Kind	Where	Decouple difficulty
@rudderjs/json-schema	hard dep	zod-to-json-schema.ts (one convertSchema() call)	Easy. Replace with Zod 4 native z.toJSONSchema() (exactly what ai-sdk Phase 2 did), keep the date-format + open-object fallbacks. Zero behavior change.
@rudderjs/router	optional peer	provider.ts, runtime/http-transport.ts	Already done. Consumed only through a duck-typed { all(path, handler, mw?) } shape via resolveOptionalPeer, with a graceful fallback. No Rudder-specific API. Keep the pattern; also expose a framework-neutral handler (see decisions).
@rudderjs/console	optional peer	commands/make-*.ts (type-only MakeSpec), doctor.ts (registerDoctorCheck)	Moderate-easy, but it stays in the binding. Scaffolders + doctor are CLI wiring, not framework. They live in @rudderjs/mcp, not in the agnostic core.
@rudderjs/core	hard dep	provider.ts (ServiceProvider, CLI rudder.command()), runtime/handle-deps.ts (reads __rudderjs_app__/__rudderjs_instance__ off globalThis), auth/oauth2.ts (MiddlewareHandler type, lazy passport)	The real work. Split into two: (a) ServiceProvider/CLI = framework wiring -> stays in the binding; (b) the DI that @Handle() performs = needs a neutral seam (below).

Tests import no undeclared @rudderjs/* packages (unlike ai-sdk, whose suite pulled cache/storage/queue) — packages/mcp already runs its full suite on zod + Node built-ins alone. The boost/ dir is Claude-skills docs, ships as-is.

The one genuine design problem: the @Handle() DI seam

Everything else is mechanical. The crux is @Handle(Logger, Cache, ...) on a tool/resource/prompt method: runtime/handle-deps.ts resolves those tokens by reading Rudder's container off globalThis. A framework-agnostic core cannot assume Rudder's container exists.

Mirror the ai-sdk Phase-2 resolution (neutral optional adapter seam; core deps collapse to zod only):

• Core defines a minimal resolver contract, e.g. McpResolver = { resolve(token): unknown }, and a setMcpResolver(resolver) (or a per-Mcp/per-server option). With no resolver set, @Handle either no-ops or throws a clear "no resolver registered" error — the core never reaches for globalThis.__rudderjs_*.
• The Rudder binding registers a resolver that delegates to Rudder's container, so @Handle(Logger) keeps working byte-for-byte in a Rudder app. Non-Rudder users either skip @Handle (plain closures/constructor injection) or register their own resolver (one adapter function over Awilix/tsyringe/InversifyJS/etc).

This keeps the decorator surface identical for existing users while making the core container-agnostic.

Decisions to confirm (A/B, like #7/#8/#9)

1. Shim shape. Confirm @rudderjs/mcp stays a thin binding package (re-export core + Rudder provider/DI/CLI/doctor) rather than a pure re-export. (Recommended: yes — it owns real Rudder surface.)
2. DI seam. Neutral setMcpResolver + binding-registers-Rudder-container (recommended), vs. drop DI from core entirely and re-add @Handle only in the binding. Trade-off: the former keeps @Handle usable by non-Rudder users with their own container; the latter keeps the core smaller but makes @Handle a Rudder-only feature.
3. HTTP transport surface. Keep the duck-typed router peer and add a framework-neutral entry (a plain (req, res) Node handler / createMcpHttpHandler()) so non-Rudder users can mount the server on Express/Hono/Vike/raw http without a Rudder router. (Recommended: add the neutral handler; it is already almost that internally.)
4. OAuth2 typing. Re-type oauth2McpMiddleware against a generic Connect-style (req, res, next) signature instead of importing core's MiddlewareHandler; keep passport as an optional peer of the binding, not the core. (Recommended.)
5. json-schema removal. Inline Zod 4 native conversion in the core, drop the @rudderjs/json-schema dep (same move as ai-sdk). (Recommended.)
6. Relationship to ai-mcp. Keep them independent: ai-mcp's mcpServerFromAgent continues to use the raw MCP SDK, not @gemstack/mcp, so neither package depends on the other. The tiny "both can produce a server" overlap is from different inputs and is expected, not duplication. (Recommended: no dependency edge between them.)
7. Versioning. @gemstack/mcp starts fresh at 0.1.0 (reset, like ai-sdk), not continuing 6.x. @rudderjs/mcp gets a minor/patch when it is repointed onto the core.

Proposed phasing

• Phase 1 — scaffold + agnostic core. Copy packages/mcp/src into gemstack/packages/mcp, rename @rudderjs/mcp -> @gemstack/mcp, strip the framework wiring (provider.ts, commands/*, doctor.ts) out of the core, drop @rudderjs/json-schema (Zod 4 native), introduce the McpResolver seam, re-type OAuth2. Core deps target: @modelcontextprotocol/sdk + zod + reflect-metadata. Publish 0.1.0.
• Phase 2 — Rudder binding. Repoint @rudderjs/mcp onto @gemstack/mcp: re-export the core, keep McpProvider, register the Rudder-container resolver, keep the CLI commands + doctor + scaffolders. Verify auto-discovery, @Handle DI, and the inspector still work in playground. Changeset for @rudderjs/mcp.
• Phase 3 (optional, later). A framework-neutral quickstart / docs showing @gemstack/mcp on raw http + Hono, proving the agnostic claim outside Rudder.

Non-goals

• No merge with @gemstack/ai-mcp — opposite taxonomy axis.
• No new transports beyond what @rudderjs/mcp ships (streamable HTTP + stdio via the SDK).
• No durable/clustered server runtime — out of scope.
• The make-* scaffolders + Rudder doctor stay Rudder-specific (binding only), not in the agnostic core.

Acceptance (for the eventual build, not this issue)

• @gemstack/mcp builds + typechecks + runs its full test suite standalone with deps = @modelcontextprotocol/sdk, zod, reflect-metadata (no @rudderjs/*).
• A server authored with only @gemstack/mcp (tools/resources/prompts/OAuth) serves over a framework-neutral HTTP handler with no Rudder present.
• @rudderjs/mcp, repointed onto the core, keeps McpProvider auto-discovery, @Handle DI, the CLI commands, and the doctor check working unchanged in playground.
• One-directional dep check: @gemstack/mcp imports nothing from @gemstack/ai-*; ai-* imports nothing from mcp.

[suleimansh]

Decisions locked (walked 2026-06-26)

Decision 1 (shim shape) -> Thin binding. @rudderjs/mcp re-exports the agnostic core from @gemstack/mcp AND keeps its genuine Rudder surface (McpProvider auto-discovery, the @Handle->Rudder-container wiring, the mcp:* CLI, the doctor check). It stays a small real package; existing imports are unchanged; the agnostic core stays free of Rudder concepts. Rejected the pure-re-export (it would drag Rudder concepts back into the core behind optional peers, which is the coupling the graduation removes).

Decision 2 (DI seam) -> Instance-scoped resolver, no global setter. The crux. Replaces today's globalThis.__rudderjs_* container read.

• Core contract: McpResolver = { resolve(token): unknown } (matches Awilix .resolve / tsyringe .resolve / Inversify .get, so any container is a one-line fit).
• The resolver is a property of the server instance, passed at construction (new McpServer({ resolver })), NOT a global setMcpResolver(). A global resolver singleton is the same anti-pattern as the globalThis coupling being removed (breaks multi-server-per-process, test isolation, serverless/SSR concurrency).
• @Handle is unchanged in surface; it only records tokens (reflect-metadata). Resolution happens at invocation time off the live instance's resolver.
• Zero-config tiers: no @Handle => needs nothing; @Handle without a container => core ships a tiny built-in createResolver() with .register(token, instance); @Handle + a real container => pass it as resolver.
• Loud failure: missing resolver / unknown token throws at call time, naming the server, the tool/resource/prompt, and the token. Never injects undefined.
• The @rudderjs/mcp binding registers Rudder's container as the resolver inside McpProvider, so @Handle(Logger) stays byte-for-byte identical in a Rudder app.

This mirrors the @gemstack/ai-sdk Phase-2 neutral-optional-adapter seam (core deps collapse to @modelcontextprotocol/sdk + zod + reflect-metadata, no @rudderjs/*).

Decisions 3-7 still open.

[suleimansh]

Decisions 3-7 locked (walked 2026-06-26)

Decision 3 (HTTP transport surface) -> Neutral handler + router peer. The agnostic core exposes createMcpHttpHandler(server) => (req, res), a plain Node (IncomingMessage, ServerResponse) handler that mounts on raw http/Express/Connect/Fastify and adapts to Hono/Vike. The transport already produces this internally; we expose it as a first-class entry. The duck-typed { all(path, handler, mw?) } router peer stays for the @rudderjs/mcp binding. Makes the agent-agnostic claim real (serve with no Rudder present).

Decision 4 (OAuth2) -> Generic in core, passport in binding. Re-type oauth2McpMiddleware against a standard Connect (req, res, next) signature (plain IncomingMessage/ServerResponse), dropping the @rudderjs/core MiddlewareHandler type. It takes a user-supplied verifyToken; @rudderjs/passport stays an optional peer of the @rudderjs/mcp binding (wired as verifyToken). OAuth 2.0 protected-resource metadata is MCP-spec, so it stays a core capability but Rudder-free, and composes with the Decision-3 handler.

Decision 5 (JSON Schema) -> Inline Zod 4 native, drop @rudderjs/json-schema. Replace the convertSchema() call with an in-core function: z.toJSONSchema() directly, strip $schema, keep the date-format mapping + open-object fallback. Zero behavior change (@rudderjs/json-schema already delegates to Zod 4 natively); zod is already a core dep. Same move as ai-sdk Phase 2. Removes a @rudderjs/* coupling for a ~30-line helper.

Decision 6 (relationship to ai-mcp) -> Independent, no dependency edge. @gemstack/ai-mcp's mcpServerFromAgent keeps using the raw @modelcontextprotocol/sdk; neither package depends on the other. Preserves the two separate taxonomy axes (ai-mcp depends on ai-sdk; mcp is agent-agnostic and depends on neither). The 'both produce a server' overlap is from different inputs (auto-wrapped agent vs hand-authored server) = expected, not duplication. A dependency edge would couple the two axes and drag the full server framework into every single-agent MCP exposure.

Decision 7 (versioning) -> Fresh 0.1.0. Reset like the rest of the family (ai-sdk/ai-mcp/ai-skills/ai-autopilot all on 0.x). New name, new agnostic surface (resolver seam, neutral HTTP handler, generic OAuth), new audience; 0.x gives room to settle the new API and keeps the family on one maturity track. @rudderjs/mcp gets a minor bump (6.2.x -> 6.3.0) when repointed onto the core. README/npm description state it is the graduation of the mature @rudderjs/mcp so maturity stays legible.

---

All 7 decisions resolved. The design is ready to build behind the 3-phase plan (scaffold agnostic core -> Rudder binding -> neutral quickstart), still gated on family alignment with the Vike team before code lands.

[suleimansh]

Build tracked as child issues (gated on family alignment)

The design is complete; the work is broken into the 3-phase plan:

• Build: @gemstack/mcp Phase 1 — scaffold the agnostic core (0.1.0) #19 — Phase 1: scaffold the agnostic core, apply Decisions 2/3/4/5, publish 0.1.0
• Build: @gemstack/mcp Phase 2 — repoint @rudderjs/mcp onto the core (thin binding) #20 — Phase 2: repoint @rudderjs/mcp onto the core as the thin Rudder binding (Decisions 1/7)
• Build: @gemstack/mcp Phase 3 — framework-neutral quickstart/docs (optional) #21 — Phase 3 (optional): framework-neutral quickstart/docs proving the agnostic claim

Phase 1 -> Phase 2 ordered (Phase 2 needs the published 0.1.0); Phase 3 is on-demand. All still gated on the Vike-team family-alignment call before code lands.

[suleimansh]

Realized: Phase 1 shipped in #22 (agnostic core, instance-scoped resolver DI, neutral node:http handler, generic verifyToken OAuth, 100 tests), Phase 2 binding landed (#20 / rudder#1448) and is smoke-verified live. Design complete — closing. Phase 3 docs remain optional in #21; public-API cleanup in #24.