diff --git a/.gitignore b/.gitignore index 3cf0bc4..cdf8daf 100644 --- a/.gitignore +++ b/.gitignore @@ -8,4 +8,5 @@ __pycache__/ *.pb.go *_pb2.py .superpowers/ +docs/superpowers/ sdk/rust/target/ diff --git a/LICENSE b/LICENSE new file mode 100644 index 0000000..935d8e3 --- /dev/null +++ b/LICENSE @@ -0,0 +1,21 @@ +MIT License + +Copyright (c) 2026 Max Silverblatt + +Permission is hereby granted, free of charge, to any person obtaining a copy +of this software and associated documentation files (the "Software"), to deal +in the Software without restriction, including without limitation the rights +to use, copy, modify, merge, publish, distribute, sublicense, and/or sell +copies of the Software, and to permit persons to whom the Software is +furnished to do so, subject to the following conditions: + +The above copyright notice and this permission notice shall be included in all +copies or substantial portions of the Software. + +THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE +AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER +LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, +OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE +SOFTWARE. diff --git a/README.md b/README.md index 2d53df2..0d0f1ec 100644 --- a/README.md +++ b/README.md @@ -21,6 +21,8 @@ protomcp is a language-agnostic MCP runtime. Write your server logic in Python, pmcp uses the [official MCP Go SDK](https://github.com/modelcontextprotocol/go-sdk) for full spec compliance. Your code registers tools, resources, and prompts through a simple protobuf protocol over a unix socket. pmcp handles everything else — protocol negotiation, transport, pagination, session management, and hot reload. +The unix socket + protobuf layer adds ~0.5ms of overhead per tool call. + ## Quick Start ### Install @@ -29,6 +31,22 @@ pmcp uses the [official MCP Go SDK](https://github.com/modelcontextprotocol/go-s brew install msilverblatt/tap/protomcp ``` +### Install SDK + +```sh +# Python +pip install protomcp + +# TypeScript +npm install protomcp + +# Go +go get github.com/msilverblatt/protomcp/sdk/go/protomcp + +# Rust +# Add to Cargo.toml: protomcp = "0.1" +``` + ### Python ```python @@ -346,7 +364,7 @@ See the [full documentation](https://msilverblatt.github.io/protomcp/) for detai protomcp is not a replacement for the official MCP SDKs — it's built on top of the [official Go SDK](https://github.com/modelcontextprotocol/go-sdk). Use protomcp when: -- **You want one server in multiple languages** — write tools in Python, prompts in TypeScript, resources in Go, all served by a single MCP server +- **You want the same API across languages** — switch between Python, TypeScript, Go, and Rust with identical concepts and patterns - **You want zero-config hot reload** — save a file, everything reloads instantly - **You don't want to learn MCP internals** — no JSON-RPC, no transport wiring, no session management - **You want a single binary** — `pmcp` is a single Go binary, no runtime dependencies for the server itself diff --git a/docs/superpowers/plans/2026-03-12-protomcp-v1.1.md b/docs/superpowers/plans/2026-03-12-protomcp-v1.1.md deleted file mode 100644 index a5d1c0a..0000000 --- a/docs/superpowers/plans/2026-03-12-protomcp-v1.1.md +++ /dev/null @@ -1,2751 +0,0 @@ -# protomcp v1.1 Implementation Plan - -> **For agentic workers:** REQUIRED: Use superpowers:subagent-driven-development (if subagents available) or superpowers:executing-plans to implement this plan. Steps use checkbox (`- [ ]`) syntax for tracking. - -**Goal:** Expand protomcp with Go and Rust SDKs, custom and auth middleware, build-time validation, CI/publishing pipelines, and comprehensive documentation. - -**Architecture:** Builds on v1.0's Go binary + protobuf-over-unix-socket architecture. Adds two new SDK implementations (Go, Rust) following the same module structure as Python/TypeScript SDKs. Extends the protobuf protocol with four new middleware messages (field numbers 24-27). Adds a `validate` subcommand and `--auth` flag to the CLI. CI/CD via GitHub Actions. - -**Tech Stack:** Go 1.25+, Rust (stable, prost, tokio), Protocol Buffers 3, GitHub Actions, GoReleaser - -**Spec:** `docs/superpowers/specs/2026-03-12-protomcp-v1.1-design.md` - ---- - -## File Structure - -``` -# New files -sdk/go/ -├── go.mod -├── protomcp/ -│ ├── tool.go # Tool() registration with functional options -│ ├── tool_test.go -│ ├── result.go # ToolResult struct -│ ├── result_test.go -│ ├── context.go # ToolContext (progress, cancellation) -│ ├── context_test.go -│ ├── manager.go # ToolManager for dynamic enable/disable -│ ├── manager_test.go -│ ├── log.go # Server logging (8 levels) -│ ├── log_test.go -│ ├── transport.go # Unix socket + envelope framing -│ ├── transport_test.go -│ └── runner.go # Main loop: connect, handshake, dispatch -│ └── runner_test.go - -sdk/rust/ -├── Cargo.toml -├── build.rs # prost-build for protobuf generation -├── src/ -│ ├── lib.rs # Public API exports -│ ├── tool.rs # Builder pattern registration -│ ├── result.rs # ToolResult struct -│ ├── context.rs # ToolContext (progress, cancellation) -│ ├── manager.rs # ToolManager for dynamic enable/disable -│ ├── log.rs # Server logging -│ ├── transport.rs # Unix socket + envelope framing -│ └── runner.rs # Main async loop -├── tests/ -│ ├── tool_test.rs -│ ├── result_test.rs -│ ├── transport_test.rs -│ └── integration_test.rs - -internal/middleware/ -├── auth.go # NEW: built-in auth middleware -├── auth_test.go # NEW -├── custom.go # NEW: user-registered middleware dispatch -├── custom_test.go # NEW - -internal/validate/ -├── validate.go # NEW: tool definition validation -├── validate_test.go # NEW - -.github/workflows/ -├── ci.yml # NEW: CI pipeline -├── release.yml # NEW: release/publishing pipeline - -examples/go/ -├── basic.go -├── real_world.go -├── full_showcase.go - -examples/rust/ -├── basic/Cargo.toml + src/main.rs -├── real_world/Cargo.toml + src/main.rs -├── full_showcase/Cargo.toml + src/main.rs - -docs/src/content/docs/guides/ -├── writing-tools-go.mdx # NEW -├── writing-tools-rust.mdx # NEW -├── middleware.mdx # NEW -├── auth.mdx # NEW -├── writing-a-language-library.mdx # NEW - -# Modified files -proto/protomcp.proto # Add middleware messages (fields 24-27) -gen/proto/protomcp/protomcp.pb.go # Regenerated -sdk/python/gen/protomcp_pb2.py # Regenerated -sdk/typescript/gen/ # Regenerated -cmd/protomcp/main.go # Add validate command, --auth flag -internal/config/config.go # Add Auth, Strict, Format fields; validate command; fix Rust runtime -internal/config/config_test.go # Tests for new fields -internal/process/manager.go # Extended handshake for middleware registration -internal/process/manager_test.go # Tests for extended handshake -internal/middleware/chain.go # Priority-based ordering with custom middleware -sdk/python/src/protomcp/runner.py # Middleware registration support -sdk/python/src/protomcp/__init__.py # Export middleware -sdk/typescript/src/runner.ts # Middleware registration support -sdk/typescript/src/index.ts # Export middleware -README.md # Badges, Go/Rust examples, features -docs/src/content/docs/index.mdx # Add Go/Rust cards -docs/src/content/docs/getting-started/quick-start.mdx # Go/Rust snippets -docs/src/content/docs/getting-started/installation.mdx # Go/Rust install -docs/src/content/docs/reference/cli.mdx # validate, --auth -``` - ---- - -## Chunk 1: Protobuf + Config + Rust Runtime Fix - -### Task 1: Add middleware messages to protobuf - -**Files:** -- Modify: `proto/protomcp.proto` - -- [ ] **Step 1: Write the new message types and envelope fields** - -Add after the `TaskCancelRequest` message (line 170) in `proto/protomcp.proto`: - -```protobuf -// --- Middleware --- - -message RegisterMiddlewareRequest { - string name = 1; - int32 priority = 2; // lower runs first -} - -message RegisterMiddlewareResponse { - bool success = 1; -} - -message MiddlewareInterceptRequest { - string middleware_name = 1; - string phase = 2; // "before" or "after" - string tool_name = 3; - string arguments_json = 4; - string result_json = 5; // empty for "before" phase - bool is_error = 6; // only set in "after" phase -} - -message MiddlewareInterceptResponse { - string arguments_json = 1; // potentially modified (before phase) - string result_json = 2; // potentially modified (after phase) - bool reject = 3; // if true, abort the call - string reject_reason = 4; -} -``` - -Add these fields to the `Envelope.oneof msg` block (after `task_cancel = 23`): - -```protobuf - RegisterMiddlewareRequest register_middleware = 24; - RegisterMiddlewareResponse register_middleware_response = 25; - MiddlewareInterceptRequest middleware_intercept = 26; - MiddlewareInterceptResponse middleware_intercept_response = 27; -``` - -- [ ] **Step 2: Regenerate all protobuf code** - -Run: `make proto` -Expected: `gen/proto/protomcp/protomcp.pb.go`, `sdk/python/gen/protomcp_pb2.py`, and `sdk/typescript/gen/` are all updated with the new middleware message types. The Makefile handles Go, Python, and TypeScript generation in one command. - -- [ ] **Step 6: Verify all existing tests still pass** - -Run: `go test ./... && cd sdk/python && python -m pytest && cd ../typescript && npx vitest run` -Expected: All tests pass (no regressions from protobuf changes). - -- [ ] **Step 7: Commit** - -```bash -git add proto/protomcp.proto gen/ sdk/python/gen/ sdk/typescript/gen/ -git commit -m "proto: add middleware messages (fields 24-27)" -``` - ---- - -### Task 2: Fix Rust runtime detection bug + add validate command to config - -**Files:** -- Modify: `internal/config/config.go` -- Modify: `internal/config/config_test.go` - -- [ ] **Step 1: Write failing test for Rust runtime fix** - -Add to `config_test.go`: - -```go -func TestRuntimeCommandRust(t *testing.T) { - cmd, args := config.RuntimeCommand("tools.rs") - if cmd != "cargo" { - t.Errorf("cmd = %q, want %q", cmd, "cargo") - } - // cargo run does NOT take a file argument; it runs the package - expected := []string{"run", "--manifest-path", "Cargo.toml"} - if !reflect.DeepEqual(args, expected) { - t.Errorf("args = %v, want %v", args, expected) - } -} -``` - -- [ ] **Step 2: Run test to verify it fails** - -Run: `go test ./internal/config/ -run TestRuntimeCommandRust -v` -Expected: FAIL — currently returns `["run", "tools.rs"]`. - -- [ ] **Step 3: Fix RuntimeCommand for .rs files** - -In `config.go`, change the `.rs` case in `RuntimeCommand`: - -```go - case ".rs": - dir := filepath.Dir(file) - manifestPath := filepath.Join(dir, "Cargo.toml") - return "cargo", []string{"run", "--manifest-path", manifestPath} -``` - -- [ ] **Step 4: Run test to verify it passes** - -Run: `go test ./internal/config/ -run TestRuntimeCommandRust -v` -Expected: PASS. - -- [ ] **Step 5: Update existing Rust test case in TestRuntimeCommand table** - -Change the `server.rs` entry to expect the new args format: - -```go - {"server.rs", "cargo", []string{"run", "--manifest-path", "Cargo.toml"}}, -``` - -- [ ] **Step 6: Run all config tests** - -Run: `go test ./internal/config/ -v` -Expected: All pass. - -- [ ] **Step 7: Write failing test for validate command parsing** - -Add to `config_test.go`: - -```go -func TestParseValidateCommand(t *testing.T) { - cfg, err := config.Parse([]string{"validate", "tools.py"}) - if err != nil { - t.Fatalf("Parse failed: %v", err) - } - if cfg.Command != "validate" { - t.Errorf("Command = %q, want %q", cfg.Command, "validate") - } - if cfg.File != "tools.py" { - t.Errorf("File = %q, want %q", cfg.File, "tools.py") - } -} - -func TestParseValidateWithStrict(t *testing.T) { - cfg, err := config.Parse([]string{"validate", "tools.py", "--strict"}) - if err != nil { - t.Fatalf("Parse failed: %v", err) - } - if !cfg.Strict { - t.Error("Strict should be true") - } -} - -func TestParseValidateWithFormatJSON(t *testing.T) { - cfg, err := config.Parse([]string{"validate", "tools.py", "--format", "json"}) - if err != nil { - t.Fatalf("Parse failed: %v", err) - } - if cfg.Format != "json" { - t.Errorf("Format = %q, want %q", cfg.Format, "json") - } -} - -func TestParseAuthFlag(t *testing.T) { - cfg, err := config.Parse([]string{"dev", "tools.py", "--auth", "token:MY_TOKEN"}) - if err != nil { - t.Fatalf("Parse failed: %v", err) - } - if len(cfg.Auth) != 1 || cfg.Auth[0] != "token:MY_TOKEN" { - t.Errorf("Auth = %v, want [\"token:MY_TOKEN\"]", cfg.Auth) - } -} - -func TestParseMultipleAuthFlags(t *testing.T) { - cfg, err := config.Parse([]string{"dev", "tools.py", "--auth", "token:T1", "--auth", "apikey:K1"}) - if err != nil { - t.Fatalf("Parse failed: %v", err) - } - if len(cfg.Auth) != 2 { - t.Errorf("Auth length = %d, want 2", len(cfg.Auth)) - } -} - -func TestParseAuthMalformed(t *testing.T) { - _, err := config.Parse([]string{"dev", "tools.py", "--auth", "invalid"}) - if err == nil { - t.Error("expected error for malformed --auth value") - } -} - -func TestParseAuthUnknownScheme(t *testing.T) { - _, err := config.Parse([]string{"dev", "tools.py", "--auth", "oauth:FOO"}) - if err == nil { - t.Error("expected error for unknown auth scheme") - } -} -``` - -- [ ] **Step 8: Run tests to verify they fail** - -Run: `go test ./internal/config/ -run TestParseValidate -v` -Expected: FAIL — "validate" not recognized as valid command. - -- [ ] **Step 9: Update Config struct and Parse function** - -In `config.go`, add fields to `Config`: - -```go -type Config struct { - Command string - File string - Transport string - HotReloadImmediate bool - CallTimeout time.Duration - LogLevel string - SocketPath string - Runtime string - Host string - Port int - Auth []string - Strict bool - Format string -} -``` - -Update `Parse`: -- Change command validation from `cmd != "dev" && cmd != "run"` to `cmd != "dev" && cmd != "run" && cmd != "validate"` -- Add `--auth` flag handling (repeatable, validates format `scheme:value` where scheme is `token` or `apikey`): - -```go - case "--auth": - i++ - if i >= len(args) { - return nil, fmt.Errorf("--auth requires a value") - } - authVal := args[i] - parts := strings.SplitN(authVal, ":", 2) - if len(parts) != 2 || parts[1] == "" { - return nil, fmt.Errorf("--auth value must be scheme:ENV_VAR (got %q)", authVal) - } - scheme := parts[0] - if scheme != "token" && scheme != "apikey" { - return nil, fmt.Errorf("unknown auth scheme %q: must be 'token' or 'apikey'", scheme) - } - cfg.Auth = append(cfg.Auth, authVal) - case "--strict": - cfg.Strict = true - case "--format": - i++ - if i >= len(args) { - return nil, fmt.Errorf("--format requires a value") - } - cfg.Format = args[i] -``` - -Add `"strings"` to imports. - -- [ ] **Step 10: Run all config tests** - -Run: `go test ./internal/config/ -v` -Expected: All pass. - -- [ ] **Step 11: Verify full build** - -Run: `go build ./...` -Expected: Clean build. - -- [ ] **Step 12: Commit** - -```bash -git add internal/config/config.go internal/config/config_test.go -git commit -m "config: add validate command, --auth flag, fix Rust runtime detection" -``` - ---- - -### Task 3: Build-time validation - -**Files:** -- Create: `internal/validate/validate.go` -- Create: `internal/validate/validate_test.go` - -- [ ] **Step 1: Write failing tests for validation logic** - -Create `internal/validate/validate_test.go`: - -```go -package validate_test - -import ( - "testing" - - pb "github.com/msilverblatt/protomcp/gen/proto/protomcp" - "github.com/msilverblatt/protomcp/internal/validate" -) - -func TestValidateValidTools(t *testing.T) { - tools := []*pb.ToolDefinition{ - {Name: "add", Description: "Add two numbers", InputSchemaJson: `{"type":"object","properties":{"a":{"type":"integer"}}}`}, - {Name: "multiply", Description: "Multiply two numbers", InputSchemaJson: `{"type":"object","properties":{"b":{"type":"integer"}}}`}, - } - result := validate.Tools(tools, false) - if !result.Pass { - t.Errorf("expected pass, got errors: %v", result.Errors) - } - if len(result.Tools) != 2 { - t.Errorf("expected 2 tools, got %d", len(result.Tools)) - } -} - -func TestValidateEmptyName(t *testing.T) { - tools := []*pb.ToolDefinition{ - {Name: "", Description: "Something", InputSchemaJson: `{"type":"object"}`}, - } - result := validate.Tools(tools, false) - if result.Pass { - t.Error("expected fail for empty name") - } -} - -func TestValidateInvalidNameChars(t *testing.T) { - tools := []*pb.ToolDefinition{ - {Name: "my-tool", Description: "Has hyphens", InputSchemaJson: `{"type":"object"}`}, - } - result := validate.Tools(tools, false) - if result.Pass { - t.Error("expected fail for invalid name characters") - } -} - -func TestValidateDuplicateNames(t *testing.T) { - tools := []*pb.ToolDefinition{ - {Name: "add", Description: "First", InputSchemaJson: `{"type":"object"}`}, - {Name: "add", Description: "Second", InputSchemaJson: `{"type":"object"}`}, - } - result := validate.Tools(tools, false) - if result.Pass { - t.Error("expected fail for duplicate names") - } -} - -func TestValidateEmptyDescription(t *testing.T) { - tools := []*pb.ToolDefinition{ - {Name: "add", Description: "", InputSchemaJson: `{"type":"object"}`}, - } - result := validate.Tools(tools, false) - if result.Pass { - t.Error("expected fail for empty description") - } -} - -func TestValidateInvalidSchema(t *testing.T) { - tools := []*pb.ToolDefinition{ - {Name: "add", Description: "Add", InputSchemaJson: `not json`}, - } - result := validate.Tools(tools, false) - if result.Pass { - t.Error("expected fail for invalid JSON schema") - } -} - -func TestValidateStrictShortDescription(t *testing.T) { - tools := []*pb.ToolDefinition{ - {Name: "add", Description: "Add", InputSchemaJson: `{"type":"object","properties":{"a":{"type":"integer"}}}`}, - } - result := validate.Tools(tools, true) - if result.Pass { - t.Error("expected fail in strict mode for short description") - } -} - -func TestValidateStrictGenericName(t *testing.T) { - tools := []*pb.ToolDefinition{ - {Name: "test", Description: "A test tool for testing", InputSchemaJson: `{"type":"object","properties":{"a":{"type":"integer"}}}`}, - } - result := validate.Tools(tools, true) - if result.Pass { - t.Error("expected fail in strict mode for generic name") - } -} - -func TestResultFormatText(t *testing.T) { - tools := []*pb.ToolDefinition{ - {Name: "add", Description: "Add two numbers", InputSchemaJson: `{"type":"object"}`}, - } - result := validate.Tools(tools, false) - output := result.FormatText() - if output == "" { - t.Error("expected non-empty text output") - } -} - -func TestResultFormatJSON(t *testing.T) { - tools := []*pb.ToolDefinition{ - {Name: "add", Description: "Add two numbers", InputSchemaJson: `{"type":"object"}`}, - } - result := validate.Tools(tools, false) - output := result.FormatJSON() - if output == "" { - t.Error("expected non-empty JSON output") - } -} -``` - -- [ ] **Step 2: Run tests to verify they fail** - -Run: `go test ./internal/validate/ -v` -Expected: FAIL — package doesn't exist yet. - -- [ ] **Step 3: Implement validation logic** - -Create `internal/validate/validate.go`: - -```go -package validate - -import ( - "encoding/json" - "fmt" - "regexp" - "strings" - - pb "github.com/msilverblatt/protomcp/gen/proto/protomcp" -) - -var validName = regexp.MustCompile(`^[a-zA-Z_][a-zA-Z0-9_]*$`) - -var genericNames = map[string]bool{ - "test": true, "tool1": true, "foo": true, "bar": true, "baz": true, - "temp": true, "tmp": true, "example": true, -} - -type ToolStatus struct { - Name string `json:"name"` - Status string `json:"status"` -} - -type ValidationError struct { - Tool string `json:"tool"` - Issue string `json:"issue"` -} - -type Result struct { - Tools []ToolStatus `json:"tools"` - Errors []ValidationError `json:"errors"` - Pass bool `json:"pass"` -} - -func Tools(tools []*pb.ToolDefinition, strict bool) Result { - result := Result{Pass: true} - seen := make(map[string]bool) - - for _, t := range tools { - status := ToolStatus{Name: t.Name, Status: "ok"} - - if t.Name == "" { - result.Errors = append(result.Errors, ValidationError{Tool: "", Issue: "empty tool name"}) - result.Pass = false - status.Status = "error" - } else if !validName.MatchString(t.Name) { - result.Errors = append(result.Errors, ValidationError{Tool: t.Name, Issue: fmt.Sprintf("invalid name %q: must match [a-zA-Z_][a-zA-Z0-9_]*", t.Name)}) - result.Pass = false - status.Status = "error" - } else if seen[t.Name] { - result.Errors = append(result.Errors, ValidationError{Tool: t.Name, Issue: "duplicate tool name"}) - result.Pass = false - status.Status = "error" - } - seen[t.Name] = true - - if t.Description == "" { - result.Errors = append(result.Errors, ValidationError{Tool: t.Name, Issue: "no description"}) - result.Pass = false - status.Status = "error" - } - - if t.InputSchemaJson != "" { - var schema map[string]interface{} - if err := json.Unmarshal([]byte(t.InputSchemaJson), &schema); err != nil { - result.Errors = append(result.Errors, ValidationError{Tool: t.Name, Issue: fmt.Sprintf("invalid input schema JSON: %v", err)}) - result.Pass = false - status.Status = "error" - } - } - - if strict { - if len(t.Description) < 10 { - result.Errors = append(result.Errors, ValidationError{Tool: t.Name, Issue: fmt.Sprintf("description too short (%d chars, minimum 10)", len(t.Description))}) - result.Pass = false - status.Status = "error" - } - if genericNames[strings.ToLower(t.Name)] { - result.Errors = append(result.Errors, ValidationError{Tool: t.Name, Issue: fmt.Sprintf("generic name %q", t.Name)}) - result.Pass = false - status.Status = "error" - } - } - - result.Tools = append(result.Tools, status) - } - - return result -} - -func (r Result) FormatText() string { - var sb strings.Builder - for _, t := range r.Tools { - if t.Status == "ok" { - sb.WriteString(fmt.Sprintf("✓ %s — OK\n", t.Name)) - } - } - if len(r.Errors) > 0 { - sb.WriteString(fmt.Sprintf("✗ — %d error(s):\n", len(r.Errors))) - for _, e := range r.Errors { - if e.Tool != "" { - sb.WriteString(fmt.Sprintf(" · %q: %s\n", e.Tool, e.Issue)) - } else { - sb.WriteString(fmt.Sprintf(" · %s\n", e.Issue)) - } - } - } - return sb.String() -} - -func (r Result) FormatJSON() string { - b, _ := json.Marshal(r) - return string(b) -} -``` - -- [ ] **Step 4: Run tests to verify they pass** - -Run: `go test ./internal/validate/ -v` -Expected: All pass. - -- [ ] **Step 5: Commit** - -```bash -git add internal/validate/ -git commit -m "feat: add build-time tool validation" -``` - ---- - -### Task 4: Wire validate command and --auth into main.go - -**Files:** -- Modify: `cmd/protomcp/main.go` - -- [ ] **Step 1: Add validate subcommand handling** - -Add an import for the validate package and handle the validate command before the existing dev/run flow. In `main()`, after parsing config, add: - -```go - if cfg.Command == "validate" { - runValidate(ctx, cfg) - return - } -``` - -Add the `runValidate` function: - -```go -func runValidate(ctx context.Context, cfg *config.Config) { - var runtimeCmd string - var runtimeArgs []string - if cfg.Runtime != "" { - runtimeCmd = cfg.Runtime - runtimeArgs = []string{cfg.File} - } else { - runtimeCmd, runtimeArgs = config.RuntimeCommand(cfg.File) - } - - pm := process.NewManager(process.ManagerConfig{ - File: cfg.File, - RuntimeCmd: runtimeCmd, - RuntimeArgs: runtimeArgs, - SocketPath: cfg.SocketPath, - MaxRetries: 1, - CallTimeout: 30 * time.Second, - }) - - tools, err := pm.Start(ctx) - if err != nil { - fmt.Fprintf(os.Stderr, "error: failed to start tool process: %v\n", err) - os.Exit(1) - } - defer pm.Stop() - - result := validate.Tools(tools, cfg.Strict) - - if cfg.Format == "json" { - fmt.Println(result.FormatJSON()) - } else { - fmt.Print(result.FormatText()) - } - - if !result.Pass { - os.Exit(1) - } -} -``` - -Add imports: `"github.com/msilverblatt/protomcp/internal/validate"`, `"time"`. - -- [ ] **Step 2: Verify build** (auth wiring is deferred to Task 5 to avoid compilation errors) - -Run: `go build ./cmd/protomcp/` -Expected: Build succeeds. - -- [ ] **Step 3: Commit** - -```bash -git add cmd/protomcp/main.go -git commit -m "feat: wire validate command into CLI" -``` - ---- - -### Task 5: Auth middleware - -**Files:** -- Create: `internal/middleware/auth.go` -- Create: `internal/middleware/auth_test.go` - -- [ ] **Step 1: Write failing tests** - -Create `internal/middleware/auth_test.go`: - -```go -package middleware_test - -import ( - "context" - "encoding/json" - "os" - "testing" - - "github.com/msilverblatt/protomcp/internal/mcp" - "github.com/msilverblatt/protomcp/internal/middleware" -) - -func TestAuthTokenValid(t *testing.T) { - os.Setenv("TEST_TOKEN", "secret123") - defer os.Unsetenv("TEST_TOKEN") - - mw, err := middleware.NewAuth([]string{"token:TEST_TOKEN"}) - if err != nil { - t.Fatal(err) - } - - next := func(ctx context.Context, req mcp.JSONRPCRequest) (*mcp.JSONRPCResponse, error) { - return &mcp.JSONRPCResponse{ID: req.ID, Result: []byte(`"ok"`)}, nil - } - - handler := mw(next) - ctx := middleware.WithAuthHeader(context.Background(), "Bearer secret123") - resp, err := handler(ctx, mcp.JSONRPCRequest{ID: json.RawMessage(`1`), Method: "tools/call"}) - if err != nil { - t.Fatal(err) - } - if resp == nil { - t.Fatal("expected response") - } -} - -func TestAuthTokenInvalid(t *testing.T) { - os.Setenv("TEST_TOKEN", "secret123") - defer os.Unsetenv("TEST_TOKEN") - - mw, err := middleware.NewAuth([]string{"token:TEST_TOKEN"}) - if err != nil { - t.Fatal(err) - } - - next := func(ctx context.Context, req mcp.JSONRPCRequest) (*mcp.JSONRPCResponse, error) { - return &mcp.JSONRPCResponse{ID: req.ID, Result: []byte(`"ok"`)}, nil - } - - handler := mw(next) - ctx := middleware.WithAuthHeader(context.Background(), "Bearer wrong") - _, err = handler(ctx, mcp.JSONRPCRequest{ID: json.RawMessage(`1`), Method: "tools/call"}) - if err == nil { - t.Fatal("expected auth error") - } -} - -func TestAuthTokenMissing(t *testing.T) { - os.Setenv("TEST_TOKEN", "secret123") - defer os.Unsetenv("TEST_TOKEN") - - mw, err := middleware.NewAuth([]string{"token:TEST_TOKEN"}) - if err != nil { - t.Fatal(err) - } - - next := func(ctx context.Context, req mcp.JSONRPCRequest) (*mcp.JSONRPCResponse, error) { - return &mcp.JSONRPCResponse{}, nil - } - - handler := mw(next) - _, err = handler(context.Background(), mcp.JSONRPCRequest{ID: json.RawMessage(`1`), Method: "tools/call"}) - if err == nil { - t.Fatal("expected auth error for missing header") - } -} - -func TestAuthApikeyValid(t *testing.T) { - os.Setenv("TEST_KEY", "mykey") - defer os.Unsetenv("TEST_KEY") - - mw, err := middleware.NewAuth([]string{"apikey:TEST_KEY"}) - if err != nil { - t.Fatal(err) - } - - next := func(ctx context.Context, req mcp.JSONRPCRequest) (*mcp.JSONRPCResponse, error) { - return &mcp.JSONRPCResponse{ID: req.ID, Result: []byte(`"ok"`)}, nil - } - - handler := mw(next) - ctx := middleware.WithAPIKeyHeader(context.Background(), "mykey") - resp, err := handler(ctx, mcp.JSONRPCRequest{ID: json.RawMessage(`1`), Method: "tools/call"}) - if err != nil { - t.Fatal(err) - } - if resp == nil { - t.Fatal("expected response") - } -} - -func TestNewAuthEnvNotSet(t *testing.T) { - os.Unsetenv("NONEXISTENT_VAR") - _, err := middleware.NewAuth([]string{"token:NONEXISTENT_VAR"}) - if err == nil { - t.Error("expected error for unset env var") - } -} - -func TestNewAuthMultiple(t *testing.T) { - os.Setenv("TEST_TOKEN2", "tok") - os.Setenv("TEST_KEY2", "key") - defer os.Unsetenv("TEST_TOKEN2") - defer os.Unsetenv("TEST_KEY2") - - mw, err := middleware.NewAuth([]string{"token:TEST_TOKEN2", "apikey:TEST_KEY2"}) - if err != nil { - t.Fatal(err) - } - - next := func(ctx context.Context, req mcp.JSONRPCRequest) (*mcp.JSONRPCResponse, error) { - return &mcp.JSONRPCResponse{ID: req.ID, Result: []byte(`"ok"`)}, nil - } - - handler := mw(next) - ctx := middleware.WithAuthHeader(context.Background(), "Bearer tok") - ctx = middleware.WithAPIKeyHeader(ctx, "key") - resp, err := handler(ctx, mcp.JSONRPCRequest{ID: json.RawMessage(`1`), Method: "tools/call"}) - if err != nil { - t.Fatal(err) - } - if resp == nil { - t.Fatal("expected response") - } -} -``` - -- [ ] **Step 2: Run tests to verify they fail** - -Run: `go test ./internal/middleware/ -v` -Expected: FAIL — `NewAuth`, `WithAuthHeader`, `WithAPIKeyHeader` don't exist. - -- [ ] **Step 3: Implement auth middleware** - -Create `internal/middleware/auth.go`: - -```go -package middleware - -import ( - "context" - "fmt" - "os" - "strings" -) - -type authContextKey string - -const ( - authHeaderKey authContextKey = "auth-header" - apiKeyHeaderKey authContextKey = "apikey-header" -) - -// WithAuthHeader adds an Authorization header value to the context. -func WithAuthHeader(ctx context.Context, value string) context.Context { - return context.WithValue(ctx, authHeaderKey, value) -} - -// WithAPIKeyHeader adds an X-API-Key header value to the context. -func WithAPIKeyHeader(ctx context.Context, value string) context.Context { - return context.WithValue(ctx, apiKeyHeaderKey, value) -} - -// GetAuthHeader retrieves the Authorization header from context. -func GetAuthHeader(ctx context.Context) string { - v, _ := ctx.Value(authHeaderKey).(string) - return v -} - -// GetAPIKeyHeader retrieves the X-API-Key header from context. -func GetAPIKeyHeader(ctx context.Context) string { - v, _ := ctx.Value(apiKeyHeaderKey).(string) - return v -} - -type authChecker struct { - scheme string - value string -} - -// NewAuth creates an auth middleware from --auth flag values. -// Each value must be "token:ENV_VAR" or "apikey:ENV_VAR". -// Returns error if env var is not set. -func NewAuth(authSpecs []string) (Middleware, error) { - var checkers []authChecker - - for _, spec := range authSpecs { - parts := strings.SplitN(spec, ":", 2) - if len(parts) != 2 { - return nil, fmt.Errorf("invalid auth spec %q", spec) - } - scheme, envVar := parts[0], parts[1] - value := os.Getenv(envVar) - if value == "" { - return nil, fmt.Errorf("environment variable %q is not set (required by --auth %s)", envVar, spec) - } - checkers = append(checkers, authChecker{scheme: scheme, value: value}) - } - - return func(next Handler) Handler { - return func(ctx context.Context, req JSONRPCRequest) (*JSONRPCResponse, error) { - for _, c := range checkers { - switch c.scheme { - case "token": - header := GetAuthHeader(ctx) - expected := "Bearer " + c.value - if header != expected { - return nil, fmt.Errorf("unauthorized: invalid or missing Bearer token") - } - case "apikey": - header := GetAPIKeyHeader(ctx) - if header != c.value { - return nil, fmt.Errorf("unauthorized: invalid or missing API key") - } - } - } - return next(ctx, req) - } - }, nil -} -``` - -Note: Import the `mcp` package types via the existing middleware package imports. The `Handler` and `Middleware` types are already defined in `chain.go`. Add the `JSONRPCRequest` and `JSONRPCResponse` type aliases or import from `mcp` package as needed — check what `chain.go` already imports. - -- [ ] **Step 4: Run tests to verify they pass** - -Run: `go test ./internal/middleware/ -v` -Expected: All pass. - -- [ ] **Step 5: Wire auth middleware into main.go** - -In `cmd/protomcp/main.go`, replace the existing middleware chain setup with: - -```go - // 6. Apply middleware - middlewares := []middleware.Middleware{ - middleware.Logging(logger), - middleware.ErrorFormatting(), - } - - if len(cfg.Auth) > 0 { - if cfg.Transport == "stdio" { - slog.Warn("--auth ignored for stdio transport") - } else { - authMw, err := middleware.NewAuth(cfg.Auth) - if err != nil { - slog.Error("invalid --auth configuration", "error", err) - os.Exit(1) - } - middlewares = append([]middleware.Middleware{authMw}, middlewares...) - } - } - - chain := middleware.Chain( - func(ctx context.Context, req mcp.JSONRPCRequest) (*mcp.JSONRPCResponse, error) { - return handler.Handle(ctx, req) - }, - middlewares..., - ) -``` - -- [ ] **Step 6: Verify full build** - -Run: `go build ./cmd/protomcp/` -Expected: Clean build. - -- [ ] **Step 7: Commit** - -```bash -git add internal/middleware/auth.go internal/middleware/auth_test.go cmd/protomcp/main.go -git commit -m "feat: add auth middleware (token + apikey) and wire into CLI" -``` - ---- - -### Task 6: Extended handshake for middleware registration - -**Files:** -- Modify: `internal/process/manager.go` -- Modify: `internal/process/manager_test.go` -- Create: `internal/middleware/custom.go` -- Create: `internal/middleware/custom_test.go` - -- [ ] **Step 1: Add middleware tracking to Manager** - -Add to `Manager` struct: - -```go - middlewares []RegisteredMiddleware -``` - -Add type: - -```go -type RegisteredMiddleware struct { - Name string - Priority int32 -} -``` - -- [ ] **Step 2: Extend handshake in Start()** - -After `listTools` returns, add a loop that waits for optional `RegisterMiddlewareRequest` messages or a `ReloadResponse` (handshake-complete signal): - -```go - // After listTools, wait for optional middleware registrations + handshake-complete signal. - m.mu.Lock() - m.tools = tools - m.mu.Unlock() - - middlewares, err := m.awaitHandshakeComplete(ctx) - if err != nil { - m.cleanup() - return nil, fmt.Errorf("handshake middleware: %w", err) - } - m.mu.Lock() - m.middlewares = middlewares - m.mu.Unlock() -``` - -Implement `awaitHandshakeComplete`: - -```go -func (m *Manager) awaitHandshakeComplete(ctx context.Context) ([]RegisteredMiddleware, error) { - var middlewares []RegisteredMiddleware - // Short timeout for handshake-complete signal. If v1.0 SDKs don't send it, - // we treat "no message within 500ms" as handshake-complete (backward compat). - timer := time.NewTimer(500 * time.Millisecond) - defer timer.Stop() - - for { - select { - case <-ctx.Done(): - return nil, ctx.Err() - case <-timer.C: - // Backward compat: v1.0 SDKs don't send handshake-complete signal. - // Treat timeout as "no middleware registered, handshake done." - return middlewares, nil - case env := <-m.handshakeCh: - if rr := env.GetReloadResponse(); rr != nil { - // Handshake complete signal - return middlewares, nil - } - if rm := env.GetRegisterMiddleware(); rm != nil { - middlewares = append(middlewares, RegisteredMiddleware{ - Name: rm.Name, - Priority: rm.Priority, - }) - // Send acknowledgment - resp := &pb.Envelope{ - Msg: &pb.Envelope_RegisterMiddlewareResponse{ - RegisterMiddlewareResponse: &pb.RegisterMiddlewareResponse{Success: true}, - }, - } - if err := envelope.Write(m.conn, resp); err != nil { - return nil, fmt.Errorf("write RegisterMiddlewareResponse: %w", err) - } - continue - } - // Unexpected message type — ignore - } - } -} -``` - -- [ ] **Step 3: Add Middlewares() accessor** - -```go -func (m *Manager) Middlewares() []RegisteredMiddleware { - m.mu.Lock() - defer m.mu.Unlock() - return m.middlewares -} -``` - -- [ ] **Step 4: Update readLoop to route RegisterMiddleware and MiddlewareIntercept messages** - -In `readLoop()`, add handling for `MiddlewareInterceptResponse` — route it via `pending` map using `request_id` (same as `CallToolResponse`). For `RegisterMiddlewareRequest` with no `request_id`, route to `handshakeCh`. - -Update the unsolicited message handling in `readLoop`: - -```go - if reqID == "" { - // Unsolicited message (ToolListResponse after reload, or middleware registration during handshake). - select { - case m.handshakeCh <- env: - default: - } - continue - } -``` - -This already works because `RegisterMiddlewareRequest` during handshake has no `request_id` and goes to `handshakeCh`. `MiddlewareInterceptResponse` has a `request_id` and routes via `pending`. - -- [ ] **Step 5: Add SendMiddlewareIntercept method to Manager** - -```go -func (m *Manager) SendMiddlewareIntercept(ctx context.Context, mwName, phase, toolName, argsJSON, resultJSON string, isError bool) (*pb.MiddlewareInterceptResponse, error) { - reqID := m.nextRequestID() - - env := &pb.Envelope{ - RequestId: reqID, - Msg: &pb.Envelope_MiddlewareIntercept{ - MiddlewareIntercept: &pb.MiddlewareInterceptRequest{ - MiddlewareName: mwName, - Phase: phase, - ToolName: toolName, - ArgumentsJson: argsJSON, - ResultJson: resultJSON, - IsError: isError, - }, - }, - } - - respCh := make(chan *pb.Envelope, 1) - m.mu.Lock() - m.pending[reqID] = respCh - m.mu.Unlock() - - defer func() { - m.mu.Lock() - delete(m.pending, reqID) - m.mu.Unlock() - }() - - if err := envelope.Write(m.conn, env); err != nil { - return nil, fmt.Errorf("write MiddlewareInterceptRequest: %w", err) - } - - timer := time.NewTimer(m.cfg.CallTimeout) - defer timer.Stop() - - select { - case <-ctx.Done(): - return nil, ctx.Err() - case <-timer.C: - return nil, fmt.Errorf("middleware intercept %q timed out", mwName) - case resp := <-respCh: - mir := resp.GetMiddlewareInterceptResponse() - if mir == nil { - return nil, fmt.Errorf("unexpected response type for MiddlewareIntercept") - } - return mir, nil - } -} -``` - -- [ ] **Step 6: Create custom middleware dispatcher** - -Create `internal/middleware/custom.go`: - -```go -package middleware - -import ( - "context" - "encoding/json" - "fmt" - "sort" - - "github.com/msilverblatt/protomcp/internal/mcp" -) - -// MiddlewareDispatcher sends intercept requests to the tool process. -type MiddlewareDispatcher interface { - SendMiddlewareIntercept(ctx context.Context, mwName, phase, toolName, argsJSON, resultJSON string, isError bool) (interceptResp interface{ GetReject() bool; GetRejectReason() string; GetArgumentsJson() string; GetResultJson() string }, error) -} - -// RegisteredMW holds a registered custom middleware from the tool process. -type RegisteredMW struct { - Name string - Priority int32 -} - -// CustomMiddleware creates a middleware that dispatches to tool-process-registered middleware. -func CustomMiddleware(dispatcher MiddlewareDispatcher, registered []RegisteredMW) Middleware { - if len(registered) == 0 { - return func(next Handler) Handler { return next } - } - - // Sort by priority (lower first) - sorted := make([]RegisteredMW, len(registered)) - copy(sorted, registered) - sort.Slice(sorted, func(i, j int) bool { - return sorted[i].Priority < sorted[j].Priority - }) - - return func(next Handler) Handler { - return func(ctx context.Context, req JSONRPCRequest) (*JSONRPCResponse, error) { - // Extract tool name and args from the request if it's a tools/call - // For non-tool-call requests, skip custom middleware - if req.Method != "tools/call" { - return next(ctx, req) - } - - toolName, argsJSON := extractToolCallParams(req) - - // Before phase - currentArgs := argsJSON - for _, mw := range sorted { - resp, err := dispatcher.SendMiddlewareIntercept(ctx, mw.Name, "before", toolName, currentArgs, "", false) - if err != nil { - return nil, fmt.Errorf("middleware %q before: %w", mw.Name, err) - } - if resp.GetReject() { - return nil, fmt.Errorf("rejected by middleware %q: %s", mw.Name, resp.GetRejectReason()) - } - if modified := resp.GetArgumentsJson(); modified != "" { - currentArgs = modified - } - } - - // Inject modified args back into the request - if currentArgs != argsJSON { - modifiedParams, _ := json.Marshal(map[string]json.RawMessage{ - "name": json.RawMessage(fmt.Sprintf("%q", toolName)), - "arguments": json.RawMessage(currentArgs), - }) - req.Params = modifiedParams - } - - result, err := next(ctx, req) - - // After phase (reverse order) - resultJSON := "" - isError := false - if result != nil { - resultJSON = string(result.Result) - isError = result.Error != nil - } - - for i := len(sorted) - 1; i >= 0; i-- { - mw := sorted[i] - resp, afterErr := dispatcher.SendMiddlewareIntercept(ctx, mw.Name, "after", toolName, currentArgs, resultJSON, isError) - if afterErr != nil { - return result, err // Return original result on after-phase error - } - if modified := resp.GetResultJson(); modified != "" { - resultJSON = modified - } - } - - return result, err - } - } -} - -func extractToolCallParams(req JSONRPCRequest) (string, string) { - var params struct { - Name string `json:"name"` - Arguments json.RawMessage `json:"arguments"` - } - if err := json.Unmarshal(req.Params, ¶ms); err != nil { - return "", "{}" - } - argsJSON := "{}" - if len(params.Arguments) > 0 { - argsJSON = string(params.Arguments) - } - return params.Name, argsJSON -} -``` - -- [ ] **Step 7: Run all tests** - -Run: `go test ./internal/... -v` -Expected: All pass. - -- [ ] **Step 8: Commit** - -```bash -git add internal/process/manager.go internal/middleware/custom.go internal/middleware/custom_test.go -git commit -m "feat: extended handshake for middleware registration + custom middleware dispatch" -``` - ---- - -### Task 7: Update SDK runners for middleware + handshake-complete signal - -**Files:** -- Modify: `sdk/python/src/protomcp/runner.py` -- Modify: `sdk/python/src/protomcp/__init__.py` -- Modify: `sdk/typescript/src/runner.ts` -- Modify: `sdk/typescript/src/index.ts` - -- [ ] **Step 1: Add middleware registry to Python SDK** - -Create `sdk/python/src/protomcp/middleware.py`: - -```python -from dataclasses import dataclass -from typing import Callable, Any - -_middleware_registry: list["MiddlewareDef"] = [] - -@dataclass -class MiddlewareDef: - name: str - priority: int - handler: Callable # (phase, tool_name, args_json, result_json, is_error) -> dict - -def middleware(name: str, priority: int = 100): - def decorator(func: Callable) -> Callable: - _middleware_registry.append(MiddlewareDef( - name=name, - priority=priority, - handler=func, - )) - return func - return decorator - -def get_registered_middleware() -> list[MiddlewareDef]: - return list(_middleware_registry) - -def clear_middleware_registry(): - _middleware_registry.clear() -``` - -- [ ] **Step 2: Update Python runner to send handshake-complete signal and handle middleware** - -In `runner.py`, after `_handle_list_tools`, send `RegisterMiddlewareRequest` for each registered middleware, then send `ReloadResponse { success: true }` as handshake-complete. Also handle `MiddlewareInterceptRequest`: - -Update the main `run()` loop setup to: -1. After sending `ToolListResponse`, send middleware registrations -2. Send `ReloadResponse(success=True)` with no `request_id` as handshake-complete -3. In the message loop, handle `middleware_intercept` messages - -Update `_handle_reload` to also send middleware registrations + handshake-complete after tool list. - -- [ ] **Step 3: Update TypeScript runner similarly** - -Add `sdk/typescript/src/middleware.ts` with equivalent registration. Update `runner.ts` to send middleware registrations + handshake-complete signal. Handle `middlewareIntercept` messages in the loop. - -- [ ] **Step 4: Export middleware from both SDKs** - -Update `sdk/python/src/protomcp/__init__.py` to export `middleware`, `get_registered_middleware`. -Update `sdk/typescript/src/index.ts` to export `middleware` from `middleware.ts`. - -- [ ] **Step 5: Run existing SDK tests to ensure no regressions** - -Run: `cd sdk/python && python -m pytest -v && cd ../typescript && npx vitest run` -Expected: All existing tests pass. - -- [ ] **Step 6: Commit** - -```bash -git add sdk/python/src/protomcp/ sdk/typescript/src/ -git commit -m "feat: add middleware support to Python and TypeScript SDKs" -``` - ---- - -## Chunk 2: Go SDK - -### Task 8: Go SDK scaffolding + transport - -**Files:** -- Create: `sdk/go/go.mod` -- Create: `sdk/go/protomcp/transport.go` -- Create: `sdk/go/protomcp/transport_test.go` - -- [ ] **Step 1: Create go.mod** - -``` -module github.com/msilverblatt/protomcp/sdk/go - -go 1.25.6 - -require github.com/msilverblatt/protomcp v0.0.0 - -replace github.com/msilverblatt/protomcp => ../.. -``` - -- [ ] **Step 2: Write failing transport test** - -Create `sdk/go/protomcp/transport_test.go`: - -```go -package protomcp_test - -import ( - "net" - "os" - "path/filepath" - "testing" - - "github.com/msilverblatt/protomcp/sdk/go/protomcp" -) - -func TestTransportConnectAndClose(t *testing.T) { - dir := t.TempDir() - sockPath := filepath.Join(dir, "test.sock") - - listener, err := net.Listen("unix", sockPath) - if err != nil { - t.Fatal(err) - } - defer listener.Close() - - os.Setenv("PROTOMCP_SOCKET", sockPath) - defer os.Unsetenv("PROTOMCP_SOCKET") - - tp := protomcp.NewTransport(sockPath) - - // Accept in background - go func() { listener.Accept() }() - - if err := tp.Connect(); err != nil { - t.Fatal(err) - } - tp.Close() -} -``` - -- [ ] **Step 3: Run test to verify it fails** - -Run: `cd sdk/go && go test ./protomcp/ -run TestTransportConnect -v` -Expected: FAIL — package doesn't exist. - -- [ ] **Step 4: Implement transport** - -Create `sdk/go/protomcp/transport.go`: - -```go -package protomcp - -import ( - "encoding/binary" - "fmt" - "io" - "net" - - pb "github.com/msilverblatt/protomcp/gen/proto/protomcp" - "google.golang.org/protobuf/proto" -) - -type Transport struct { - socketPath string - conn net.Conn -} - -func NewTransport(socketPath string) *Transport { - return &Transport{socketPath: socketPath} -} - -func (t *Transport) Connect() error { - conn, err := net.Dial("unix", t.socketPath) - if err != nil { - return fmt.Errorf("connect to socket: %w", err) - } - t.conn = conn - return nil -} - -func (t *Transport) Send(env *pb.Envelope) error { - data, err := proto.Marshal(env) - if err != nil { - return fmt.Errorf("marshal envelope: %w", err) - } - length := make([]byte, 4) - binary.BigEndian.PutUint32(length, uint32(len(data))) - if _, err := t.conn.Write(length); err != nil { - return err - } - _, err = t.conn.Write(data) - return err -} - -func (t *Transport) Recv() (*pb.Envelope, error) { - lengthBuf := make([]byte, 4) - if _, err := io.ReadFull(t.conn, lengthBuf); err != nil { - return nil, err - } - length := binary.BigEndian.Uint32(lengthBuf) - data := make([]byte, length) - if _, err := io.ReadFull(t.conn, data); err != nil { - return nil, err - } - env := &pb.Envelope{} - if err := proto.Unmarshal(data, env); err != nil { - return nil, fmt.Errorf("unmarshal envelope: %w", err) - } - return env, nil -} - -func (t *Transport) Close() { - if t.conn != nil { - t.conn.Close() - } -} -``` - -- [ ] **Step 5: Run test to verify it passes** - -Run: `cd sdk/go && go test ./protomcp/ -run TestTransportConnect -v` -Expected: PASS. - -- [ ] **Step 6: Commit** - -```bash -git add sdk/go/ -git commit -m "feat(sdk/go): add transport layer" -``` - ---- - -### Task 9: Go SDK result + context - -**Files:** -- Create: `sdk/go/protomcp/result.go` -- Create: `sdk/go/protomcp/result_test.go` -- Create: `sdk/go/protomcp/context.go` -- Create: `sdk/go/protomcp/context_test.go` - -- [ ] **Step 1: Write failing result test** - -```go -package protomcp_test - -import "testing" - -func TestToolResultBasic(t *testing.T) { - r := protomcp.Result("hello") - if r.ResultText != "hello" { - t.Errorf("ResultText = %q, want %q", r.ResultText, "hello") - } - if r.IsError { - t.Error("should not be error") - } -} - -func TestToolResultError(t *testing.T) { - r := protomcp.ErrorResult("failed", "INVALID", "try again", true) - if !r.IsError { - t.Error("should be error") - } - if r.ErrorCode != "INVALID" { - t.Errorf("ErrorCode = %q, want %q", r.ErrorCode, "INVALID") - } -} - -func TestToolResultEnableDisable(t *testing.T) { - r := protomcp.Result("ok") - r.EnableTools = []string{"admin_panel"} - r.DisableTools = []string{"login"} - if len(r.EnableTools) != 1 { - t.Error("expected 1 enable tool") - } -} -``` - -- [ ] **Step 2: Implement result.go** - -```go -package protomcp - -type ToolResult struct { - ResultText string - IsError bool - ErrorCode string - Message string - Suggestion string - Retryable bool - EnableTools []string - DisableTools []string -} - -func Result(text string) ToolResult { - return ToolResult{ResultText: text} -} - -func ErrorResult(text, errorCode, suggestion string, retryable bool) ToolResult { - return ToolResult{ - ResultText: text, - IsError: true, - ErrorCode: errorCode, - Message: text, - Suggestion: suggestion, - Retryable: retryable, - } -} -``` - -- [ ] **Step 3: Write context tests and implement** - -Context wraps progress reporting and cancellation. Uses `context.Context` for cancellation: - -```go -package protomcp - -import ( - "context" - - pb "github.com/msilverblatt/protomcp/gen/proto/protomcp" -) - -type ToolContext struct { - Ctx context.Context - ProgressToken string - sendFn func(*pb.Envelope) error -} - -func (tc *ToolContext) ReportProgress(progress, total int64, message string) error { - env := &pb.Envelope{ - Msg: &pb.Envelope_Progress{ - Progress: &pb.ProgressNotification{ - ProgressToken: tc.ProgressToken, - Progress: progress, - Total: total, - Message: message, - }, - }, - } - return tc.sendFn(env) -} - -func (tc *ToolContext) IsCancelled() bool { - return tc.Ctx.Err() != nil -} -``` - -- [ ] **Step 4: Run tests** - -Run: `cd sdk/go && go test ./protomcp/ -v` -Expected: All pass. - -- [ ] **Step 5: Commit** - -```bash -git add sdk/go/protomcp/result.go sdk/go/protomcp/result_test.go sdk/go/protomcp/context.go sdk/go/protomcp/context_test.go -git commit -m "feat(sdk/go): add ToolResult and ToolContext" -``` - ---- - -### Task 10: Go SDK tool registration - -**Files:** -- Create: `sdk/go/protomcp/tool.go` -- Create: `sdk/go/protomcp/tool_test.go` - -- [ ] **Step 1: Write failing test** - -```go -package protomcp_test - -import ( - "testing" - - "github.com/msilverblatt/protomcp/sdk/go/protomcp" -) - -func TestToolRegistration(t *testing.T) { - protomcp.ClearRegistry() - protomcp.Tool("add", - protomcp.Description("Add two numbers"), - protomcp.Args(protomcp.IntArg("a"), protomcp.IntArg("b")), - protomcp.Handler(func(ctx protomcp.ToolContext, args map[string]interface{}) protomcp.ToolResult { - return protomcp.Result("3") - }), - ) - - tools := protomcp.GetRegisteredTools() - if len(tools) != 1 { - t.Fatalf("expected 1 tool, got %d", len(tools)) - } - if tools[0].Name != "add" { - t.Errorf("name = %q, want %q", tools[0].Name, "add") - } - if tools[0].Desc != "Add two numbers" { - t.Errorf("description = %q, want %q", tools[0].Desc, "Add two numbers") - } -} - -func TestToolMetadata(t *testing.T) { - protomcp.ClearRegistry() - protomcp.Tool("delete_user", - protomcp.Description("Delete a user account"), - protomcp.DestructiveHint(true), - protomcp.Handler(func(ctx protomcp.ToolContext, args map[string]interface{}) protomcp.ToolResult { - return protomcp.Result("deleted") - }), - ) - - tools := protomcp.GetRegisteredTools() - if !tools[0].Destructive { - t.Error("expected destructive hint") - } -} -``` - -- [ ] **Step 2: Implement tool.go with functional options** - -```go -package protomcp - -import ( - "encoding/json" -) - -type ToolDef struct { - Name string - Desc string - InputSchema map[string]interface{} - OutputSchema map[string]interface{} - HandlerFn func(ToolContext, map[string]interface{}) ToolResult - Title string - Destructive bool - Idempotent bool - ReadOnly bool - OpenWorld bool - TaskSupport bool -} - -type ToolOption func(*ToolDef) - -var registry []ToolDef - -func Tool(name string, opts ...ToolOption) { - td := ToolDef{ - Name: name, - InputSchema: map[string]interface{}{"type": "object", "properties": map[string]interface{}{}}, - } - for _, opt := range opts { - opt(&td) - } - registry = append(registry, td) -} - -func Description(desc string) ToolOption { - return func(td *ToolDef) { td.Desc = desc } -} - -type ArgDef struct { - Name string - Type string -} - -func IntArg(name string) ArgDef { return ArgDef{Name: name, Type: "integer"} } -func StrArg(name string) ArgDef { return ArgDef{Name: name, Type: "string"} } -func NumArg(name string) ArgDef { return ArgDef{Name: name, Type: "number"} } -func BoolArg(name string) ArgDef { return ArgDef{Name: name, Type: "boolean"} } - -func Args(args ...ArgDef) ToolOption { - return func(td *ToolDef) { - props := map[string]interface{}{} - required := []string{} - for _, a := range args { - props[a.Name] = map[string]interface{}{"type": a.Type} - required = append(required, a.Name) - } - td.InputSchema = map[string]interface{}{ - "type": "object", - "properties": props, - "required": required, - } - } -} - -func Handler(fn func(ToolContext, map[string]interface{}) ToolResult) ToolOption { - return func(td *ToolDef) { td.HandlerFn = fn } -} - -func DestructiveHint(v bool) ToolOption { return func(td *ToolDef) { td.Destructive = v } } -func IdempotentHint(v bool) ToolOption { return func(td *ToolDef) { td.Idempotent = v } } -func ReadOnlyHint(v bool) ToolOption { return func(td *ToolDef) { td.ReadOnly = v } } -func OpenWorldHint(v bool) ToolOption { return func(td *ToolDef) { td.OpenWorld = v } } -func TaskSupportHint(v bool) ToolOption { return func(td *ToolDef) { td.TaskSupport = v } } - -func (td ToolDef) InputSchemaJSON() string { - b, _ := json.Marshal(td.InputSchema) - return string(b) -} - -func GetRegisteredTools() []ToolDef { return append([]ToolDef{}, registry...) } -func ClearRegistry() { registry = nil } -``` - -- [ ] **Step 3: Run tests** - -Run: `cd sdk/go && go test ./protomcp/ -run TestTool -v` -Expected: All pass. - -- [ ] **Step 4: Commit** - -```bash -git add sdk/go/protomcp/tool.go sdk/go/protomcp/tool_test.go -git commit -m "feat(sdk/go): add tool registration with functional options" -``` - ---- - -### Task 11: Go SDK log + manager + runner - -**Files:** -- Create: `sdk/go/protomcp/log.go` -- Create: `sdk/go/protomcp/manager.go` -- Create: `sdk/go/protomcp/runner.go` - -- [ ] **Step 1: Implement log.go** - -Server logging with 8 RFC 5424 levels. Follow the same pattern as Python SDK's `log.py`: - -```go -package protomcp - -import ( - pb "github.com/msilverblatt/protomcp/gen/proto/protomcp" -) - -type ServerLogger struct { - sendFn func(*pb.Envelope) error - logger string -} - -func NewServerLogger(sendFn func(*pb.Envelope) error, logger string) *ServerLogger { - return &ServerLogger{sendFn: sendFn, logger: logger} -} - -func (l *ServerLogger) log(level, dataJSON string) { - if l.sendFn == nil { - return - } - l.sendFn(&pb.Envelope{ - Msg: &pb.Envelope_Log{ - Log: &pb.LogMessage{ - Level: level, - Logger: l.logger, - DataJson: dataJSON, - }, - }, - }) -} - -func (l *ServerLogger) Debug(msg string) { l.log("debug", msg) } -func (l *ServerLogger) Info(msg string) { l.log("info", msg) } -func (l *ServerLogger) Notice(msg string) { l.log("notice", msg) } -func (l *ServerLogger) Warning(msg string) { l.log("warning", msg) } -func (l *ServerLogger) Error(msg string) { l.log("error", msg) } -func (l *ServerLogger) Critical(msg string) { l.log("critical", msg) } -func (l *ServerLogger) Alert(msg string) { l.log("alert", msg) } -func (l *ServerLogger) Emergency(msg string) { l.log("emergency", msg) } -``` - -- [ ] **Step 2: Implement manager.go** - -Dynamic tool list management — `ToolManager` with Enable/Disable/SetAllowed/SetBlocked: - -```go -package protomcp - -import ( - pb "github.com/msilverblatt/protomcp/gen/proto/protomcp" -) - -type ToolManager struct { - sendFn func(*pb.Envelope) error -} - -func newToolManager(sendFn func(*pb.Envelope) error) *ToolManager { - return &ToolManager{sendFn: sendFn} -} - -func (m *ToolManager) Enable(names ...string) { - m.sendFn(&pb.Envelope{ - Msg: &pb.Envelope_EnableTools{ - EnableTools: &pb.EnableToolsRequest{ToolNames: names}, - }, - }) -} - -func (m *ToolManager) Disable(names ...string) { - m.sendFn(&pb.Envelope{ - Msg: &pb.Envelope_DisableTools{ - DisableTools: &pb.DisableToolsRequest{ToolNames: names}, - }, - }) -} -``` - -- [ ] **Step 3: Implement runner.go** - -Main loop: connect to socket, send tool list, handle calls, handle reload. Follow the same pattern as Python/TypeScript runners but with handshake-complete signal: - -```go -package protomcp - -import ( - "encoding/json" - "fmt" - "os" - - pb "github.com/msilverblatt/protomcp/gen/proto/protomcp" -) - -var Log *ServerLogger - -func Run() { - socketPath := os.Getenv("PROTOMCP_SOCKET") - if socketPath == "" { - fmt.Fprintln(os.Stderr, "PROTOMCP_SOCKET not set — run via 'pmcp dev'") - os.Exit(1) - } - - tp := NewTransport(socketPath) - if err := tp.Connect(); err != nil { - fmt.Fprintf(os.Stderr, "connect: %v\n", err) - os.Exit(1) - } - defer tp.Close() - - Log = NewServerLogger(func(env *pb.Envelope) error { return tp.Send(env) }, "protomcp-go") - - for { - env, err := tp.Recv() - if err != nil { - break - } - - reqID := env.GetRequestId() - - switch { - case env.GetListTools() != nil: - handleListTools(tp, reqID) - sendHandshakeComplete(tp) - case env.GetCallTool() != nil: - handleCallTool(tp, env.GetCallTool(), reqID) - case env.GetReload() != nil: - handleReload(tp, reqID) - case env.GetMiddlewareIntercept() != nil: - handleMiddlewareIntercept(tp, env.GetMiddlewareIntercept(), reqID) - } - } -} - -func handleListTools(tp *Transport, reqID string) { - tools := GetRegisteredTools() - var defs []*pb.ToolDefinition - for _, t := range tools { - defs = append(defs, &pb.ToolDefinition{ - Name: t.Name, - Description: t.Desc, - InputSchemaJson: t.InputSchemaJSON(), - DestructiveHint: t.Destructive, - IdempotentHint: t.Idempotent, - ReadOnlyHint: t.ReadOnly, - OpenWorldHint: t.OpenWorld, - TaskSupport: t.TaskSupport, - }) - } - tp.Send(&pb.Envelope{ - RequestId: reqID, - Msg: &pb.Envelope_ToolList{ - ToolList: &pb.ToolListResponse{Tools: defs}, - }, - }) -} - -func sendHandshakeComplete(tp *Transport) { - // Send middleware registrations if any - // TODO: add middleware registration here when middleware support is added - - // Send handshake-complete signal - tp.Send(&pb.Envelope{ - Msg: &pb.Envelope_ReloadResponse{ - ReloadResponse: &pb.ReloadResponse{Success: true}, - }, - }) -} - -func handleCallTool(tp *Transport, req *pb.CallToolRequest, reqID string) { - tools := GetRegisteredTools() - var handler func(ToolContext, map[string]interface{}) ToolResult - for _, t := range tools { - if t.Name == req.Name { - handler = t.HandlerFn - break - } - } - - if handler == nil { - tp.Send(&pb.Envelope{ - RequestId: reqID, - Msg: &pb.Envelope_CallResult{ - CallResult: &pb.CallToolResponse{ - IsError: true, - ResultJson: fmt.Sprintf(`[{"type":"text","text":"Tool not found: %s"}]`, req.Name), - }, - }, - }) - return - } - - var args map[string]interface{} - if req.ArgumentsJson != "" { - json.Unmarshal([]byte(req.ArgumentsJson), &args) - } - if args == nil { - args = map[string]interface{}{} - } - - ctx := ToolContext{ - ProgressToken: req.ProgressToken, - sendFn: func(env *pb.Envelope) error { return tp.Send(env) }, - } - - result := handler(ctx, args) - - resp := &pb.CallToolResponse{ - IsError: result.IsError, - ResultJson: fmt.Sprintf(`[{"type":"text","text":"%s"}]`, result.ResultText), - EnableTools: result.EnableTools, - DisableTools: result.DisableTools, - } - if result.IsError && result.ErrorCode != "" { - resp.Error = &pb.ToolError{ - ErrorCode: result.ErrorCode, - Message: result.Message, - Suggestion: result.Suggestion, - Retryable: result.Retryable, - } - } - - tp.Send(&pb.Envelope{ - RequestId: reqID, - Msg: &pb.Envelope_CallResult{CallResult: resp}, - }) -} - -func handleReload(tp *Transport, reqID string) { - tp.Send(&pb.Envelope{ - RequestId: reqID, - Msg: &pb.Envelope_ReloadResponse{ - ReloadResponse: &pb.ReloadResponse{Success: true}, - }, - }) - handleListTools(tp, "") - sendHandshakeComplete(tp) -} - -func handleMiddlewareIntercept(tp *Transport, req *pb.MiddlewareInterceptRequest, reqID string) { - // TODO: dispatch to registered middleware handlers - tp.Send(&pb.Envelope{ - RequestId: reqID, - Msg: &pb.Envelope_MiddlewareInterceptResponse{ - MiddlewareInterceptResponse: &pb.MiddlewareInterceptResponse{ - ArgumentsJson: req.ArgumentsJson, - ResultJson: req.ResultJson, - }, - }, - }) -} -``` - -- [ ] **Step 4: Run Go SDK tests** - -Run: `cd sdk/go && go test ./protomcp/ -v` -Expected: All pass. - -- [ ] **Step 5: Verify build** - -Run: `cd sdk/go && go build ./protomcp/` -Expected: Clean build. - -- [ ] **Step 6: Commit** - -```bash -git add sdk/go/protomcp/ -git commit -m "feat(sdk/go): add log, manager, and runner" -``` - ---- - -## Chunk 3: Rust SDK - -### Task 12: Rust SDK scaffolding - -**Files:** -- Create: `sdk/rust/Cargo.toml` -- Create: `sdk/rust/build.rs` -- Create: `sdk/rust/src/lib.rs` - -- [ ] **Step 1: Create Cargo.toml** - -```toml -[package] -name = "protomcp" -version = "0.1.0" -edition = "2021" -description = "Rust SDK for protomcp — write MCP tools in Rust" -license = "MIT" - -[dependencies] -prost = "0.13" -tokio = { version = "1", features = ["full"] } -serde = { version = "1", features = ["derive"] } -serde_json = "1" - -[build-dependencies] -prost-build = "0.13" -``` - -- [ ] **Step 2: Create build.rs** - -```rust -fn main() { - prost_build::compile_protos(&["../../proto/protomcp.proto"], &["../../proto/"]).unwrap(); -} -``` - -- [ ] **Step 3: Create src/lib.rs with module declarations** - -```rust -pub mod proto { - include!(concat!(env!("OUT_DIR"), "/protomcp.rs")); -} - -mod tool; -mod result; -mod context; -mod manager; -mod log; -mod transport; -mod runner; - -pub use tool::{tool, ToolDef, ArgDef, clear_registry, get_registered_tools}; -pub use result::ToolResult; -pub use context::ToolContext; -pub use runner::run; -``` - -- [ ] **Step 4: Verify protobuf generation builds** - -Run: `cd sdk/rust && cargo build` -Expected: May fail on missing modules — that's fine. Proto generation should succeed. - -- [ ] **Step 5: Commit** - -```bash -git add sdk/rust/ -git commit -m "feat(sdk/rust): scaffold crate with prost protobuf generation" -``` - ---- - -### Task 13-16: Rust SDK modules - -Follow the same pattern as the Go SDK tasks (9-11), implementing: -- `result.rs` — `ToolResult` struct -- `context.rs` — `ToolContext` with progress, cancellation token -- `tool.rs` — builder pattern registration -- `log.rs` — `ServerLogger` -- `manager.rs` — `ToolManager` (enable/disable) -- `transport.rs` — Unix socket + envelope framing -- `runner.rs` — async main loop with tokio - -Each module mirrors the Go SDK's functionality. Implementation follows the API surface from the spec. Tests use `#[cfg(test)]` inline modules. - -Key differences from Go SDK: -- Uses `tokio::net::UnixStream` instead of `net.Conn` -- Uses `tokio::sync::CancellationToken` instead of `context.Context` -- Builder pattern for tool registration instead of functional options -- All handlers are `async` - -- [ ] **Step 1: Implement all modules** -- [ ] **Step 2: Run tests**: `cd sdk/rust && cargo test` -- [ ] **Step 3: Commit** - -```bash -git add sdk/rust/src/ -git commit -m "feat(sdk/rust): implement full SDK (tool, result, context, transport, runner)" -``` - ---- - -## Chunk 4: CI/CD + Badges - -### Task 17: CI Pipeline - -**Files:** -- Create: `.github/workflows/ci.yml` - -- [ ] **Step 1: Create CI workflow** - -```yaml -name: CI - -on: - push: - branches: [master] - pull_request: - branches: [master] - -jobs: - go: - runs-on: ubuntu-latest - steps: - - uses: actions/checkout@v4 - - uses: actions/setup-go@v5 - with: - go-version: '1.25' - - run: go vet ./... - - run: go test ./... - - run: go build ./cmd/protomcp/ - - go-sdk: - runs-on: ubuntu-latest - steps: - - uses: actions/checkout@v4 - - uses: actions/setup-go@v5 - with: - go-version: '1.25' - - run: cd sdk/go && go test ./... - - python-sdk: - runs-on: ubuntu-latest - strategy: - matrix: - python-version: ['3.10', '3.11', '3.12'] - steps: - - uses: actions/checkout@v4 - - uses: actions/setup-python@v5 - with: - python-version: ${{ matrix.python-version }} - - run: cd sdk/python && pip install -e ".[dev]" && python -m pytest - - typescript-sdk: - runs-on: ubuntu-latest - steps: - - uses: actions/checkout@v4 - - uses: actions/setup-node@v4 - with: - node-version: '20' - - run: cd sdk/typescript && npm ci && npx vitest run - - rust-sdk: - runs-on: ubuntu-latest - steps: - - uses: actions/checkout@v4 - - uses: dtolnay/rust-toolchain@stable - - run: cd sdk/rust && cargo test - - run: cd sdk/rust && cargo clippy -- -D warnings - - e2e: - runs-on: ubuntu-latest - needs: [go] - steps: - - uses: actions/checkout@v4 - - uses: actions/setup-go@v5 - with: - go-version: '1.25' - - uses: actions/setup-python@v5 - with: - python-version: '3.12' - - uses: actions/setup-node@v4 - with: - node-version: '20' - - run: go build -o pmcp ./cmd/protomcp/ - - run: cd sdk/python && pip install -e . - - run: cd sdk/typescript && npm ci - - run: go test ./test/e2e/ -v -``` - -- [ ] **Step 2: Commit** - -```bash -git add .github/workflows/ci.yml -git commit -m "ci: add GitHub Actions CI pipeline" -``` - ---- - -### Task 18: Release Pipeline - -**Files:** -- Create: `.github/workflows/release.yml` - -- [ ] **Step 1: Create release workflow** - -```yaml -name: Release - -on: - push: - tags: ['v*'] - -permissions: - contents: write - -jobs: - goreleaser: - runs-on: ubuntu-latest - steps: - - uses: actions/checkout@v4 - with: - fetch-depth: 0 - - uses: actions/setup-go@v5 - with: - go-version: '1.25' - - uses: goreleaser/goreleaser-action@v6 - with: - args: release --clean - env: - GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }} - - pypi: - runs-on: ubuntu-latest - steps: - - uses: actions/checkout@v4 - - uses: actions/setup-python@v5 - with: - python-version: '3.12' - - run: pip install build twine - - run: cd sdk/python && python -m build - - run: cd sdk/python && twine upload dist/* - env: - TWINE_USERNAME: __token__ - TWINE_PASSWORD: ${{ secrets.PYPI_API_TOKEN }} - - npm: - runs-on: ubuntu-latest - steps: - - uses: actions/checkout@v4 - - uses: actions/setup-node@v4 - with: - node-version: '20' - registry-url: 'https://registry.npmjs.org' - - run: cd sdk/typescript && npm ci && npm publish - env: - NODE_AUTH_TOKEN: ${{ secrets.NPM_TOKEN }} - - crates: - runs-on: ubuntu-latest - steps: - - uses: actions/checkout@v4 - - uses: dtolnay/rust-toolchain@stable - - run: cd sdk/rust && cargo publish - env: - CARGO_REGISTRY_TOKEN: ${{ secrets.CARGO_REGISTRY_TOKEN }} - - homebrew: - runs-on: ubuntu-latest - needs: [goreleaser] - steps: - - uses: actions/checkout@v4 - with: - repository: protomcp/homebrew-tap - token: ${{ secrets.HOMEBREW_TAP_TOKEN }} - - name: Update formula - run: | - VERSION=${GITHUB_REF#refs/tags/v} - # GoReleaser generates checksums — update the formula with new version + SHA256 - sed -i "s/version \".*\"/version \"${VERSION}\"/" Formula/protomcp.rb - git add Formula/protomcp.rb - git commit -m "protomcp ${VERSION}" - git push -``` - -- [ ] **Step 2: Commit** - -```bash -git add .github/workflows/release.yml -git commit -m "ci: add release pipeline for Go binary, PyPI, npm, crates.io" -``` - ---- - -### Task 19: Fix README badges - -**Files:** -- Modify: `README.md` - -- [ ] **Step 1: Update badges** - -Replace the current badge block with: - -```markdown -[![CI](https://github.com/msilverblatt/protomcp/actions/workflows/ci.yml/badge.svg)](https://github.com/msilverblatt/protomcp/actions/workflows/ci.yml) -[![License: MIT](https://img.shields.io/badge/License-MIT-blue.svg)](LICENSE) -[![Go](https://img.shields.io/badge/Go-1.25+-00ADD8?logo=go)](https://go.dev) -[![PyPI](https://img.shields.io/pypi/v/protomcp)](https://pypi.org/project/protomcp/) -[![npm](https://img.shields.io/npm/v/protomcp)](https://www.npmjs.com/package/protomcp) -[![crates.io](https://img.shields.io/crates/v/protomcp)](https://crates.io/crates/protomcp) -``` - -Changes: renamed Build→CI to match workflow name, added crates.io badge, reordered. - -- [ ] **Step 2: Commit** - -```bash -git add README.md -git commit -m "docs: fix README badges to match CI workflow" -``` - ---- - -## Chunk 5: Examples + Documentation - -### Task 20: Go examples - -**Files:** -- Create: `examples/go/basic.go` -- Create: `examples/go/real_world.go` -- Create: `examples/go/full_showcase.go` - -- [ ] **Step 1: Create basic example** - -```go -// examples/go/basic.go -// A minimal protomcp tool — adds and multiplies numbers. -// Run: pmcp dev examples/go/basic.go -package main - -import ( - "fmt" - - "github.com/msilverblatt/protomcp/sdk/go/protomcp" -) - -func main() { - protomcp.Tool("add", - protomcp.Description("Add two numbers"), - protomcp.Args(protomcp.IntArg("a"), protomcp.IntArg("b")), - protomcp.Handler(func(ctx protomcp.ToolContext, args map[string]interface{}) protomcp.ToolResult { - a := int(args["a"].(float64)) - b := int(args["b"].(float64)) - return protomcp.Result(fmt.Sprintf("%d", a+b)) - }), - ) - - protomcp.Tool("multiply", - protomcp.Description("Multiply two numbers"), - protomcp.Args(protomcp.IntArg("a"), protomcp.IntArg("b")), - protomcp.Handler(func(ctx protomcp.ToolContext, args map[string]interface{}) protomcp.ToolResult { - a := int(args["a"].(float64)) - b := int(args["b"].(float64)) - return protomcp.Result(fmt.Sprintf("%d", a*b)) - }), - ) - - protomcp.Run() -} -``` - -- [ ] **Step 2: Create real_world and full_showcase examples** - -Follow the same patterns as the Python examples but in Go, adding progress reporting, cancellation, logging, dynamic tool lists, structured output, and metadata. - -- [ ] **Step 3: Verify examples parse**: `cd examples/go && go vet ./...` - -- [ ] **Step 4: Commit** - -```bash -git add examples/go/ -git commit -m "examples: add Go examples at 3 tiers" -``` - ---- - -### Task 21: Rust examples - -**Files:** -- Create: `examples/rust/basic/Cargo.toml` + `examples/rust/basic/src/main.rs` -- Create: `examples/rust/real_world/Cargo.toml` + `examples/rust/real_world/src/main.rs` -- Create: `examples/rust/full_showcase/Cargo.toml` + `examples/rust/full_showcase/src/main.rs` - -Each is a standalone Cargo project that depends on the `protomcp` crate via path dependency. - -- [ ] **Step 1: Create basic example** -- [ ] **Step 2: Create real_world and full_showcase** -- [ ] **Step 3: Verify**: `cd examples/rust/basic && cargo check` -- [ ] **Step 4: Commit** - -```bash -git add examples/rust/ -git commit -m "examples: add Rust examples at 3 tiers" -``` - ---- - -### Task 22: Documentation — new guide pages - -**Files:** -- Create: `docs/src/content/docs/guides/writing-tools-go.mdx` -- Create: `docs/src/content/docs/guides/writing-tools-rust.mdx` -- Create: `docs/src/content/docs/guides/middleware.mdx` -- Create: `docs/src/content/docs/guides/auth.mdx` -- Create: `docs/src/content/docs/guides/writing-a-language-library.mdx` - -Each guide follows the same structure as the existing Python/TypeScript guides: -1. Installation -2. Basic usage (decorator/builder pattern) -3. Type hints / schema definition -4. Optional parameters -5. Progress reporting -6. Cancellation -7. Logging -8. Error handling -9. Advanced features (middleware, dynamic tools) - -For the middleware and auth guides, focus on configuration and usage patterns with examples in all four languages. - -The "Writing a Language Library" guide covers: -1. The protobuf contract (envelope format, message types) -2. Unix socket connection -3. Length-prefixed framing -4. Handshake protocol (ListToolsRequest → ToolListResponse → optional middleware → ReloadResponse) -5. Tool call dispatch -6. Reload handling -7. Testing against the Go binary - -- [ ] **Step 1: Write all five guide pages** -- [ ] **Step 2: Verify docs build**: `cd docs && npm run build` -- [ ] **Step 3: Commit** - -```bash -git add docs/src/content/docs/guides/ -git commit -m "docs: add Go, Rust, middleware, auth, and language library guides" -``` - ---- - -### Task 23: Documentation — update existing pages - -**Files:** -- Modify: `docs/src/content/docs/index.mdx` -- Modify: `docs/src/content/docs/getting-started/quick-start.mdx` -- Modify: `docs/src/content/docs/getting-started/installation.mdx` -- Modify: `docs/src/content/docs/reference/cli.mdx` - -- [ ] **Step 1: Update index.mdx** - -Add Go and Rust cards to the CardGrid. - -- [ ] **Step 2: Update quick-start.mdx** - -Add Go and Rust quick start snippets (install SDK, write tool file, run). - -- [ ] **Step 3: Update installation.mdx** - -Add Go SDK install (`go get github.com/msilverblatt/protomcp/sdk/go`) and Rust SDK install (`cargo add protomcp`). - -- [ ] **Step 4: Update cli.mdx** - -Add `validate` subcommand docs and `--auth` flag docs. - -- [ ] **Step 5: Verify docs build**: `cd docs && npm run build` - -- [ ] **Step 6: Commit** - -```bash -git add docs/src/content/docs/ -git commit -m "docs: update index, quick-start, installation, and CLI reference for v1.1" -``` - ---- - -### Task 24: Update README - -**Files:** -- Modify: `README.md` - -- [ ] **Step 1: Add Go and Rust quick start examples** - -After the TypeScript example, add Go and Rust sections: - -````markdown -### Go - -```go -// tools.go -package main - -import ( - "fmt" - "github.com/msilverblatt/protomcp/sdk/go/protomcp" -) - -func main() { - protomcp.Tool("add", - protomcp.Description("Add two numbers"), - protomcp.Args(protomcp.IntArg("a"), protomcp.IntArg("b")), - protomcp.Handler(func(ctx protomcp.ToolContext, args map[string]interface{}) protomcp.ToolResult { - a := int(args["a"].(float64)) - b := int(args["b"].(float64)) - return protomcp.Result(fmt.Sprintf("%d", a+b)) - }), - ) - protomcp.Run() -} -``` - -```sh -pmcp dev tools.go -``` - -### Rust - -```rust -// src/main.rs -use protomcp::{tool, ToolResult}; - -#[tokio::main] -async fn main() { - tool("add") - .description("Add two numbers") - .arg::("a") - .arg::("b") - .handler(|_ctx, args| { - let a = args["a"].as_i64().unwrap_or(0); - let b = args["b"].as_i64().unwrap_or(0); - ToolResult::new(format!("{}", a + b)) - }) - .register(); - - protomcp::run().await; -} -``` - -```sh -pmcp dev src/main.rs -``` -```` - -- [ ] **Step 2: Update features list** - -Add after "Tool Metadata": - -```markdown -- **Middleware** — intercept tool calls before/after with custom logic -- **Auth** — built-in token and API key authentication for network transports -- **Validation** — `pmcp validate` checks tool definitions before deployment -``` - -- [ ] **Step 3: Update comparison table** - -Add middleware and validation rows. Update language support to mention Go/Rust explicitly. - -- [ ] **Step 4: Verify README renders** - -Visually inspect or use a markdown previewer. - -- [ ] **Step 5: Commit** - -```bash -git add README.md -git commit -m "docs: update README with Go/Rust examples, middleware, validation" -``` - ---- - -### Task 25: Update sidebar config - -**Files:** -- Modify: `docs/astro.config.mjs` - -- [ ] **Step 1: Add new guide pages to sidebar** - -Add entries for the new guides (writing-tools-go, writing-tools-rust, middleware, auth, writing-a-language-library) in the sidebar configuration. - -- [ ] **Step 2: Verify docs build**: `cd docs && npm run build` - -- [ ] **Step 3: Commit** - -```bash -git add docs/astro.config.mjs -git commit -m "docs: add v1.1 guide pages to sidebar" -``` - ---- - -### Task 26: Final verification - -- [ ] **Step 1: Run all tests** - -```bash -go test ./... -cd sdk/go && go test ./... -cd ../rust && cargo test -cd ../../sdk/python && python -m pytest -cd ../typescript && npx vitest run -``` - -- [ ] **Step 2: Build binary** - -Run: `go build -o pmcp ./cmd/protomcp/` - -- [ ] **Step 3: Build docs** - -Run: `cd docs && npm run build` - -- [ ] **Step 4: Verify README badges will resolve** - -Check that `.github/workflows/ci.yml` has `name: CI` matching the badge URL. - -- [ ] **Step 5: Final commit if any cleanup needed** - -```bash -git add -A -git commit -m "chore: v1.1 final cleanup" -``` diff --git a/docs/superpowers/plans/2026-03-12-protomcp-v1.md b/docs/superpowers/plans/2026-03-12-protomcp-v1.md deleted file mode 100644 index d6d5604..0000000 --- a/docs/superpowers/plans/2026-03-12-protomcp-v1.md +++ /dev/null @@ -1,4992 +0,0 @@ -# protomcp v1.0 Implementation Plan - -> **For agentic workers:** REQUIRED: Use superpowers:subagent-driven-development (if subagents available) or superpowers:executing-plans to implement this plan. Steps use checkbox (`- [ ]`) syntax for tracking. - -**Goal:** Build a language-agnostic MCP runtime — a Go binary that proxies MCP protocol to tool processes in any language via protobuf over unix socket, with hot-reload, dynamic tool list management, progress notifications, async tasks, cancellation, server logging, structured output, and tool metadata. - -**Architecture:** A precompiled Go binary handles all MCP transport/protocol. It communicates with a spawned tool process over a unix socket using length-prefixed protobuf envelopes. First-class Python and TypeScript SDKs provide decorator APIs. The binary proxies progress, cancellation, logging, and async task lifecycle. Starlight documentation site. - -**Tech Stack:** Go 1.22+, Protocol Buffers 3, Python 3.10+ (with protobuf, inspect), TypeScript 5+ (with Zod, protobuf-ts), Astro Starlight, fsnotify (Go file watching) - -**Spec:** `docs/superpowers/specs/2026-03-12-protomcp-design.md` - ---- - -## File Structure - -``` -protomcp/ -├── proto/ -│ └── protomcp.proto # Single source of truth for internal protocol -├── cmd/ -│ └── protomcp/ -│ └── main.go # CLI entry point (dev, run commands) -├── internal/ -│ ├── envelope/ -│ │ ├── envelope.go # Length-prefixed protobuf read/write -│ │ └── envelope_test.go -│ ├── toollist/ -│ │ ├── manager.go # Tool list state machine (open/allow/block) -│ │ └── manager_test.go -│ ├── process/ -│ │ ├── manager.go # Tool process spawn, handshake, crash recovery -│ │ └── manager_test.go -│ ├── reload/ -│ │ ├── watcher.go # File watcher + reload orchestration -│ │ └── watcher_test.go -│ ├── mcp/ -│ │ ├── types.go # MCP JSON-RPC types (requests, responses, notifications) -│ │ ├── handler.go # MCP request handler (tools/list, tools/call dispatch) -│ │ └── handler_test.go -│ ├── transport/ -│ │ ├── transport.go # Transport interface -│ │ ├── stdio.go # stdio transport -│ │ ├── stdio_test.go -│ │ ├── http.go # Streamable HTTP transport -│ │ ├── http_test.go -│ │ ├── sse.go # SSE transport -│ │ ├── sse_test.go -│ │ ├── grpc.go # gRPC transport -│ │ ├── grpc_test.go -│ │ ├── ws.go # WebSocket transport -│ │ └── ws_test.go -│ ├── middleware/ -│ │ ├── chain.go # Middleware interceptor chain -│ │ ├── logging.go # Structured logging middleware -│ │ └── errors.go # Error formatting middleware -│ ├── progress/ -│ │ ├── progress.go # Progress notification proxy -│ │ └── progress_test.go -│ ├── tasks/ -│ │ ├── manager.go # Async task lifecycle manager -│ │ └── manager_test.go -│ ├── cancel/ -│ │ ├── tracker.go # Cancellation tracking for in-flight calls -│ │ └── tracker_test.go -│ ├── serverlog/ -│ │ ├── forwarder.go # Server log forwarding to MCP notifications/message -│ │ └── forwarder_test.go -│ └── config/ -│ └── config.go # CLI flag parsing and config struct -├── gen/ -│ └── proto/ -│ └── protomcp/ -│ ├── protomcp.pb.go # Generated Go protobuf types -│ └── protomcp_grpc.pb.go # Generated gRPC service (for external gRPC transport) -├── go.mod -├── go.sum -├── sdk/ -│ ├── python/ -│ │ ├── pyproject.toml -│ │ ├── src/ -│ │ │ └── protomcp/ -│ │ │ ├── __init__.py # Public API exports -│ │ │ ├── tool.py # @tool decorator + schema generation -│ │ │ ├── result.py # ToolResult type -│ │ │ ├── manager.py # tool_manager client -│ │ │ ├── context.py # ToolContext (progress, cancellation) -│ │ │ ├── log.py # Server logging API -│ │ │ ├── transport.py # Unix socket + envelope framing -│ │ │ └── runner.py # Main loop: connect, listen, reload, dispatch -│ │ ├── tests/ -│ │ │ ├── test_tool.py -│ │ │ ├── test_result.py -│ │ │ ├── test_manager.py -│ │ │ ├── test_transport.py -│ │ │ └── test_integration.py # End-to-end with Go binary -│ │ └── gen/ -│ │ └── protomcp_pb2.py # Generated Python protobuf types -│ └── typescript/ -│ ├── package.json -│ ├── tsconfig.json -│ ├── vitest.config.ts -│ ├── src/ -│ │ ├── index.ts # Public API exports -│ │ ├── tool.ts # tool() function + Zod schema generation -│ │ ├── result.ts # ToolResult type -│ │ ├── manager.ts # tool_manager client -│ │ ├── context.ts # ToolContext (progress, cancellation) -│ │ ├── log.ts # Server logging API -│ │ ├── transport.ts # Unix socket + envelope framing -│ │ └── runner.ts # Main loop -│ ├── tests/ -│ │ ├── tool.test.ts -│ │ ├── result.test.ts -│ │ ├── manager.test.ts -│ │ ├── transport.test.ts -│ │ └── integration.test.ts # End-to-end with Go binary -│ └── gen/ -│ └── protomcp.ts # Generated TS protobuf types -├── test/ -│ ├── e2e/ -│ │ ├── e2e_test.go # Full pipeline: Go binary + Python tool + MCP client -│ │ ├── fixtures/ -│ │ │ ├── simple_tool.py # Basic Python tool for testing -│ │ │ ├── dynamic_tool.py # Tool with enable/disable mutations -│ │ │ ├── simple_tool.ts # Basic TS tool for testing -│ │ │ ├── crash_tool.py # Tool that crashes (for recovery testing) -│ │ │ ├── progress_tool.py # Tool that reports progress -│ │ │ ├── async_tool.py # Tool with task_support for async testing -│ │ │ ├── logging_tool.py # Tool that emits server logs -│ │ │ └── structured_output_tool.py # Tool with output_type for structured output -│ │ └── helpers.go # Test helpers (start binary, MCP client mock) -│ └── proto/ -│ └── proto_test.go # Validate proto generates correctly for all languages -├── docs/ -│ ├── astro.config.mjs -│ ├── package.json -│ └── src/ -│ └── content/ -│ └── docs/ -│ ├── index.mdx -│ ├── getting-started/ -│ │ ├── installation.mdx -│ │ ├── quick-start.mdx -│ │ └── how-it-works.mdx -│ ├── guides/ -│ │ ├── writing-tools-python.mdx -│ │ ├── writing-tools-typescript.mdx -│ │ ├── dynamic-tool-lists.mdx -│ │ ├── hot-reload.mdx -│ │ ├── progress-notifications.mdx -│ │ ├── async-tasks.mdx -│ │ ├── cancellation.mdx -│ │ ├── server-logging.mdx -│ │ ├── structured-output.mdx -│ │ ├── error-handling.mdx -│ │ └── production-deployment.mdx -│ ├── reference/ -│ │ ├── cli.mdx -│ │ ├── protobuf-spec.mdx -│ │ ├── python-api.mdx -│ │ └── typescript-api.mdx -│ └── concepts/ -│ ├── architecture.mdx -│ ├── tool-list-modes.mdx -│ └── transports.mdx -├── Makefile # proto-gen, build, test, docs commands -└── .goreleaser.yml # Cross-platform binary releases -``` - ---- - -## Chunk 1: Foundation — Project Setup, Protobuf, Envelope, Tool List Manager - -### Task 1: Project Initialization - -**Files:** -- Create: `go.mod` -- Create: `Makefile` -- Create: `.gitignore` -- Create: `.goreleaser.yml` - -- [ ] **Step 1: Initialize Go module** - -```bash -cd /Users/msilverblatt/hotmcp -go mod init github.com/msilverblatt/protomcp -``` - -- [ ] **Step 2: Create Makefile** - -Create `Makefile` with targets: -```makefile -.PHONY: proto build test clean - -PROTO_DIR := proto -GEN_DIR := gen/proto/protomcp -PYTHON_GEN_DIR := sdk/python/gen -TS_GEN_DIR := sdk/typescript/gen - -proto: - mkdir -p $(GEN_DIR) $(PYTHON_GEN_DIR) $(TS_GEN_DIR) - protoc --go_out=$(GEN_DIR) --go_opt=paths=source_relative \ - --go-grpc_out=$(GEN_DIR) --go-grpc_opt=paths=source_relative \ - -I$(PROTO_DIR) $(PROTO_DIR)/protomcp.proto - protoc --python_out=$(PYTHON_GEN_DIR) \ - -I$(PROTO_DIR) $(PROTO_DIR)/protomcp.proto - protoc --plugin=protoc-gen-ts=$$(which protoc-gen-ts) \ - --ts_out=$(TS_GEN_DIR) \ - -I$(PROTO_DIR) $(PROTO_DIR)/protomcp.proto - -build: - go build -o bin/protomcp ./cmd/protomcp - -test: - go test ./... - -test-python: - cd sdk/python && python -m pytest tests/ -v - -test-ts: - cd sdk/typescript && npx vitest run - -test-all: test test-python test-ts - -clean: - rm -rf bin/ gen/ -``` - -- [ ] **Step 3: Create .gitignore** - -``` -bin/ -gen/ -dist/ -node_modules/ -__pycache__/ -*.egg-info/ -.venv/ -*.pb.go -*_pb2.py -``` - -- [ ] **Step 4: Create .goreleaser.yml** - -```yaml -version: 2 -builds: - - main: ./cmd/protomcp - binary: protomcp - goos: - - linux - - darwin - - windows - goarch: - - amd64 - - arm64 - env: - - CGO_ENABLED=0 - -archives: - - format: tar.gz - name_template: "protomcp_{{ .Os }}_{{ .Arch }}" - -brews: - - repository: - owner: msilverblatt - name: homebrew-tap - homepage: "https://github.com/msilverblatt/protomcp" - description: "Language-agnostic MCP runtime" -``` - -- [ ] **Step 5: Commit** - -```bash -git add go.mod Makefile .gitignore .goreleaser.yml -git commit -m "feat: initialize project with Go module, Makefile, and release config" -``` - ---- - -### Task 2: Protobuf Spec - -**Files:** -- Create: `proto/protomcp.proto` - -- [ ] **Step 1: Write the protobuf spec** - -Create `proto/protomcp.proto` — this is the single source of truth for the internal protocol between the Go binary and tool processes: - -```protobuf -syntax = "proto3"; - -package protomcp; - -option go_package = "github.com/msilverblatt/protomcp/gen/proto/protomcp"; - -// Envelope wraps all messages with length-prefixed framing. -// Wire format: 4-byte big-endian uint32 length prefix, then serialized Envelope. -message Envelope { - oneof msg { - // Go -> Tool Process - ReloadRequest reload = 1; - ListToolsRequest list_tools = 2; - CallToolRequest call_tool = 3; - - // Tool Process -> Go - ReloadResponse reload_response = 4; - ToolListResponse tool_list = 5; - CallToolResponse call_result = 6; - EnableToolsRequest enable_tools = 7; - DisableToolsRequest disable_tools = 8; - SetAllowedRequest set_allowed = 9; - SetBlockedRequest set_blocked = 10; - GetActiveToolsRequest get_active_tools = 11; - BatchUpdateRequest batch = 12; - ActiveToolsResponse active_tools = 13; - - // Progress, cancellation, logging, tasks - ProgressNotification progress = 16; - CancelRequest cancel = 17; - LogMessage log = 18; - CreateTaskResponse create_task = 19; - TaskStatusRequest task_status = 20; - TaskStatusResponse task_status_response = 21; - TaskResultRequest task_result = 22; - TaskCancelRequest task_cancel = 23; - } - // Correlation ID for matching requests to responses. - // Required for CallToolRequest/CallToolResponse to support concurrent calls. - // Optional for other message types. - string request_id = 14; - // Namespace for future multi-process support. Ignored in v1. - string namespace = 15; -} - -// --- Go -> Tool Process --- - -message ReloadRequest {} - -message ListToolsRequest {} - -message CallToolRequest { - string name = 1; - // JSON-encoded arguments from the MCP client - string arguments_json = 2; - // Progress token from client's _meta.progressToken (empty if none) - string progress_token = 3; -} - -// --- Tool Process -> Go --- - -message ReloadResponse { - bool success = 1; - string error = 2; // Non-empty on failure (e.g., syntax error, import error) -} - -message ToolListResponse { - repeated ToolDefinition tools = 1; -} - -message ToolDefinition { - string name = 1; - string description = 2; - // JSON Schema for the tool's input parameters - string input_schema_json = 3; - // JSON Schema for the tool's output (empty if unstructured) - string output_schema_json = 4; - // Tool metadata/annotations - string title = 5; - bool read_only_hint = 6; - bool destructive_hint = 7; - bool idempotent_hint = 8; - bool open_world_hint = 9; - bool task_support = 10; // Advertises async task capability -} - -message CallToolResponse { - bool is_error = 1; - // JSON-encoded result content (MCP content array) - string result_json = 2; - // Optional tool list mutations — Go binary intercepts these before proxying to host - repeated string enable_tools = 3; - repeated string disable_tools = 4; - // Structured error details (if is_error is true) - ToolError error = 5; - // Structured output (JSON, validated against outputSchema) - string structured_content_json = 6; -} - -message ToolError { - string error_code = 1; - string message = 2; - string suggestion = 3; - bool retryable = 4; -} - -// --- Tool List Control (Tool Process -> Go) --- - -message EnableToolsRequest { - repeated string tool_names = 1; -} - -message DisableToolsRequest { - repeated string tool_names = 1; -} - -message SetAllowedRequest { - repeated string tool_names = 1; -} - -message SetBlockedRequest { - repeated string tool_names = 1; -} - -message GetActiveToolsRequest {} - -message BatchUpdateRequest { - repeated string enable = 1; - repeated string disable = 2; - repeated string allow = 3; // Set allowed list (mutually exclusive with block) - repeated string block = 4; // Set blocked list (mutually exclusive with allow) -} - -message ActiveToolsResponse { - repeated string tool_names = 1; -} - -// --- Progress, Cancellation, Logging --- - -message ProgressNotification { - string progress_token = 1; // Token from CallToolRequest - int64 progress = 2; // Current progress value - int64 total = 3; // Total expected (0 = indeterminate) - string message = 4; // Human-readable status -} - -message CancelRequest { - string request_id = 1; // ID of the call to cancel -} - -message LogMessage { - string level = 1; // RFC 5424: emergency, alert, critical, error, warning, notice, info, debug - string logger = 2; // Source component (optional) - string data_json = 3; // JSON-encoded log data -} - -// --- Task (Async) Lifecycle --- - -message CreateTaskResponse { - string task_id = 1; -} - -message TaskStatusRequest { - string task_id = 1; -} - -message TaskStatusResponse { - string task_id = 1; - string state = 2; // running, completed, failed, cancelled - int64 progress = 3; // Optional progress - int64 total = 4; // Optional total - string message = 5; // Optional status message -} - -message TaskResultRequest { - string task_id = 1; -} - -message TaskCancelRequest { - string task_id = 1; -} -``` - -- [ ] **Step 2: Generate Go code from proto** - -```bash -make proto -``` - -Verify: `gen/proto/protomcp/protomcp.pb.go` exists and compiles. - -- [ ] **Step 3: Commit** - -```bash -git add proto/protomcp.proto -git commit -m "feat: add protobuf spec defining internal protocol between Go binary and tool processes" -``` - ---- - -### Task 3: Envelope — Length-Prefixed Protobuf Read/Write - -**Files:** -- Create: `internal/envelope/envelope.go` -- Create: `internal/envelope/envelope_test.go` - -- [ ] **Step 1: Write failing tests for envelope** - -Create `internal/envelope/envelope_test.go`: - -```go -package envelope_test - -import ( - "bytes" - "testing" - - "github.com/msilverblatt/protomcp/internal/envelope" - pb "github.com/msilverblatt/protomcp/gen/proto/protomcp" -) - -func TestWriteAndReadEnvelope(t *testing.T) { - // Create a test envelope with a ListToolsRequest - env := &pb.Envelope{ - Msg: &pb.Envelope_ListTools{ - ListTools: &pb.ListToolsRequest{}, - }, - RequestId: "test-123", - } - - var buf bytes.Buffer - err := envelope.Write(&buf, env) - if err != nil { - t.Fatalf("Write failed: %v", err) - } - - got, err := envelope.Read(&buf) - if err != nil { - t.Fatalf("Read failed: %v", err) - } - - if got.RequestId != "test-123" { - t.Errorf("RequestId = %q, want %q", got.RequestId, "test-123") - } - - if got.GetListTools() == nil { - t.Error("expected ListToolsRequest, got nil") - } -} - -func TestWriteAndReadCallTool(t *testing.T) { - env := &pb.Envelope{ - Msg: &pb.Envelope_CallTool{ - CallTool: &pb.CallToolRequest{ - Name: "search", - ArgumentsJson: `{"query": "hello"}`, - }, - }, - RequestId: "call-456", - } - - var buf bytes.Buffer - err := envelope.Write(&buf, env) - if err != nil { - t.Fatalf("Write failed: %v", err) - } - - got, err := envelope.Read(&buf) - if err != nil { - t.Fatalf("Read failed: %v", err) - } - - ct := got.GetCallTool() - if ct == nil { - t.Fatal("expected CallToolRequest, got nil") - } - if ct.Name != "search" { - t.Errorf("Name = %q, want %q", ct.Name, "search") - } - if ct.ArgumentsJson != `{"query": "hello"}` { - t.Errorf("ArgumentsJson = %q, want %q", ct.ArgumentsJson, `{"query": "hello"}`) - } -} - -func TestReadEmptyBuffer(t *testing.T) { - var buf bytes.Buffer - _, err := envelope.Read(&buf) - if err == nil { - t.Error("expected error reading empty buffer, got nil") - } -} - -func TestMultipleEnvelopes(t *testing.T) { - var buf bytes.Buffer - - envs := []*pb.Envelope{ - {Msg: &pb.Envelope_ListTools{ListTools: &pb.ListToolsRequest{}}, RequestId: "1"}, - {Msg: &pb.Envelope_Reload{Reload: &pb.ReloadRequest{}}, RequestId: "2"}, - {Msg: &pb.Envelope_ListTools{ListTools: &pb.ListToolsRequest{}}, RequestId: "3"}, - } - - for _, env := range envs { - if err := envelope.Write(&buf, env); err != nil { - t.Fatalf("Write failed: %v", err) - } - } - - for i, want := range envs { - got, err := envelope.Read(&buf) - if err != nil { - t.Fatalf("Read %d failed: %v", i, err) - } - if got.RequestId != want.RequestId { - t.Errorf("envelope %d: RequestId = %q, want %q", i, got.RequestId, want.RequestId) - } - } -} -``` - -- [ ] **Step 2: Run tests to verify they fail** - -```bash -go test ./internal/envelope/... -v -``` - -Expected: compilation error — package `envelope` doesn't exist. - -- [ ] **Step 3: Implement envelope** - -Create `internal/envelope/envelope.go`: - -```go -package envelope - -import ( - "encoding/binary" - "fmt" - "io" - - pb "github.com/msilverblatt/protomcp/gen/proto/protomcp" - "google.golang.org/protobuf/proto" -) - -const maxMessageSize = 10 * 1024 * 1024 // 10MB - -// Write serializes an Envelope and writes it with a 4-byte big-endian length prefix. -func Write(w io.Writer, env *pb.Envelope) error { - data, err := proto.Marshal(env) - if err != nil { - return fmt.Errorf("marshal envelope: %w", err) - } - - length := uint32(len(data)) - if err := binary.Write(w, binary.BigEndian, length); err != nil { - return fmt.Errorf("write length prefix: %w", err) - } - - if _, err := w.Write(data); err != nil { - return fmt.Errorf("write envelope data: %w", err) - } - - return nil -} - -// Read reads a length-prefixed Envelope from the reader. -func Read(r io.Reader) (*pb.Envelope, error) { - var length uint32 - if err := binary.Read(r, binary.BigEndian, &length); err != nil { - return nil, fmt.Errorf("read length prefix: %w", err) - } - - if length > maxMessageSize { - return nil, fmt.Errorf("message size %d exceeds max %d", length, maxMessageSize) - } - - data := make([]byte, length) - if _, err := io.ReadFull(r, data); err != nil { - return nil, fmt.Errorf("read envelope data: %w", err) - } - - env := &pb.Envelope{} - if err := proto.Unmarshal(data, env); err != nil { - return nil, fmt.Errorf("unmarshal envelope: %w", err) - } - - return env, nil -} -``` - -- [ ] **Step 4: Run tests to verify they pass** - -```bash -go test ./internal/envelope/... -v -``` - -Expected: all 4 tests PASS. - -- [ ] **Step 5: Commit** - -```bash -git add internal/envelope/ -git commit -m "feat: implement length-prefixed protobuf envelope read/write" -``` - ---- - -### Task 4: Tool List Manager — State Machine - -**Files:** -- Create: `internal/toollist/manager.go` -- Create: `internal/toollist/manager_test.go` - -- [ ] **Step 1: Write failing tests for tool list manager** - -Create `internal/toollist/manager_test.go`: - -```go -package toollist_test - -import ( - "testing" - - "github.com/msilverblatt/protomcp/internal/toollist" -) - -func TestOpenMode_AllToolsActive(t *testing.T) { - m := toollist.New() - m.SetRegistered([]string{"a", "b", "c"}) - - got := m.Active() - want := []string{"a", "b", "c"} - if !slicesEqual(got, want) { - t.Errorf("Active() = %v, want %v", got, want) - } -} - -func TestOpenMode_DisableTool(t *testing.T) { - m := toollist.New() - m.SetRegistered([]string{"a", "b", "c"}) - - changed := m.Disable([]string{"b"}) - if !changed { - t.Error("Disable should report change") - } - - got := m.Active() - want := []string{"a", "c"} - if !slicesEqual(got, want) { - t.Errorf("Active() = %v, want %v", got, want) - } -} - -func TestOpenMode_EnableReAdds(t *testing.T) { - m := toollist.New() - m.SetRegistered([]string{"a", "b", "c"}) - m.Disable([]string{"b"}) - - changed := m.Enable([]string{"b"}) - if !changed { - t.Error("Enable should report change") - } - - got := m.Active() - want := []string{"a", "b", "c"} - if !slicesEqual(got, want) { - t.Errorf("Active() = %v, want %v", got, want) - } -} - -func TestAllowlistMode(t *testing.T) { - m := toollist.New() - m.SetRegistered([]string{"a", "b", "c", "d"}) - - changed := m.SetAllowed([]string{"a", "c"}) - if !changed { - t.Error("SetAllowed should report change") - } - - got := m.Active() - want := []string{"a", "c"} - if !slicesEqual(got, want) { - t.Errorf("Active() = %v, want %v", got, want) - } -} - -func TestAllowlistMode_EnableAddsToAllowlist(t *testing.T) { - m := toollist.New() - m.SetRegistered([]string{"a", "b", "c"}) - m.SetAllowed([]string{"a"}) - - m.Enable([]string{"b"}) - - got := m.Active() - want := []string{"a", "b"} - if !slicesEqual(got, want) { - t.Errorf("Active() = %v, want %v", got, want) - } -} - -func TestAllowlistMode_DisableRemovesFromAllowlist(t *testing.T) { - m := toollist.New() - m.SetRegistered([]string{"a", "b", "c"}) - m.SetAllowed([]string{"a", "b"}) - - m.Disable([]string{"a"}) - - got := m.Active() - want := []string{"b"} - if !slicesEqual(got, want) { - t.Errorf("Active() = %v, want %v", got, want) - } -} - -func TestBlocklistMode(t *testing.T) { - m := toollist.New() - m.SetRegistered([]string{"a", "b", "c", "d"}) - - changed := m.SetBlocked([]string{"b", "d"}) - if !changed { - t.Error("SetBlocked should report change") - } - - got := m.Active() - want := []string{"a", "c"} - if !slicesEqual(got, want) { - t.Errorf("Active() = %v, want %v", got, want) - } -} - -func TestBlocklistMode_DisableAddsToBlocklist(t *testing.T) { - m := toollist.New() - m.SetRegistered([]string{"a", "b", "c"}) - m.SetBlocked([]string{"c"}) - - m.Disable([]string{"a"}) - - got := m.Active() - want := []string{"b"} - if !slicesEqual(got, want) { - t.Errorf("Active() = %v, want %v", got, want) - } -} - -func TestBlocklistMode_EnableRemovesFromBlocklist(t *testing.T) { - m := toollist.New() - m.SetRegistered([]string{"a", "b", "c"}) - m.SetBlocked([]string{"b", "c"}) - - m.Enable([]string{"c"}) - - got := m.Active() - want := []string{"a", "c"} - if !slicesEqual(got, want) { - t.Errorf("Active() = %v, want %v", got, want) - } -} - -func TestSetAllowedSwitchesFromBlocklist(t *testing.T) { - m := toollist.New() - m.SetRegistered([]string{"a", "b", "c"}) - m.SetBlocked([]string{"c"}) - m.SetAllowed([]string{"a"}) - - got := m.Active() - want := []string{"a"} - if !slicesEqual(got, want) { - t.Errorf("Active() = %v, want %v", got, want) - } -} - -func TestEmptyAllowlistResetsToOpen(t *testing.T) { - m := toollist.New() - m.SetRegistered([]string{"a", "b", "c"}) - m.SetAllowed([]string{"a"}) - m.SetAllowed([]string{}) - - got := m.Active() - want := []string{"a", "b", "c"} - if !slicesEqual(got, want) { - t.Errorf("Active() = %v, want %v", got, want) - } -} - -func TestEmptyBlocklistResetsToOpen(t *testing.T) { - m := toollist.New() - m.SetRegistered([]string{"a", "b", "c"}) - m.SetBlocked([]string{"a"}) - m.SetBlocked([]string{}) - - got := m.Active() - want := []string{"a", "b", "c"} - if !slicesEqual(got, want) { - t.Errorf("Active() = %v, want %v", got, want) - } -} - -func TestBatch_EnableAndDisable(t *testing.T) { - m := toollist.New() - m.SetRegistered([]string{"a", "b", "c"}) - m.Disable([]string{"a", "b"}) - - changed, err := m.Batch([]string{"a"}, []string{"c"}, nil, nil) - if err != nil { - t.Fatalf("Batch failed: %v", err) - } - if !changed { - t.Error("Batch should report change") - } - - got := m.Active() - want := []string{"a"} - if !slicesEqual(got, want) { - t.Errorf("Active() = %v, want %v", got, want) - } -} - -func TestBatch_AllowAndBlockRejects(t *testing.T) { - m := toollist.New() - m.SetRegistered([]string{"a", "b", "c"}) - - _, err := m.Batch(nil, nil, []string{"a"}, []string{"b"}) - if err == nil { - t.Error("Batch with both allow and block should return error") - } -} - -func TestBatch_AllowThenEnableDelta(t *testing.T) { - m := toollist.New() - m.SetRegistered([]string{"a", "b", "c"}) - - // Set allow to ["a"], then enable "b" — should result in ["a", "b"] - changed, err := m.Batch([]string{"b"}, nil, []string{"a"}, nil) - if err != nil { - t.Fatalf("Batch failed: %v", err) - } - if !changed { - t.Error("Batch should report change") - } - - got := m.Active() - want := []string{"a", "b"} - if !slicesEqual(got, want) { - t.Errorf("Active() = %v, want %v", got, want) - } -} - -func TestNoChangeReturnsFalse(t *testing.T) { - m := toollist.New() - m.SetRegistered([]string{"a", "b", "c"}) - - changed := m.Enable([]string{"a"}) // already active - if changed { - t.Error("Enable of already-active tool should return false") - } -} - -func slicesEqual(a, b []string) bool { - if len(a) != len(b) { - return false - } - m := make(map[string]bool, len(a)) - for _, v := range a { - m[v] = true - } - for _, v := range b { - if !m[v] { - return false - } - } - return true -} -``` - -- [ ] **Step 2: Run tests to verify they fail** - -```bash -go test ./internal/toollist/... -v -``` - -Expected: compilation error — package `toollist` doesn't exist. - -- [ ] **Step 3: Implement tool list manager** - -Create `internal/toollist/manager.go`: - -```go -package toollist - -import ( - "fmt" - "sort" - "sync" -) - -type mode int - -const ( - modeOpen mode = iota - modeAllowlist - modeBlocklist -) - -// Manager tracks which tools are active using open/allowlist/blocklist modes. -// All methods are safe for concurrent use. -type Manager struct { - mu sync.RWMutex - registered map[string]bool // all tools the tool process has registered - current mode - set map[string]bool // allowlist entries, blocklist entries, or disabled set (open mode) -} - -func New() *Manager { - return &Manager{ - registered: make(map[string]bool), - current: modeOpen, - set: make(map[string]bool), - } -} - -// SetRegistered updates the full set of registered tools (called after reload). -func (m *Manager) SetRegistered(tools []string) { - m.mu.Lock() - defer m.mu.Unlock() - m.registered = make(map[string]bool, len(tools)) - for _, t := range tools { - m.registered[t] = true - } -} - -// Active returns the currently active tool names. -func (m *Manager) Active() []string { - m.mu.RLock() - defer m.mu.RUnlock() - return m.activeLocked() -} - -func (m *Manager) activeLocked() []string { - var result []string - switch m.current { - case modeOpen: - for t := range m.registered { - if !m.set[t] { // set contains disabled tools in open mode - result = append(result, t) - } - } - case modeAllowlist: - for t := range m.set { - if m.registered[t] { - result = append(result, t) - } - } - case modeBlocklist: - for t := range m.registered { - if !m.set[t] { - result = append(result, t) - } - } - } - sort.Strings(result) - return result -} - -// Enable adds tools. Behavior depends on mode: -// Open: re-enables disabled tools. Allowlist: adds to allowlist. Blocklist: removes from blocklist. -func (m *Manager) Enable(tools []string) bool { - m.mu.Lock() - defer m.mu.Unlock() - before := m.activeLocked() - - switch m.current { - case modeOpen: - for _, t := range tools { - delete(m.set, t) - } - case modeAllowlist: - for _, t := range tools { - m.set[t] = true - } - case modeBlocklist: - for _, t := range tools { - delete(m.set, t) - } - } - - after := m.activeLocked() - return !slicesEqual(before, after) -} - -// Disable removes tools. Behavior depends on mode: -// Open: disables tools. Allowlist: removes from allowlist. Blocklist: adds to blocklist. -func (m *Manager) Disable(tools []string) bool { - m.mu.Lock() - defer m.mu.Unlock() - before := m.activeLocked() - - switch m.current { - case modeOpen: - for _, t := range tools { - m.set[t] = true - } - case modeAllowlist: - for _, t := range tools { - delete(m.set, t) - } - case modeBlocklist: - for _, t := range tools { - m.set[t] = true - } - } - - after := m.activeLocked() - return !slicesEqual(before, after) -} - -// SetAllowed switches to allowlist mode. Empty list resets to open mode. -func (m *Manager) SetAllowed(tools []string) bool { - m.mu.Lock() - defer m.mu.Unlock() - before := m.activeLocked() - - if len(tools) == 0 { - m.current = modeOpen - m.set = make(map[string]bool) - } else { - m.current = modeAllowlist - m.set = make(map[string]bool, len(tools)) - for _, t := range tools { - m.set[t] = true - } - } - - after := m.activeLocked() - return !slicesEqual(before, after) -} - -// SetBlocked switches to blocklist mode. Empty list resets to open mode. -func (m *Manager) SetBlocked(tools []string) bool { - m.mu.Lock() - defer m.mu.Unlock() - before := m.activeLocked() - - if len(tools) == 0 { - m.current = modeOpen - m.set = make(map[string]bool) - } else { - m.current = modeBlocklist - m.set = make(map[string]bool, len(tools)) - for _, t := range tools { - m.set[t] = true - } - } - - after := m.activeLocked() - return !slicesEqual(before, after) -} - -// Batch applies multiple operations atomically. Allow and block are mutually exclusive. -// Order: mode-setting (allow/block) first, then deltas (enable/disable). -func (m *Manager) Batch(enable, disable, allow, block []string) (bool, error) { - if len(allow) > 0 && len(block) > 0 { - return false, fmt.Errorf("batch: allow and block are mutually exclusive") - } - - m.mu.Lock() - defer m.mu.Unlock() - before := m.activeLocked() - - // Mode-setting first - if len(allow) > 0 { - m.current = modeAllowlist - m.set = make(map[string]bool, len(allow)) - for _, t := range allow { - m.set[t] = true - } - } else if len(block) > 0 { - m.current = modeBlocklist - m.set = make(map[string]bool, len(block)) - for _, t := range block { - m.set[t] = true - } - } - - // Then deltas (reuse logic without locks since we hold mu) - if len(enable) > 0 { - switch m.current { - case modeOpen: - for _, t := range enable { - delete(m.set, t) - } - case modeAllowlist: - for _, t := range enable { - m.set[t] = true - } - case modeBlocklist: - for _, t := range enable { - delete(m.set, t) - } - } - } - if len(disable) > 0 { - switch m.current { - case modeOpen: - for _, t := range disable { - m.set[t] = true - } - case modeAllowlist: - for _, t := range disable { - delete(m.set, t) - } - case modeBlocklist: - for _, t := range disable { - m.set[t] = true - } - } - } - - after := m.activeLocked() - return !slicesEqual(before, after), nil -} - -func slicesEqual(a, b []string) bool { - if len(a) != len(b) { - return false - } - m := make(map[string]bool, len(a)) - for _, v := range a { - m[v] = true - } - for _, v := range b { - if !m[v] { - return false - } - } - return true -} -``` - -- [ ] **Step 4: Run tests to verify they pass** - -```bash -go test ./internal/toollist/... -v -``` - -Expected: all 16 tests PASS. - -- [ ] **Step 5: Commit** - -```bash -git add internal/toollist/ -git commit -m "feat: implement tool list state machine with open/allowlist/blocklist modes" -``` - ---- - -## Chunk 2: Go Binary Core — Process Manager, MCP Handler, Config - -### Task 5: Config — CLI Flag Parsing - -**Files:** -- Create: `internal/config/config.go` -- Create: `internal/config/config_test.go` - -- [ ] **Step 1: Write failing tests for config parsing** - -Create `internal/config/config_test.go`: - -```go -package config_test - -import ( - "reflect" - "testing" - "time" - - "github.com/msilverblatt/protomcp/internal/config" -) - -func TestParseDefaults(t *testing.T) { - cfg, err := config.Parse([]string{"dev", "server.py"}) - if err != nil { - t.Fatalf("Parse failed: %v", err) - } - if cfg.Command != "dev" { - t.Errorf("Command = %q, want %q", cfg.Command, "dev") - } - if cfg.File != "server.py" { - t.Errorf("File = %q, want %q", cfg.File, "server.py") - } - if cfg.Transport != "stdio" { - t.Errorf("Transport = %q, want %q", cfg.Transport, "stdio") - } - if cfg.CallTimeout != 5*time.Minute { - t.Errorf("CallTimeout = %v, want %v", cfg.CallTimeout, 5*time.Minute) - } - if cfg.HotReloadImmediate { - t.Error("HotReloadImmediate should default to false") - } - if cfg.LogLevel != "info" { - t.Errorf("LogLevel = %q, want %q", cfg.LogLevel, "info") - } -} - -func TestParseWithFlags(t *testing.T) { - cfg, err := config.Parse([]string{ - "run", "tools.ts", - "--transport", "grpc", - "--hot-reload", "immediate", - "--call-timeout", "30s", - "--log-level", "debug", - "--socket", "/tmp/test.sock", - "--runtime", "python3.12", - }) - if err != nil { - t.Fatalf("Parse failed: %v", err) - } - if cfg.Command != "run" { - t.Errorf("Command = %q, want %q", cfg.Command, "run") - } - if cfg.File != "tools.ts" { - t.Errorf("File = %q, want %q", cfg.File, "tools.ts") - } - if cfg.Transport != "grpc" { - t.Errorf("Transport = %q, want %q", cfg.Transport, "grpc") - } - if !cfg.HotReloadImmediate { - t.Error("HotReloadImmediate should be true") - } - if cfg.CallTimeout != 30*time.Second { - t.Errorf("CallTimeout = %v, want %v", cfg.CallTimeout, 30*time.Second) - } - if cfg.SocketPath != "/tmp/test.sock" { - t.Errorf("SocketPath = %q, want %q", cfg.SocketPath, "/tmp/test.sock") - } - if cfg.Runtime != "python3.12" { - t.Errorf("Runtime = %q, want %q", cfg.Runtime, "python3.12") - } -} - -func TestParseMissingFile(t *testing.T) { - _, err := config.Parse([]string{"dev"}) - if err == nil { - t.Error("expected error for missing file argument") - } -} - -func TestParseInvalidCommand(t *testing.T) { - _, err := config.Parse([]string{"foo", "server.py"}) - if err == nil { - t.Error("expected error for invalid command") - } -} - -func TestRuntimeCommand(t *testing.T) { - tests := []struct { - file string - wantCmd string - wantArgs []string - }{ - {"server.py", "python3", []string{"server.py"}}, - {"server.ts", "npx", []string{"tsx", "server.ts"}}, - {"server.js", "node", []string{"server.js"}}, - {"server.go", "go", []string{"run", "server.go"}}, - {"server.rs", "cargo", []string{"run", "server.rs"}}, - {"server", "server", nil}, - } - for _, tt := range tests { - cmd, args := config.RuntimeCommand(tt.file) - if cmd != tt.wantCmd { - t.Errorf("RuntimeCommand(%q) cmd = %q, want %q", tt.file, cmd, tt.wantCmd) - } - if !reflect.DeepEqual(args, tt.wantArgs) { - t.Errorf("RuntimeCommand(%q) args = %v, want %v", tt.file, args, tt.wantArgs) - } - } -} -``` - -- [ ] **Step 2: Run tests to verify they fail** - -```bash -go test ./internal/config/... -v -``` - -- [ ] **Step 3: Implement config** - -Create `internal/config/config.go`: - -```go -package config - -import ( - "fmt" - "os" - "path/filepath" - "time" -) - -type Config struct { - Command string - File string - Transport string - HotReloadImmediate bool - CallTimeout time.Duration - LogLevel string - SocketPath string - Runtime string - Host string // For network transports (default: "localhost") - Port int // For network transports (default: 8080) -} - -func Parse(args []string) (*Config, error) { - if len(args) < 1 { - return nil, fmt.Errorf("usage: protomcp [flags]") - } - - cmd := args[0] - if cmd != "dev" && cmd != "run" { - return nil, fmt.Errorf("unknown command %q: must be 'dev' or 'run'", cmd) - } - - cfg := &Config{ - Command: cmd, - Transport: "stdio", - CallTimeout: 5 * time.Minute, - LogLevel: "info", - Host: "localhost", - Port: 8080, - } - - // Find file argument (first non-flag after command) - i := 1 - if i >= len(args) || args[i] == "--" { - return nil, fmt.Errorf("missing file argument") - } - if args[i][0] != '-' { - cfg.File = args[i] - i++ - } else { - return nil, fmt.Errorf("missing file argument") - } - - // Parse flags - for i < len(args) { - switch args[i] { - case "--transport": - i++ - if i >= len(args) { - return nil, fmt.Errorf("--transport requires a value") - } - cfg.Transport = args[i] - case "--hot-reload": - i++ - if i >= len(args) { - return nil, fmt.Errorf("--hot-reload requires a value") - } - if args[i] == "immediate" { - cfg.HotReloadImmediate = true - } - case "--call-timeout": - i++ - if i >= len(args) { - return nil, fmt.Errorf("--call-timeout requires a value") - } - d, err := time.ParseDuration(args[i]) - if err != nil { - return nil, fmt.Errorf("invalid --call-timeout: %w", err) - } - cfg.CallTimeout = d - case "--log-level": - i++ - if i >= len(args) { - return nil, fmt.Errorf("--log-level requires a value") - } - cfg.LogLevel = args[i] - case "--socket": - i++ - if i >= len(args) { - return nil, fmt.Errorf("--socket requires a value") - } - cfg.SocketPath = args[i] - case "--runtime": - i++ - if i >= len(args) { - return nil, fmt.Errorf("--runtime requires a value") - } - cfg.Runtime = args[i] - case "--host": - i++ - if i >= len(args) { - return nil, fmt.Errorf("--host requires a value") - } - cfg.Host = args[i] - case "--port": - i++ - if i >= len(args) { - return nil, fmt.Errorf("--port requires a value") - } - p, err := strconv.Atoi(args[i]) - if err != nil { - return nil, fmt.Errorf("invalid --port: %w", err) - } - cfg.Port = p - default: - return nil, fmt.Errorf("unknown flag %q", args[i]) - } - i++ - } - - // Default socket path - if cfg.SocketPath == "" { - dir := os.Getenv("XDG_RUNTIME_DIR") - if dir == "" { - dir = os.TempDir() - } - cfg.SocketPath = filepath.Join(dir, "protomcp", fmt.Sprintf("%d.sock", os.Getpid())) - } - - return cfg, nil -} - -// RuntimeCommand returns the command and args to run a tool file. -// Returns (command, args) to be used with exec.Command(command, args...). -func RuntimeCommand(file string) (string, []string) { - ext := filepath.Ext(file) - switch ext { - case ".py": - cmd := "python3" - if env := os.Getenv("PROTOMCP_PYTHON"); env != "" { - cmd = env - } - return cmd, []string{file} - case ".ts": - cmd := "npx" - if env := os.Getenv("PROTOMCP_NODE"); env != "" { - return env, []string{file} - } - return cmd, []string{"tsx", file} - case ".js": - cmd := "node" - if env := os.Getenv("PROTOMCP_NODE"); env != "" { - cmd = env - } - return cmd, []string{file} - case ".go": - return "go", []string{"run", file} - case ".rs": - return "cargo", []string{"run", file} - default: - return file, nil // Treat as executable binary - } -} -``` - -- [ ] **Step 4: Run tests to verify they pass** - -```bash -go test ./internal/config/... -v -``` - -- [ ] **Step 5: Commit** - -```bash -git add internal/config/ -git commit -m "feat: implement CLI config parsing with flag defaults and runtime detection" -``` - ---- - -### Task 6: Process Manager — Spawn, Handshake, Crash Recovery - -**Files:** -- Create: `internal/process/manager.go` -- Create: `internal/process/manager_test.go` - -- [ ] **Step 1: Write failing tests for process manager** - -Create `internal/process/manager_test.go`. Tests use a mock tool process (a small Go program compiled as a test helper) that speaks the protobuf protocol over the unix socket: - -```go -package process_test - -import ( - "context" - "os" - "path/filepath" - "testing" - "time" - - "github.com/msilverblatt/protomcp/internal/process" - pb "github.com/msilverblatt/protomcp/gen/proto/protomcp" -) - -// TestStartAndHandshake verifies the process manager can spawn a tool process, -// perform the handshake, and retrieve the tool list. -func TestStartAndHandshake(t *testing.T) { - socketPath := filepath.Join(t.TempDir(), "test.sock") - - pm := process.NewManager(process.ManagerConfig{ - File: "testdata/echo_tool.py", - Runtime: "python3", - SocketPath: socketPath, - MaxRetries: 1, - }) - - ctx, cancel := context.WithTimeout(context.Background(), 10*time.Second) - defer cancel() - - tools, err := pm.Start(ctx) - if err != nil { - t.Fatalf("Start failed: %v", err) - } - defer pm.Stop() - - if len(tools) == 0 { - t.Fatal("expected at least one tool from handshake") - } -} - -// TestCallTool verifies tool call dispatch and response. -func TestCallTool(t *testing.T) { - socketPath := filepath.Join(t.TempDir(), "test.sock") - - pm := process.NewManager(process.ManagerConfig{ - File: "testdata/echo_tool.py", - Runtime: "python3", - SocketPath: socketPath, - MaxRetries: 1, - }) - - ctx, cancel := context.WithTimeout(context.Background(), 10*time.Second) - defer cancel() - - _, err := pm.Start(ctx) - if err != nil { - t.Fatalf("Start failed: %v", err) - } - defer pm.Stop() - - resp, err := pm.CallTool(ctx, "echo", `{"message": "hello"}`) - if err != nil { - t.Fatalf("CallTool failed: %v", err) - } - if resp.IsError { - t.Errorf("unexpected error: %s", resp.ResultJson) - } -} - -// TestReload verifies the reload cycle. -func TestReload(t *testing.T) { - socketPath := filepath.Join(t.TempDir(), "test.sock") - - pm := process.NewManager(process.ManagerConfig{ - File: "testdata/echo_tool.py", - Runtime: "python3", - SocketPath: socketPath, - MaxRetries: 1, - }) - - ctx, cancel := context.WithTimeout(context.Background(), 10*time.Second) - defer cancel() - - _, err := pm.Start(ctx) - if err != nil { - t.Fatalf("Start failed: %v", err) - } - defer pm.Stop() - - tools, err := pm.Reload(ctx) - if err != nil { - t.Fatalf("Reload failed: %v", err) - } - if len(tools) == 0 { - t.Fatal("expected tools after reload") - } -} -``` - -Note: `testdata/echo_tool.py` is a minimal Python tool process that speaks the protobuf protocol. It will be created as part of the Python SDK task but a simple test fixture version is needed here. Create `internal/process/testdata/echo_tool.py` — a minimal script that: -1. Reads `PROTOMCP_SOCKET` env var -2. Connects to the unix socket -3. Responds to `ListToolsRequest` with one tool ("echo") -4. Responds to `CallToolRequest` by echoing the args back -5. Responds to `ReloadRequest` with success - -- [ ] **Step 2: Implement process manager** - -Create `internal/process/manager.go`: - -The process manager must: -1. Create the unix socket and listen -2. Spawn the tool process with `PROTOMCP_SOCKET` env var -3. Accept the connection from the tool process -4. Send `ListToolsRequest`, receive `ToolListResponse` (handshake) -5. Handle `CallTool` by sending `CallToolRequest` and waiting for `CallToolResponse` (with request_id correlation) -6. Handle `Reload` by sending `ReloadRequest`, waiting for `ReloadResponse`, then re-fetching tools -7. Monitor the child process — on unexpected exit, attempt restart with exponential backoff (up to MaxRetries) -8. Handle concurrent calls using request_id for correlation -9. Provide a `Stop()` method for clean shutdown - -Key interfaces: -```go -type ManagerConfig struct { - File string - Runtime string // e.g., "python3" — if empty, detect from extension - SocketPath string - MaxRetries int // default 3 - CallTimeout time.Duration // default 5m -} - -type Manager struct { /* ... */ } - -func NewManager(cfg ManagerConfig) *Manager -func (m *Manager) Start(ctx context.Context) ([]*pb.ToolDefinition, error) -func (m *Manager) Stop() error -func (m *Manager) CallTool(ctx context.Context, name string, argsJSON string) (*pb.CallToolResponse, error) -func (m *Manager) Reload(ctx context.Context) ([]*pb.ToolDefinition, error) -func (m *Manager) HandleToolListControl(ctx context.Context, env *pb.Envelope) (*pb.ActiveToolsResponse, error) -func (m *Manager) OnCrash() <-chan error // channel that receives crash events -``` - -- [ ] **Step 3: Create test fixture echo_tool.py** - -Create `internal/process/testdata/echo_tool.py` — a minimal Python script that speaks the protobuf envelope protocol. This is a standalone test fixture (not the SDK). It: -- Reads `PROTOMCP_SOCKET` from env -- Connects to the unix socket -- Reads length-prefixed envelopes -- Responds to `ListToolsRequest` with one tool: `echo(message: str) -> str` -- Responds to `CallToolRequest` by echoing args -- Responds to `ReloadRequest` with `success=true` - -This requires the generated Python protobuf code. For the test fixture, use raw protobuf encoding with the `protobuf` pip package and the generated `protomcp_pb2.py`. - -- [ ] **Step 4: Run tests to verify they pass** - -```bash -go test ./internal/process/... -v -timeout 30s -``` - -- [ ] **Step 5: Commit** - -```bash -git add internal/process/ -git commit -m "feat: implement process manager with spawn, handshake, call dispatch, and crash recovery" -``` - ---- - -### Task 7: MCP Protocol Types and Handler - -**Files:** -- Create: `internal/mcp/types.go` -- Create: `internal/mcp/handler.go` -- Create: `internal/mcp/handler_test.go` - -- [ ] **Step 1: Write MCP JSON-RPC types** - -Create `internal/mcp/types.go` with types for: -- `JSONRPCRequest` / `JSONRPCResponse` / `JSONRPCNotification` (generic JSON-RPC 2.0) -- `InitializeRequest` / `InitializeResponse` (MCP initialization with capabilities) -- `ToolsListRequest` / `ToolsListResponse` (tools/list) -- `ToolsCallRequest` / `ToolsCallResponse` (tools/call) -- `ToolsListChangedNotification` (notifications/tools/list_changed) -- MCP capability structs (declaring `tools.listChanged: true`) - -These are standard MCP protocol types serialized as JSON, not protobuf. - -- [ ] **Step 2: Write failing tests for MCP handler** - -Create `internal/mcp/handler_test.go`: - -```go -package mcp_test - -import ( - "context" - "encoding/json" - "testing" - - "github.com/msilverblatt/protomcp/internal/mcp" - pb "github.com/msilverblatt/protomcp/gen/proto/protomcp" -) - -// mockToolBackend implements the interface the handler uses to talk to the tool process -type mockToolBackend struct { - tools []*pb.ToolDefinition -} - -func (m *mockToolBackend) ActiveTools() []*pb.ToolDefinition { - return m.tools -} - -func (m *mockToolBackend) CallTool(ctx context.Context, name, argsJSON string) (*pb.CallToolResponse, error) { - return &pb.CallToolResponse{ - ResultJson: `[{"type":"text","text":"result"}]`, - }, nil -} - -func TestHandleInitialize(t *testing.T) { - h := mcp.NewHandler(&mockToolBackend{}) - - req := mcp.JSONRPCRequest{ - JSONRPC: "2.0", - ID: json.RawMessage(`1`), - Method: "initialize", - } - - resp, err := h.Handle(context.Background(), req) - if err != nil { - t.Fatalf("Handle initialize failed: %v", err) - } - - // Should return capabilities with tools.listChanged = true - var result mcp.InitializeResult - if err := json.Unmarshal(resp.Result, &result); err != nil { - t.Fatalf("unmarshal result: %v", err) - } - if !result.Capabilities.Tools.ListChanged { - t.Error("expected tools.listChanged = true") - } -} - -func TestHandleToolsList(t *testing.T) { - backend := &mockToolBackend{ - tools: []*pb.ToolDefinition{ - {Name: "search", Description: "Search docs", InputSchemaJson: `{"type":"object","properties":{"query":{"type":"string"}}}`}, - }, - } - h := mcp.NewHandler(backend) - - req := mcp.JSONRPCRequest{ - JSONRPC: "2.0", - ID: json.RawMessage(`2`), - Method: "tools/list", - } - - resp, err := h.Handle(context.Background(), req) - if err != nil { - t.Fatalf("Handle tools/list failed: %v", err) - } - - var result mcp.ToolsListResult - if err := json.Unmarshal(resp.Result, &result); err != nil { - t.Fatalf("unmarshal result: %v", err) - } - if len(result.Tools) != 1 { - t.Fatalf("expected 1 tool, got %d", len(result.Tools)) - } - if result.Tools[0].Name != "search" { - t.Errorf("tool name = %q, want %q", result.Tools[0].Name, "search") - } -} - -func TestHandleToolsCall(t *testing.T) { - backend := &mockToolBackend{ - tools: []*pb.ToolDefinition{ - {Name: "search", Description: "Search docs", InputSchemaJson: `{}`}, - }, - } - h := mcp.NewHandler(backend) - - params, _ := json.Marshal(map[string]interface{}{ - "name": "search", - "arguments": map[string]string{"query": "hello"}, - }) - req := mcp.JSONRPCRequest{ - JSONRPC: "2.0", - ID: json.RawMessage(`3`), - Method: "tools/call", - Params: params, - } - - resp, err := h.Handle(context.Background(), req) - if err != nil { - t.Fatalf("Handle tools/call failed: %v", err) - } - - if resp.Error != nil { - t.Errorf("unexpected error: %v", resp.Error) - } -} -``` - -- [ ] **Step 3: Implement MCP handler** - -Create `internal/mcp/handler.go`: - -The handler: -- Implements `Handle(ctx, JSONRPCRequest) -> JSONRPCResponse` -- Routes by method: `initialize`, `tools/list`, `tools/call`, `notifications/initialized` -- For `initialize`: returns server info and capabilities (`tools.listChanged: true`) -- For `tools/list`: calls `backend.ActiveTools()` and formats as MCP response -- For `tools/call`: extracts tool name and args, calls `backend.CallTool()`, intercepts `enable_tools`/`disable_tools` from the response, and returns the MCP-formatted result -- Provides `ListChangedNotification() -> JSONRPCNotification` for the transport to send -- Unknown methods return JSON-RPC method-not-found error - -- [ ] **Step 4: Run tests to verify they pass** - -```bash -go test ./internal/mcp/... -v -``` - -- [ ] **Step 5: Commit** - -```bash -git add internal/mcp/ -git commit -m "feat: implement MCP JSON-RPC handler with initialize, tools/list, and tools/call" -``` - ---- - -### Task 8: Middleware — Logging and Error Handling - -**Files:** -- Create: `internal/middleware/chain.go` -- Create: `internal/middleware/logging.go` -- Create: `internal/middleware/errors.go` - -- [ ] **Step 1: Implement middleware chain** - -Create `internal/middleware/chain.go`: - -```go -package middleware - -import ( - "context" - - "github.com/msilverblatt/protomcp/internal/mcp" -) - -// Handler processes an MCP request and returns a response. -type Handler func(ctx context.Context, req mcp.JSONRPCRequest) (*mcp.JSONRPCResponse, error) - -// Middleware wraps a Handler with additional behavior. -type Middleware func(next Handler) Handler - -// Chain applies middleware in order (first middleware is outermost). -func Chain(handler Handler, middlewares ...Middleware) Handler { - for i := len(middlewares) - 1; i >= 0; i-- { - handler = middlewares[i](handler) - } - return handler -} -``` - -- [ ] **Step 2: Implement logging middleware** - -Create `internal/middleware/logging.go`: - -Logs each request (method, request ID) and response (duration, error status). Uses `log/slog` for structured logging. Log level controlled by config. - -- [ ] **Step 3: Implement error formatting middleware** - -Create `internal/middleware/errors.go`: - -Catches panics and unhandled errors from the handler, formats them as agent-friendly structured errors with `error_code`, `message`, `suggestion`, and `retryable` fields. - -- [ ] **Step 4: Commit** - -```bash -git add internal/middleware/ -git commit -m "feat: implement middleware chain with structured logging and error formatting" -``` - ---- - -## Chunk 3: Transports, File Watcher, CLI Entry Point - -### Task 9: Transport Interface and stdio Transport - -**Files:** -- Create: `internal/transport/transport.go` -- Create: `internal/transport/stdio.go` -- Create: `internal/transport/stdio_test.go` - -- [ ] **Step 1: Define transport interface** - -Create `internal/transport/transport.go`: - -```go -package transport - -import ( - "context" - - "github.com/msilverblatt/protomcp/internal/mcp" -) - -// Transport handles bidirectional MCP communication with the host/client. -type Transport interface { - // Start begins listening for MCP requests. Calls handler for each request. - // Blocks until ctx is cancelled or an error occurs. - Start(ctx context.Context, handler mcp.RequestHandler) error - - // SendNotification pushes a server-initiated notification to the client. - SendNotification(notification mcp.JSONRPCNotification) error - - // Close shuts down the transport. - Close() error -} - -// mcp.RequestHandler is: func(ctx context.Context, req JSONRPCRequest) (*JSONRPCResponse, error) -``` - -- [ ] **Step 2: Write failing tests for stdio transport** - -Create `internal/transport/stdio_test.go`: - -Test that the stdio transport reads JSON-RPC requests from stdin, passes them to the handler, and writes responses to stdout. Use `io.Pipe` to simulate stdin/stdout. - -- [ ] **Step 3: Implement stdio transport** - -Create `internal/transport/stdio.go`: - -Reads newline-delimited JSON-RPC messages from stdin, dispatches to handler, writes JSON-RPC responses to stdout. Handles `notifications/initialized` (no response needed). - -- [ ] **Step 4: Run tests** - -```bash -go test ./internal/transport/... -run TestStdio -v -``` - -- [ ] **Step 5: Commit** - -```bash -git add internal/transport/transport.go internal/transport/stdio.go internal/transport/stdio_test.go -git commit -m "feat: implement transport interface and stdio transport" -``` - ---- - -### Task 10: Streamable HTTP Transport - -**Files:** -- Create: `internal/transport/http.go` -- Create: `internal/transport/http_test.go` - -- [ ] **Step 1: Write failing tests** - -Test that: -- POST to `/mcp` with a JSON-RPC request returns a JSON-RPC response -- GET to `/mcp` opens an SSE stream for server-initiated notifications -- `SendNotification` pushes events to connected SSE clients - -- [ ] **Step 2: Implement HTTP transport** - -Uses `net/http`. Single endpoint `/mcp`. POST for requests, GET for SSE stream. Maintains a list of SSE clients for notifications. - -- [ ] **Step 3: Run tests and commit** - -```bash -go test ./internal/transport/... -run TestHTTP -v -git add internal/transport/http.go internal/transport/http_test.go -git commit -m "feat: implement streamable HTTP transport with SSE notifications" -``` - ---- - -### Task 11: SSE Transport - -**Files:** -- Create: `internal/transport/sse.go` -- Create: `internal/transport/sse_test.go` - -- [ ] **Step 1: Implement SSE transport** - -Legacy SSE transport per MCP spec: separate SSE endpoint for server-to-client, POST endpoint for client-to-server. Largely reuses HTTP transport internals. - -- [ ] **Step 2: Test and commit** - -```bash -go test ./internal/transport/... -run TestSSE -v -git add internal/transport/sse.go internal/transport/sse_test.go -git commit -m "feat: implement legacy SSE transport" -``` - ---- - -### Task 12: WebSocket Transport - -**Files:** -- Create: `internal/transport/ws.go` -- Create: `internal/transport/ws_test.go` - -- [ ] **Step 1: Implement WebSocket transport** - -Uses `nhooyr.io/websocket` (or `gorilla/websocket`). Single WebSocket connection for bidirectional JSON-RPC. Notifications are pushed as WebSocket messages. - -- [ ] **Step 2: Test and commit** - -```bash -go test ./internal/transport/... -run TestWS -v -git add internal/transport/ws.go internal/transport/ws_test.go -git commit -m "feat: implement WebSocket transport" -``` - ---- - -### Task 13: gRPC Transport - -**Files:** -- Create: `internal/transport/grpc.go` -- Create: `internal/transport/grpc_test.go` -- Modify: `proto/protomcp.proto` (add external gRPC service definition) - -- [ ] **Step 1: Add external gRPC service to proto** - -Add a separate service definition in `proto/protomcp.proto` for the external gRPC transport (client-facing, separate from the internal envelope protocol): - -```protobuf -// External gRPC service — client-facing MCP transport -service MCPService { - rpc Request(MCPRequest) returns (MCPResponse); - rpc Subscribe(MCPSubscribeRequest) returns (stream MCPNotification); -} - -message MCPRequest { - string jsonrpc_request = 1; // JSON-encoded MCP JSON-RPC request -} -message MCPResponse { - string jsonrpc_response = 1; // JSON-encoded MCP JSON-RPC response -} -message MCPSubscribeRequest {} -message MCPNotification { - string jsonrpc_notification = 1; // JSON-encoded notification -} -``` - -- [ ] **Step 2: Regenerate proto and implement gRPC transport** - -```bash -make proto -``` - -Implement gRPC server that wraps JSON-RPC messages in protobuf, dispatches to handler, and streams notifications. - -- [ ] **Step 3: Test and commit** - -```bash -go test ./internal/transport/... -run TestGRPC -v -git add internal/transport/grpc.go internal/transport/grpc_test.go proto/protomcp.proto -git commit -m "feat: implement gRPC transport with external service definition" -``` - ---- - -### Task 14: File Watcher — Reload Orchestration - -**Files:** -- Create: `internal/reload/watcher.go` -- Create: `internal/reload/watcher_test.go` - -- [ ] **Step 1: Write failing tests** - -Test that: -- Watcher detects file modifications and calls the reload callback -- Watcher debounces rapid changes (only one reload per debounce window) -- Watcher supports both single file and directory watching -- Watcher filters by relevant file extensions - -- [ ] **Step 2: Implement file watcher** - -Create `internal/reload/watcher.go`: - -Uses `github.com/fsnotify/fsnotify`. Key behavior: -- Watches the specified file or directory -- On change, debounces (100ms) then calls the reload callback -- Reload callback signature: `func() error` -- The watcher does NOT do the reload itself — it signals the main server loop -- In `run` mode (production), the watcher is never started - -```go -type Watcher struct { /* ... */ } - -func NewWatcher(path string, extensions []string, onChange func()) (*Watcher, error) -func (w *Watcher) Start(ctx context.Context) error -func (w *Watcher) Stop() error -``` - -- [ ] **Step 3: Test and commit** - -```bash -go test ./internal/reload/... -v -git add internal/reload/ -git commit -m "feat: implement file watcher with debouncing and extension filtering" -``` - ---- - -### Task 15: CLI Entry Point — Wiring It All Together - -**Files:** -- Create: `cmd/protomcp/main.go` - -- [ ] **Step 1: Implement main.go** - -Create `cmd/protomcp/main.go` that wires everything together: - -```go -package main - -import ( - "context" - "fmt" - "log/slog" - "os" - "os/signal" - - "github.com/msilverblatt/protomcp/internal/config" - "github.com/msilverblatt/protomcp/internal/mcp" - "github.com/msilverblatt/protomcp/internal/middleware" - "github.com/msilverblatt/protomcp/internal/process" - "github.com/msilverblatt/protomcp/internal/reload" - "github.com/msilverblatt/protomcp/internal/toollist" - "github.com/msilverblatt/protomcp/internal/transport" -) - -func main() { - cfg, err := config.Parse(os.Args[1:]) - if err != nil { - fmt.Fprintf(os.Stderr, "error: %v\n", err) - fmt.Fprintf(os.Stderr, "usage: protomcp [flags]\n") - os.Exit(1) - } - - // Setup structured logging - logLevel := parseLogLevel(cfg.LogLevel) - logger := slog.New(slog.NewTextHandler(os.Stderr, &slog.HandlerOptions{Level: logLevel})) - slog.SetDefault(logger) - - ctx, cancel := signal.NotifyContext(context.Background(), os.Interrupt) - defer cancel() - - // 1. Create tool list manager - tlm := toollist.New() - - // 2. Determine runtime command - var runtimeCmd string - var runtimeArgs []string - if cfg.Runtime != "" { - runtimeCmd = cfg.Runtime - runtimeArgs = []string{cfg.File} - } else { - runtimeCmd, runtimeArgs = config.RuntimeCommand(cfg.File) - } - - // 3. Start process manager - pm := process.NewManager(process.ManagerConfig{ - File: cfg.File, - RuntimeCmd: runtimeCmd, - RuntimeArgs: runtimeArgs, - SocketPath: cfg.SocketPath, - MaxRetries: 3, - CallTimeout: cfg.CallTimeout, - }) - - tools, err := pm.Start(ctx) - if err != nil { - slog.Error("failed to start tool process", "error", err) - os.Exit(1) - } - - // Register tools and log them - toolNames := make([]string, len(tools)) - for i, t := range tools { - toolNames[i] = t.Name - slog.Info("tool registered", "name", t.Name, "description", t.Description) - } - tlm.SetRegistered(toolNames) - - // 4. Create MCP handler with tool backend - backend := newToolBackend(pm, tlm, tools) - handler := mcp.NewHandler(backend) - - // 5. Apply middleware - chain := middleware.Chain( - handler.Handle, - middleware.Logging(logger), - middleware.ErrorFormatting(), - ) - - // 6. Create transport - tp, err := createTransport(cfg.Transport) - if err != nil { - slog.Error("failed to create transport", "error", err) - os.Exit(1) - } - - // 7. Handle tool list control messages from tool process - go handleToolListControl(ctx, pm, tlm, tp) - - // 8. Handle crash recovery - go handleCrashRecovery(ctx, pm, tlm, tools, tp) - - // 9. Start file watcher (dev mode only) - if cfg.Command == "dev" { - w, err := reload.NewWatcher(cfg.File, nil, func() { - // Wait for in-flight calls unless immediate mode - if !cfg.HotReloadImmediate { - pm.WaitForInFlight() - } - newTools, err := pm.Reload(ctx) - if err != nil { - slog.Error("reload failed", "error", err) - return - } - // Update tool list - newNames := make([]string, len(newTools)) - for i, t := range newTools { - newNames[i] = t.Name - } - oldActive := tlm.Active() - tlm.SetRegistered(newNames) - backend.UpdateTools(newTools) - newActive := tlm.Active() - if !slicesEqual(oldActive, newActive) { - slog.Info("tool list changed", "added", diff(newActive, oldActive), "removed", diff(oldActive, newActive)) - tp.SendNotification(mcp.ListChangedNotification()) - } - }) - if err != nil { - slog.Error("failed to create file watcher", "error", err) - os.Exit(1) - } - go w.Start(ctx) - defer w.Stop() - } - - // 10. Start transport (blocks until ctx cancelled) - slog.Info("protomcp started", "command", cfg.Command, "file", cfg.File, "transport", cfg.Transport) - if err := tp.Start(ctx, chain); err != nil { - slog.Error("transport error", "error", err) - os.Exit(1) - } -} -``` - -Note: `createTransport()`, `newToolBackend()`, `handleToolListControl()`, `handleCrashRecovery()`, `parseLogLevel()`, `slicesEqual()`, `diff()` are helper functions in the same file or a `helpers.go` file. Keep it focused — these are thin wiring functions, not business logic. - -- [ ] **Step 2: Build and verify it compiles** - -```bash -make build -``` - -Expected: `bin/protomcp` binary created. - -- [ ] **Step 3: Smoke test with a Python fixture** - -```bash -echo '{"jsonrpc":"2.0","id":1,"method":"initialize","params":{}}' | bin/protomcp dev internal/process/testdata/echo_tool.py -``` - -Expected: JSON-RPC response with capabilities. - -- [ ] **Step 4: Commit** - -```bash -git add cmd/protomcp/ -git commit -m "feat: implement CLI entry point wiring all components together" -``` - ---- - -## Chunk 4: Python SDK - -### Task 16: Python Project Setup - -**Files:** -- Create: `sdk/python/pyproject.toml` -- Create: `sdk/python/src/protomcp/__init__.py` - -- [ ] **Step 1: Create pyproject.toml** - -```toml -[build-system] -requires = ["hatchling"] -build-backend = "hatchling.build" - -[project] -name = "protomcp" -version = "0.1.0" -description = "Write MCP tools in Python. No MCP knowledge required." -requires-python = ">=3.10" -dependencies = [ - "protobuf>=4.0", -] - -[project.optional-dependencies] -dev = [ - "pytest>=7.0", - "pytest-asyncio>=0.21", -] - -[tool.hatch.build.targets.wheel] -packages = ["src/protomcp"] -``` - -- [ ] **Step 2: Create __init__.py with public API** - -```python -from protomcp.tool import tool -from protomcp.result import ToolResult -from protomcp import manager as tool_manager - -__all__ = ["tool", "ToolResult", "tool_manager"] -``` - -- [ ] **Step 3: Generate Python protobuf code** - -```bash -make proto -``` - -Verify: `sdk/python/gen/protomcp_pb2.py` exists. - -- [ ] **Step 4: Commit** - -```bash -git add sdk/python/pyproject.toml sdk/python/src/protomcp/__init__.py -git commit -m "feat: initialize Python SDK project" -``` - ---- - -### Task 17: Python — @tool Decorator and Schema Generation - -**Files:** -- Create: `sdk/python/src/protomcp/tool.py` -- Create: `sdk/python/tests/test_tool.py` - -- [ ] **Step 1: Write failing tests** - -Create `sdk/python/tests/test_tool.py`: - -```python -import json -from protomcp.tool import tool, get_registered_tools - - -def test_tool_decorator_registers(): - """@tool() should register the function as a tool.""" - clear_registry() - @tool(description="Add two numbers") - def add(a: int, b: int) -> int: - return a + b - - tools = get_registered_tools() - assert any(t.name == "add" for t in tools) - - -def test_tool_schema_from_type_hints(): - """Schema should be generated from Python type annotations.""" - clear_registry() - @tool(description="Search documents") - def search(query: str, limit: int = 10) -> list: - return [] - - tools = get_registered_tools() - t = next(t for t in tools if t.name == "search") - schema = json.loads(t.input_schema_json) - - assert schema["type"] == "object" - assert "query" in schema["properties"] - assert schema["properties"]["query"]["type"] == "string" - assert "limit" in schema["properties"] - assert schema["properties"]["limit"]["type"] == "integer" - assert schema["properties"]["limit"]["default"] == 10 - assert "query" in schema["required"] - assert "limit" not in schema["required"] # has default - - -def test_tool_callable(): - """Decorated function should still be callable normally.""" - clear_registry() - @tool(description="Double a number") - def double(n: int) -> int: - return n * 2 - - assert double(5) == 10 - - -def test_tool_optional_params(): - """Optional params should not be in required.""" - clear_registry() - from typing import Optional - - @tool(description="Greet") - def greet(name: str, greeting: Optional[str] = None) -> str: - return f"{greeting or 'Hello'}, {name}!" - - tools = get_registered_tools() - t = next(t for t in tools if t.name == "greet") - schema = json.loads(t.input_schema_json) - - assert "name" in schema["required"] - assert "greeting" not in schema["required"] - -def test_tool_optional_without_default(): - """Optional[X] without a default should NOT be required.""" - clear_registry() - from typing import Optional - - @tool(description="Maybe filter") - def search(query: str, tag: Optional[str] = None) -> list: - return [] - - tools = get_registered_tools() - t = next(t for t in tools if t.name == "search") - schema = json.loads(t.input_schema_json) - assert "tag" not in schema.get("required", []) -``` - -- [ ] **Step 2: Run tests to verify they fail** - -```bash -cd sdk/python && python -m pytest tests/test_tool.py -v -``` - -- [ ] **Step 3: Implement @tool decorator** - -Create `sdk/python/src/protomcp/tool.py`: - -```python -import inspect -import json -from dataclasses import dataclass -from typing import Any, Callable, Optional, get_type_hints - -# Global registry of tools -_registry: list["ToolDef"] = [] - - -@dataclass -class ToolDef: - name: str - description: str - input_schema_json: str - handler: Callable - - -def tool(description: str): - """Decorator that registers a function as an MCP tool. - - Schema is generated automatically from Python type annotations. - """ - def decorator(func: Callable) -> Callable: - schema = _generate_schema(func) - _registry.append(ToolDef( - name=func.__name__, - description=description, - input_schema_json=json.dumps(schema), - handler=func, - )) - return func - return decorator - - -def get_registered_tools() -> list[ToolDef]: - return list(_registry) - - -def clear_registry(): - _registry.clear() - - -_PYTHON_TYPE_TO_JSON_SCHEMA = { - str: "string", - int: "integer", - float: "number", - bool: "boolean", - list: "array", - dict: "object", -} - - -def _generate_schema(func: Callable) -> dict: - """Generate JSON Schema from function signature and type hints.""" - hints = get_type_hints(func) - sig = inspect.signature(func) - - properties = {} - required = [] - - for name, param in sig.parameters.items(): - if name in ("self", "cls", "ctx"): - continue - - hint = hints.get(name, Any) - # Skip ToolContext parameters - if hint.__name__ == "ToolContext" if hasattr(hint, "__name__") else False: - continue - json_type = _python_type_to_json(hint) - prop: dict[str, Any] = {"type": json_type} - - if param.default is not inspect.Parameter.empty: - prop["default"] = param.default - elif not _is_optional_type(hint): - required.append(name) - - properties[name] = prop - - schema: dict[str, Any] = { - "type": "object", - "properties": properties, - } - if required: - schema["required"] = required - - return schema - - -def _is_optional_type(hint) -> bool: - """Check if a type hint is Optional[X] (Union[X, None]).""" - import typing - origin = getattr(hint, "__origin__", None) - if origin is typing.Union: - return type(None) in hint.__args__ - return False - - -def _python_type_to_json(hint) -> str: - """Convert a Python type hint to a JSON Schema type string.""" - # Handle Optional[X] (Union[X, None]) - origin = getattr(hint, "__origin__", None) - if origin is type(None): - return "null" - - # Handle Optional (typing.Union with None) - import typing - if origin is typing.Union: - args = hint.__args__ - non_none = [a for a in args if a is not type(None)] - if len(non_none) == 1: - return _python_type_to_json(non_none[0]) - - return _PYTHON_TYPE_TO_JSON_SCHEMA.get(hint, "string") -``` - -- [ ] **Step 4: Run tests to verify they pass** - -```bash -cd sdk/python && python -m pytest tests/test_tool.py -v -``` - -- [ ] **Step 5: Commit** - -```bash -git add sdk/python/src/protomcp/tool.py sdk/python/tests/test_tool.py -git commit -m "feat: implement Python @tool decorator with automatic schema generation from type hints" -``` - ---- - -### Task 18: Python — ToolResult Type - -**Files:** -- Create: `sdk/python/src/protomcp/result.py` -- Create: `sdk/python/tests/test_result.py` - -- [ ] **Step 1: Write failing tests** - -```python -from protomcp.result import ToolResult - - -def test_tool_result_basic(): - r = ToolResult(result="success") - assert r.result == "success" - assert r.enable_tools is None - assert r.disable_tools is None - - -def test_tool_result_with_mutations(): - r = ToolResult( - result="connected", - enable_tools=["query_db"], - disable_tools=["connect_db"], - ) - assert r.enable_tools == ["query_db"] - assert r.disable_tools == ["connect_db"] - - -def test_tool_result_with_error(): - r = ToolResult( - is_error=True, - error_code="NOT_FOUND", - message="User not found", - suggestion="Try searching by email", - retryable=False, - ) - assert r.is_error is True - assert r.error_code == "NOT_FOUND" -``` - -- [ ] **Step 2: Implement ToolResult** - -```python -from dataclasses import dataclass, field -from typing import Optional - - -@dataclass -class ToolResult: - result: str = "" - is_error: bool = False - enable_tools: Optional[list[str]] = None - disable_tools: Optional[list[str]] = None - # Structured error fields - error_code: Optional[str] = None - message: Optional[str] = None - suggestion: Optional[str] = None - retryable: bool = False -``` - -- [ ] **Step 3: Test and commit** - -```bash -cd sdk/python && python -m pytest tests/test_result.py -v -git add sdk/python/src/protomcp/result.py sdk/python/tests/test_result.py -git commit -m "feat: implement Python ToolResult type with tool list mutations and structured errors" -``` - ---- - -### Task 19: Python — Transport Layer (Unix Socket + Envelope) - -**Files:** -- Create: `sdk/python/src/protomcp/transport.py` -- Create: `sdk/python/tests/test_transport.py` - -- [ ] **Step 1: Write failing tests** - -Test that `Transport` can: -- Connect to a unix socket -- Write a length-prefixed protobuf envelope -- Read a length-prefixed protobuf envelope -- Handle multiple sequential messages - -- [ ] **Step 2: Implement transport** - -```python -import socket -import struct -from protomcp.gen import protomcp_pb2 as pb - - -class Transport: - """Speaks length-prefixed protobuf envelopes over a unix socket.""" - - def __init__(self, socket_path: str): - self._socket_path = socket_path - self._sock: socket.socket | None = None - - def connect(self): - self._sock = socket.socket(socket.AF_UNIX, socket.SOCK_STREAM) - self._sock.connect(self._socket_path) - - def send(self, envelope: pb.Envelope): - data = envelope.SerializeToString() - length = struct.pack(">I", len(data)) - self._sock.sendall(length + data) - - def recv(self) -> pb.Envelope: - length_bytes = self._recv_exactly(4) - length = struct.unpack(">I", length_bytes)[0] - data = self._recv_exactly(length) - env = pb.Envelope() - env.ParseFromString(data) - return env - - def close(self): - if self._sock: - self._sock.close() - - def _recv_exactly(self, n: int) -> bytes: - buf = bytearray() - while len(buf) < n: - chunk = self._sock.recv(n - len(buf)) - if not chunk: - raise ConnectionError("socket closed") - buf.extend(chunk) - return bytes(buf) -``` - -- [ ] **Step 3: Test and commit** - -```bash -cd sdk/python && python -m pytest tests/test_transport.py -v -git add sdk/python/src/protomcp/transport.py sdk/python/tests/test_transport.py -git commit -m "feat: implement Python transport layer for unix socket + envelope protocol" -``` - ---- - -### Task 20: Python — tool_manager Client - -**Files:** -- Create: `sdk/python/src/protomcp/manager.py` -- Create: `sdk/python/tests/test_manager.py` - -- [ ] **Step 1: Write failing tests** - -Test that `tool_manager` methods send the correct protobuf messages over the transport and return the response. - -- [ ] **Step 2: Implement manager** - -```python -from protomcp.transport import Transport -from protomcp.gen import protomcp_pb2 as pb - -_transport: Transport | None = None - - -def _get_transport() -> Transport: - if _transport is None: - raise RuntimeError("protomcp not connected — are you running via 'protomcp dev'?") - return _transport - - -def _init(transport: Transport): - global _transport - _transport = transport - - -def enable(tool_names: list[str]) -> list[str]: - t = _get_transport() - env = pb.Envelope( - enable_tools=pb.EnableToolsRequest(tool_names=tool_names) - ) - t.send(env) - resp = t.recv() - return list(resp.active_tools.tool_names) - - -def disable(tool_names: list[str]) -> list[str]: - t = _get_transport() - env = pb.Envelope( - disable_tools=pb.DisableToolsRequest(tool_names=tool_names) - ) - t.send(env) - resp = t.recv() - return list(resp.active_tools.tool_names) - - -def set_allowed(tool_names: list[str]) -> list[str]: - t = _get_transport() - env = pb.Envelope( - set_allowed=pb.SetAllowedRequest(tool_names=tool_names) - ) - t.send(env) - resp = t.recv() - return list(resp.active_tools.tool_names) - - -def set_blocked(tool_names: list[str]) -> list[str]: - t = _get_transport() - env = pb.Envelope( - set_blocked=pb.SetBlockedRequest(tool_names=tool_names) - ) - t.send(env) - resp = t.recv() - return list(resp.active_tools.tool_names) - - -def get_active_tools() -> list[str]: - t = _get_transport() - env = pb.Envelope( - get_active_tools=pb.GetActiveToolsRequest() - ) - t.send(env) - resp = t.recv() - return list(resp.active_tools.tool_names) - - -def batch( - enable: list[str] | None = None, - disable: list[str] | None = None, - allow: list[str] | None = None, - block: list[str] | None = None, -) -> list[str]: - t = _get_transport() - env = pb.Envelope( - batch=pb.BatchUpdateRequest( - enable=enable or [], - disable=disable or [], - allow=allow or [], - block=block or [], - ) - ) - t.send(env) - resp = t.recv() - return list(resp.active_tools.tool_names) -``` - -- [ ] **Step 3: Test and commit** - -```bash -cd sdk/python && python -m pytest tests/test_manager.py -v -git add sdk/python/src/protomcp/manager.py sdk/python/tests/test_manager.py -git commit -m "feat: implement Python tool_manager client for dynamic tool list control" -``` - ---- - -### Task 21: Python — Runner (Main Loop) - -**Files:** -- Create: `sdk/python/src/protomcp/runner.py` - -- [ ] **Step 1: Implement runner** - -The runner is the main loop that the tool process runs. It: -1. Reads `PROTOMCP_SOCKET` from env -2. Connects to the unix socket via `Transport` -3. Initializes `tool_manager` with the transport -4. Listens for envelopes from the Go binary -5. Dispatches `ListToolsRequest` → returns registered tools from `tool.get_registered_tools()` -6. Dispatches `CallToolRequest` → finds the handler, calls it, returns `CallToolResponse` -7. Dispatches `ReloadRequest` → calls `importlib.reload()` on the user's module, clears and re-registers tools, returns `ReloadResponse` - -The user's tool file imports this runner at module level (or the SDK auto-runs it): - -```python -# At the bottom of a user's tool file, or auto-detected: -if __name__ == "__main__": - from protomcp.runner import run - run() -``` - -Actually, the runner should auto-start. When the tool process is spawned, the Python SDK needs to: -1. Import the user's module (which registers tools via `@tool()`) -2. Start the runner loop - -This is handled by the SDK's entry point. The Go binary runs: `python -c "import protomcp.runner; protomcp.runner.start('server')"` where `server` is the module path. Or simpler: the user's file calls `protomcp.run()` at the bottom. - -- [ ] **Step 2: Commit** - -```bash -git add sdk/python/src/protomcp/runner.py -git commit -m "feat: implement Python runner main loop with reload and tool dispatch" -``` - ---- - -## Chunk 5: TypeScript SDK - -### Task 22: TypeScript Project Setup - -**Files:** -- Create: `sdk/typescript/package.json` -- Create: `sdk/typescript/tsconfig.json` -- Create: `sdk/typescript/vitest.config.ts` -- Create: `sdk/typescript/src/index.ts` - -- [ ] **Step 1: Create package.json** - -```json -{ - "name": "protomcp", - "version": "0.1.0", - "description": "Write MCP tools in TypeScript. No MCP knowledge required.", - "type": "module", - "main": "dist/index.js", - "types": "dist/index.d.ts", - "scripts": { - "build": "tsc", - "test": "vitest run", - "test:watch": "vitest" - }, - "dependencies": { - "protobufjs": "^7.0.0", - "zod": "^3.22.0", - "zod-to-json-schema": "^3.22.0" - }, - "devDependencies": { - "typescript": "^5.3.0", - "vitest": "^1.0.0", - "@types/node": "^20.0.0" - } -} -``` - -- [ ] **Step 2: Create tsconfig.json, vitest.config.ts, index.ts** - -Standard TypeScript config with ESM output. Index exports `tool`, `ToolResult`, `toolManager`. - -- [ ] **Step 3: Install dependencies and commit** - -```bash -cd sdk/typescript && npm install -git add sdk/typescript/package.json sdk/typescript/tsconfig.json sdk/typescript/vitest.config.ts sdk/typescript/src/index.ts -git commit -m "feat: initialize TypeScript SDK project" -``` - ---- - -### Task 23: TypeScript — tool() Function with Zod Schema - -**Files:** -- Create: `sdk/typescript/src/tool.ts` -- Create: `sdk/typescript/tests/tool.test.ts` - -- [ ] **Step 1: Write failing tests** - -```typescript -import { describe, it, expect } from 'vitest'; -import { z } from 'zod'; -import { tool, getRegisteredTools, clearRegistry } from '../src/tool'; - -describe('tool()', () => { - beforeEach(() => clearRegistry()); - - it('registers a tool with Zod schema', () => { - const add = tool({ - name: 'add', - description: 'Add two numbers', - args: z.object({ - a: z.number().describe('First number'), - b: z.number().describe('Second number'), - }), - handler: (args) => args.a + args.b, - }); - - const tools = getRegisteredTools(); - expect(tools).toHaveLength(1); - expect(tools[0].name).toBe('add'); - expect(tools[0].description).toBe('Add two numbers'); - }); - - it('requires name for arrow function handlers', () => { - const t = tool({ - description: 'No name', - args: z.object({ x: z.number() }), - handler: (args) => args.x, - }); - // Arrow functions have empty .name, so falls back to tool_N - expect(t.name).toMatch(/^tool_\d+$/); - }); - - it('generates JSON Schema from Zod', () => { - tool({ - name: 'search', - description: 'Search', - args: z.object({ - query: z.string().describe('Search query'), - limit: z.number().default(10).describe('Max results'), - }), - handler: (args) => [], - }); - - const tools = getRegisteredTools(); - const schema = JSON.parse(tools[0].inputSchemaJson); - expect(schema.type).toBe('object'); - expect(schema.properties.query.type).toBe('string'); - expect(schema.properties.limit.default).toBe(10); - expect(schema.required).toContain('query'); - expect(schema.required).not.toContain('limit'); - }); - - it('handler is callable', () => { - const add = tool({ - description: 'Add', - args: z.object({ a: z.number(), b: z.number() }), - handler: (args) => args.a + args.b, - }); - - expect(add.handler({ a: 2, b: 3 })).toBe(5); - }); -}); -``` - -- [ ] **Step 2: Implement tool()** - -```typescript -import { z } from 'zod'; -import { zodToJsonSchema } from 'zod-to-json-schema'; - -interface ToolDef { - name: string; - description: string; - inputSchemaJson: string; - handler: (args: z.infer) => any; -} - -const registry: ToolDef[] = []; - -interface ToolOptions> { - description: string; - args: T; - handler: (args: z.infer) => any; - name?: string; // defaults to handler function name or export name -} - -export function tool>(options: ToolOptions): ToolDef { - const schema = zodToJsonSchema(options.args, { target: 'openApi3' }); - const def: ToolDef = { - name: options.name || options.handler.name || `tool_${registry.length}`, - description: options.description, - inputSchemaJson: JSON.stringify(schema), - handler: options.handler, - }; - registry.push(def); - return def; -} - -export function getRegisteredTools(): ToolDef[] { - return [...registry]; -} - -export function clearRegistry(): void { - registry.length = 0; -} -``` - -- [ ] **Step 3: Test and commit** - -```bash -cd sdk/typescript && npx vitest run tests/tool.test.ts -git add sdk/typescript/src/tool.ts sdk/typescript/tests/tool.test.ts -git commit -m "feat: implement TypeScript tool() function with Zod schema generation" -``` - ---- - -### Task 24: TypeScript — ToolResult, Transport, Manager, Runner - -**Files:** -- Create: `sdk/typescript/src/result.ts` -- Create: `sdk/typescript/src/transport.ts` -- Create: `sdk/typescript/src/manager.ts` -- Create: `sdk/typescript/src/runner.ts` -- Create: `sdk/typescript/tests/result.test.ts` -- Create: `sdk/typescript/tests/transport.test.ts` -- Create: `sdk/typescript/tests/manager.test.ts` - -These mirror the Python implementations exactly: - -- **result.ts**: `ToolResult` interface with `result`, `isError`, `enableTools`, `disableTools`, `errorCode`, `message`, `suggestion`, `retryable` -- **transport.ts**: Unix socket client with length-prefixed protobuf envelope read/write using `net.Socket` and `protobufjs` -- **manager.ts**: `toolManager` with `enable()`, `disable()`, `setAllowed()`, `setBlocked()`, `getActiveTools()`, `batch()` -- **runner.ts**: Main loop that connects to socket, listens for envelopes, dispatches tool calls, handles reload via Node module cache invalidation - -Follow the same TDD pattern as the Python tasks: write tests first, verify they fail, implement, verify they pass, commit. - -- [ ] **Step 1: Implement and test ToolResult** -- [ ] **Step 2: Implement and test transport** -- [ ] **Step 3: Implement and test manager** -- [ ] **Step 4: Implement runner** -- [ ] **Step 5: Commit all** - -```bash -git add sdk/typescript/src/ sdk/typescript/tests/ -git commit -m "feat: implement TypeScript SDK — ToolResult, transport, manager, runner" -``` - ---- - -## Chunk 6: Integration Tests and Documentation - -### Task 25: End-to-End Integration Tests - -**Files:** -- Create: `test/e2e/e2e_test.go` -- Create: `test/e2e/helpers.go` -- Create: `test/e2e/fixtures/simple_tool.py` -- Create: `test/e2e/fixtures/dynamic_tool.py` -- Create: `test/e2e/fixtures/simple_tool.ts` -- Create: `test/e2e/fixtures/crash_tool.py` - -- [ ] **Step 1: Create test fixtures** - -`simple_tool.py`: -```python -from protomcp import tool - -@tool(description="Echo a message back") -def echo(message: str) -> str: - return message - -@tool(description="Add two numbers") -def add(a: int, b: int) -> int: - return a + b -``` - -`dynamic_tool.py`: -```python -from protomcp import tool, ToolResult, tool_manager - -# Start with admin_action hidden — it gets enabled after auth -tool_manager.set_blocked(["admin_action"]) - -@tool(description="Authenticate") -def auth(token: str) -> ToolResult: - if token == "valid": - return ToolResult( - result="Authenticated", - enable_tools=["admin_action"], - disable_tools=["auth"], - ) - return ToolResult(is_error=True, message="Invalid token", suggestion="Use 'valid' as token") - -@tool(description="Admin action (hidden until auth)") -def admin_action() -> str: - return "admin stuff done" -``` - -`crash_tool.py`: -```python -from protomcp import tool -import sys - -@tool(description="Crash the process") -def crash() -> str: - sys.exit(1) -``` - -`simple_tool.ts`: -```typescript -import { tool } from 'protomcp'; -import { z } from 'zod'; - -export const echo = tool({ - description: 'Echo a message back', - args: z.object({ message: z.string() }), - handler: (args) => args.message, -}); -``` - -- [ ] **Step 2: Write e2e test helpers** - -Create `test/e2e/helpers.go`: - -```go -package e2e - -import ( - "bufio" - "encoding/json" - "io" - "os/exec" - "testing" - - "github.com/msilverblatt/protomcp/internal/mcp" -) - -// StartProtomcp starts the protomcp binary with the given args. -// Returns stdin writer, stdout reader, and cleanup function. -func StartProtomcp(t *testing.T, args ...string) (io.Writer, *bufio.Scanner, func()) { - t.Helper() - cmd := exec.Command("../../bin/protomcp", args...) - stdin, _ := cmd.StdinPipe() - stdout, _ := cmd.StdoutPipe() - cmd.Stderr = nil // suppress stderr in tests - - if err := cmd.Start(); err != nil { - t.Fatalf("failed to start protomcp: %v", err) - } - - scanner := bufio.NewScanner(stdout) - cleanup := func() { - stdin.Close() - cmd.Process.Kill() - cmd.Wait() - } - - return stdin, scanner, cleanup -} - -// SendRequest sends a JSON-RPC request and reads the response. -func SendRequest(t *testing.T, w io.Writer, r *bufio.Scanner, method string, params interface{}) mcp.JSONRPCResponse { - t.Helper() - id := json.RawMessage(`1`) - p, _ := json.Marshal(params) - req := mcp.JSONRPCRequest{ - JSONRPC: "2.0", - ID: id, - Method: method, - Params: p, - } - data, _ := json.Marshal(req) - w.Write(append(data, '\n')) - - if !r.Scan() { - t.Fatal("no response from protomcp") - } - - var resp mcp.JSONRPCResponse - json.Unmarshal(r.Bytes(), &resp) - return resp -} -``` - -- [ ] **Step 3: Write e2e tests** - -Create `test/e2e/e2e_test.go`: - -```go -package e2e - -import ( - "encoding/json" - "testing" - - "github.com/msilverblatt/protomcp/internal/mcp" -) - -func TestE2E_Initialize(t *testing.T) { - w, r, cleanup := StartProtomcp(t, "dev", "fixtures/simple_tool.py") - defer cleanup() - - resp := SendRequest(t, w, r, "initialize", nil) - if resp.Error != nil { - t.Fatalf("initialize error: %v", resp.Error) - } - - var result mcp.InitializeResult - json.Unmarshal(resp.Result, &result) - if !result.Capabilities.Tools.ListChanged { - t.Error("expected tools.listChanged = true") - } -} - -func TestE2E_ToolsList(t *testing.T) { - w, r, cleanup := StartProtomcp(t, "dev", "fixtures/simple_tool.py") - defer cleanup() - - SendRequest(t, w, r, "initialize", nil) - resp := SendRequest(t, w, r, "tools/list", nil) - - var result mcp.ToolsListResult - json.Unmarshal(resp.Result, &result) - if len(result.Tools) != 2 { - t.Fatalf("expected 2 tools, got %d", len(result.Tools)) - } -} - -func TestE2E_ToolsCall(t *testing.T) { - w, r, cleanup := StartProtomcp(t, "dev", "fixtures/simple_tool.py") - defer cleanup() - - SendRequest(t, w, r, "initialize", nil) - resp := SendRequest(t, w, r, "tools/call", map[string]interface{}{ - "name": "echo", - "arguments": map[string]string{"message": "hello"}, - }) - - if resp.Error != nil { - t.Fatalf("tools/call error: %v", resp.Error) - } -} - -func TestE2E_DynamicToolList(t *testing.T) { - w, r, cleanup := StartProtomcp(t, "dev", "fixtures/dynamic_tool.py") - defer cleanup() - - SendRequest(t, w, r, "initialize", nil) - - // Initially, auth should be visible but admin_action should not - // (admin_action starts disabled — depends on how the fixture sets initial state) - // Call auth with valid token - resp := SendRequest(t, w, r, "tools/call", map[string]interface{}{ - "name": "auth", - "arguments": map[string]string{"token": "valid"}, - }) - if resp.Error != nil { - t.Fatalf("auth call error: %v", resp.Error) - } - - // After auth, admin_action should now be visible in tools/list - listResp := SendRequest(t, w, r, "tools/list", nil) - var result mcp.ToolsListResult - json.Unmarshal(listResp.Result, &result) - - found := false - for _, tool := range result.Tools { - if tool.Name == "admin_action" { - found = true - } - } - if !found { - t.Error("admin_action should be visible after auth") - } -} - -func TestE2E_TypeScript(t *testing.T) { - w, r, cleanup := StartProtomcp(t, "dev", "fixtures/simple_tool.ts") - defer cleanup() - - SendRequest(t, w, r, "initialize", nil) - resp := SendRequest(t, w, r, "tools/list", nil) - - var result mcp.ToolsListResult - json.Unmarshal(resp.Result, &result) - if len(result.Tools) != 1 { - t.Fatalf("expected 1 tool, got %d", len(result.Tools)) - } - if result.Tools[0].Name != "echo" { - t.Errorf("tool name = %q, want %q", result.Tools[0].Name, "echo") - } -} -``` - -- [ ] **Step 4: Build binary and run e2e tests** - -```bash -make build -go test ./test/e2e/... -v -timeout 60s -``` - -- [ ] **Step 5: Commit** - -```bash -git add test/ -git commit -m "feat: add end-to-end integration tests for Python, TypeScript, and dynamic tool lists" -``` - ---- - -### Task 26: Documentation Site Setup - -**Files:** -- Create: `docs/package.json` -- Create: `docs/astro.config.mjs` -- Create: `docs/src/content/docs/index.mdx` - -- [ ] **Step 1: Scaffold Starlight site** - -```bash -cd docs && npm create astro@latest -- --template starlight -``` - -Or manually create `docs/package.json`: -```json -{ - "name": "protomcp-docs", - "scripts": { - "dev": "astro dev", - "build": "astro build" - }, - "dependencies": { - "@astrojs/starlight": "latest", - "astro": "latest" - } -} -``` - -- [ ] **Step 2: Configure astro.config.mjs** - -```javascript -import { defineConfig } from 'astro/config'; -import starlight from '@astrojs/starlight'; - -export default defineConfig({ - integrations: [ - starlight({ - title: 'protomcp', - description: 'Language-agnostic MCP runtime', - social: { - github: 'https://github.com/msilverblatt/protomcp', - }, - sidebar: [ - { - label: 'Getting Started', - items: [ - { label: 'Installation', slug: 'getting-started/installation' }, - { label: 'Quick Start', slug: 'getting-started/quick-start' }, - { label: 'How It Works', slug: 'getting-started/how-it-works' }, - ], - }, - { - label: 'Guides', - items: [ - { label: 'Writing Tools (Python)', slug: 'guides/writing-tools-python' }, - { label: 'Writing Tools (TypeScript)', slug: 'guides/writing-tools-typescript' }, - { label: 'Dynamic Tool Lists', slug: 'guides/dynamic-tool-lists' }, - { label: 'Hot Reload', slug: 'guides/hot-reload' }, - { label: 'Error Handling', slug: 'guides/error-handling' }, - { label: 'Production Deployment', slug: 'guides/production-deployment' }, - ], - }, - { - label: 'Reference', - items: [ - { label: 'CLI', slug: 'reference/cli' }, - { label: 'Protobuf Spec', slug: 'reference/protobuf-spec' }, - { label: 'Python API', slug: 'reference/python-api' }, - { label: 'TypeScript API', slug: 'reference/typescript-api' }, - ], - }, - { - label: 'Concepts', - items: [ - { label: 'Architecture', slug: 'concepts/architecture' }, - { label: 'Tool List Modes', slug: 'concepts/tool-list-modes' }, - { label: 'Transports', slug: 'concepts/transports' }, - ], - }, - ], - }), - ], -}); -``` - -- [ ] **Step 3: Install dependencies and verify build** - -```bash -cd docs && npm install && npm run build -``` - -- [ ] **Step 4: Commit** - -```bash -git add docs/package.json docs/astro.config.mjs docs/src/ -git commit -m "feat: scaffold Starlight documentation site with sidebar structure" -``` - ---- - -### Task 27: Documentation — Getting Started Pages - -**Files:** -- Create: `docs/src/content/docs/getting-started/installation.mdx` -- Create: `docs/src/content/docs/getting-started/quick-start.mdx` -- Create: `docs/src/content/docs/getting-started/how-it-works.mdx` - -- [ ] **Step 1: Write Installation page** - -Covers: Homebrew, binary download, building from source. Platform-specific instructions for macOS, Linux, Windows. Python and TypeScript SDK installation via pip/npm. - -- [ ] **Step 2: Write Quick Start page** - -5-minute tutorial: -1. Install protomcp -2. Write a Python tool file (3 lines) -3. Run `protomcp dev server.py` -4. Configure your MCP client -5. Use the tool — edit, save, see it update live - -Include exact terminal output at each step. - -- [ ] **Step 3: Write How It Works page** - -Architecture explainer with Mermaid diagrams: -- The three layers (host, Go binary, tool process) -- How hot-reload works (sequence diagram) -- How dynamic tool lists work (state diagram) -- Why the Go binary never needs to change - -- [ ] **Step 4: Commit** - -```bash -git add docs/src/content/docs/getting-started/ -git commit -m "docs: add Getting Started pages — installation, quick start, architecture" -``` - ---- - -### Task 28: Documentation — Guides - -**Files:** -- Create: `docs/src/content/docs/guides/writing-tools-python.mdx` -- Create: `docs/src/content/docs/guides/writing-tools-typescript.mdx` -- Create: `docs/src/content/docs/guides/dynamic-tool-lists.mdx` -- Create: `docs/src/content/docs/guides/hot-reload.mdx` -- Create: `docs/src/content/docs/guides/error-handling.mdx` -- Create: `docs/src/content/docs/guides/production-deployment.mdx` - -- [ ] **Step 1: Write Python guide** - -Covers: `@tool()` decorator, type hints for schema generation, `ToolResult`, `tool_manager`, async tools, testing tools outside protomcp. All examples pulled from test files. - -- [ ] **Step 2: Write TypeScript guide** - -Covers: `tool()` function, Zod schemas, `ToolResult`, `toolManager`. Parallel structure to Python guide. - -- [ ] **Step 3: Write Dynamic Tool Lists guide** - -Covers: inline mutations (enable_tools/disable_tools in ToolResult), programmatic control (tool_manager), modes (open/allowlist/blocklist), batch operations, event-driven examples (locks, auth). - -- [ ] **Step 4: Write Hot Reload guide** - -Covers: how it works, what triggers a reload, in-flight call handling (default vs immediate), reload logs, common gotchas (module-level side effects, import caching). - -- [ ] **Step 5: Write Error Handling guide** - -Covers: structured errors (error_code, message, suggestion, retryable), how errors are formatted by the Go binary, best practices for agent-friendly errors. - -- [ ] **Step 6: Write Production Deployment guide** - -Covers: `protomcp run` vs `protomcp dev`, transport selection for production, systemd/supervisor config, Docker (optional), monitoring via logs. - -- [ ] **Step 7: Commit** - -```bash -git add docs/src/content/docs/guides/ -git commit -m "docs: add all v1.0 guides — Python, TypeScript, dynamic tools, hot reload, errors, production" -``` - ---- - -### Task 29: Documentation — Reference Pages - -**Files:** -- Create: `docs/src/content/docs/reference/cli.mdx` -- Create: `docs/src/content/docs/reference/protobuf-spec.mdx` -- Create: `docs/src/content/docs/reference/python-api.mdx` -- Create: `docs/src/content/docs/reference/typescript-api.mdx` - -- [ ] **Step 1: Write CLI Reference** - -Exhaustive documentation of `protomcp dev`, `protomcp run`, and all flags with examples. - -- [ ] **Step 2: Write Protobuf Spec Reference** - -Full `.proto` file with annotated explanations of every message type, field, and the wire format. - -- [ ] **Step 3: Write Python API Reference** - -Complete API docs for `@tool()`, `ToolResult`, `tool_manager` — all parameters, return types, examples. - -- [ ] **Step 4: Write TypeScript API Reference** - -Same as Python reference but for TS — `tool()`, `ToolResult`, `toolManager`. - -- [ ] **Step 5: Commit** - -```bash -git add docs/src/content/docs/reference/ -git commit -m "docs: add reference pages — CLI, protobuf spec, Python API, TypeScript API" -``` - ---- - -### Task 30: Documentation — Concepts Pages - -**Files:** -- Create: `docs/src/content/docs/concepts/architecture.mdx` -- Create: `docs/src/content/docs/concepts/tool-list-modes.mdx` -- Create: `docs/src/content/docs/concepts/transports.mdx` - -- [ ] **Step 1: Write Architecture page** - -Deep dive into the three-layer architecture with Mermaid diagrams. Explains why the Go binary is language-agnostic, how protobuf enables any-language support, and the unix socket communication pattern. - -- [ ] **Step 2: Write Tool List Modes page** - -Explains open/allowlist/blocklist modes, mode transitions, interaction semantics, batch operations. Includes state diagram. - -- [ ] **Step 3: Write Transports page** - -Explains all five transports, when to use each, configuration, and client compatibility matrix. - -- [ ] **Step 4: Build docs and verify** - -```bash -cd docs && npm run build -``` - -- [ ] **Step 5: Commit** - -```bash -git add docs/src/content/docs/concepts/ -git commit -m "docs: add concept pages — architecture, tool list modes, transports" -``` - ---- - -## Chunk 7: MCP Advanced Features — Progress, Tasks, Cancellation, Logging, Structured Output, Tool Metadata - -### Task 32: Go — Progress Notification Proxy - -**Files:** -- Create: `internal/progress/progress.go` -- Create: `internal/progress/progress_test.go` - -- [ ] **Step 1: Write failing test for progress proxy** - -```go -// internal/progress/progress_test.go -package progress - -import ( - "testing" - pb "github.com/msilverblatt/protomcp/gen/proto/protomcp" -) - -func TestProgressProxy_ForwardsToMCP(t *testing.T) { - var sent []map[string]any - proxy := NewProxy(func(notification map[string]any) { - sent = append(sent, notification) - }) - proxy.HandleProgress(&pb.ProgressNotification{ - ProgressToken: "tok-1", - Progress: 5, - Total: 10, - Message: "Processing item 5", - }) - if len(sent) != 1 { - t.Fatalf("expected 1 notification, got %d", len(sent)) - } - if sent[0]["method"] != "notifications/progress" { - t.Fatalf("expected notifications/progress, got %v", sent[0]["method"]) - } -} - -func TestProgressProxy_DropsWhenNoToken(t *testing.T) { - var sent []map[string]any - proxy := NewProxy(func(notification map[string]any) { - sent = append(sent, notification) - }) - proxy.HandleProgress(&pb.ProgressNotification{ - ProgressToken: "", - Progress: 1, - }) - if len(sent) != 0 { - t.Fatalf("expected 0 notifications for empty token, got %d", len(sent)) - } -} -``` - -- [ ] **Step 2: Run test to verify it fails** - -Run: `go test ./internal/progress/ -v` -Expected: FAIL — package doesn't exist yet. - -- [ ] **Step 3: Implement progress proxy** - -```go -// internal/progress/progress.go -package progress - -import pb "github.com/msilverblatt/protomcp/gen/proto/protomcp" - -type NotifySender func(notification map[string]any) - -type Proxy struct { - send NotifySender -} - -func NewProxy(send NotifySender) *Proxy { - return &Proxy{send: send} -} - -func (p *Proxy) HandleProgress(msg *pb.ProgressNotification) { - if msg.ProgressToken == "" { - return // Client didn't request progress — silently drop - } - params := map[string]any{ - "progressToken": msg.ProgressToken, - "progress": msg.Progress, - } - if msg.Total > 0 { - params["total"] = msg.Total - } - if msg.Message != "" { - params["message"] = msg.Message - } - p.send(map[string]any{ - "method": "notifications/progress", - "params": params, - }) -} -``` - -- [ ] **Step 4: Run test to verify it passes** - -Run: `go test ./internal/progress/ -v` -Expected: PASS - -- [ ] **Step 5: Commit** - -```bash -git add internal/progress/ -git commit -m "feat: add progress notification proxy for MCP notifications/progress" -``` - ---- - -### Task 33: Go — Cancellation Tracker - -**Files:** -- Create: `internal/cancel/tracker.go` -- Create: `internal/cancel/tracker_test.go` - -- [ ] **Step 1: Write failing test for cancellation tracker** - -```go -// internal/cancel/tracker_test.go -package cancel - -import ( - "testing" -) - -func TestTracker_SetAndCheck(t *testing.T) { - tracker := NewTracker() - tracker.TrackCall("req-1") - if tracker.IsCancelled("req-1") { - t.Fatal("should not be cancelled yet") - } - tracker.Cancel("req-1") - if !tracker.IsCancelled("req-1") { - t.Fatal("should be cancelled after Cancel()") - } -} - -func TestTracker_CancelUnknownRequestIsNoop(t *testing.T) { - tracker := NewTracker() - tracker.Cancel("nonexistent") // Should not panic -} - -func TestTracker_CompleteRemovesTracking(t *testing.T) { - tracker := NewTracker() - tracker.TrackCall("req-1") - tracker.Complete("req-1") - if tracker.IsCancelled("req-1") { - t.Fatal("completed call should not report cancelled") - } -} -``` - -- [ ] **Step 2: Run test to verify it fails** - -Run: `go test ./internal/cancel/ -v` -Expected: FAIL - -- [ ] **Step 3: Implement cancellation tracker** - -```go -// internal/cancel/tracker.go -package cancel - -import "sync" - -type Tracker struct { - mu sync.RWMutex - cancelled map[string]bool -} - -func NewTracker() *Tracker { - return &Tracker{cancelled: make(map[string]bool)} -} - -func (t *Tracker) TrackCall(requestID string) { - t.mu.Lock() - defer t.mu.Unlock() - t.cancelled[requestID] = false -} - -func (t *Tracker) Cancel(requestID string) { - t.mu.Lock() - defer t.mu.Unlock() - if _, exists := t.cancelled[requestID]; exists { - t.cancelled[requestID] = true - } -} - -func (t *Tracker) IsCancelled(requestID string) bool { - t.mu.RLock() - defer t.mu.RUnlock() - return t.cancelled[requestID] -} - -func (t *Tracker) Complete(requestID string) { - t.mu.Lock() - defer t.mu.Unlock() - delete(t.cancelled, requestID) -} -``` - -- [ ] **Step 4: Run test to verify it passes** - -Run: `go test ./internal/cancel/ -v` -Expected: PASS - -- [ ] **Step 5: Commit** - -```bash -git add internal/cancel/ -git commit -m "feat: add cancellation tracker for cooperative tool call cancellation" -``` - ---- - -### Task 34: Go — Server Log Forwarder - -**Files:** -- Create: `internal/serverlog/forwarder.go` -- Create: `internal/serverlog/forwarder_test.go` - -- [ ] **Step 1: Write failing test for log forwarder** - -```go -// internal/serverlog/forwarder_test.go -package serverlog - -import ( - "testing" - pb "github.com/msilverblatt/protomcp/gen/proto/protomcp" -) - -func TestForwarder_ForwardsAboveLevel(t *testing.T) { - var sent []map[string]any - fwd := NewForwarder("info", func(n map[string]any) { sent = append(sent, n) }) - - fwd.HandleLog(&pb.LogMessage{Level: "warning", DataJson: `{"msg":"rate limit"}`}) - if len(sent) != 1 { - t.Fatalf("expected 1, got %d", len(sent)) - } - if sent[0]["method"] != "notifications/message" { - t.Fatalf("wrong method: %v", sent[0]["method"]) - } -} - -func TestForwarder_FiltersBelowLevel(t *testing.T) { - var sent []map[string]any - fwd := NewForwarder("warning", func(n map[string]any) { sent = append(sent, n) }) - - fwd.HandleLog(&pb.LogMessage{Level: "info", DataJson: `{"msg":"hello"}`}) - if len(sent) != 0 { - t.Fatalf("expected 0 (filtered), got %d", len(sent)) - } -} -``` - -- [ ] **Step 2: Run test to verify it fails** - -Run: `go test ./internal/serverlog/ -v` -Expected: FAIL - -- [ ] **Step 3: Implement log forwarder** - -```go -// internal/serverlog/forwarder.go -package serverlog - -import ( - "encoding/json" - pb "github.com/msilverblatt/protomcp/gen/proto/protomcp" -) - -var levelPriority = map[string]int{ - "debug": 0, - "info": 1, - "notice": 2, - "warning": 3, - "error": 4, - "critical": 5, - "alert": 6, - "emergency": 7, -} - -type NotifySender func(notification map[string]any) - -type Forwarder struct { - minLevel int - send NotifySender -} - -func NewForwarder(minLevelStr string, send NotifySender) *Forwarder { - return &Forwarder{ - minLevel: levelPriority[minLevelStr], - send: send, - } -} - -func (f *Forwarder) HandleLog(msg *pb.LogMessage) { - priority, ok := levelPriority[msg.Level] - if !ok { - priority = 1 // default to info for unknown levels - } - if priority < f.minLevel { - return - } - params := map[string]any{ - "level": msg.Level, - } - if msg.Logger != "" { - params["logger"] = msg.Logger - } - if msg.DataJson != "" { - var data any - if err := json.Unmarshal([]byte(msg.DataJson), &data); err == nil { - params["data"] = data - } else { - params["data"] = msg.DataJson - } - } - f.send(map[string]any{ - "method": "notifications/message", - "params": params, - }) -} -``` - -- [ ] **Step 4: Run test to verify it passes** - -Run: `go test ./internal/serverlog/ -v` -Expected: PASS - -- [ ] **Step 5: Commit** - -```bash -git add internal/serverlog/ -git commit -m "feat: add server log forwarder for MCP notifications/message" -``` - ---- - -### Task 35: Go — Async Task Manager - -**Files:** -- Create: `internal/tasks/manager.go` -- Create: `internal/tasks/manager_test.go` - -- [ ] **Step 1: Write failing test for task manager** - -```go -// internal/tasks/manager_test.go -package tasks - -import ( - "testing" -) - -func TestTaskManager_CreateAndGet(t *testing.T) { - mgr := NewManager() - mgr.Register("task-1", "req-1") - state, err := mgr.GetStatus("task-1") - if err != nil { - t.Fatal(err) - } - if state.State != "running" { - t.Fatalf("expected running, got %s", state.State) - } -} - -func TestTaskManager_FailOnCrash(t *testing.T) { - mgr := NewManager() - mgr.Register("task-1", "req-1") - mgr.FailAll("tool process crashed") - state, err := mgr.GetStatus("task-1") - if err != nil { - t.Fatal(err) - } - if state.State != "failed" { - t.Fatalf("expected failed, got %s", state.State) - } -} - -func TestTaskManager_GetUnknownTask(t *testing.T) { - mgr := NewManager() - _, err := mgr.GetStatus("nonexistent") - if err == nil { - t.Fatal("expected error for unknown task") - } -} -``` - -- [ ] **Step 2: Run test to verify it fails** - -Run: `go test ./internal/tasks/ -v` -Expected: FAIL - -- [ ] **Step 3: Implement task manager** - -```go -// internal/tasks/manager.go -package tasks - -import ( - "fmt" - "sync" -) - -type TaskState struct { - State string // running, completed, failed, cancelled - Message string -} - -type Manager struct { - mu sync.RWMutex - tasks map[string]*TaskState -} - -func NewManager() *Manager { - return &Manager{tasks: make(map[string]*TaskState)} -} - -func (m *Manager) Register(taskID, requestID string) { - m.mu.Lock() - defer m.mu.Unlock() - m.tasks[taskID] = &TaskState{State: "running"} -} - -func (m *Manager) GetStatus(taskID string) (*TaskState, error) { - m.mu.RLock() - defer m.mu.RUnlock() - state, ok := m.tasks[taskID] - if !ok { - return nil, fmt.Errorf("unknown task: %s", taskID) - } - return state, nil -} - -func (m *Manager) UpdateStatus(taskID, state, message string) error { - m.mu.Lock() - defer m.mu.Unlock() - ts, ok := m.tasks[taskID] - if !ok { - return fmt.Errorf("unknown task: %s", taskID) - } - ts.State = state - ts.Message = message - return nil -} - -func (m *Manager) FailAll(reason string) { - m.mu.Lock() - defer m.mu.Unlock() - for _, ts := range m.tasks { - if ts.State == "running" { - ts.State = "failed" - ts.Message = reason - } - } -} -``` - -- [ ] **Step 4: Run test to verify it passes** - -Run: `go test ./internal/tasks/ -v` -Expected: PASS - -- [ ] **Step 5: Commit** - -```bash -git add internal/tasks/ -git commit -m "feat: add async task manager for MCP task lifecycle" -``` - ---- - -### Task 36: Go — Integrate New Features into MCP Handler - -**Files:** -- Modify: `internal/mcp/handler.go` -- Modify: `internal/mcp/handler_test.go` - -- [ ] **Step 1: Write failing tests for new MCP handler capabilities** - -Add tests to `internal/mcp/handler_test.go`: - -```go -func TestHandler_ToolsCallWithProgressToken(t *testing.T) { - // Test that _meta.progressToken is extracted and forwarded in CallToolRequest - handler := newTestHandler() - req := makeToolCallRequest("test_tool", `{"x": 1}`, map[string]any{"progressToken": "pt-1"}) - handler.HandleRequest(req) - // Verify CallToolRequest sent to tool process includes progress_token = "pt-1" - sentMsg := handler.lastSentToToolProcess() - callReq := sentMsg.GetCallTool() - if callReq.ProgressToken != "pt-1" { - t.Fatalf("expected progress_token pt-1, got %s", callReq.ProgressToken) - } -} - -func TestHandler_NotificationsCancelled(t *testing.T) { - // Test that notifications/cancelled is forwarded as CancelRequest - handler := newTestHandler() - handler.startToolCall("req-42") - handler.HandleNotification("notifications/cancelled", map[string]any{"requestId": "req-42"}) - if !handler.cancelTracker.IsCancelled("req-42") { - t.Fatal("expected req-42 to be cancelled") - } -} - -func TestHandler_ToolsCallReturnsStructuredContent(t *testing.T) { - // Test that structuredContent is included when outputSchema is defined - handler := newTestHandler() - handler.setToolOutputSchema("search", `{"type":"array"}`) - resp := handler.callTool("search", `{"q":"test"}`) - if resp["structuredContent"] == nil { - t.Fatal("expected structuredContent in response") - } -} - -func TestHandler_ToolDefinitionIncludesMetadata(t *testing.T) { - // Test that tool metadata (title, hints, task_support) is included in tools/list - handler := newTestHandler() - tools := handler.handleToolsList() - for _, tool := range tools { - toolMap := tool.(map[string]any) - if toolMap["name"] == "delete_doc" { - annotations := toolMap["annotations"].(map[string]any) - if annotations["destructiveHint"] != true { - t.Fatal("expected destructiveHint on delete_doc") - } - } - } -} -``` - -- [ ] **Step 2: Run tests to verify they fail** - -Run: `go test ./internal/mcp/ -v -run "ProgressToken|Cancelled|StructuredContent|Metadata"` -Expected: FAIL - -- [ ] **Step 3: Update MCP handler implementation** - -Update `internal/mcp/handler.go` to: -1. Extract `_meta.progressToken` from `tools/call` requests and include it in `CallToolRequest.progress_token` -2. Handle `notifications/cancelled` by forwarding `CancelRequest` to tool process and marking in cancel tracker -3. Include `outputSchema` in tool definitions from `ToolDefinition.output_schema_json` -4. Include `structuredContent` in `tools/call` responses from `CallToolResponse.structured_content_json` -5. Include `annotations` (title, hints) in tool definitions from `ToolDefinition` metadata fields -6. Include `execution.taskSupport` in tool definitions when `ToolDefinition.task_support` is true -7. Handle `tasks/get`, `tasks/result`, `tasks/cancel` by forwarding to tool process -8. Advertise `tasks` capability in `initialize` response when any tool has `task_support` -9. Handle `notifications/message` from tool process by forwarding via server log forwarder - -- [ ] **Step 4: Run tests to verify they pass** - -Run: `go test ./internal/mcp/ -v` -Expected: PASS - -- [ ] **Step 5: Commit** - -```bash -git add internal/mcp/ -git commit -m "feat: integrate progress, cancellation, tasks, logging, structured output, and tool metadata into MCP handler" -``` - ---- - -### Task 37: Python SDK — ToolContext (Progress + Cancellation) - -**Files:** -- Create: `sdk/python/src/protomcp/context.py` -- Create: `sdk/python/tests/test_context.py` - -- [ ] **Step 1: Write failing test** - -```python -# sdk/python/tests/test_context.py -from protomcp.context import ToolContext - -def test_report_progress_sends_notification(): - sent = [] - ctx = ToolContext(progress_token="pt-1", send_fn=lambda msg: sent.append(msg)) - ctx.report_progress(progress=5, total=10, message="Working") - assert len(sent) == 1 - assert sent[0].progress.progress_token == "pt-1" - assert sent[0].progress.progress == 5 - assert sent[0].progress.total == 10 - -def test_report_progress_noop_without_token(): - sent = [] - ctx = ToolContext(progress_token="", send_fn=lambda msg: sent.append(msg)) - ctx.report_progress(progress=1) - assert len(sent) == 0 - -def test_is_cancelled(): - ctx = ToolContext(progress_token="", send_fn=lambda msg: None) - assert not ctx.is_cancelled() - ctx._cancelled = True - assert ctx.is_cancelled() -``` - -- [ ] **Step 2: Run test to verify it fails** - -Run: `cd sdk/python && python -m pytest tests/test_context.py -v` -Expected: FAIL - -- [ ] **Step 3: Implement ToolContext** - -```python -# sdk/python/src/protomcp/context.py -import threading -from protomcp.gen import protomcp_pb2 as pb - -class ToolContext: - def __init__(self, progress_token: str, send_fn): - self._progress_token = progress_token - self._send_fn = send_fn - self._cancelled = False - self._lock = threading.Lock() - - def report_progress(self, progress: int, total: int = 0, message: str = ""): - if not self._progress_token: - return - envelope = pb.Envelope( - progress=pb.ProgressNotification( - progress_token=self._progress_token, - progress=progress, - total=total, - message=message, - ) - ) - self._send_fn(envelope) - - def is_cancelled(self) -> bool: - with self._lock: - return self._cancelled - - def _set_cancelled(self): - with self._lock: - self._cancelled = True -``` - -- [ ] **Step 4: Run test to verify it passes** - -Run: `cd sdk/python && python -m pytest tests/test_context.py -v` -Expected: PASS - -- [ ] **Step 5: Commit** - -```bash -git add sdk/python/src/protomcp/context.py sdk/python/tests/test_context.py -git commit -m "feat(python): add ToolContext with progress reporting and cancellation" -``` - ---- - -### Task 38: Python SDK — Server Logging API - -**Files:** -- Create: `sdk/python/src/protomcp/log.py` -- Create: `sdk/python/tests/test_log.py` - -- [ ] **Step 1: Write failing test** - -```python -# sdk/python/tests/test_log.py -from protomcp.log import ServerLogger - -def test_log_info(): - sent = [] - logger = ServerLogger(send_fn=lambda msg: sent.append(msg)) - logger.info("hello", data={"count": 5}) - assert len(sent) == 1 - assert sent[0].log.level == "info" - assert '"count": 5' in sent[0].log.data_json or '"count":5' in sent[0].log.data_json - -def test_log_with_logger_name(): - sent = [] - logger = ServerLogger(send_fn=lambda msg: sent.append(msg), name="cache") - logger.debug("hit") - assert sent[0].log.logger == "cache" - -def test_all_levels(): - sent = [] - logger = ServerLogger(send_fn=lambda msg: sent.append(msg)) - for level in ["debug", "info", "notice", "warning", "error", "critical", "alert", "emergency"]: - getattr(logger, level)("test") - assert len(sent) == 8 -``` - -- [ ] **Step 2: Run test to verify it fails** - -Run: `cd sdk/python && python -m pytest tests/test_log.py -v` -Expected: FAIL - -- [ ] **Step 3: Implement ServerLogger** - -```python -# sdk/python/src/protomcp/log.py -import json -from protomcp.gen import protomcp_pb2 as pb - -class ServerLogger: - def __init__(self, send_fn, name: str = ""): - self._send_fn = send_fn - self._name = name - - def _log(self, level: str, message: str, data: dict | None = None): - data_json = json.dumps(data) if data else json.dumps({"message": message}) - envelope = pb.Envelope( - log=pb.LogMessage( - level=level, - logger=self._name, - data_json=data_json, - ) - ) - self._send_fn(envelope) - - def debug(self, message: str, **kwargs): self._log("debug", message, kwargs.get("data")) - def info(self, message: str, **kwargs): self._log("info", message, kwargs.get("data")) - def notice(self, message: str, **kwargs): self._log("notice", message, kwargs.get("data")) - def warning(self, message: str, **kwargs): self._log("warning", message, kwargs.get("data")) - def error(self, message: str, **kwargs): self._log("error", message, kwargs.get("data")) - def critical(self, message: str, **kwargs): self._log("critical", message, kwargs.get("data")) - def alert(self, message: str, **kwargs): self._log("alert", message, kwargs.get("data")) - def emergency(self, message: str, **kwargs): self._log("emergency", message, kwargs.get("data")) -``` - -- [ ] **Step 4: Run test to verify it passes** - -Run: `cd sdk/python && python -m pytest tests/test_log.py -v` -Expected: PASS - -- [ ] **Step 5: Commit** - -```bash -git add sdk/python/src/protomcp/log.py sdk/python/tests/test_log.py -git commit -m "feat(python): add server logging API for MCP notifications/message" -``` - ---- - -### Task 39: Python SDK — Structured Output and Tool Metadata - -**Files:** -- Modify: `sdk/python/src/protomcp/tool.py` -- Modify: `sdk/python/tests/test_tool.py` - -- [ ] **Step 1: Write failing tests for output_type and metadata** - -Add to `sdk/python/tests/test_tool.py`: - -```python -from dataclasses import dataclass - -def test_tool_with_output_type(): - clear_registry() - - @dataclass - class SearchResult: - title: str - score: float - - @tool(description="Search", output_type=SearchResult) - def search(query: str) -> SearchResult: - return SearchResult(title="test", score=0.9) - - tools = get_registered_tools() - assert tools[0].output_schema_json != "" - schema = json.loads(tools[0].output_schema_json) - assert "title" in schema["properties"] - assert "score" in schema["properties"] - -def test_tool_with_metadata(): - clear_registry() - - @tool( - description="Delete doc", - title="Delete Document", - destructive=True, - idempotent=True, - ) - def delete_doc(doc_id: str) -> str: - return "deleted" - - tools = get_registered_tools() - assert tools[0].title == "Delete Document" - assert tools[0].destructive_hint is True - assert tools[0].idempotent_hint is True - -def test_tool_with_task_support(): - clear_registry() - - @tool(description="Long task", task_support=True) - async def long_task(data: str) -> str: - return "done" - - tools = get_registered_tools() - assert tools[0].task_support is True -``` - -- [ ] **Step 2: Run tests to verify they fail** - -Run: `cd sdk/python && python -m pytest tests/test_tool.py -v -k "output_type or metadata or task_support"` -Expected: FAIL - -- [ ] **Step 3: Update @tool decorator** - -Update `sdk/python/src/protomcp/tool.py` to accept `output_type`, `title`, `destructive`, `idempotent`, `read_only`, `open_world`, and `task_support` parameters. Generate `output_schema_json` from the output type's fields using the same schema generation logic. Set the corresponding fields on `ToolDefinition`. - -- [ ] **Step 4: Run tests to verify they pass** - -Run: `cd sdk/python && python -m pytest tests/test_tool.py -v` -Expected: PASS - -- [ ] **Step 5: Commit** - -```bash -git add sdk/python/src/protomcp/tool.py sdk/python/tests/test_tool.py -git commit -m "feat(python): add output_type, tool metadata, and task_support to @tool decorator" -``` - ---- - -### Task 40: Python SDK — Update Runner for New Features - -**Files:** -- Modify: `sdk/python/src/protomcp/runner.py` -- Modify: `sdk/python/src/protomcp/__init__.py` - -- [ ] **Step 1: Update runner to handle new message types** - -Update `sdk/python/src/protomcp/runner.py` to: -1. Create `ToolContext` with `progress_token` from `CallToolRequest` and pass it to tool handlers that accept a `ctx` parameter (detected via `inspect.signature`) -2. Handle incoming `CancelRequest` by calling `ctx._set_cancelled()` on the matching in-flight call -3. Initialize `ServerLogger` and expose as module-level `log` in `__init__.py` -4. For tools with `task_support=True`, run handler in a background thread, immediately return `CreateTaskResponse`, and handle `TaskStatusRequest`/`TaskResultRequest`/`TaskCancelRequest` by querying the running task - -- [ ] **Step 2: Update __init__.py exports** - -```python -# sdk/python/src/protomcp/__init__.py -from protomcp.tool import tool -from protomcp.result import ToolResult -from protomcp.manager import tool_manager -from protomcp.context import ToolContext -from protomcp.log import ServerLogger - -log = ServerLogger(send_fn=None) # Initialized by runner on connect -``` - -- [ ] **Step 3: Run all Python tests** - -Run: `cd sdk/python && python -m pytest tests/ -v` -Expected: PASS - -- [ ] **Step 4: Commit** - -```bash -git add sdk/python/src/protomcp/ -git commit -m "feat(python): integrate progress, cancellation, logging, and async tasks into runner" -``` - ---- - -### Task 41: TypeScript SDK — ToolContext, Logging, Structured Output, Metadata - -**Files:** -- Create: `sdk/typescript/src/context.ts` -- Create: `sdk/typescript/src/log.ts` -- Create: `sdk/typescript/tests/context.test.ts` -- Create: `sdk/typescript/tests/log.test.ts` -- Modify: `sdk/typescript/src/tool.ts` -- Modify: `sdk/typescript/tests/tool.test.ts` -- Modify: `sdk/typescript/src/runner.ts` -- Modify: `sdk/typescript/src/index.ts` - -- [ ] **Step 1: Write failing tests for ToolContext** - -```typescript -// sdk/typescript/tests/context.test.ts -import { describe, it, expect } from 'vitest'; -import { ToolContext } from '../src/context'; - -describe('ToolContext', () => { - it('sends progress notification', () => { - const sent: any[] = []; - const ctx = new ToolContext('pt-1', (msg) => sent.push(msg)); - ctx.reportProgress(5, 10, 'Working'); - expect(sent).toHaveLength(1); - expect(sent[0].progress.progressToken).toBe('pt-1'); - }); - - it('is noop without progress token', () => { - const sent: any[] = []; - const ctx = new ToolContext('', (msg) => sent.push(msg)); - ctx.reportProgress(1); - expect(sent).toHaveLength(0); - }); - - it('tracks cancellation', () => { - const ctx = new ToolContext('', () => {}); - expect(ctx.isCancelled()).toBe(false); - ctx.setCancelled(); - expect(ctx.isCancelled()).toBe(true); - }); -}); -``` - -- [ ] **Step 2: Write failing tests for ServerLogger** - -```typescript -// sdk/typescript/tests/log.test.ts -import { describe, it, expect } from 'vitest'; -import { ServerLogger } from '../src/log'; - -describe('ServerLogger', () => { - it('sends log message', () => { - const sent: any[] = []; - const logger = new ServerLogger((msg) => sent.push(msg)); - logger.info('hello', { count: 5 }); - expect(sent).toHaveLength(1); - expect(sent[0].log.level).toBe('info'); - }); - - it('supports all RFC 5424 levels', () => { - const sent: any[] = []; - const logger = new ServerLogger((msg) => sent.push(msg)); - const levels = ['debug', 'info', 'notice', 'warning', 'error', 'critical', 'alert', 'emergency'] as const; - for (const level of levels) { - logger[level]('test'); - } - expect(sent).toHaveLength(8); - }); -}); -``` - -- [ ] **Step 3: Run tests to verify they fail** - -Run: `cd sdk/typescript && npx vitest run --reporter=verbose` -Expected: FAIL - -- [ ] **Step 4: Implement ToolContext** - -```typescript -// sdk/typescript/src/context.ts -export class ToolContext { - private _cancelled = false; - constructor( - private readonly progressToken: string, - private readonly sendFn: (msg: any) => void, - ) {} - - reportProgress(progress: number, total?: number, message?: string) { - if (!this.progressToken) return; - this.sendFn({ - progress: { - progressToken: this.progressToken, - progress, - ...(total !== undefined && { total }), - ...(message !== undefined && { message }), - }, - }); - } - - isCancelled(): boolean { return this._cancelled; } - setCancelled() { this._cancelled = true; } -} -``` - -- [ ] **Step 5: Implement ServerLogger** - -```typescript -// sdk/typescript/src/log.ts -const LEVELS = ['debug', 'info', 'notice', 'warning', 'error', 'critical', 'alert', 'emergency'] as const; -type LogLevel = typeof LEVELS[number]; - -export class ServerLogger { - constructor( - private sendFn: (msg: any) => void, - private name?: string, - ) {} - - private _log(level: LogLevel, message: string, data?: Record) { - this.sendFn({ - log: { - level, - logger: this.name ?? '', - dataJson: JSON.stringify(data ?? { message }), - }, - }); - } - - debug(msg: string, data?: Record) { this._log('debug', msg, data); } - info(msg: string, data?: Record) { this._log('info', msg, data); } - notice(msg: string, data?: Record) { this._log('notice', msg, data); } - warning(msg: string, data?: Record) { this._log('warning', msg, data); } - error(msg: string, data?: Record) { this._log('error', msg, data); } - critical(msg: string, data?: Record) { this._log('critical', msg, data); } - alert(msg: string, data?: Record) { this._log('alert', msg, data); } - emergency(msg: string, data?: Record) { this._log('emergency', msg, data); } -} -``` - -- [ ] **Step 6: Update tool() to support output, metadata, and task_support** - -Update `sdk/typescript/src/tool.ts` to accept `output` (Zod schema), `title`, `destructiveHint`, `idempotentHint`, `readOnlyHint`, `openWorldHint`, and `taskSupport` options. Generate `output_schema_json` from the Zod output schema. Set corresponding fields on the `ToolDefinition`. - -Add tests to `sdk/typescript/tests/tool.test.ts`: - -```typescript -it('generates output schema from Zod', () => { - const OutputSchema = z.object({ title: z.string(), score: z.number() }); - const t = tool({ - name: 'search', - description: 'Search', - args: z.object({ q: z.string() }), - output: OutputSchema, - handler: (args) => ({ title: 'test', score: 0.9 }), - }); - expect(t.definition.outputSchemaJson).toBeTruthy(); - const schema = JSON.parse(t.definition.outputSchemaJson); - expect(schema.properties.title).toBeTruthy(); -}); - -it('includes tool metadata', () => { - const t = tool({ - name: 'delete_doc', - description: 'Delete', - args: z.object({ id: z.string() }), - title: 'Delete Document', - destructiveHint: true, - handler: (args) => 'deleted', - }); - expect(t.definition.title).toBe('Delete Document'); - expect(t.definition.destructiveHint).toBe(true); -}); -``` - -- [ ] **Step 7: Update runner and index exports** - -Update `sdk/typescript/src/runner.ts` to handle new message types (same as Python runner — ToolContext injection, cancel handling, async task lifecycle, log initialization). - -Update `sdk/typescript/src/index.ts`: - -```typescript -export { tool, ToolResult } from './tool'; -export { toolManager } from './manager'; -export { ToolContext } from './context'; -export { ServerLogger } from './log'; -``` - -- [ ] **Step 8: Run all TypeScript tests** - -Run: `cd sdk/typescript && npx vitest run --reporter=verbose` -Expected: PASS - -- [ ] **Step 9: Commit** - -```bash -git add sdk/typescript/ -git commit -m "feat(typescript): add progress, cancellation, logging, structured output, and tool metadata" -``` - ---- - -### Task 42: E2E Tests for New Features - -**Files:** -- Create: `test/e2e/fixtures/progress_tool.py` -- Create: `test/e2e/fixtures/async_tool.py` -- Create: `test/e2e/fixtures/logging_tool.py` -- Create: `test/e2e/fixtures/structured_output_tool.py` -- Modify: `test/e2e/e2e_test.go` - -- [ ] **Step 1: Create test fixtures** - -`progress_tool.py`: -```python -from protomcp import tool, ToolContext - -@tool(description="Count with progress") -def count_with_progress(n: int, ctx: ToolContext) -> str: - for i in range(n): - ctx.report_progress(i + 1, n, f"Step {i + 1}") - return f"Counted to {n}" -``` - -`async_tool.py`: -```python -import asyncio -from protomcp import tool - -@tool(description="Async task", task_support=True) -async def slow_task(duration_ms: int) -> str: - await asyncio.sleep(duration_ms / 1000) - return f"Completed after {duration_ms}ms" -``` - -`logging_tool.py`: -```python -from protomcp import tool, log - -@tool(description="Tool that logs") -def logging_tool(message: str) -> str: - log.info("Tool called", data={"message": message}) - log.debug("Debug detail") - return f"Logged: {message}" -``` - -`structured_output_tool.py`: -```python -from dataclasses import dataclass -from protomcp import tool - -@dataclass -class SearchResult: - title: str - score: float - -@tool(description="Structured search", output_type=SearchResult) -def search(query: str) -> SearchResult: - return SearchResult(title=f"Result for {query}", score=0.95) -``` - -- [ ] **Step 2: Write e2e tests** - -Add to `test/e2e/e2e_test.go`: - -```go -func TestE2E_ProgressNotifications(t *testing.T) { - binary, cleanup := buildBinary(t) - defer cleanup() - client := startWithFixture(t, binary, "progress_tool.py") - defer client.Close() - - notifications := client.CollectNotifications() - result := client.CallTool("count_with_progress", map[string]any{ - "n": 3, - }, map[string]any{"progressToken": "pt-test"}) - - assertNoError(t, result) - progressNotifs := filterNotifications(notifications, "notifications/progress") - if len(progressNotifs) != 3 { - t.Fatalf("expected 3 progress notifications, got %d", len(progressNotifs)) - } -} - -func TestE2E_AsyncTask(t *testing.T) { - binary, cleanup := buildBinary(t) - defer cleanup() - client := startWithFixture(t, binary, "async_tool.py") - defer client.Close() - - result := client.CallTool("slow_task", map[string]any{"duration_ms": 100}) - // Should get a CreateTaskResult with taskId - taskID := result["taskId"].(string) - if taskID == "" { - t.Fatal("expected taskId in response") - } - - // Poll until complete - var status map[string]any - for i := 0; i < 20; i++ { - status = client.TaskGet(taskID) - if status["state"] == "completed" { - break - } - time.Sleep(50 * time.Millisecond) - } - if status["state"] != "completed" { - t.Fatalf("expected completed, got %s", status["state"]) - } - - // Get result - taskResult := client.TaskResult(taskID) - assertNoError(t, taskResult) -} - -func TestE2E_ServerLogging(t *testing.T) { - binary, cleanup := buildBinary(t) - defer cleanup() - client := startWithFixture(t, binary, "logging_tool.py") - defer client.Close() - - notifications := client.CollectNotifications() - client.CallTool("logging_tool", map[string]any{"message": "test"}) - - logNotifs := filterNotifications(notifications, "notifications/message") - if len(logNotifs) == 0 { - t.Fatal("expected at least one log notification") - } -} - -func TestE2E_StructuredOutput(t *testing.T) { - binary, cleanup := buildBinary(t) - defer cleanup() - client := startWithFixture(t, binary, "structured_output_tool.py") - defer client.Close() - - // Verify tools/list includes outputSchema - tools := client.ListTools() - searchTool := findTool(tools, "search") - if searchTool["outputSchema"] == nil { - t.Fatal("expected outputSchema on search tool") - } - - // Verify call result includes structuredContent - result := client.CallTool("search", map[string]any{"query": "test"}) - if result["structuredContent"] == nil { - t.Fatal("expected structuredContent in result") - } -} -``` - -- [ ] **Step 3: Run e2e tests** - -Run: `go test ./test/e2e/... -v -timeout 60s -run "Progress|Async|Logging|Structured"` -Expected: PASS - -- [ ] **Step 4: Commit** - -```bash -git add test/e2e/ -git commit -m "test: add e2e tests for progress, async tasks, server logging, and structured output" -``` - ---- - -### Task 43: Documentation — New Feature Guides - -**Files:** -- Create: `docs/src/content/docs/guides/progress-notifications.mdx` -- Create: `docs/src/content/docs/guides/async-tasks.mdx` -- Create: `docs/src/content/docs/guides/cancellation.mdx` -- Create: `docs/src/content/docs/guides/server-logging.mdx` -- Create: `docs/src/content/docs/guides/structured-output.mdx` - -- [ ] **Step 1: Write progress notifications guide** - -Cover: how to use `ToolContext.report_progress()`, what happens with/without progressToken, Python and TS examples. - -- [ ] **Step 2: Write async tasks guide** - -Cover: `task_support=True`, how async tools work, task lifecycle (create → poll → result), cancellation of tasks, Python and TS examples. - -- [ ] **Step 3: Write cancellation guide** - -Cover: `ctx.is_cancelled()`, cooperative cancellation pattern, how it interacts with async tasks, Python and TS examples. - -- [ ] **Step 4: Write server logging guide** - -Cover: log levels (RFC 5424), `log.info()`/`log.debug()` etc., how `--log-level` filters, named loggers, Python and TS examples. - -- [ ] **Step 5: Write structured output guide** - -Cover: `output_type`/`output` parameter, how `outputSchema` appears in tool definitions, `structuredContent` in results, Python and TS examples. - -- [ ] **Step 6: Update Starlight sidebar config** - -Add new guide pages to the sidebar in `docs/astro.config.mjs`. - -- [ ] **Step 7: Build docs to verify** - -Run: `cd docs && npm run build` -Expected: builds successfully. - -- [ ] **Step 8: Commit** - -```bash -git add docs/ -git commit -m "docs: add guides for progress, async tasks, cancellation, logging, and structured output" -``` - ---- - -### Task 31: Final Verification - -- [ ] **Step 1: Run all Go tests** - -```bash -make test -``` - -Expected: all tests pass. - -- [ ] **Step 2: Run Python SDK tests** - -```bash -make test-python -``` - -Expected: all tests pass. - -- [ ] **Step 3: Run TypeScript SDK tests** - -```bash -make test-ts -``` - -Expected: all tests pass. - -- [ ] **Step 4: Run e2e tests** - -```bash -go test ./test/e2e/... -v -timeout 60s -``` - -Expected: all tests pass. - -- [ ] **Step 5: Build documentation** - -```bash -cd docs && npm run build -``` - -Expected: builds successfully. - -- [ ] **Step 6: Build release binary** - -```bash -goreleaser build --snapshot --clean -``` - -Expected: binaries for linux/darwin/windows amd64/arm64. - -- [ ] **Step 7: Final commit** - -```bash -git add -A -git commit -m "chore: final v1.0 verification — all tests pass, docs build, release builds" -``` diff --git a/docs/superpowers/plans/2026-03-12-readme-demos-interactive.md b/docs/superpowers/plans/2026-03-12-readme-demos-interactive.md deleted file mode 100644 index 71874a1..0000000 --- a/docs/superpowers/plans/2026-03-12-readme-demos-interactive.md +++ /dev/null @@ -1,1695 +0,0 @@ -# README, Working Demos & Interactive Demo Page — Implementation Plan - -> **For agentic workers:** REQUIRED: Use superpowers:subagent-driven-development (if subagents available) or superpowers:executing-plans to implement this plan. Steps use checkbox (`- [ ]`) syntax for tracking. - -**Goal:** Create a README, runnable examples at three tiers in Python+TypeScript, and an animated interactive demo page in the Starlight docs site. - -**Architecture:** Three independent deliverables. Examples use the protomcp Python/TypeScript SDKs directly. The demo page uses Astro components embedded in a Starlight MDX page with CSS animations and minimal JS for tab toggling. - -**Tech Stack:** Markdown (README), Python/TypeScript (examples), Astro/Starlight (demo page), CSS @keyframes (animations), bash (demo runner) - ---- - -## Chunk 1: Working Code Examples - -### Task 1: Basic Python Example - -**Files:** -- Create: `examples/python/basic.py` - -- [ ] **Step 1: Create the basic Python example** - -```python -# examples/python/basic.py -# A minimal protomcp tool — adds two numbers. -# Run: pmcp dev examples/python/basic.py - -from protomcp import tool, ToolResult - -@tool("Add two numbers") -def add(a: int, b: int) -> ToolResult: - return ToolResult(result=str(a + b)) - -@tool("Multiply two numbers") -def multiply(a: int, b: int) -> ToolResult: - return ToolResult(result=str(a * b)) -``` - -- [ ] **Step 2: Verify it parses** - -Run: `python -c "import ast; ast.parse(open('examples/python/basic.py').read()); print('OK')"` -Expected: `OK` - -- [ ] **Step 3: Commit** - -```bash -git add examples/python/basic.py -git commit -m "examples: add basic Python example" -``` - ---- - -### Task 2: Basic TypeScript Example - -**Files:** -- Create: `examples/typescript/basic.ts` - -- [ ] **Step 1: Create the basic TypeScript example** - -```typescript -// examples/typescript/basic.ts -// A minimal protomcp tool — adds two numbers. -// Run: pmcp dev examples/typescript/basic.ts - -import { tool, ToolResult } from 'protomcp'; -import { z } from 'zod'; - -tool({ - name: 'add', - description: 'Add two numbers', - args: z.object({ a: z.number(), b: z.number() }), - handler({ a, b }) { - return new ToolResult({ result: String(a + b) }); - }, -}); - -tool({ - name: 'multiply', - description: 'Multiply two numbers', - args: z.object({ a: z.number(), b: z.number() }), - handler({ a, b }) { - return new ToolResult({ result: String(a * b) }); - }, -}); -``` - -- [ ] **Step 2: Commit** - -```bash -git add examples/typescript/basic.ts -git commit -m "examples: add basic TypeScript example" -``` - ---- - -### Task 3: Real-World Python Example - -**Files:** -- Create: `examples/python/real_world.py` - -- [ ] **Step 1: Create the real-world Python example** - -This is a file search tool that demonstrates progress reporting, logging, and cancellation. - -```python -# examples/python/real_world.py -# A file search tool demonstrating progress, logging, and cancellation. -# Run: pmcp dev examples/python/real_world.py - -import os -import fnmatch -from protomcp import tool, ToolResult, ToolContext, log - -@tool("Search files in a directory by glob pattern", read_only=True) -def search_files(ctx: ToolContext, directory: str, pattern: str, max_results: int = 50) -> ToolResult: - log.info(f"Searching {directory} for '{pattern}'") - - if not os.path.isdir(directory): - return ToolResult( - result=f"Directory not found: {directory}", - is_error=True, - error_code="INVALID_PATH", - message="The specified directory does not exist", - suggestion="Check the path and try again", - ) - - matches = [] - all_files = [] - for root, dirs, files in os.walk(directory): - for f in files: - all_files.append(os.path.join(root, f)) - - total = len(all_files) - log.debug(f"Found {total} files to scan") - - for i, filepath in enumerate(all_files): - if ctx.is_cancelled(): - log.warning("Search cancelled by client") - return ToolResult( - result=f"Cancelled after scanning {i}/{total} files. Found {len(matches)} matches so far.", - is_error=True, - error_code="CANCELLED", - retryable=True, - ) - - if i % 100 == 0: - ctx.report_progress(i, total, f"Scanning... {i}/{total}") - - if fnmatch.fnmatch(os.path.basename(filepath), pattern): - matches.append(filepath) - if len(matches) >= max_results: - log.info(f"Hit max_results={max_results}, stopping early") - break - - ctx.report_progress(total, total, "Complete") - log.info(f"Search complete: {len(matches)} matches") - return ToolResult(result="\n".join(matches) if matches else "No files found") -``` - -- [ ] **Step 2: Verify it parses** - -Run: `python -c "import ast; ast.parse(open('examples/python/real_world.py').read()); print('OK')"` -Expected: `OK` - -- [ ] **Step 3: Commit** - -```bash -git add examples/python/real_world.py -git commit -m "examples: add real-world Python example with progress and cancellation" -``` - ---- - -### Task 4: Real-World TypeScript Example - -**Files:** -- Create: `examples/typescript/real-world.ts` - -- [ ] **Step 1: Create the real-world TypeScript example** - -```typescript -// examples/typescript/real-world.ts -// A file search tool demonstrating progress, logging, and cancellation. -// Run: pmcp dev examples/typescript/real-world.ts - -import { tool, ToolResult, ToolContext, ServerLogger } from 'protomcp'; -import { z } from 'zod'; -import * as fs from 'fs'; -import * as path from 'path'; - -// Note: ServerLogger requires a transport send function. In a real pmcp process, -// this is wired up automatically by the runner. For demonstration purposes, -// we show the API shape — logging calls are forwarded to the MCP host. - -tool({ - name: 'search_files', - description: 'Search files in a directory by glob pattern', - readOnlyHint: true, - args: z.object({ - directory: z.string(), - pattern: z.string(), - max_results: z.number().default(50), - }), - handler({ directory, pattern, max_results }, ctx: ToolContext) { - if (!fs.existsSync(directory)) { - return new ToolResult({ - result: `Directory not found: ${directory}`, - isError: true, - errorCode: 'INVALID_PATH', - message: 'The specified directory does not exist', - suggestion: 'Check the path and try again', - }); - } - - const matches: string[] = []; - const allFiles: string[] = []; - - function walk(dir: string) { - for (const entry of fs.readdirSync(dir, { withFileTypes: true })) { - const full = path.join(dir, entry.name); - if (entry.isDirectory()) walk(full); - else allFiles.push(full); - } - } - walk(directory); - - const total = allFiles.length; - - for (let i = 0; i < total; i++) { - if (ctx.isCancelled()) { - return new ToolResult({ - result: `Cancelled after scanning ${i}/${total} files. Found ${matches.length} matches so far.`, - isError: true, - errorCode: 'CANCELLED', - retryable: true, - }); - } - - if (i % 100 === 0) { - ctx.reportProgress(i, total, `Scanning... ${i}/${total}`); - } - - if (matchGlob(path.basename(allFiles[i]), pattern)) { - matches.push(allFiles[i]); - if (matches.length >= max_results) break; - } - } - - ctx.reportProgress(total, total, 'Complete'); - return new ToolResult({ - result: matches.length > 0 ? matches.join('\n') : 'No files found', - }); - }, -}); - -function matchGlob(filename: string, pattern: string): boolean { - const regex = new RegExp( - '^' + pattern.replace(/\*/g, '.*').replace(/\?/g, '.') + '$' - ); - return regex.test(filename); -} -``` - -- [ ] **Step 2: Commit** - -```bash -git add examples/typescript/real-world.ts -git commit -m "examples: add real-world TypeScript example with progress and cancellation" -``` - ---- - -### Task 5: Full Showcase Python Example - -**Files:** -- Create: `examples/python/full_showcase.py` - -- [ ] **Step 1: Create the full showcase Python example** - -Multi-tool server demonstrating structured output, dynamic tool lists, metadata, progress, cancellation, and logging. - -```python -# examples/python/full_showcase.py -# Full-featured protomcp demo — multiple tools showcasing the complete API. -# Run: pmcp dev examples/python/full_showcase.py - -import json -import time -from dataclasses import dataclass -from protomcp import tool, ToolResult, ToolContext, log -from protomcp import tool_manager - -# --- Tool 1: Structured output with output schema --- - -@dataclass -class WeatherData: - location: str - temperature_f: float - conditions: str - humidity: int - -@tool( - "Get current weather for a location", - output_type=WeatherData, - read_only=True, - title="Weather Lookup", -) -def get_weather(location: str) -> ToolResult: - log.info(f"Weather lookup for {location}") - # Simulated weather data - data = WeatherData( - location=location, - temperature_f=72.5, - conditions="Partly cloudy", - humidity=45, - ) - return ToolResult(result=json.dumps({ - "location": data.location, - "temperature_f": data.temperature_f, - "conditions": data.conditions, - "humidity": data.humidity, - })) - -# --- Tool 2: Long-running operation with progress --- - -@tool( - "Analyze a dataset (simulated long-running task)", - title="Dataset Analyzer", - idempotent=True, - task_support=True, -) -def analyze_dataset(ctx: ToolContext, dataset_name: str, depth: str = "basic") -> ToolResult: - log.info(f"Starting analysis of {dataset_name} at depth={depth}") - steps = 10 - - for i in range(steps): - if ctx.is_cancelled(): - log.warning(f"Analysis cancelled at step {i}/{steps}") - return ToolResult( - result=f"Analysis cancelled at step {i}/{steps}", - is_error=True, - error_code="CANCELLED", - retryable=True, - ) - ctx.report_progress(i, steps, f"Analyzing step {i+1}/{steps}...") - time.sleep(0.1) # Simulate work - - ctx.report_progress(steps, steps, "Analysis complete") - log.info("Analysis finished successfully") - return ToolResult(result=json.dumps({ - "dataset": dataset_name, - "depth": depth, - "rows_analyzed": 15000, - "anomalies_found": 3, - "summary": "Dataset is healthy with 3 minor anomalies detected.", - })) - -# --- Tool 3: Dynamic tool list management --- - -@tool( - "Enable or disable tools at runtime", - title="Tool Manager", - destructive=True, -) -def manage_tools(action: str, tool_names: str) -> ToolResult: - names = [n.strip() for n in tool_names.split(",")] - log.info(f"manage_tools: action={action}, names={names}") - - if action == "enable": - active = tool_manager.enable(names) - elif action == "disable": - active = tool_manager.disable(names) - elif action == "list": - active = tool_manager.get_active_tools() - else: - return ToolResult( - result=f"Unknown action: {action}", - is_error=True, - error_code="INVALID_ACTION", - suggestion="Use 'enable', 'disable', or 'list'", - ) - - return ToolResult(result=json.dumps({"active_tools": active})) - -# --- Tool 4: Demonstrates error handling and logging levels --- - -@tool( - "Validate data against a schema (demonstrates error handling)", - title="Data Validator", - read_only=True, - idempotent=True, -) -def validate_data(data_json: str, strict: bool = False) -> ToolResult: - log.debug("Starting validation") - - try: - data = json.loads(data_json) - except json.JSONDecodeError as e: - log.error(f"Invalid JSON: {e}") - return ToolResult( - result=f"Invalid JSON: {e}", - is_error=True, - error_code="PARSE_ERROR", - message="The input is not valid JSON", - suggestion="Check for syntax errors and try again", - retryable=True, - ) - - issues = [] - if not isinstance(data, dict): - issues.append("Root must be an object") - elif "name" not in data: - issues.append("Missing required field: name") - - if strict and isinstance(data, dict): - allowed = {"name", "value", "tags"} - extra = set(data.keys()) - allowed - if extra: - issues.append(f"Unknown fields: {', '.join(extra)}") - - if issues: - log.warning(f"Validation failed: {issues}") - return ToolResult( - result=json.dumps({"valid": False, "issues": issues}), - is_error=True, - error_code="VALIDATION_FAILED", - ) - - log.info("Validation passed") - return ToolResult(result=json.dumps({"valid": True, "issues": []})) -``` - -- [ ] **Step 2: Verify it parses** - -Run: `python -c "import ast; ast.parse(open('examples/python/full_showcase.py').read()); print('OK')"` -Expected: `OK` - -- [ ] **Step 3: Commit** - -```bash -git add examples/python/full_showcase.py -git commit -m "examples: add full showcase Python example with structured output, dynamic tools, and error handling" -``` - ---- - -### Task 6: Full Showcase TypeScript Example - -**Files:** -- Create: `examples/typescript/full-showcase.ts` - -- [ ] **Step 1: Create the full showcase TypeScript example** - -```typescript -// examples/typescript/full-showcase.ts -// Full-featured protomcp demo — multiple tools showcasing the complete API. -// Run: pmcp dev examples/typescript/full-showcase.ts - -import { tool, ToolResult, ToolContext, toolManager, ServerLogger } from 'protomcp'; -import { z } from 'zod'; - -// Note: ServerLogger is wired to the MCP host transport automatically by the runner. -// For demonstration, we show how to create one — in practice, use the runner-provided instance. - -// --- Tool 1: Structured output with output schema --- - -const WeatherOutput = z.object({ - location: z.string(), - temperature_f: z.number(), - conditions: z.string(), - humidity: z.number(), -}); - -tool({ - name: 'get_weather', - description: 'Get current weather for a location', - title: 'Weather Lookup', - readOnlyHint: true, - output: WeatherOutput, - args: z.object({ location: z.string() }), - handler({ location }) { - const data = { - location, - temperature_f: 72.5, - conditions: 'Partly cloudy', - humidity: 45, - }; - return new ToolResult({ result: JSON.stringify(data) }); - }, -}); - -// --- Tool 2: Long-running operation with progress + task support --- - -tool({ - name: 'analyze_dataset', - description: 'Analyze a dataset (simulated long-running task)', - title: 'Dataset Analyzer', - idempotentHint: true, - taskSupport: true, - args: z.object({ - dataset_name: z.string(), - depth: z.enum(['basic', 'deep']).default('basic'), - }), - async handler({ dataset_name, depth }, ctx: ToolContext) { - const steps = 10; - for (let i = 0; i < steps; i++) { - if (ctx.isCancelled()) { - return new ToolResult({ - result: `Analysis cancelled at step ${i}/${steps}`, - isError: true, - errorCode: 'CANCELLED', - retryable: true, - }); - } - ctx.reportProgress(i, steps, `Analyzing step ${i + 1}/${steps}...`); - await new Promise(r => setTimeout(r, 100)); // Simulate work - } - ctx.reportProgress(steps, steps, 'Analysis complete'); - return new ToolResult({ - result: JSON.stringify({ - dataset: dataset_name, - depth, - rows_analyzed: 15000, - anomalies_found: 3, - summary: 'Dataset is healthy with 3 minor anomalies detected.', - }), - }); - }, -}); - -// --- Tool 3: Dynamic tool list management --- - -tool({ - name: 'manage_tools', - description: 'Enable or disable tools at runtime', - title: 'Tool Manager', - destructiveHint: true, - args: z.object({ - action: z.enum(['enable', 'disable', 'list']), - tool_names: z.string().describe('Comma-separated tool names'), - }), - async handler({ action, tool_names }) { - const names = tool_names.split(',').map(n => n.trim()); - let active: string[]; - switch (action) { - case 'enable': - active = await toolManager.enable(names); - break; - case 'disable': - active = await toolManager.disable(names); - break; - case 'list': - active = await toolManager.getActiveTools(); - break; - } - return new ToolResult({ result: JSON.stringify({ active_tools: active }) }); - }, -}); - -// --- Tool 4: Error handling, validation, and logging --- - -tool({ - name: 'validate_data', - description: 'Validate data against a schema (demonstrates error handling)', - title: 'Data Validator', - readOnlyHint: true, - idempotentHint: true, - args: z.object({ - data_json: z.string(), - strict: z.boolean().default(false), - }), - handler({ data_json, strict }) { - let data: unknown; - try { - data = JSON.parse(data_json); - } catch (e) { - return new ToolResult({ - result: `Invalid JSON: ${e}`, - isError: true, - errorCode: 'PARSE_ERROR', - message: 'The input is not valid JSON', - suggestion: 'Check for syntax errors and try again', - retryable: true, - }); - } - - const issues: string[] = []; - if (typeof data !== 'object' || data === null || Array.isArray(data)) { - issues.push('Root must be an object'); - } else { - if (!('name' in data)) issues.push('Missing required field: name'); - if (strict) { - const allowed = new Set(['name', 'value', 'tags']); - const extra = Object.keys(data).filter(k => !allowed.has(k)); - if (extra.length) issues.push(`Unknown fields: ${extra.join(', ')}`); - } - } - - if (issues.length) { - return new ToolResult({ - result: JSON.stringify({ valid: false, issues }), - isError: true, - errorCode: 'VALIDATION_FAILED', - }); - } - - return new ToolResult({ result: JSON.stringify({ valid: true, issues: [] }) }); - }, -}); -``` - -- [ ] **Step 2: Commit** - -```bash -git add examples/typescript/full-showcase.ts -git commit -m "examples: add full showcase TypeScript example with structured output, dynamic tools, and error handling" -``` - ---- - -### Task 7: Example Dependency Files - -**Files:** -- Create: `examples/python/requirements.txt` -- Create: `examples/typescript/package.json` -- Create: `examples/typescript/tsconfig.json` - -- [ ] **Step 1: Create Python requirements.txt** - -``` -protomcp -``` - -- [ ] **Step 2: Create TypeScript package.json** - -```json -{ - "name": "protomcp-examples", - "private": true, - "type": "module", - "dependencies": { - "protomcp": "file:../../sdk/typescript", - "zod": "^3.22.0" - } -} -``` - -Note: Uses `file:` reference to the local SDK so examples work without publishing to npm. - -- [ ] **Step 3: Create TypeScript tsconfig.json** - -```json -{ - "compilerOptions": { - "target": "ES2022", - "module": "ES2022", - "moduleResolution": "node", - "esModuleInterop": true, - "strict": true, - "outDir": "dist" - }, - "include": ["*.ts"] -} -``` - -- [ ] **Step 4: Commit** - -```bash -git add examples/python/requirements.txt examples/typescript/package.json examples/typescript/tsconfig.json -git commit -m "examples: add dependency files for Python and TypeScript examples" -``` - ---- - -### Task 8: Demo Runner Script - -**Files:** -- Create: `examples/run-demo.sh` - -- [ ] **Step 1: Create the demo runner script** - -This script starts `pmcp dev` for each example, sends JSON-RPC messages over stdio, and prints human-readable output. It performs the full MCP handshake: `initialize` → `initialized` notification → `tools/list` → `tools/call`. - -```bash -#!/usr/bin/env bash -set -euo pipefail - -# examples/run-demo.sh -# Runs each example through pmcp, demonstrating the MCP protocol interaction. -# Requires: pmcp installed and on PATH - -RED='\033[0;31m' -GREEN='\033[0;32m' -CYAN='\033[0;36m' -YELLOW='\033[1;33m' -NC='\033[0m' - -SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)" -PASSED=0 -FAILED=0 - -if ! command -v pmcp &> /dev/null; then - echo -e "${RED}Error: pmcp not found. Install it first: brew install protomcp/tap/protomcp${NC}" - exit 1 -fi - -# Send a JSON-RPC request to a pmcp process and capture the response. -# Usage: run_example