From 775cbdb822c171de631072c60bb126134b836d43 Mon Sep 17 00:00:00 2001 From: Michael S Date: Sat, 14 Mar 2026 23:54:50 -0400 Subject: [PATCH 1/7] fix(process): lock m.streams map access in readLoop to prevent data race The stream reassembly map was accessed in readLoop without holding m.mu, causing potential "concurrent map read and map write" panics when CallToolStream runs concurrently. --- .../plans/2026-03-12-protomcp-v1.1.md | 2751 --------- .../plans/2026-03-12-protomcp-v1.md | 4992 ----------------- .../2026-03-12-readme-demos-interactive.md | 1695 ------ .../plans/2026-03-13-chunked-streaming.md | 2054 ------- .../plans/2026-03-14-harness-ml-migration.md | 1775 ------ .../2026-03-14-server-defined-workflows.md | 1405 ----- .../2026-03-14-test-engine-playground.md | 1584 ------ .../specs/2026-03-12-protomcp-design.md | 839 --- .../specs/2026-03-12-protomcp-v1.1-design.md | 368 -- ...6-03-12-readme-demos-interactive-design.md | 200 - .../2026-03-13-chunked-streaming-design.md | 283 - ...6-03-14-server-defined-workflows-design.md | 483 -- ...026-03-14-test-engine-playground-design.md | 492 -- internal/process/manager.go | 8 + .../__pycache__/protomcp_pb2.cpython-312.pyc | Bin 13365 -> 0 bytes .../__pycache__/__init__.cpython-312.pyc | Bin 2214 -> 0 bytes .../__pycache__/completion.cpython-312.pyc | Bin 1628 -> 0 bytes .../__pycache__/context.cpython-312.pyc | Bin 5392 -> 0 bytes .../protomcp/__pycache__/log.cpython-312.pyc | Bin 2905 -> 0 bytes .../__pycache__/manager.cpython-312.pyc | Bin 3784 -> 0 bytes .../__pycache__/middleware.cpython-312.pyc | Bin 1501 -> 0 bytes .../__pycache__/prompt.cpython-312.pyc | Bin 2128 -> 0 bytes .../__pycache__/resource.cpython-312.pyc | Bin 3214 -> 0 bytes .../__pycache__/result.cpython-312.pyc | Bin 841 -> 0 bytes .../__pycache__/runner.cpython-312.pyc | Bin 21429 -> 0 bytes .../protomcp/__pycache__/tool.cpython-312.pyc | Bin 7501 -> 0 bytes .../__pycache__/transport.cpython-312.pyc | Bin 5190 -> 0 bytes 27 files changed, 8 insertions(+), 18921 deletions(-) delete mode 100644 docs/superpowers/plans/2026-03-12-protomcp-v1.1.md delete mode 100644 docs/superpowers/plans/2026-03-12-protomcp-v1.md delete mode 100644 docs/superpowers/plans/2026-03-12-readme-demos-interactive.md delete mode 100644 docs/superpowers/plans/2026-03-13-chunked-streaming.md delete mode 100644 docs/superpowers/plans/2026-03-14-harness-ml-migration.md delete mode 100644 docs/superpowers/plans/2026-03-14-server-defined-workflows.md delete mode 100644 docs/superpowers/plans/2026-03-14-test-engine-playground.md delete mode 100644 docs/superpowers/specs/2026-03-12-protomcp-design.md delete mode 100644 docs/superpowers/specs/2026-03-12-protomcp-v1.1-design.md delete mode 100644 docs/superpowers/specs/2026-03-12-readme-demos-interactive-design.md delete mode 100644 docs/superpowers/specs/2026-03-13-chunked-streaming-design.md delete mode 100644 docs/superpowers/specs/2026-03-14-server-defined-workflows-design.md delete mode 100644 docs/superpowers/specs/2026-03-14-test-engine-playground-design.md delete mode 100644 sdk/python/gen/__pycache__/protomcp_pb2.cpython-312.pyc delete mode 100644 sdk/python/src/protomcp/__pycache__/__init__.cpython-312.pyc delete mode 100644 sdk/python/src/protomcp/__pycache__/completion.cpython-312.pyc delete mode 100644 sdk/python/src/protomcp/__pycache__/context.cpython-312.pyc delete mode 100644 sdk/python/src/protomcp/__pycache__/log.cpython-312.pyc delete mode 100644 sdk/python/src/protomcp/__pycache__/manager.cpython-312.pyc delete mode 100644 sdk/python/src/protomcp/__pycache__/middleware.cpython-312.pyc delete mode 100644 sdk/python/src/protomcp/__pycache__/prompt.cpython-312.pyc delete mode 100644 sdk/python/src/protomcp/__pycache__/resource.cpython-312.pyc delete mode 100644 sdk/python/src/protomcp/__pycache__/result.cpython-312.pyc delete mode 100644 sdk/python/src/protomcp/__pycache__/runner.cpython-312.pyc delete mode 100644 sdk/python/src/protomcp/__pycache__/tool.cpython-312.pyc delete mode 100644 sdk/python/src/protomcp/__pycache__/transport.cpython-312.pyc diff --git a/docs/superpowers/plans/2026-03-12-protomcp-v1.1.md b/docs/superpowers/plans/2026-03-12-protomcp-v1.1.md deleted file mode 100644 index a5d1c0a..0000000 --- a/docs/superpowers/plans/2026-03-12-protomcp-v1.1.md +++ /dev/null @@ -1,2751 +0,0 @@ -# protomcp v1.1 Implementation Plan - -> **For agentic workers:** REQUIRED: Use superpowers:subagent-driven-development (if subagents available) or superpowers:executing-plans to implement this plan. Steps use checkbox (`- [ ]`) syntax for tracking. - -**Goal:** Expand protomcp with Go and Rust SDKs, custom and auth middleware, build-time validation, CI/publishing pipelines, and comprehensive documentation. - -**Architecture:** Builds on v1.0's Go binary + protobuf-over-unix-socket architecture. Adds two new SDK implementations (Go, Rust) following the same module structure as Python/TypeScript SDKs. Extends the protobuf protocol with four new middleware messages (field numbers 24-27). Adds a `validate` subcommand and `--auth` flag to the CLI. CI/CD via GitHub Actions. - -**Tech Stack:** Go 1.25+, Rust (stable, prost, tokio), Protocol Buffers 3, GitHub Actions, GoReleaser - -**Spec:** `docs/superpowers/specs/2026-03-12-protomcp-v1.1-design.md` - ---- - -## File Structure - -``` -# New files -sdk/go/ -├── go.mod -├── protomcp/ -│ ├── tool.go # Tool() registration with functional options -│ ├── tool_test.go -│ ├── result.go # ToolResult struct -│ ├── result_test.go -│ ├── context.go # ToolContext (progress, cancellation) -│ ├── context_test.go -│ ├── manager.go # ToolManager for dynamic enable/disable -│ ├── manager_test.go -│ ├── log.go # Server logging (8 levels) -│ ├── log_test.go -│ ├── transport.go # Unix socket + envelope framing -│ ├── transport_test.go -│ └── runner.go # Main loop: connect, handshake, dispatch -│ └── runner_test.go - -sdk/rust/ -├── Cargo.toml -├── build.rs # prost-build for protobuf generation -├── src/ -│ ├── lib.rs # Public API exports -│ ├── tool.rs # Builder pattern registration -│ ├── result.rs # ToolResult struct -│ ├── context.rs # ToolContext (progress, cancellation) -│ ├── manager.rs # ToolManager for dynamic enable/disable -│ ├── log.rs # Server logging -│ ├── transport.rs # Unix socket + envelope framing -│ └── runner.rs # Main async loop -├── tests/ -│ ├── tool_test.rs -│ ├── result_test.rs -│ ├── transport_test.rs -│ └── integration_test.rs - -internal/middleware/ -├── auth.go # NEW: built-in auth middleware -├── auth_test.go # NEW -├── custom.go # NEW: user-registered middleware dispatch -├── custom_test.go # NEW - -internal/validate/ -├── validate.go # NEW: tool definition validation -├── validate_test.go # NEW - -.github/workflows/ -├── ci.yml # NEW: CI pipeline -├── release.yml # NEW: release/publishing pipeline - -examples/go/ -├── basic.go -├── real_world.go -├── full_showcase.go - -examples/rust/ -├── basic/Cargo.toml + src/main.rs -├── real_world/Cargo.toml + src/main.rs -├── full_showcase/Cargo.toml + src/main.rs - -docs/src/content/docs/guides/ -├── writing-tools-go.mdx # NEW -├── writing-tools-rust.mdx # NEW -├── middleware.mdx # NEW -├── auth.mdx # NEW -├── writing-a-language-library.mdx # NEW - -# Modified files -proto/protomcp.proto # Add middleware messages (fields 24-27) -gen/proto/protomcp/protomcp.pb.go # Regenerated -sdk/python/gen/protomcp_pb2.py # Regenerated -sdk/typescript/gen/ # Regenerated -cmd/protomcp/main.go # Add validate command, --auth flag -internal/config/config.go # Add Auth, Strict, Format fields; validate command; fix Rust runtime -internal/config/config_test.go # Tests for new fields -internal/process/manager.go # Extended handshake for middleware registration -internal/process/manager_test.go # Tests for extended handshake -internal/middleware/chain.go # Priority-based ordering with custom middleware -sdk/python/src/protomcp/runner.py # Middleware registration support -sdk/python/src/protomcp/__init__.py # Export middleware -sdk/typescript/src/runner.ts # Middleware registration support -sdk/typescript/src/index.ts # Export middleware -README.md # Badges, Go/Rust examples, features -docs/src/content/docs/index.mdx # Add Go/Rust cards -docs/src/content/docs/getting-started/quick-start.mdx # Go/Rust snippets -docs/src/content/docs/getting-started/installation.mdx # Go/Rust install -docs/src/content/docs/reference/cli.mdx # validate, --auth -``` - ---- - -## Chunk 1: Protobuf + Config + Rust Runtime Fix - -### Task 1: Add middleware messages to protobuf - -**Files:** -- Modify: `proto/protomcp.proto` - -- [ ] **Step 1: Write the new message types and envelope fields** - -Add after the `TaskCancelRequest` message (line 170) in `proto/protomcp.proto`: - -```protobuf -// --- Middleware --- - -message RegisterMiddlewareRequest { - string name = 1; - int32 priority = 2; // lower runs first -} - -message RegisterMiddlewareResponse { - bool success = 1; -} - -message MiddlewareInterceptRequest { - string middleware_name = 1; - string phase = 2; // "before" or "after" - string tool_name = 3; - string arguments_json = 4; - string result_json = 5; // empty for "before" phase - bool is_error = 6; // only set in "after" phase -} - -message MiddlewareInterceptResponse { - string arguments_json = 1; // potentially modified (before phase) - string result_json = 2; // potentially modified (after phase) - bool reject = 3; // if true, abort the call - string reject_reason = 4; -} -``` - -Add these fields to the `Envelope.oneof msg` block (after `task_cancel = 23`): - -```protobuf - RegisterMiddlewareRequest register_middleware = 24; - RegisterMiddlewareResponse register_middleware_response = 25; - MiddlewareInterceptRequest middleware_intercept = 26; - MiddlewareInterceptResponse middleware_intercept_response = 27; -``` - -- [ ] **Step 2: Regenerate all protobuf code** - -Run: `make proto` -Expected: `gen/proto/protomcp/protomcp.pb.go`, `sdk/python/gen/protomcp_pb2.py`, and `sdk/typescript/gen/` are all updated with the new middleware message types. The Makefile handles Go, Python, and TypeScript generation in one command. - -- [ ] **Step 6: Verify all existing tests still pass** - -Run: `go test ./... && cd sdk/python && python -m pytest && cd ../typescript && npx vitest run` -Expected: All tests pass (no regressions from protobuf changes). - -- [ ] **Step 7: Commit** - -```bash -git add proto/protomcp.proto gen/ sdk/python/gen/ sdk/typescript/gen/ -git commit -m "proto: add middleware messages (fields 24-27)" -``` - ---- - -### Task 2: Fix Rust runtime detection bug + add validate command to config - -**Files:** -- Modify: `internal/config/config.go` -- Modify: `internal/config/config_test.go` - -- [ ] **Step 1: Write failing test for Rust runtime fix** - -Add to `config_test.go`: - -```go -func TestRuntimeCommandRust(t *testing.T) { - cmd, args := config.RuntimeCommand("tools.rs") - if cmd != "cargo" { - t.Errorf("cmd = %q, want %q", cmd, "cargo") - } - // cargo run does NOT take a file argument; it runs the package - expected := []string{"run", "--manifest-path", "Cargo.toml"} - if !reflect.DeepEqual(args, expected) { - t.Errorf("args = %v, want %v", args, expected) - } -} -``` - -- [ ] **Step 2: Run test to verify it fails** - -Run: `go test ./internal/config/ -run TestRuntimeCommandRust -v` -Expected: FAIL — currently returns `["run", "tools.rs"]`. - -- [ ] **Step 3: Fix RuntimeCommand for .rs files** - -In `config.go`, change the `.rs` case in `RuntimeCommand`: - -```go - case ".rs": - dir := filepath.Dir(file) - manifestPath := filepath.Join(dir, "Cargo.toml") - return "cargo", []string{"run", "--manifest-path", manifestPath} -``` - -- [ ] **Step 4: Run test to verify it passes** - -Run: `go test ./internal/config/ -run TestRuntimeCommandRust -v` -Expected: PASS. - -- [ ] **Step 5: Update existing Rust test case in TestRuntimeCommand table** - -Change the `server.rs` entry to expect the new args format: - -```go - {"server.rs", "cargo", []string{"run", "--manifest-path", "Cargo.toml"}}, -``` - -- [ ] **Step 6: Run all config tests** - -Run: `go test ./internal/config/ -v` -Expected: All pass. - -- [ ] **Step 7: Write failing test for validate command parsing** - -Add to `config_test.go`: - -```go -func TestParseValidateCommand(t *testing.T) { - cfg, err := config.Parse([]string{"validate", "tools.py"}) - if err != nil { - t.Fatalf("Parse failed: %v", err) - } - if cfg.Command != "validate" { - t.Errorf("Command = %q, want %q", cfg.Command, "validate") - } - if cfg.File != "tools.py" { - t.Errorf("File = %q, want %q", cfg.File, "tools.py") - } -} - -func TestParseValidateWithStrict(t *testing.T) { - cfg, err := config.Parse([]string{"validate", "tools.py", "--strict"}) - if err != nil { - t.Fatalf("Parse failed: %v", err) - } - if !cfg.Strict { - t.Error("Strict should be true") - } -} - -func TestParseValidateWithFormatJSON(t *testing.T) { - cfg, err := config.Parse([]string{"validate", "tools.py", "--format", "json"}) - if err != nil { - t.Fatalf("Parse failed: %v", err) - } - if cfg.Format != "json" { - t.Errorf("Format = %q, want %q", cfg.Format, "json") - } -} - -func TestParseAuthFlag(t *testing.T) { - cfg, err := config.Parse([]string{"dev", "tools.py", "--auth", "token:MY_TOKEN"}) - if err != nil { - t.Fatalf("Parse failed: %v", err) - } - if len(cfg.Auth) != 1 || cfg.Auth[0] != "token:MY_TOKEN" { - t.Errorf("Auth = %v, want [\"token:MY_TOKEN\"]", cfg.Auth) - } -} - -func TestParseMultipleAuthFlags(t *testing.T) { - cfg, err := config.Parse([]string{"dev", "tools.py", "--auth", "token:T1", "--auth", "apikey:K1"}) - if err != nil { - t.Fatalf("Parse failed: %v", err) - } - if len(cfg.Auth) != 2 { - t.Errorf("Auth length = %d, want 2", len(cfg.Auth)) - } -} - -func TestParseAuthMalformed(t *testing.T) { - _, err := config.Parse([]string{"dev", "tools.py", "--auth", "invalid"}) - if err == nil { - t.Error("expected error for malformed --auth value") - } -} - -func TestParseAuthUnknownScheme(t *testing.T) { - _, err := config.Parse([]string{"dev", "tools.py", "--auth", "oauth:FOO"}) - if err == nil { - t.Error("expected error for unknown auth scheme") - } -} -``` - -- [ ] **Step 8: Run tests to verify they fail** - -Run: `go test ./internal/config/ -run TestParseValidate -v` -Expected: FAIL — "validate" not recognized as valid command. - -- [ ] **Step 9: Update Config struct and Parse function** - -In `config.go`, add fields to `Config`: - -```go -type Config struct { - Command string - File string - Transport string - HotReloadImmediate bool - CallTimeout time.Duration - LogLevel string - SocketPath string - Runtime string - Host string - Port int - Auth []string - Strict bool - Format string -} -``` - -Update `Parse`: -- Change command validation from `cmd != "dev" && cmd != "run"` to `cmd != "dev" && cmd != "run" && cmd != "validate"` -- Add `--auth` flag handling (repeatable, validates format `scheme:value` where scheme is `token` or `apikey`): - -```go - case "--auth": - i++ - if i >= len(args) { - return nil, fmt.Errorf("--auth requires a value") - } - authVal := args[i] - parts := strings.SplitN(authVal, ":", 2) - if len(parts) != 2 || parts[1] == "" { - return nil, fmt.Errorf("--auth value must be scheme:ENV_VAR (got %q)", authVal) - } - scheme := parts[0] - if scheme != "token" && scheme != "apikey" { - return nil, fmt.Errorf("unknown auth scheme %q: must be 'token' or 'apikey'", scheme) - } - cfg.Auth = append(cfg.Auth, authVal) - case "--strict": - cfg.Strict = true - case "--format": - i++ - if i >= len(args) { - return nil, fmt.Errorf("--format requires a value") - } - cfg.Format = args[i] -``` - -Add `"strings"` to imports. - -- [ ] **Step 10: Run all config tests** - -Run: `go test ./internal/config/ -v` -Expected: All pass. - -- [ ] **Step 11: Verify full build** - -Run: `go build ./...` -Expected: Clean build. - -- [ ] **Step 12: Commit** - -```bash -git add internal/config/config.go internal/config/config_test.go -git commit -m "config: add validate command, --auth flag, fix Rust runtime detection" -``` - ---- - -### Task 3: Build-time validation - -**Files:** -- Create: `internal/validate/validate.go` -- Create: `internal/validate/validate_test.go` - -- [ ] **Step 1: Write failing tests for validation logic** - -Create `internal/validate/validate_test.go`: - -```go -package validate_test - -import ( - "testing" - - pb "github.com/msilverblatt/protomcp/gen/proto/protomcp" - "github.com/msilverblatt/protomcp/internal/validate" -) - -func TestValidateValidTools(t *testing.T) { - tools := []*pb.ToolDefinition{ - {Name: "add", Description: "Add two numbers", InputSchemaJson: `{"type":"object","properties":{"a":{"type":"integer"}}}`}, - {Name: "multiply", Description: "Multiply two numbers", InputSchemaJson: `{"type":"object","properties":{"b":{"type":"integer"}}}`}, - } - result := validate.Tools(tools, false) - if !result.Pass { - t.Errorf("expected pass, got errors: %v", result.Errors) - } - if len(result.Tools) != 2 { - t.Errorf("expected 2 tools, got %d", len(result.Tools)) - } -} - -func TestValidateEmptyName(t *testing.T) { - tools := []*pb.ToolDefinition{ - {Name: "", Description: "Something", InputSchemaJson: `{"type":"object"}`}, - } - result := validate.Tools(tools, false) - if result.Pass { - t.Error("expected fail for empty name") - } -} - -func TestValidateInvalidNameChars(t *testing.T) { - tools := []*pb.ToolDefinition{ - {Name: "my-tool", Description: "Has hyphens", InputSchemaJson: `{"type":"object"}`}, - } - result := validate.Tools(tools, false) - if result.Pass { - t.Error("expected fail for invalid name characters") - } -} - -func TestValidateDuplicateNames(t *testing.T) { - tools := []*pb.ToolDefinition{ - {Name: "add", Description: "First", InputSchemaJson: `{"type":"object"}`}, - {Name: "add", Description: "Second", InputSchemaJson: `{"type":"object"}`}, - } - result := validate.Tools(tools, false) - if result.Pass { - t.Error("expected fail for duplicate names") - } -} - -func TestValidateEmptyDescription(t *testing.T) { - tools := []*pb.ToolDefinition{ - {Name: "add", Description: "", InputSchemaJson: `{"type":"object"}`}, - } - result := validate.Tools(tools, false) - if result.Pass { - t.Error("expected fail for empty description") - } -} - -func TestValidateInvalidSchema(t *testing.T) { - tools := []*pb.ToolDefinition{ - {Name: "add", Description: "Add", InputSchemaJson: `not json`}, - } - result := validate.Tools(tools, false) - if result.Pass { - t.Error("expected fail for invalid JSON schema") - } -} - -func TestValidateStrictShortDescription(t *testing.T) { - tools := []*pb.ToolDefinition{ - {Name: "add", Description: "Add", InputSchemaJson: `{"type":"object","properties":{"a":{"type":"integer"}}}`}, - } - result := validate.Tools(tools, true) - if result.Pass { - t.Error("expected fail in strict mode for short description") - } -} - -func TestValidateStrictGenericName(t *testing.T) { - tools := []*pb.ToolDefinition{ - {Name: "test", Description: "A test tool for testing", InputSchemaJson: `{"type":"object","properties":{"a":{"type":"integer"}}}`}, - } - result := validate.Tools(tools, true) - if result.Pass { - t.Error("expected fail in strict mode for generic name") - } -} - -func TestResultFormatText(t *testing.T) { - tools := []*pb.ToolDefinition{ - {Name: "add", Description: "Add two numbers", InputSchemaJson: `{"type":"object"}`}, - } - result := validate.Tools(tools, false) - output := result.FormatText() - if output == "" { - t.Error("expected non-empty text output") - } -} - -func TestResultFormatJSON(t *testing.T) { - tools := []*pb.ToolDefinition{ - {Name: "add", Description: "Add two numbers", InputSchemaJson: `{"type":"object"}`}, - } - result := validate.Tools(tools, false) - output := result.FormatJSON() - if output == "" { - t.Error("expected non-empty JSON output") - } -} -``` - -- [ ] **Step 2: Run tests to verify they fail** - -Run: `go test ./internal/validate/ -v` -Expected: FAIL — package doesn't exist yet. - -- [ ] **Step 3: Implement validation logic** - -Create `internal/validate/validate.go`: - -```go -package validate - -import ( - "encoding/json" - "fmt" - "regexp" - "strings" - - pb "github.com/msilverblatt/protomcp/gen/proto/protomcp" -) - -var validName = regexp.MustCompile(`^[a-zA-Z_][a-zA-Z0-9_]*$`) - -var genericNames = map[string]bool{ - "test": true, "tool1": true, "foo": true, "bar": true, "baz": true, - "temp": true, "tmp": true, "example": true, -} - -type ToolStatus struct { - Name string `json:"name"` - Status string `json:"status"` -} - -type ValidationError struct { - Tool string `json:"tool"` - Issue string `json:"issue"` -} - -type Result struct { - Tools []ToolStatus `json:"tools"` - Errors []ValidationError `json:"errors"` - Pass bool `json:"pass"` -} - -func Tools(tools []*pb.ToolDefinition, strict bool) Result { - result := Result{Pass: true} - seen := make(map[string]bool) - - for _, t := range tools { - status := ToolStatus{Name: t.Name, Status: "ok"} - - if t.Name == "" { - result.Errors = append(result.Errors, ValidationError{Tool: "", Issue: "empty tool name"}) - result.Pass = false - status.Status = "error" - } else if !validName.MatchString(t.Name) { - result.Errors = append(result.Errors, ValidationError{Tool: t.Name, Issue: fmt.Sprintf("invalid name %q: must match [a-zA-Z_][a-zA-Z0-9_]*", t.Name)}) - result.Pass = false - status.Status = "error" - } else if seen[t.Name] { - result.Errors = append(result.Errors, ValidationError{Tool: t.Name, Issue: "duplicate tool name"}) - result.Pass = false - status.Status = "error" - } - seen[t.Name] = true - - if t.Description == "" { - result.Errors = append(result.Errors, ValidationError{Tool: t.Name, Issue: "no description"}) - result.Pass = false - status.Status = "error" - } - - if t.InputSchemaJson != "" { - var schema map[string]interface{} - if err := json.Unmarshal([]byte(t.InputSchemaJson), &schema); err != nil { - result.Errors = append(result.Errors, ValidationError{Tool: t.Name, Issue: fmt.Sprintf("invalid input schema JSON: %v", err)}) - result.Pass = false - status.Status = "error" - } - } - - if strict { - if len(t.Description) < 10 { - result.Errors = append(result.Errors, ValidationError{Tool: t.Name, Issue: fmt.Sprintf("description too short (%d chars, minimum 10)", len(t.Description))}) - result.Pass = false - status.Status = "error" - } - if genericNames[strings.ToLower(t.Name)] { - result.Errors = append(result.Errors, ValidationError{Tool: t.Name, Issue: fmt.Sprintf("generic name %q", t.Name)}) - result.Pass = false - status.Status = "error" - } - } - - result.Tools = append(result.Tools, status) - } - - return result -} - -func (r Result) FormatText() string { - var sb strings.Builder - for _, t := range r.Tools { - if t.Status == "ok" { - sb.WriteString(fmt.Sprintf("✓ %s — OK\n", t.Name)) - } - } - if len(r.Errors) > 0 { - sb.WriteString(fmt.Sprintf("✗ — %d error(s):\n", len(r.Errors))) - for _, e := range r.Errors { - if e.Tool != "" { - sb.WriteString(fmt.Sprintf(" · %q: %s\n", e.Tool, e.Issue)) - } else { - sb.WriteString(fmt.Sprintf(" · %s\n", e.Issue)) - } - } - } - return sb.String() -} - -func (r Result) FormatJSON() string { - b, _ := json.Marshal(r) - return string(b) -} -``` - -- [ ] **Step 4: Run tests to verify they pass** - -Run: `go test ./internal/validate/ -v` -Expected: All pass. - -- [ ] **Step 5: Commit** - -```bash -git add internal/validate/ -git commit -m "feat: add build-time tool validation" -``` - ---- - -### Task 4: Wire validate command and --auth into main.go - -**Files:** -- Modify: `cmd/protomcp/main.go` - -- [ ] **Step 1: Add validate subcommand handling** - -Add an import for the validate package and handle the validate command before the existing dev/run flow. In `main()`, after parsing config, add: - -```go - if cfg.Command == "validate" { - runValidate(ctx, cfg) - return - } -``` - -Add the `runValidate` function: - -```go -func runValidate(ctx context.Context, cfg *config.Config) { - var runtimeCmd string - var runtimeArgs []string - if cfg.Runtime != "" { - runtimeCmd = cfg.Runtime - runtimeArgs = []string{cfg.File} - } else { - runtimeCmd, runtimeArgs = config.RuntimeCommand(cfg.File) - } - - pm := process.NewManager(process.ManagerConfig{ - File: cfg.File, - RuntimeCmd: runtimeCmd, - RuntimeArgs: runtimeArgs, - SocketPath: cfg.SocketPath, - MaxRetries: 1, - CallTimeout: 30 * time.Second, - }) - - tools, err := pm.Start(ctx) - if err != nil { - fmt.Fprintf(os.Stderr, "error: failed to start tool process: %v\n", err) - os.Exit(1) - } - defer pm.Stop() - - result := validate.Tools(tools, cfg.Strict) - - if cfg.Format == "json" { - fmt.Println(result.FormatJSON()) - } else { - fmt.Print(result.FormatText()) - } - - if !result.Pass { - os.Exit(1) - } -} -``` - -Add imports: `"github.com/msilverblatt/protomcp/internal/validate"`, `"time"`. - -- [ ] **Step 2: Verify build** (auth wiring is deferred to Task 5 to avoid compilation errors) - -Run: `go build ./cmd/protomcp/` -Expected: Build succeeds. - -- [ ] **Step 3: Commit** - -```bash -git add cmd/protomcp/main.go -git commit -m "feat: wire validate command into CLI" -``` - ---- - -### Task 5: Auth middleware - -**Files:** -- Create: `internal/middleware/auth.go` -- Create: `internal/middleware/auth_test.go` - -- [ ] **Step 1: Write failing tests** - -Create `internal/middleware/auth_test.go`: - -```go -package middleware_test - -import ( - "context" - "encoding/json" - "os" - "testing" - - "github.com/msilverblatt/protomcp/internal/mcp" - "github.com/msilverblatt/protomcp/internal/middleware" -) - -func TestAuthTokenValid(t *testing.T) { - os.Setenv("TEST_TOKEN", "secret123") - defer os.Unsetenv("TEST_TOKEN") - - mw, err := middleware.NewAuth([]string{"token:TEST_TOKEN"}) - if err != nil { - t.Fatal(err) - } - - next := func(ctx context.Context, req mcp.JSONRPCRequest) (*mcp.JSONRPCResponse, error) { - return &mcp.JSONRPCResponse{ID: req.ID, Result: []byte(`"ok"`)}, nil - } - - handler := mw(next) - ctx := middleware.WithAuthHeader(context.Background(), "Bearer secret123") - resp, err := handler(ctx, mcp.JSONRPCRequest{ID: json.RawMessage(`1`), Method: "tools/call"}) - if err != nil { - t.Fatal(err) - } - if resp == nil { - t.Fatal("expected response") - } -} - -func TestAuthTokenInvalid(t *testing.T) { - os.Setenv("TEST_TOKEN", "secret123") - defer os.Unsetenv("TEST_TOKEN") - - mw, err := middleware.NewAuth([]string{"token:TEST_TOKEN"}) - if err != nil { - t.Fatal(err) - } - - next := func(ctx context.Context, req mcp.JSONRPCRequest) (*mcp.JSONRPCResponse, error) { - return &mcp.JSONRPCResponse{ID: req.ID, Result: []byte(`"ok"`)}, nil - } - - handler := mw(next) - ctx := middleware.WithAuthHeader(context.Background(), "Bearer wrong") - _, err = handler(ctx, mcp.JSONRPCRequest{ID: json.RawMessage(`1`), Method: "tools/call"}) - if err == nil { - t.Fatal("expected auth error") - } -} - -func TestAuthTokenMissing(t *testing.T) { - os.Setenv("TEST_TOKEN", "secret123") - defer os.Unsetenv("TEST_TOKEN") - - mw, err := middleware.NewAuth([]string{"token:TEST_TOKEN"}) - if err != nil { - t.Fatal(err) - } - - next := func(ctx context.Context, req mcp.JSONRPCRequest) (*mcp.JSONRPCResponse, error) { - return &mcp.JSONRPCResponse{}, nil - } - - handler := mw(next) - _, err = handler(context.Background(), mcp.JSONRPCRequest{ID: json.RawMessage(`1`), Method: "tools/call"}) - if err == nil { - t.Fatal("expected auth error for missing header") - } -} - -func TestAuthApikeyValid(t *testing.T) { - os.Setenv("TEST_KEY", "mykey") - defer os.Unsetenv("TEST_KEY") - - mw, err := middleware.NewAuth([]string{"apikey:TEST_KEY"}) - if err != nil { - t.Fatal(err) - } - - next := func(ctx context.Context, req mcp.JSONRPCRequest) (*mcp.JSONRPCResponse, error) { - return &mcp.JSONRPCResponse{ID: req.ID, Result: []byte(`"ok"`)}, nil - } - - handler := mw(next) - ctx := middleware.WithAPIKeyHeader(context.Background(), "mykey") - resp, err := handler(ctx, mcp.JSONRPCRequest{ID: json.RawMessage(`1`), Method: "tools/call"}) - if err != nil { - t.Fatal(err) - } - if resp == nil { - t.Fatal("expected response") - } -} - -func TestNewAuthEnvNotSet(t *testing.T) { - os.Unsetenv("NONEXISTENT_VAR") - _, err := middleware.NewAuth([]string{"token:NONEXISTENT_VAR"}) - if err == nil { - t.Error("expected error for unset env var") - } -} - -func TestNewAuthMultiple(t *testing.T) { - os.Setenv("TEST_TOKEN2", "tok") - os.Setenv("TEST_KEY2", "key") - defer os.Unsetenv("TEST_TOKEN2") - defer os.Unsetenv("TEST_KEY2") - - mw, err := middleware.NewAuth([]string{"token:TEST_TOKEN2", "apikey:TEST_KEY2"}) - if err != nil { - t.Fatal(err) - } - - next := func(ctx context.Context, req mcp.JSONRPCRequest) (*mcp.JSONRPCResponse, error) { - return &mcp.JSONRPCResponse{ID: req.ID, Result: []byte(`"ok"`)}, nil - } - - handler := mw(next) - ctx := middleware.WithAuthHeader(context.Background(), "Bearer tok") - ctx = middleware.WithAPIKeyHeader(ctx, "key") - resp, err := handler(ctx, mcp.JSONRPCRequest{ID: json.RawMessage(`1`), Method: "tools/call"}) - if err != nil { - t.Fatal(err) - } - if resp == nil { - t.Fatal("expected response") - } -} -``` - -- [ ] **Step 2: Run tests to verify they fail** - -Run: `go test ./internal/middleware/ -v` -Expected: FAIL — `NewAuth`, `WithAuthHeader`, `WithAPIKeyHeader` don't exist. - -- [ ] **Step 3: Implement auth middleware** - -Create `internal/middleware/auth.go`: - -```go -package middleware - -import ( - "context" - "fmt" - "os" - "strings" -) - -type authContextKey string - -const ( - authHeaderKey authContextKey = "auth-header" - apiKeyHeaderKey authContextKey = "apikey-header" -) - -// WithAuthHeader adds an Authorization header value to the context. -func WithAuthHeader(ctx context.Context, value string) context.Context { - return context.WithValue(ctx, authHeaderKey, value) -} - -// WithAPIKeyHeader adds an X-API-Key header value to the context. -func WithAPIKeyHeader(ctx context.Context, value string) context.Context { - return context.WithValue(ctx, apiKeyHeaderKey, value) -} - -// GetAuthHeader retrieves the Authorization header from context. -func GetAuthHeader(ctx context.Context) string { - v, _ := ctx.Value(authHeaderKey).(string) - return v -} - -// GetAPIKeyHeader retrieves the X-API-Key header from context. -func GetAPIKeyHeader(ctx context.Context) string { - v, _ := ctx.Value(apiKeyHeaderKey).(string) - return v -} - -type authChecker struct { - scheme string - value string -} - -// NewAuth creates an auth middleware from --auth flag values. -// Each value must be "token:ENV_VAR" or "apikey:ENV_VAR". -// Returns error if env var is not set. -func NewAuth(authSpecs []string) (Middleware, error) { - var checkers []authChecker - - for _, spec := range authSpecs { - parts := strings.SplitN(spec, ":", 2) - if len(parts) != 2 { - return nil, fmt.Errorf("invalid auth spec %q", spec) - } - scheme, envVar := parts[0], parts[1] - value := os.Getenv(envVar) - if value == "" { - return nil, fmt.Errorf("environment variable %q is not set (required by --auth %s)", envVar, spec) - } - checkers = append(checkers, authChecker{scheme: scheme, value: value}) - } - - return func(next Handler) Handler { - return func(ctx context.Context, req JSONRPCRequest) (*JSONRPCResponse, error) { - for _, c := range checkers { - switch c.scheme { - case "token": - header := GetAuthHeader(ctx) - expected := "Bearer " + c.value - if header != expected { - return nil, fmt.Errorf("unauthorized: invalid or missing Bearer token") - } - case "apikey": - header := GetAPIKeyHeader(ctx) - if header != c.value { - return nil, fmt.Errorf("unauthorized: invalid or missing API key") - } - } - } - return next(ctx, req) - } - }, nil -} -``` - -Note: Import the `mcp` package types via the existing middleware package imports. The `Handler` and `Middleware` types are already defined in `chain.go`. Add the `JSONRPCRequest` and `JSONRPCResponse` type aliases or import from `mcp` package as needed — check what `chain.go` already imports. - -- [ ] **Step 4: Run tests to verify they pass** - -Run: `go test ./internal/middleware/ -v` -Expected: All pass. - -- [ ] **Step 5: Wire auth middleware into main.go** - -In `cmd/protomcp/main.go`, replace the existing middleware chain setup with: - -```go - // 6. Apply middleware - middlewares := []middleware.Middleware{ - middleware.Logging(logger), - middleware.ErrorFormatting(), - } - - if len(cfg.Auth) > 0 { - if cfg.Transport == "stdio" { - slog.Warn("--auth ignored for stdio transport") - } else { - authMw, err := middleware.NewAuth(cfg.Auth) - if err != nil { - slog.Error("invalid --auth configuration", "error", err) - os.Exit(1) - } - middlewares = append([]middleware.Middleware{authMw}, middlewares...) - } - } - - chain := middleware.Chain( - func(ctx context.Context, req mcp.JSONRPCRequest) (*mcp.JSONRPCResponse, error) { - return handler.Handle(ctx, req) - }, - middlewares..., - ) -``` - -- [ ] **Step 6: Verify full build** - -Run: `go build ./cmd/protomcp/` -Expected: Clean build. - -- [ ] **Step 7: Commit** - -```bash -git add internal/middleware/auth.go internal/middleware/auth_test.go cmd/protomcp/main.go -git commit -m "feat: add auth middleware (token + apikey) and wire into CLI" -``` - ---- - -### Task 6: Extended handshake for middleware registration - -**Files:** -- Modify: `internal/process/manager.go` -- Modify: `internal/process/manager_test.go` -- Create: `internal/middleware/custom.go` -- Create: `internal/middleware/custom_test.go` - -- [ ] **Step 1: Add middleware tracking to Manager** - -Add to `Manager` struct: - -```go - middlewares []RegisteredMiddleware -``` - -Add type: - -```go -type RegisteredMiddleware struct { - Name string - Priority int32 -} -``` - -- [ ] **Step 2: Extend handshake in Start()** - -After `listTools` returns, add a loop that waits for optional `RegisterMiddlewareRequest` messages or a `ReloadResponse` (handshake-complete signal): - -```go - // After listTools, wait for optional middleware registrations + handshake-complete signal. - m.mu.Lock() - m.tools = tools - m.mu.Unlock() - - middlewares, err := m.awaitHandshakeComplete(ctx) - if err != nil { - m.cleanup() - return nil, fmt.Errorf("handshake middleware: %w", err) - } - m.mu.Lock() - m.middlewares = middlewares - m.mu.Unlock() -``` - -Implement `awaitHandshakeComplete`: - -```go -func (m *Manager) awaitHandshakeComplete(ctx context.Context) ([]RegisteredMiddleware, error) { - var middlewares []RegisteredMiddleware - // Short timeout for handshake-complete signal. If v1.0 SDKs don't send it, - // we treat "no message within 500ms" as handshake-complete (backward compat). - timer := time.NewTimer(500 * time.Millisecond) - defer timer.Stop() - - for { - select { - case <-ctx.Done(): - return nil, ctx.Err() - case <-timer.C: - // Backward compat: v1.0 SDKs don't send handshake-complete signal. - // Treat timeout as "no middleware registered, handshake done." - return middlewares, nil - case env := <-m.handshakeCh: - if rr := env.GetReloadResponse(); rr != nil { - // Handshake complete signal - return middlewares, nil - } - if rm := env.GetRegisterMiddleware(); rm != nil { - middlewares = append(middlewares, RegisteredMiddleware{ - Name: rm.Name, - Priority: rm.Priority, - }) - // Send acknowledgment - resp := &pb.Envelope{ - Msg: &pb.Envelope_RegisterMiddlewareResponse{ - RegisterMiddlewareResponse: &pb.RegisterMiddlewareResponse{Success: true}, - }, - } - if err := envelope.Write(m.conn, resp); err != nil { - return nil, fmt.Errorf("write RegisterMiddlewareResponse: %w", err) - } - continue - } - // Unexpected message type — ignore - } - } -} -``` - -- [ ] **Step 3: Add Middlewares() accessor** - -```go -func (m *Manager) Middlewares() []RegisteredMiddleware { - m.mu.Lock() - defer m.mu.Unlock() - return m.middlewares -} -``` - -- [ ] **Step 4: Update readLoop to route RegisterMiddleware and MiddlewareIntercept messages** - -In `readLoop()`, add handling for `MiddlewareInterceptResponse` — route it via `pending` map using `request_id` (same as `CallToolResponse`). For `RegisterMiddlewareRequest` with no `request_id`, route to `handshakeCh`. - -Update the unsolicited message handling in `readLoop`: - -```go - if reqID == "" { - // Unsolicited message (ToolListResponse after reload, or middleware registration during handshake). - select { - case m.handshakeCh <- env: - default: - } - continue - } -``` - -This already works because `RegisterMiddlewareRequest` during handshake has no `request_id` and goes to `handshakeCh`. `MiddlewareInterceptResponse` has a `request_id` and routes via `pending`. - -- [ ] **Step 5: Add SendMiddlewareIntercept method to Manager** - -```go -func (m *Manager) SendMiddlewareIntercept(ctx context.Context, mwName, phase, toolName, argsJSON, resultJSON string, isError bool) (*pb.MiddlewareInterceptResponse, error) { - reqID := m.nextRequestID() - - env := &pb.Envelope{ - RequestId: reqID, - Msg: &pb.Envelope_MiddlewareIntercept{ - MiddlewareIntercept: &pb.MiddlewareInterceptRequest{ - MiddlewareName: mwName, - Phase: phase, - ToolName: toolName, - ArgumentsJson: argsJSON, - ResultJson: resultJSON, - IsError: isError, - }, - }, - } - - respCh := make(chan *pb.Envelope, 1) - m.mu.Lock() - m.pending[reqID] = respCh - m.mu.Unlock() - - defer func() { - m.mu.Lock() - delete(m.pending, reqID) - m.mu.Unlock() - }() - - if err := envelope.Write(m.conn, env); err != nil { - return nil, fmt.Errorf("write MiddlewareInterceptRequest: %w", err) - } - - timer := time.NewTimer(m.cfg.CallTimeout) - defer timer.Stop() - - select { - case <-ctx.Done(): - return nil, ctx.Err() - case <-timer.C: - return nil, fmt.Errorf("middleware intercept %q timed out", mwName) - case resp := <-respCh: - mir := resp.GetMiddlewareInterceptResponse() - if mir == nil { - return nil, fmt.Errorf("unexpected response type for MiddlewareIntercept") - } - return mir, nil - } -} -``` - -- [ ] **Step 6: Create custom middleware dispatcher** - -Create `internal/middleware/custom.go`: - -```go -package middleware - -import ( - "context" - "encoding/json" - "fmt" - "sort" - - "github.com/msilverblatt/protomcp/internal/mcp" -) - -// MiddlewareDispatcher sends intercept requests to the tool process. -type MiddlewareDispatcher interface { - SendMiddlewareIntercept(ctx context.Context, mwName, phase, toolName, argsJSON, resultJSON string, isError bool) (interceptResp interface{ GetReject() bool; GetRejectReason() string; GetArgumentsJson() string; GetResultJson() string }, error) -} - -// RegisteredMW holds a registered custom middleware from the tool process. -type RegisteredMW struct { - Name string - Priority int32 -} - -// CustomMiddleware creates a middleware that dispatches to tool-process-registered middleware. -func CustomMiddleware(dispatcher MiddlewareDispatcher, registered []RegisteredMW) Middleware { - if len(registered) == 0 { - return func(next Handler) Handler { return next } - } - - // Sort by priority (lower first) - sorted := make([]RegisteredMW, len(registered)) - copy(sorted, registered) - sort.Slice(sorted, func(i, j int) bool { - return sorted[i].Priority < sorted[j].Priority - }) - - return func(next Handler) Handler { - return func(ctx context.Context, req JSONRPCRequest) (*JSONRPCResponse, error) { - // Extract tool name and args from the request if it's a tools/call - // For non-tool-call requests, skip custom middleware - if req.Method != "tools/call" { - return next(ctx, req) - } - - toolName, argsJSON := extractToolCallParams(req) - - // Before phase - currentArgs := argsJSON - for _, mw := range sorted { - resp, err := dispatcher.SendMiddlewareIntercept(ctx, mw.Name, "before", toolName, currentArgs, "", false) - if err != nil { - return nil, fmt.Errorf("middleware %q before: %w", mw.Name, err) - } - if resp.GetReject() { - return nil, fmt.Errorf("rejected by middleware %q: %s", mw.Name, resp.GetRejectReason()) - } - if modified := resp.GetArgumentsJson(); modified != "" { - currentArgs = modified - } - } - - // Inject modified args back into the request - if currentArgs != argsJSON { - modifiedParams, _ := json.Marshal(map[string]json.RawMessage{ - "name": json.RawMessage(fmt.Sprintf("%q", toolName)), - "arguments": json.RawMessage(currentArgs), - }) - req.Params = modifiedParams - } - - result, err := next(ctx, req) - - // After phase (reverse order) - resultJSON := "" - isError := false - if result != nil { - resultJSON = string(result.Result) - isError = result.Error != nil - } - - for i := len(sorted) - 1; i >= 0; i-- { - mw := sorted[i] - resp, afterErr := dispatcher.SendMiddlewareIntercept(ctx, mw.Name, "after", toolName, currentArgs, resultJSON, isError) - if afterErr != nil { - return result, err // Return original result on after-phase error - } - if modified := resp.GetResultJson(); modified != "" { - resultJSON = modified - } - } - - return result, err - } - } -} - -func extractToolCallParams(req JSONRPCRequest) (string, string) { - var params struct { - Name string `json:"name"` - Arguments json.RawMessage `json:"arguments"` - } - if err := json.Unmarshal(req.Params, ¶ms); err != nil { - return "", "{}" - } - argsJSON := "{}" - if len(params.Arguments) > 0 { - argsJSON = string(params.Arguments) - } - return params.Name, argsJSON -} -``` - -- [ ] **Step 7: Run all tests** - -Run: `go test ./internal/... -v` -Expected: All pass. - -- [ ] **Step 8: Commit** - -```bash -git add internal/process/manager.go internal/middleware/custom.go internal/middleware/custom_test.go -git commit -m "feat: extended handshake for middleware registration + custom middleware dispatch" -``` - ---- - -### Task 7: Update SDK runners for middleware + handshake-complete signal - -**Files:** -- Modify: `sdk/python/src/protomcp/runner.py` -- Modify: `sdk/python/src/protomcp/__init__.py` -- Modify: `sdk/typescript/src/runner.ts` -- Modify: `sdk/typescript/src/index.ts` - -- [ ] **Step 1: Add middleware registry to Python SDK** - -Create `sdk/python/src/protomcp/middleware.py`: - -```python -from dataclasses import dataclass -from typing import Callable, Any - -_middleware_registry: list["MiddlewareDef"] = [] - -@dataclass -class MiddlewareDef: - name: str - priority: int - handler: Callable # (phase, tool_name, args_json, result_json, is_error) -> dict - -def middleware(name: str, priority: int = 100): - def decorator(func: Callable) -> Callable: - _middleware_registry.append(MiddlewareDef( - name=name, - priority=priority, - handler=func, - )) - return func - return decorator - -def get_registered_middleware() -> list[MiddlewareDef]: - return list(_middleware_registry) - -def clear_middleware_registry(): - _middleware_registry.clear() -``` - -- [ ] **Step 2: Update Python runner to send handshake-complete signal and handle middleware** - -In `runner.py`, after `_handle_list_tools`, send `RegisterMiddlewareRequest` for each registered middleware, then send `ReloadResponse { success: true }` as handshake-complete. Also handle `MiddlewareInterceptRequest`: - -Update the main `run()` loop setup to: -1. After sending `ToolListResponse`, send middleware registrations -2. Send `ReloadResponse(success=True)` with no `request_id` as handshake-complete -3. In the message loop, handle `middleware_intercept` messages - -Update `_handle_reload` to also send middleware registrations + handshake-complete after tool list. - -- [ ] **Step 3: Update TypeScript runner similarly** - -Add `sdk/typescript/src/middleware.ts` with equivalent registration. Update `runner.ts` to send middleware registrations + handshake-complete signal. Handle `middlewareIntercept` messages in the loop. - -- [ ] **Step 4: Export middleware from both SDKs** - -Update `sdk/python/src/protomcp/__init__.py` to export `middleware`, `get_registered_middleware`. -Update `sdk/typescript/src/index.ts` to export `middleware` from `middleware.ts`. - -- [ ] **Step 5: Run existing SDK tests to ensure no regressions** - -Run: `cd sdk/python && python -m pytest -v && cd ../typescript && npx vitest run` -Expected: All existing tests pass. - -- [ ] **Step 6: Commit** - -```bash -git add sdk/python/src/protomcp/ sdk/typescript/src/ -git commit -m "feat: add middleware support to Python and TypeScript SDKs" -``` - ---- - -## Chunk 2: Go SDK - -### Task 8: Go SDK scaffolding + transport - -**Files:** -- Create: `sdk/go/go.mod` -- Create: `sdk/go/protomcp/transport.go` -- Create: `sdk/go/protomcp/transport_test.go` - -- [ ] **Step 1: Create go.mod** - -``` -module github.com/msilverblatt/protomcp/sdk/go - -go 1.25.6 - -require github.com/msilverblatt/protomcp v0.0.0 - -replace github.com/msilverblatt/protomcp => ../.. -``` - -- [ ] **Step 2: Write failing transport test** - -Create `sdk/go/protomcp/transport_test.go`: - -```go -package protomcp_test - -import ( - "net" - "os" - "path/filepath" - "testing" - - "github.com/msilverblatt/protomcp/sdk/go/protomcp" -) - -func TestTransportConnectAndClose(t *testing.T) { - dir := t.TempDir() - sockPath := filepath.Join(dir, "test.sock") - - listener, err := net.Listen("unix", sockPath) - if err != nil { - t.Fatal(err) - } - defer listener.Close() - - os.Setenv("PROTOMCP_SOCKET", sockPath) - defer os.Unsetenv("PROTOMCP_SOCKET") - - tp := protomcp.NewTransport(sockPath) - - // Accept in background - go func() { listener.Accept() }() - - if err := tp.Connect(); err != nil { - t.Fatal(err) - } - tp.Close() -} -``` - -- [ ] **Step 3: Run test to verify it fails** - -Run: `cd sdk/go && go test ./protomcp/ -run TestTransportConnect -v` -Expected: FAIL — package doesn't exist. - -- [ ] **Step 4: Implement transport** - -Create `sdk/go/protomcp/transport.go`: - -```go -package protomcp - -import ( - "encoding/binary" - "fmt" - "io" - "net" - - pb "github.com/msilverblatt/protomcp/gen/proto/protomcp" - "google.golang.org/protobuf/proto" -) - -type Transport struct { - socketPath string - conn net.Conn -} - -func NewTransport(socketPath string) *Transport { - return &Transport{socketPath: socketPath} -} - -func (t *Transport) Connect() error { - conn, err := net.Dial("unix", t.socketPath) - if err != nil { - return fmt.Errorf("connect to socket: %w", err) - } - t.conn = conn - return nil -} - -func (t *Transport) Send(env *pb.Envelope) error { - data, err := proto.Marshal(env) - if err != nil { - return fmt.Errorf("marshal envelope: %w", err) - } - length := make([]byte, 4) - binary.BigEndian.PutUint32(length, uint32(len(data))) - if _, err := t.conn.Write(length); err != nil { - return err - } - _, err = t.conn.Write(data) - return err -} - -func (t *Transport) Recv() (*pb.Envelope, error) { - lengthBuf := make([]byte, 4) - if _, err := io.ReadFull(t.conn, lengthBuf); err != nil { - return nil, err - } - length := binary.BigEndian.Uint32(lengthBuf) - data := make([]byte, length) - if _, err := io.ReadFull(t.conn, data); err != nil { - return nil, err - } - env := &pb.Envelope{} - if err := proto.Unmarshal(data, env); err != nil { - return nil, fmt.Errorf("unmarshal envelope: %w", err) - } - return env, nil -} - -func (t *Transport) Close() { - if t.conn != nil { - t.conn.Close() - } -} -``` - -- [ ] **Step 5: Run test to verify it passes** - -Run: `cd sdk/go && go test ./protomcp/ -run TestTransportConnect -v` -Expected: PASS. - -- [ ] **Step 6: Commit** - -```bash -git add sdk/go/ -git commit -m "feat(sdk/go): add transport layer" -``` - ---- - -### Task 9: Go SDK result + context - -**Files:** -- Create: `sdk/go/protomcp/result.go` -- Create: `sdk/go/protomcp/result_test.go` -- Create: `sdk/go/protomcp/context.go` -- Create: `sdk/go/protomcp/context_test.go` - -- [ ] **Step 1: Write failing result test** - -```go -package protomcp_test - -import "testing" - -func TestToolResultBasic(t *testing.T) { - r := protomcp.Result("hello") - if r.ResultText != "hello" { - t.Errorf("ResultText = %q, want %q", r.ResultText, "hello") - } - if r.IsError { - t.Error("should not be error") - } -} - -func TestToolResultError(t *testing.T) { - r := protomcp.ErrorResult("failed", "INVALID", "try again", true) - if !r.IsError { - t.Error("should be error") - } - if r.ErrorCode != "INVALID" { - t.Errorf("ErrorCode = %q, want %q", r.ErrorCode, "INVALID") - } -} - -func TestToolResultEnableDisable(t *testing.T) { - r := protomcp.Result("ok") - r.EnableTools = []string{"admin_panel"} - r.DisableTools = []string{"login"} - if len(r.EnableTools) != 1 { - t.Error("expected 1 enable tool") - } -} -``` - -- [ ] **Step 2: Implement result.go** - -```go -package protomcp - -type ToolResult struct { - ResultText string - IsError bool - ErrorCode string - Message string - Suggestion string - Retryable bool - EnableTools []string - DisableTools []string -} - -func Result(text string) ToolResult { - return ToolResult{ResultText: text} -} - -func ErrorResult(text, errorCode, suggestion string, retryable bool) ToolResult { - return ToolResult{ - ResultText: text, - IsError: true, - ErrorCode: errorCode, - Message: text, - Suggestion: suggestion, - Retryable: retryable, - } -} -``` - -- [ ] **Step 3: Write context tests and implement** - -Context wraps progress reporting and cancellation. Uses `context.Context` for cancellation: - -```go -package protomcp - -import ( - "context" - - pb "github.com/msilverblatt/protomcp/gen/proto/protomcp" -) - -type ToolContext struct { - Ctx context.Context - ProgressToken string - sendFn func(*pb.Envelope) error -} - -func (tc *ToolContext) ReportProgress(progress, total int64, message string) error { - env := &pb.Envelope{ - Msg: &pb.Envelope_Progress{ - Progress: &pb.ProgressNotification{ - ProgressToken: tc.ProgressToken, - Progress: progress, - Total: total, - Message: message, - }, - }, - } - return tc.sendFn(env) -} - -func (tc *ToolContext) IsCancelled() bool { - return tc.Ctx.Err() != nil -} -``` - -- [ ] **Step 4: Run tests** - -Run: `cd sdk/go && go test ./protomcp/ -v` -Expected: All pass. - -- [ ] **Step 5: Commit** - -```bash -git add sdk/go/protomcp/result.go sdk/go/protomcp/result_test.go sdk/go/protomcp/context.go sdk/go/protomcp/context_test.go -git commit -m "feat(sdk/go): add ToolResult and ToolContext" -``` - ---- - -### Task 10: Go SDK tool registration - -**Files:** -- Create: `sdk/go/protomcp/tool.go` -- Create: `sdk/go/protomcp/tool_test.go` - -- [ ] **Step 1: Write failing test** - -```go -package protomcp_test - -import ( - "testing" - - "github.com/msilverblatt/protomcp/sdk/go/protomcp" -) - -func TestToolRegistration(t *testing.T) { - protomcp.ClearRegistry() - protomcp.Tool("add", - protomcp.Description("Add two numbers"), - protomcp.Args(protomcp.IntArg("a"), protomcp.IntArg("b")), - protomcp.Handler(func(ctx protomcp.ToolContext, args map[string]interface{}) protomcp.ToolResult { - return protomcp.Result("3") - }), - ) - - tools := protomcp.GetRegisteredTools() - if len(tools) != 1 { - t.Fatalf("expected 1 tool, got %d", len(tools)) - } - if tools[0].Name != "add" { - t.Errorf("name = %q, want %q", tools[0].Name, "add") - } - if tools[0].Desc != "Add two numbers" { - t.Errorf("description = %q, want %q", tools[0].Desc, "Add two numbers") - } -} - -func TestToolMetadata(t *testing.T) { - protomcp.ClearRegistry() - protomcp.Tool("delete_user", - protomcp.Description("Delete a user account"), - protomcp.DestructiveHint(true), - protomcp.Handler(func(ctx protomcp.ToolContext, args map[string]interface{}) protomcp.ToolResult { - return protomcp.Result("deleted") - }), - ) - - tools := protomcp.GetRegisteredTools() - if !tools[0].Destructive { - t.Error("expected destructive hint") - } -} -``` - -- [ ] **Step 2: Implement tool.go with functional options** - -```go -package protomcp - -import ( - "encoding/json" -) - -type ToolDef struct { - Name string - Desc string - InputSchema map[string]interface{} - OutputSchema map[string]interface{} - HandlerFn func(ToolContext, map[string]interface{}) ToolResult - Title string - Destructive bool - Idempotent bool - ReadOnly bool - OpenWorld bool - TaskSupport bool -} - -type ToolOption func(*ToolDef) - -var registry []ToolDef - -func Tool(name string, opts ...ToolOption) { - td := ToolDef{ - Name: name, - InputSchema: map[string]interface{}{"type": "object", "properties": map[string]interface{}{}}, - } - for _, opt := range opts { - opt(&td) - } - registry = append(registry, td) -} - -func Description(desc string) ToolOption { - return func(td *ToolDef) { td.Desc = desc } -} - -type ArgDef struct { - Name string - Type string -} - -func IntArg(name string) ArgDef { return ArgDef{Name: name, Type: "integer"} } -func StrArg(name string) ArgDef { return ArgDef{Name: name, Type: "string"} } -func NumArg(name string) ArgDef { return ArgDef{Name: name, Type: "number"} } -func BoolArg(name string) ArgDef { return ArgDef{Name: name, Type: "boolean"} } - -func Args(args ...ArgDef) ToolOption { - return func(td *ToolDef) { - props := map[string]interface{}{} - required := []string{} - for _, a := range args { - props[a.Name] = map[string]interface{}{"type": a.Type} - required = append(required, a.Name) - } - td.InputSchema = map[string]interface{}{ - "type": "object", - "properties": props, - "required": required, - } - } -} - -func Handler(fn func(ToolContext, map[string]interface{}) ToolResult) ToolOption { - return func(td *ToolDef) { td.HandlerFn = fn } -} - -func DestructiveHint(v bool) ToolOption { return func(td *ToolDef) { td.Destructive = v } } -func IdempotentHint(v bool) ToolOption { return func(td *ToolDef) { td.Idempotent = v } } -func ReadOnlyHint(v bool) ToolOption { return func(td *ToolDef) { td.ReadOnly = v } } -func OpenWorldHint(v bool) ToolOption { return func(td *ToolDef) { td.OpenWorld = v } } -func TaskSupportHint(v bool) ToolOption { return func(td *ToolDef) { td.TaskSupport = v } } - -func (td ToolDef) InputSchemaJSON() string { - b, _ := json.Marshal(td.InputSchema) - return string(b) -} - -func GetRegisteredTools() []ToolDef { return append([]ToolDef{}, registry...) } -func ClearRegistry() { registry = nil } -``` - -- [ ] **Step 3: Run tests** - -Run: `cd sdk/go && go test ./protomcp/ -run TestTool -v` -Expected: All pass. - -- [ ] **Step 4: Commit** - -```bash -git add sdk/go/protomcp/tool.go sdk/go/protomcp/tool_test.go -git commit -m "feat(sdk/go): add tool registration with functional options" -``` - ---- - -### Task 11: Go SDK log + manager + runner - -**Files:** -- Create: `sdk/go/protomcp/log.go` -- Create: `sdk/go/protomcp/manager.go` -- Create: `sdk/go/protomcp/runner.go` - -- [ ] **Step 1: Implement log.go** - -Server logging with 8 RFC 5424 levels. Follow the same pattern as Python SDK's `log.py`: - -```go -package protomcp - -import ( - pb "github.com/msilverblatt/protomcp/gen/proto/protomcp" -) - -type ServerLogger struct { - sendFn func(*pb.Envelope) error - logger string -} - -func NewServerLogger(sendFn func(*pb.Envelope) error, logger string) *ServerLogger { - return &ServerLogger{sendFn: sendFn, logger: logger} -} - -func (l *ServerLogger) log(level, dataJSON string) { - if l.sendFn == nil { - return - } - l.sendFn(&pb.Envelope{ - Msg: &pb.Envelope_Log{ - Log: &pb.LogMessage{ - Level: level, - Logger: l.logger, - DataJson: dataJSON, - }, - }, - }) -} - -func (l *ServerLogger) Debug(msg string) { l.log("debug", msg) } -func (l *ServerLogger) Info(msg string) { l.log("info", msg) } -func (l *ServerLogger) Notice(msg string) { l.log("notice", msg) } -func (l *ServerLogger) Warning(msg string) { l.log("warning", msg) } -func (l *ServerLogger) Error(msg string) { l.log("error", msg) } -func (l *ServerLogger) Critical(msg string) { l.log("critical", msg) } -func (l *ServerLogger) Alert(msg string) { l.log("alert", msg) } -func (l *ServerLogger) Emergency(msg string) { l.log("emergency", msg) } -``` - -- [ ] **Step 2: Implement manager.go** - -Dynamic tool list management — `ToolManager` with Enable/Disable/SetAllowed/SetBlocked: - -```go -package protomcp - -import ( - pb "github.com/msilverblatt/protomcp/gen/proto/protomcp" -) - -type ToolManager struct { - sendFn func(*pb.Envelope) error -} - -func newToolManager(sendFn func(*pb.Envelope) error) *ToolManager { - return &ToolManager{sendFn: sendFn} -} - -func (m *ToolManager) Enable(names ...string) { - m.sendFn(&pb.Envelope{ - Msg: &pb.Envelope_EnableTools{ - EnableTools: &pb.EnableToolsRequest{ToolNames: names}, - }, - }) -} - -func (m *ToolManager) Disable(names ...string) { - m.sendFn(&pb.Envelope{ - Msg: &pb.Envelope_DisableTools{ - DisableTools: &pb.DisableToolsRequest{ToolNames: names}, - }, - }) -} -``` - -- [ ] **Step 3: Implement runner.go** - -Main loop: connect to socket, send tool list, handle calls, handle reload. Follow the same pattern as Python/TypeScript runners but with handshake-complete signal: - -```go -package protomcp - -import ( - "encoding/json" - "fmt" - "os" - - pb "github.com/msilverblatt/protomcp/gen/proto/protomcp" -) - -var Log *ServerLogger - -func Run() { - socketPath := os.Getenv("PROTOMCP_SOCKET") - if socketPath == "" { - fmt.Fprintln(os.Stderr, "PROTOMCP_SOCKET not set — run via 'pmcp dev'") - os.Exit(1) - } - - tp := NewTransport(socketPath) - if err := tp.Connect(); err != nil { - fmt.Fprintf(os.Stderr, "connect: %v\n", err) - os.Exit(1) - } - defer tp.Close() - - Log = NewServerLogger(func(env *pb.Envelope) error { return tp.Send(env) }, "protomcp-go") - - for { - env, err := tp.Recv() - if err != nil { - break - } - - reqID := env.GetRequestId() - - switch { - case env.GetListTools() != nil: - handleListTools(tp, reqID) - sendHandshakeComplete(tp) - case env.GetCallTool() != nil: - handleCallTool(tp, env.GetCallTool(), reqID) - case env.GetReload() != nil: - handleReload(tp, reqID) - case env.GetMiddlewareIntercept() != nil: - handleMiddlewareIntercept(tp, env.GetMiddlewareIntercept(), reqID) - } - } -} - -func handleListTools(tp *Transport, reqID string) { - tools := GetRegisteredTools() - var defs []*pb.ToolDefinition - for _, t := range tools { - defs = append(defs, &pb.ToolDefinition{ - Name: t.Name, - Description: t.Desc, - InputSchemaJson: t.InputSchemaJSON(), - DestructiveHint: t.Destructive, - IdempotentHint: t.Idempotent, - ReadOnlyHint: t.ReadOnly, - OpenWorldHint: t.OpenWorld, - TaskSupport: t.TaskSupport, - }) - } - tp.Send(&pb.Envelope{ - RequestId: reqID, - Msg: &pb.Envelope_ToolList{ - ToolList: &pb.ToolListResponse{Tools: defs}, - }, - }) -} - -func sendHandshakeComplete(tp *Transport) { - // Send middleware registrations if any - // TODO: add middleware registration here when middleware support is added - - // Send handshake-complete signal - tp.Send(&pb.Envelope{ - Msg: &pb.Envelope_ReloadResponse{ - ReloadResponse: &pb.ReloadResponse{Success: true}, - }, - }) -} - -func handleCallTool(tp *Transport, req *pb.CallToolRequest, reqID string) { - tools := GetRegisteredTools() - var handler func(ToolContext, map[string]interface{}) ToolResult - for _, t := range tools { - if t.Name == req.Name { - handler = t.HandlerFn - break - } - } - - if handler == nil { - tp.Send(&pb.Envelope{ - RequestId: reqID, - Msg: &pb.Envelope_CallResult{ - CallResult: &pb.CallToolResponse{ - IsError: true, - ResultJson: fmt.Sprintf(`[{"type":"text","text":"Tool not found: %s"}]`, req.Name), - }, - }, - }) - return - } - - var args map[string]interface{} - if req.ArgumentsJson != "" { - json.Unmarshal([]byte(req.ArgumentsJson), &args) - } - if args == nil { - args = map[string]interface{}{} - } - - ctx := ToolContext{ - ProgressToken: req.ProgressToken, - sendFn: func(env *pb.Envelope) error { return tp.Send(env) }, - } - - result := handler(ctx, args) - - resp := &pb.CallToolResponse{ - IsError: result.IsError, - ResultJson: fmt.Sprintf(`[{"type":"text","text":"%s"}]`, result.ResultText), - EnableTools: result.EnableTools, - DisableTools: result.DisableTools, - } - if result.IsError && result.ErrorCode != "" { - resp.Error = &pb.ToolError{ - ErrorCode: result.ErrorCode, - Message: result.Message, - Suggestion: result.Suggestion, - Retryable: result.Retryable, - } - } - - tp.Send(&pb.Envelope{ - RequestId: reqID, - Msg: &pb.Envelope_CallResult{CallResult: resp}, - }) -} - -func handleReload(tp *Transport, reqID string) { - tp.Send(&pb.Envelope{ - RequestId: reqID, - Msg: &pb.Envelope_ReloadResponse{ - ReloadResponse: &pb.ReloadResponse{Success: true}, - }, - }) - handleListTools(tp, "") - sendHandshakeComplete(tp) -} - -func handleMiddlewareIntercept(tp *Transport, req *pb.MiddlewareInterceptRequest, reqID string) { - // TODO: dispatch to registered middleware handlers - tp.Send(&pb.Envelope{ - RequestId: reqID, - Msg: &pb.Envelope_MiddlewareInterceptResponse{ - MiddlewareInterceptResponse: &pb.MiddlewareInterceptResponse{ - ArgumentsJson: req.ArgumentsJson, - ResultJson: req.ResultJson, - }, - }, - }) -} -``` - -- [ ] **Step 4: Run Go SDK tests** - -Run: `cd sdk/go && go test ./protomcp/ -v` -Expected: All pass. - -- [ ] **Step 5: Verify build** - -Run: `cd sdk/go && go build ./protomcp/` -Expected: Clean build. - -- [ ] **Step 6: Commit** - -```bash -git add sdk/go/protomcp/ -git commit -m "feat(sdk/go): add log, manager, and runner" -``` - ---- - -## Chunk 3: Rust SDK - -### Task 12: Rust SDK scaffolding - -**Files:** -- Create: `sdk/rust/Cargo.toml` -- Create: `sdk/rust/build.rs` -- Create: `sdk/rust/src/lib.rs` - -- [ ] **Step 1: Create Cargo.toml** - -```toml -[package] -name = "protomcp" -version = "0.1.0" -edition = "2021" -description = "Rust SDK for protomcp — write MCP tools in Rust" -license = "MIT" - -[dependencies] -prost = "0.13" -tokio = { version = "1", features = ["full"] } -serde = { version = "1", features = ["derive"] } -serde_json = "1" - -[build-dependencies] -prost-build = "0.13" -``` - -- [ ] **Step 2: Create build.rs** - -```rust -fn main() { - prost_build::compile_protos(&["../../proto/protomcp.proto"], &["../../proto/"]).unwrap(); -} -``` - -- [ ] **Step 3: Create src/lib.rs with module declarations** - -```rust -pub mod proto { - include!(concat!(env!("OUT_DIR"), "/protomcp.rs")); -} - -mod tool; -mod result; -mod context; -mod manager; -mod log; -mod transport; -mod runner; - -pub use tool::{tool, ToolDef, ArgDef, clear_registry, get_registered_tools}; -pub use result::ToolResult; -pub use context::ToolContext; -pub use runner::run; -``` - -- [ ] **Step 4: Verify protobuf generation builds** - -Run: `cd sdk/rust && cargo build` -Expected: May fail on missing modules — that's fine. Proto generation should succeed. - -- [ ] **Step 5: Commit** - -```bash -git add sdk/rust/ -git commit -m "feat(sdk/rust): scaffold crate with prost protobuf generation" -``` - ---- - -### Task 13-16: Rust SDK modules - -Follow the same pattern as the Go SDK tasks (9-11), implementing: -- `result.rs` — `ToolResult` struct -- `context.rs` — `ToolContext` with progress, cancellation token -- `tool.rs` — builder pattern registration -- `log.rs` — `ServerLogger` -- `manager.rs` — `ToolManager` (enable/disable) -- `transport.rs` — Unix socket + envelope framing -- `runner.rs` — async main loop with tokio - -Each module mirrors the Go SDK's functionality. Implementation follows the API surface from the spec. Tests use `#[cfg(test)]` inline modules. - -Key differences from Go SDK: -- Uses `tokio::net::UnixStream` instead of `net.Conn` -- Uses `tokio::sync::CancellationToken` instead of `context.Context` -- Builder pattern for tool registration instead of functional options -- All handlers are `async` - -- [ ] **Step 1: Implement all modules** -- [ ] **Step 2: Run tests**: `cd sdk/rust && cargo test` -- [ ] **Step 3: Commit** - -```bash -git add sdk/rust/src/ -git commit -m "feat(sdk/rust): implement full SDK (tool, result, context, transport, runner)" -``` - ---- - -## Chunk 4: CI/CD + Badges - -### Task 17: CI Pipeline - -**Files:** -- Create: `.github/workflows/ci.yml` - -- [ ] **Step 1: Create CI workflow** - -```yaml -name: CI - -on: - push: - branches: [master] - pull_request: - branches: [master] - -jobs: - go: - runs-on: ubuntu-latest - steps: - - uses: actions/checkout@v4 - - uses: actions/setup-go@v5 - with: - go-version: '1.25' - - run: go vet ./... - - run: go test ./... - - run: go build ./cmd/protomcp/ - - go-sdk: - runs-on: ubuntu-latest - steps: - - uses: actions/checkout@v4 - - uses: actions/setup-go@v5 - with: - go-version: '1.25' - - run: cd sdk/go && go test ./... - - python-sdk: - runs-on: ubuntu-latest - strategy: - matrix: - python-version: ['3.10', '3.11', '3.12'] - steps: - - uses: actions/checkout@v4 - - uses: actions/setup-python@v5 - with: - python-version: ${{ matrix.python-version }} - - run: cd sdk/python && pip install -e ".[dev]" && python -m pytest - - typescript-sdk: - runs-on: ubuntu-latest - steps: - - uses: actions/checkout@v4 - - uses: actions/setup-node@v4 - with: - node-version: '20' - - run: cd sdk/typescript && npm ci && npx vitest run - - rust-sdk: - runs-on: ubuntu-latest - steps: - - uses: actions/checkout@v4 - - uses: dtolnay/rust-toolchain@stable - - run: cd sdk/rust && cargo test - - run: cd sdk/rust && cargo clippy -- -D warnings - - e2e: - runs-on: ubuntu-latest - needs: [go] - steps: - - uses: actions/checkout@v4 - - uses: actions/setup-go@v5 - with: - go-version: '1.25' - - uses: actions/setup-python@v5 - with: - python-version: '3.12' - - uses: actions/setup-node@v4 - with: - node-version: '20' - - run: go build -o pmcp ./cmd/protomcp/ - - run: cd sdk/python && pip install -e . - - run: cd sdk/typescript && npm ci - - run: go test ./test/e2e/ -v -``` - -- [ ] **Step 2: Commit** - -```bash -git add .github/workflows/ci.yml -git commit -m "ci: add GitHub Actions CI pipeline" -``` - ---- - -### Task 18: Release Pipeline - -**Files:** -- Create: `.github/workflows/release.yml` - -- [ ] **Step 1: Create release workflow** - -```yaml -name: Release - -on: - push: - tags: ['v*'] - -permissions: - contents: write - -jobs: - goreleaser: - runs-on: ubuntu-latest - steps: - - uses: actions/checkout@v4 - with: - fetch-depth: 0 - - uses: actions/setup-go@v5 - with: - go-version: '1.25' - - uses: goreleaser/goreleaser-action@v6 - with: - args: release --clean - env: - GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }} - - pypi: - runs-on: ubuntu-latest - steps: - - uses: actions/checkout@v4 - - uses: actions/setup-python@v5 - with: - python-version: '3.12' - - run: pip install build twine - - run: cd sdk/python && python -m build - - run: cd sdk/python && twine upload dist/* - env: - TWINE_USERNAME: __token__ - TWINE_PASSWORD: ${{ secrets.PYPI_API_TOKEN }} - - npm: - runs-on: ubuntu-latest - steps: - - uses: actions/checkout@v4 - - uses: actions/setup-node@v4 - with: - node-version: '20' - registry-url: 'https://registry.npmjs.org' - - run: cd sdk/typescript && npm ci && npm publish - env: - NODE_AUTH_TOKEN: ${{ secrets.NPM_TOKEN }} - - crates: - runs-on: ubuntu-latest - steps: - - uses: actions/checkout@v4 - - uses: dtolnay/rust-toolchain@stable - - run: cd sdk/rust && cargo publish - env: - CARGO_REGISTRY_TOKEN: ${{ secrets.CARGO_REGISTRY_TOKEN }} - - homebrew: - runs-on: ubuntu-latest - needs: [goreleaser] - steps: - - uses: actions/checkout@v4 - with: - repository: protomcp/homebrew-tap - token: ${{ secrets.HOMEBREW_TAP_TOKEN }} - - name: Update formula - run: | - VERSION=${GITHUB_REF#refs/tags/v} - # GoReleaser generates checksums — update the formula with new version + SHA256 - sed -i "s/version \".*\"/version \"${VERSION}\"/" Formula/protomcp.rb - git add Formula/protomcp.rb - git commit -m "protomcp ${VERSION}" - git push -``` - -- [ ] **Step 2: Commit** - -```bash -git add .github/workflows/release.yml -git commit -m "ci: add release pipeline for Go binary, PyPI, npm, crates.io" -``` - ---- - -### Task 19: Fix README badges - -**Files:** -- Modify: `README.md` - -- [ ] **Step 1: Update badges** - -Replace the current badge block with: - -```markdown -[![CI](https://github.com/msilverblatt/protomcp/actions/workflows/ci.yml/badge.svg)](https://github.com/msilverblatt/protomcp/actions/workflows/ci.yml) -[![License: MIT](https://img.shields.io/badge/License-MIT-blue.svg)](LICENSE) -[![Go](https://img.shields.io/badge/Go-1.25+-00ADD8?logo=go)](https://go.dev) -[![PyPI](https://img.shields.io/pypi/v/protomcp)](https://pypi.org/project/protomcp/) -[![npm](https://img.shields.io/npm/v/protomcp)](https://www.npmjs.com/package/protomcp) -[![crates.io](https://img.shields.io/crates/v/protomcp)](https://crates.io/crates/protomcp) -``` - -Changes: renamed Build→CI to match workflow name, added crates.io badge, reordered. - -- [ ] **Step 2: Commit** - -```bash -git add README.md -git commit -m "docs: fix README badges to match CI workflow" -``` - ---- - -## Chunk 5: Examples + Documentation - -### Task 20: Go examples - -**Files:** -- Create: `examples/go/basic.go` -- Create: `examples/go/real_world.go` -- Create: `examples/go/full_showcase.go` - -- [ ] **Step 1: Create basic example** - -```go -// examples/go/basic.go -// A minimal protomcp tool — adds and multiplies numbers. -// Run: pmcp dev examples/go/basic.go -package main - -import ( - "fmt" - - "github.com/msilverblatt/protomcp/sdk/go/protomcp" -) - -func main() { - protomcp.Tool("add", - protomcp.Description("Add two numbers"), - protomcp.Args(protomcp.IntArg("a"), protomcp.IntArg("b")), - protomcp.Handler(func(ctx protomcp.ToolContext, args map[string]interface{}) protomcp.ToolResult { - a := int(args["a"].(float64)) - b := int(args["b"].(float64)) - return protomcp.Result(fmt.Sprintf("%d", a+b)) - }), - ) - - protomcp.Tool("multiply", - protomcp.Description("Multiply two numbers"), - protomcp.Args(protomcp.IntArg("a"), protomcp.IntArg("b")), - protomcp.Handler(func(ctx protomcp.ToolContext, args map[string]interface{}) protomcp.ToolResult { - a := int(args["a"].(float64)) - b := int(args["b"].(float64)) - return protomcp.Result(fmt.Sprintf("%d", a*b)) - }), - ) - - protomcp.Run() -} -``` - -- [ ] **Step 2: Create real_world and full_showcase examples** - -Follow the same patterns as the Python examples but in Go, adding progress reporting, cancellation, logging, dynamic tool lists, structured output, and metadata. - -- [ ] **Step 3: Verify examples parse**: `cd examples/go && go vet ./...` - -- [ ] **Step 4: Commit** - -```bash -git add examples/go/ -git commit -m "examples: add Go examples at 3 tiers" -``` - ---- - -### Task 21: Rust examples - -**Files:** -- Create: `examples/rust/basic/Cargo.toml` + `examples/rust/basic/src/main.rs` -- Create: `examples/rust/real_world/Cargo.toml` + `examples/rust/real_world/src/main.rs` -- Create: `examples/rust/full_showcase/Cargo.toml` + `examples/rust/full_showcase/src/main.rs` - -Each is a standalone Cargo project that depends on the `protomcp` crate via path dependency. - -- [ ] **Step 1: Create basic example** -- [ ] **Step 2: Create real_world and full_showcase** -- [ ] **Step 3: Verify**: `cd examples/rust/basic && cargo check` -- [ ] **Step 4: Commit** - -```bash -git add examples/rust/ -git commit -m "examples: add Rust examples at 3 tiers" -``` - ---- - -### Task 22: Documentation — new guide pages - -**Files:** -- Create: `docs/src/content/docs/guides/writing-tools-go.mdx` -- Create: `docs/src/content/docs/guides/writing-tools-rust.mdx` -- Create: `docs/src/content/docs/guides/middleware.mdx` -- Create: `docs/src/content/docs/guides/auth.mdx` -- Create: `docs/src/content/docs/guides/writing-a-language-library.mdx` - -Each guide follows the same structure as the existing Python/TypeScript guides: -1. Installation -2. Basic usage (decorator/builder pattern) -3. Type hints / schema definition -4. Optional parameters -5. Progress reporting -6. Cancellation -7. Logging -8. Error handling -9. Advanced features (middleware, dynamic tools) - -For the middleware and auth guides, focus on configuration and usage patterns with examples in all four languages. - -The "Writing a Language Library" guide covers: -1. The protobuf contract (envelope format, message types) -2. Unix socket connection -3. Length-prefixed framing -4. Handshake protocol (ListToolsRequest → ToolListResponse → optional middleware → ReloadResponse) -5. Tool call dispatch -6. Reload handling -7. Testing against the Go binary - -- [ ] **Step 1: Write all five guide pages** -- [ ] **Step 2: Verify docs build**: `cd docs && npm run build` -- [ ] **Step 3: Commit** - -```bash -git add docs/src/content/docs/guides/ -git commit -m "docs: add Go, Rust, middleware, auth, and language library guides" -``` - ---- - -### Task 23: Documentation — update existing pages - -**Files:** -- Modify: `docs/src/content/docs/index.mdx` -- Modify: `docs/src/content/docs/getting-started/quick-start.mdx` -- Modify: `docs/src/content/docs/getting-started/installation.mdx` -- Modify: `docs/src/content/docs/reference/cli.mdx` - -- [ ] **Step 1: Update index.mdx** - -Add Go and Rust cards to the CardGrid. - -- [ ] **Step 2: Update quick-start.mdx** - -Add Go and Rust quick start snippets (install SDK, write tool file, run). - -- [ ] **Step 3: Update installation.mdx** - -Add Go SDK install (`go get github.com/msilverblatt/protomcp/sdk/go`) and Rust SDK install (`cargo add protomcp`). - -- [ ] **Step 4: Update cli.mdx** - -Add `validate` subcommand docs and `--auth` flag docs. - -- [ ] **Step 5: Verify docs build**: `cd docs && npm run build` - -- [ ] **Step 6: Commit** - -```bash -git add docs/src/content/docs/ -git commit -m "docs: update index, quick-start, installation, and CLI reference for v1.1" -``` - ---- - -### Task 24: Update README - -**Files:** -- Modify: `README.md` - -- [ ] **Step 1: Add Go and Rust quick start examples** - -After the TypeScript example, add Go and Rust sections: - -````markdown -### Go - -```go -// tools.go -package main - -import ( - "fmt" - "github.com/msilverblatt/protomcp/sdk/go/protomcp" -) - -func main() { - protomcp.Tool("add", - protomcp.Description("Add two numbers"), - protomcp.Args(protomcp.IntArg("a"), protomcp.IntArg("b")), - protomcp.Handler(func(ctx protomcp.ToolContext, args map[string]interface{}) protomcp.ToolResult { - a := int(args["a"].(float64)) - b := int(args["b"].(float64)) - return protomcp.Result(fmt.Sprintf("%d", a+b)) - }), - ) - protomcp.Run() -} -``` - -```sh -pmcp dev tools.go -``` - -### Rust - -```rust -// src/main.rs -use protomcp::{tool, ToolResult}; - -#[tokio::main] -async fn main() { - tool("add") - .description("Add two numbers") - .arg::("a") - .arg::("b") - .handler(|_ctx, args| { - let a = args["a"].as_i64().unwrap_or(0); - let b = args["b"].as_i64().unwrap_or(0); - ToolResult::new(format!("{}", a + b)) - }) - .register(); - - protomcp::run().await; -} -``` - -```sh -pmcp dev src/main.rs -``` -```` - -- [ ] **Step 2: Update features list** - -Add after "Tool Metadata": - -```markdown -- **Middleware** — intercept tool calls before/after with custom logic -- **Auth** — built-in token and API key authentication for network transports -- **Validation** — `pmcp validate` checks tool definitions before deployment -``` - -- [ ] **Step 3: Update comparison table** - -Add middleware and validation rows. Update language support to mention Go/Rust explicitly. - -- [ ] **Step 4: Verify README renders** - -Visually inspect or use a markdown previewer. - -- [ ] **Step 5: Commit** - -```bash -git add README.md -git commit -m "docs: update README with Go/Rust examples, middleware, validation" -``` - ---- - -### Task 25: Update sidebar config - -**Files:** -- Modify: `docs/astro.config.mjs` - -- [ ] **Step 1: Add new guide pages to sidebar** - -Add entries for the new guides (writing-tools-go, writing-tools-rust, middleware, auth, writing-a-language-library) in the sidebar configuration. - -- [ ] **Step 2: Verify docs build**: `cd docs && npm run build` - -- [ ] **Step 3: Commit** - -```bash -git add docs/astro.config.mjs -git commit -m "docs: add v1.1 guide pages to sidebar" -``` - ---- - -### Task 26: Final verification - -- [ ] **Step 1: Run all tests** - -```bash -go test ./... -cd sdk/go && go test ./... -cd ../rust && cargo test -cd ../../sdk/python && python -m pytest -cd ../typescript && npx vitest run -``` - -- [ ] **Step 2: Build binary** - -Run: `go build -o pmcp ./cmd/protomcp/` - -- [ ] **Step 3: Build docs** - -Run: `cd docs && npm run build` - -- [ ] **Step 4: Verify README badges will resolve** - -Check that `.github/workflows/ci.yml` has `name: CI` matching the badge URL. - -- [ ] **Step 5: Final commit if any cleanup needed** - -```bash -git add -A -git commit -m "chore: v1.1 final cleanup" -``` diff --git a/docs/superpowers/plans/2026-03-12-protomcp-v1.md b/docs/superpowers/plans/2026-03-12-protomcp-v1.md deleted file mode 100644 index d6d5604..0000000 --- a/docs/superpowers/plans/2026-03-12-protomcp-v1.md +++ /dev/null @@ -1,4992 +0,0 @@ -# protomcp v1.0 Implementation Plan - -> **For agentic workers:** REQUIRED: Use superpowers:subagent-driven-development (if subagents available) or superpowers:executing-plans to implement this plan. Steps use checkbox (`- [ ]`) syntax for tracking. - -**Goal:** Build a language-agnostic MCP runtime — a Go binary that proxies MCP protocol to tool processes in any language via protobuf over unix socket, with hot-reload, dynamic tool list management, progress notifications, async tasks, cancellation, server logging, structured output, and tool metadata. - -**Architecture:** A precompiled Go binary handles all MCP transport/protocol. It communicates with a spawned tool process over a unix socket using length-prefixed protobuf envelopes. First-class Python and TypeScript SDKs provide decorator APIs. The binary proxies progress, cancellation, logging, and async task lifecycle. Starlight documentation site. - -**Tech Stack:** Go 1.22+, Protocol Buffers 3, Python 3.10+ (with protobuf, inspect), TypeScript 5+ (with Zod, protobuf-ts), Astro Starlight, fsnotify (Go file watching) - -**Spec:** `docs/superpowers/specs/2026-03-12-protomcp-design.md` - ---- - -## File Structure - -``` -protomcp/ -├── proto/ -│ └── protomcp.proto # Single source of truth for internal protocol -├── cmd/ -│ └── protomcp/ -│ └── main.go # CLI entry point (dev, run commands) -├── internal/ -│ ├── envelope/ -│ │ ├── envelope.go # Length-prefixed protobuf read/write -│ │ └── envelope_test.go -│ ├── toollist/ -│ │ ├── manager.go # Tool list state machine (open/allow/block) -│ │ └── manager_test.go -│ ├── process/ -│ │ ├── manager.go # Tool process spawn, handshake, crash recovery -│ │ └── manager_test.go -│ ├── reload/ -│ │ ├── watcher.go # File watcher + reload orchestration -│ │ └── watcher_test.go -│ ├── mcp/ -│ │ ├── types.go # MCP JSON-RPC types (requests, responses, notifications) -│ │ ├── handler.go # MCP request handler (tools/list, tools/call dispatch) -│ │ └── handler_test.go -│ ├── transport/ -│ │ ├── transport.go # Transport interface -│ │ ├── stdio.go # stdio transport -│ │ ├── stdio_test.go -│ │ ├── http.go # Streamable HTTP transport -│ │ ├── http_test.go -│ │ ├── sse.go # SSE transport -│ │ ├── sse_test.go -│ │ ├── grpc.go # gRPC transport -│ │ ├── grpc_test.go -│ │ ├── ws.go # WebSocket transport -│ │ └── ws_test.go -│ ├── middleware/ -│ │ ├── chain.go # Middleware interceptor chain -│ │ ├── logging.go # Structured logging middleware -│ │ └── errors.go # Error formatting middleware -│ ├── progress/ -│ │ ├── progress.go # Progress notification proxy -│ │ └── progress_test.go -│ ├── tasks/ -│ │ ├── manager.go # Async task lifecycle manager -│ │ └── manager_test.go -│ ├── cancel/ -│ │ ├── tracker.go # Cancellation tracking for in-flight calls -│ │ └── tracker_test.go -│ ├── serverlog/ -│ │ ├── forwarder.go # Server log forwarding to MCP notifications/message -│ │ └── forwarder_test.go -│ └── config/ -│ └── config.go # CLI flag parsing and config struct -├── gen/ -│ └── proto/ -│ └── protomcp/ -│ ├── protomcp.pb.go # Generated Go protobuf types -│ └── protomcp_grpc.pb.go # Generated gRPC service (for external gRPC transport) -├── go.mod -├── go.sum -├── sdk/ -│ ├── python/ -│ │ ├── pyproject.toml -│ │ ├── src/ -│ │ │ └── protomcp/ -│ │ │ ├── __init__.py # Public API exports -│ │ │ ├── tool.py # @tool decorator + schema generation -│ │ │ ├── result.py # ToolResult type -│ │ │ ├── manager.py # tool_manager client -│ │ │ ├── context.py # ToolContext (progress, cancellation) -│ │ │ ├── log.py # Server logging API -│ │ │ ├── transport.py # Unix socket + envelope framing -│ │ │ └── runner.py # Main loop: connect, listen, reload, dispatch -│ │ ├── tests/ -│ │ │ ├── test_tool.py -│ │ │ ├── test_result.py -│ │ │ ├── test_manager.py -│ │ │ ├── test_transport.py -│ │ │ └── test_integration.py # End-to-end with Go binary -│ │ └── gen/ -│ │ └── protomcp_pb2.py # Generated Python protobuf types -│ └── typescript/ -│ ├── package.json -│ ├── tsconfig.json -│ ├── vitest.config.ts -│ ├── src/ -│ │ ├── index.ts # Public API exports -│ │ ├── tool.ts # tool() function + Zod schema generation -│ │ ├── result.ts # ToolResult type -│ │ ├── manager.ts # tool_manager client -│ │ ├── context.ts # ToolContext (progress, cancellation) -│ │ ├── log.ts # Server logging API -│ │ ├── transport.ts # Unix socket + envelope framing -│ │ └── runner.ts # Main loop -│ ├── tests/ -│ │ ├── tool.test.ts -│ │ ├── result.test.ts -│ │ ├── manager.test.ts -│ │ ├── transport.test.ts -│ │ └── integration.test.ts # End-to-end with Go binary -│ └── gen/ -│ └── protomcp.ts # Generated TS protobuf types -├── test/ -│ ├── e2e/ -│ │ ├── e2e_test.go # Full pipeline: Go binary + Python tool + MCP client -│ │ ├── fixtures/ -│ │ │ ├── simple_tool.py # Basic Python tool for testing -│ │ │ ├── dynamic_tool.py # Tool with enable/disable mutations -│ │ │ ├── simple_tool.ts # Basic TS tool for testing -│ │ │ ├── crash_tool.py # Tool that crashes (for recovery testing) -│ │ │ ├── progress_tool.py # Tool that reports progress -│ │ │ ├── async_tool.py # Tool with task_support for async testing -│ │ │ ├── logging_tool.py # Tool that emits server logs -│ │ │ └── structured_output_tool.py # Tool with output_type for structured output -│ │ └── helpers.go # Test helpers (start binary, MCP client mock) -│ └── proto/ -│ └── proto_test.go # Validate proto generates correctly for all languages -├── docs/ -│ ├── astro.config.mjs -│ ├── package.json -│ └── src/ -│ └── content/ -│ └── docs/ -│ ├── index.mdx -│ ├── getting-started/ -│ │ ├── installation.mdx -│ │ ├── quick-start.mdx -│ │ └── how-it-works.mdx -│ ├── guides/ -│ │ ├── writing-tools-python.mdx -│ │ ├── writing-tools-typescript.mdx -│ │ ├── dynamic-tool-lists.mdx -│ │ ├── hot-reload.mdx -│ │ ├── progress-notifications.mdx -│ │ ├── async-tasks.mdx -│ │ ├── cancellation.mdx -│ │ ├── server-logging.mdx -│ │ ├── structured-output.mdx -│ │ ├── error-handling.mdx -│ │ └── production-deployment.mdx -│ ├── reference/ -│ │ ├── cli.mdx -│ │ ├── protobuf-spec.mdx -│ │ ├── python-api.mdx -│ │ └── typescript-api.mdx -│ └── concepts/ -│ ├── architecture.mdx -│ ├── tool-list-modes.mdx -│ └── transports.mdx -├── Makefile # proto-gen, build, test, docs commands -└── .goreleaser.yml # Cross-platform binary releases -``` - ---- - -## Chunk 1: Foundation — Project Setup, Protobuf, Envelope, Tool List Manager - -### Task 1: Project Initialization - -**Files:** -- Create: `go.mod` -- Create: `Makefile` -- Create: `.gitignore` -- Create: `.goreleaser.yml` - -- [ ] **Step 1: Initialize Go module** - -```bash -cd /Users/msilverblatt/hotmcp -go mod init github.com/msilverblatt/protomcp -``` - -- [ ] **Step 2: Create Makefile** - -Create `Makefile` with targets: -```makefile -.PHONY: proto build test clean - -PROTO_DIR := proto -GEN_DIR := gen/proto/protomcp -PYTHON_GEN_DIR := sdk/python/gen -TS_GEN_DIR := sdk/typescript/gen - -proto: - mkdir -p $(GEN_DIR) $(PYTHON_GEN_DIR) $(TS_GEN_DIR) - protoc --go_out=$(GEN_DIR) --go_opt=paths=source_relative \ - --go-grpc_out=$(GEN_DIR) --go-grpc_opt=paths=source_relative \ - -I$(PROTO_DIR) $(PROTO_DIR)/protomcp.proto - protoc --python_out=$(PYTHON_GEN_DIR) \ - -I$(PROTO_DIR) $(PROTO_DIR)/protomcp.proto - protoc --plugin=protoc-gen-ts=$$(which protoc-gen-ts) \ - --ts_out=$(TS_GEN_DIR) \ - -I$(PROTO_DIR) $(PROTO_DIR)/protomcp.proto - -build: - go build -o bin/protomcp ./cmd/protomcp - -test: - go test ./... - -test-python: - cd sdk/python && python -m pytest tests/ -v - -test-ts: - cd sdk/typescript && npx vitest run - -test-all: test test-python test-ts - -clean: - rm -rf bin/ gen/ -``` - -- [ ] **Step 3: Create .gitignore** - -``` -bin/ -gen/ -dist/ -node_modules/ -__pycache__/ -*.egg-info/ -.venv/ -*.pb.go -*_pb2.py -``` - -- [ ] **Step 4: Create .goreleaser.yml** - -```yaml -version: 2 -builds: - - main: ./cmd/protomcp - binary: protomcp - goos: - - linux - - darwin - - windows - goarch: - - amd64 - - arm64 - env: - - CGO_ENABLED=0 - -archives: - - format: tar.gz - name_template: "protomcp_{{ .Os }}_{{ .Arch }}" - -brews: - - repository: - owner: msilverblatt - name: homebrew-tap - homepage: "https://github.com/msilverblatt/protomcp" - description: "Language-agnostic MCP runtime" -``` - -- [ ] **Step 5: Commit** - -```bash -git add go.mod Makefile .gitignore .goreleaser.yml -git commit -m "feat: initialize project with Go module, Makefile, and release config" -``` - ---- - -### Task 2: Protobuf Spec - -**Files:** -- Create: `proto/protomcp.proto` - -- [ ] **Step 1: Write the protobuf spec** - -Create `proto/protomcp.proto` — this is the single source of truth for the internal protocol between the Go binary and tool processes: - -```protobuf -syntax = "proto3"; - -package protomcp; - -option go_package = "github.com/msilverblatt/protomcp/gen/proto/protomcp"; - -// Envelope wraps all messages with length-prefixed framing. -// Wire format: 4-byte big-endian uint32 length prefix, then serialized Envelope. -message Envelope { - oneof msg { - // Go -> Tool Process - ReloadRequest reload = 1; - ListToolsRequest list_tools = 2; - CallToolRequest call_tool = 3; - - // Tool Process -> Go - ReloadResponse reload_response = 4; - ToolListResponse tool_list = 5; - CallToolResponse call_result = 6; - EnableToolsRequest enable_tools = 7; - DisableToolsRequest disable_tools = 8; - SetAllowedRequest set_allowed = 9; - SetBlockedRequest set_blocked = 10; - GetActiveToolsRequest get_active_tools = 11; - BatchUpdateRequest batch = 12; - ActiveToolsResponse active_tools = 13; - - // Progress, cancellation, logging, tasks - ProgressNotification progress = 16; - CancelRequest cancel = 17; - LogMessage log = 18; - CreateTaskResponse create_task = 19; - TaskStatusRequest task_status = 20; - TaskStatusResponse task_status_response = 21; - TaskResultRequest task_result = 22; - TaskCancelRequest task_cancel = 23; - } - // Correlation ID for matching requests to responses. - // Required for CallToolRequest/CallToolResponse to support concurrent calls. - // Optional for other message types. - string request_id = 14; - // Namespace for future multi-process support. Ignored in v1. - string namespace = 15; -} - -// --- Go -> Tool Process --- - -message ReloadRequest {} - -message ListToolsRequest {} - -message CallToolRequest { - string name = 1; - // JSON-encoded arguments from the MCP client - string arguments_json = 2; - // Progress token from client's _meta.progressToken (empty if none) - string progress_token = 3; -} - -// --- Tool Process -> Go --- - -message ReloadResponse { - bool success = 1; - string error = 2; // Non-empty on failure (e.g., syntax error, import error) -} - -message ToolListResponse { - repeated ToolDefinition tools = 1; -} - -message ToolDefinition { - string name = 1; - string description = 2; - // JSON Schema for the tool's input parameters - string input_schema_json = 3; - // JSON Schema for the tool's output (empty if unstructured) - string output_schema_json = 4; - // Tool metadata/annotations - string title = 5; - bool read_only_hint = 6; - bool destructive_hint = 7; - bool idempotent_hint = 8; - bool open_world_hint = 9; - bool task_support = 10; // Advertises async task capability -} - -message CallToolResponse { - bool is_error = 1; - // JSON-encoded result content (MCP content array) - string result_json = 2; - // Optional tool list mutations — Go binary intercepts these before proxying to host - repeated string enable_tools = 3; - repeated string disable_tools = 4; - // Structured error details (if is_error is true) - ToolError error = 5; - // Structured output (JSON, validated against outputSchema) - string structured_content_json = 6; -} - -message ToolError { - string error_code = 1; - string message = 2; - string suggestion = 3; - bool retryable = 4; -} - -// --- Tool List Control (Tool Process -> Go) --- - -message EnableToolsRequest { - repeated string tool_names = 1; -} - -message DisableToolsRequest { - repeated string tool_names = 1; -} - -message SetAllowedRequest { - repeated string tool_names = 1; -} - -message SetBlockedRequest { - repeated string tool_names = 1; -} - -message GetActiveToolsRequest {} - -message BatchUpdateRequest { - repeated string enable = 1; - repeated string disable = 2; - repeated string allow = 3; // Set allowed list (mutually exclusive with block) - repeated string block = 4; // Set blocked list (mutually exclusive with allow) -} - -message ActiveToolsResponse { - repeated string tool_names = 1; -} - -// --- Progress, Cancellation, Logging --- - -message ProgressNotification { - string progress_token = 1; // Token from CallToolRequest - int64 progress = 2; // Current progress value - int64 total = 3; // Total expected (0 = indeterminate) - string message = 4; // Human-readable status -} - -message CancelRequest { - string request_id = 1; // ID of the call to cancel -} - -message LogMessage { - string level = 1; // RFC 5424: emergency, alert, critical, error, warning, notice, info, debug - string logger = 2; // Source component (optional) - string data_json = 3; // JSON-encoded log data -} - -// --- Task (Async) Lifecycle --- - -message CreateTaskResponse { - string task_id = 1; -} - -message TaskStatusRequest { - string task_id = 1; -} - -message TaskStatusResponse { - string task_id = 1; - string state = 2; // running, completed, failed, cancelled - int64 progress = 3; // Optional progress - int64 total = 4; // Optional total - string message = 5; // Optional status message -} - -message TaskResultRequest { - string task_id = 1; -} - -message TaskCancelRequest { - string task_id = 1; -} -``` - -- [ ] **Step 2: Generate Go code from proto** - -```bash -make proto -``` - -Verify: `gen/proto/protomcp/protomcp.pb.go` exists and compiles. - -- [ ] **Step 3: Commit** - -```bash -git add proto/protomcp.proto -git commit -m "feat: add protobuf spec defining internal protocol between Go binary and tool processes" -``` - ---- - -### Task 3: Envelope — Length-Prefixed Protobuf Read/Write - -**Files:** -- Create: `internal/envelope/envelope.go` -- Create: `internal/envelope/envelope_test.go` - -- [ ] **Step 1: Write failing tests for envelope** - -Create `internal/envelope/envelope_test.go`: - -```go -package envelope_test - -import ( - "bytes" - "testing" - - "github.com/msilverblatt/protomcp/internal/envelope" - pb "github.com/msilverblatt/protomcp/gen/proto/protomcp" -) - -func TestWriteAndReadEnvelope(t *testing.T) { - // Create a test envelope with a ListToolsRequest - env := &pb.Envelope{ - Msg: &pb.Envelope_ListTools{ - ListTools: &pb.ListToolsRequest{}, - }, - RequestId: "test-123", - } - - var buf bytes.Buffer - err := envelope.Write(&buf, env) - if err != nil { - t.Fatalf("Write failed: %v", err) - } - - got, err := envelope.Read(&buf) - if err != nil { - t.Fatalf("Read failed: %v", err) - } - - if got.RequestId != "test-123" { - t.Errorf("RequestId = %q, want %q", got.RequestId, "test-123") - } - - if got.GetListTools() == nil { - t.Error("expected ListToolsRequest, got nil") - } -} - -func TestWriteAndReadCallTool(t *testing.T) { - env := &pb.Envelope{ - Msg: &pb.Envelope_CallTool{ - CallTool: &pb.CallToolRequest{ - Name: "search", - ArgumentsJson: `{"query": "hello"}`, - }, - }, - RequestId: "call-456", - } - - var buf bytes.Buffer - err := envelope.Write(&buf, env) - if err != nil { - t.Fatalf("Write failed: %v", err) - } - - got, err := envelope.Read(&buf) - if err != nil { - t.Fatalf("Read failed: %v", err) - } - - ct := got.GetCallTool() - if ct == nil { - t.Fatal("expected CallToolRequest, got nil") - } - if ct.Name != "search" { - t.Errorf("Name = %q, want %q", ct.Name, "search") - } - if ct.ArgumentsJson != `{"query": "hello"}` { - t.Errorf("ArgumentsJson = %q, want %q", ct.ArgumentsJson, `{"query": "hello"}`) - } -} - -func TestReadEmptyBuffer(t *testing.T) { - var buf bytes.Buffer - _, err := envelope.Read(&buf) - if err == nil { - t.Error("expected error reading empty buffer, got nil") - } -} - -func TestMultipleEnvelopes(t *testing.T) { - var buf bytes.Buffer - - envs := []*pb.Envelope{ - {Msg: &pb.Envelope_ListTools{ListTools: &pb.ListToolsRequest{}}, RequestId: "1"}, - {Msg: &pb.Envelope_Reload{Reload: &pb.ReloadRequest{}}, RequestId: "2"}, - {Msg: &pb.Envelope_ListTools{ListTools: &pb.ListToolsRequest{}}, RequestId: "3"}, - } - - for _, env := range envs { - if err := envelope.Write(&buf, env); err != nil { - t.Fatalf("Write failed: %v", err) - } - } - - for i, want := range envs { - got, err := envelope.Read(&buf) - if err != nil { - t.Fatalf("Read %d failed: %v", i, err) - } - if got.RequestId != want.RequestId { - t.Errorf("envelope %d: RequestId = %q, want %q", i, got.RequestId, want.RequestId) - } - } -} -``` - -- [ ] **Step 2: Run tests to verify they fail** - -```bash -go test ./internal/envelope/... -v -``` - -Expected: compilation error — package `envelope` doesn't exist. - -- [ ] **Step 3: Implement envelope** - -Create `internal/envelope/envelope.go`: - -```go -package envelope - -import ( - "encoding/binary" - "fmt" - "io" - - pb "github.com/msilverblatt/protomcp/gen/proto/protomcp" - "google.golang.org/protobuf/proto" -) - -const maxMessageSize = 10 * 1024 * 1024 // 10MB - -// Write serializes an Envelope and writes it with a 4-byte big-endian length prefix. -func Write(w io.Writer, env *pb.Envelope) error { - data, err := proto.Marshal(env) - if err != nil { - return fmt.Errorf("marshal envelope: %w", err) - } - - length := uint32(len(data)) - if err := binary.Write(w, binary.BigEndian, length); err != nil { - return fmt.Errorf("write length prefix: %w", err) - } - - if _, err := w.Write(data); err != nil { - return fmt.Errorf("write envelope data: %w", err) - } - - return nil -} - -// Read reads a length-prefixed Envelope from the reader. -func Read(r io.Reader) (*pb.Envelope, error) { - var length uint32 - if err := binary.Read(r, binary.BigEndian, &length); err != nil { - return nil, fmt.Errorf("read length prefix: %w", err) - } - - if length > maxMessageSize { - return nil, fmt.Errorf("message size %d exceeds max %d", length, maxMessageSize) - } - - data := make([]byte, length) - if _, err := io.ReadFull(r, data); err != nil { - return nil, fmt.Errorf("read envelope data: %w", err) - } - - env := &pb.Envelope{} - if err := proto.Unmarshal(data, env); err != nil { - return nil, fmt.Errorf("unmarshal envelope: %w", err) - } - - return env, nil -} -``` - -- [ ] **Step 4: Run tests to verify they pass** - -```bash -go test ./internal/envelope/... -v -``` - -Expected: all 4 tests PASS. - -- [ ] **Step 5: Commit** - -```bash -git add internal/envelope/ -git commit -m "feat: implement length-prefixed protobuf envelope read/write" -``` - ---- - -### Task 4: Tool List Manager — State Machine - -**Files:** -- Create: `internal/toollist/manager.go` -- Create: `internal/toollist/manager_test.go` - -- [ ] **Step 1: Write failing tests for tool list manager** - -Create `internal/toollist/manager_test.go`: - -```go -package toollist_test - -import ( - "testing" - - "github.com/msilverblatt/protomcp/internal/toollist" -) - -func TestOpenMode_AllToolsActive(t *testing.T) { - m := toollist.New() - m.SetRegistered([]string{"a", "b", "c"}) - - got := m.Active() - want := []string{"a", "b", "c"} - if !slicesEqual(got, want) { - t.Errorf("Active() = %v, want %v", got, want) - } -} - -func TestOpenMode_DisableTool(t *testing.T) { - m := toollist.New() - m.SetRegistered([]string{"a", "b", "c"}) - - changed := m.Disable([]string{"b"}) - if !changed { - t.Error("Disable should report change") - } - - got := m.Active() - want := []string{"a", "c"} - if !slicesEqual(got, want) { - t.Errorf("Active() = %v, want %v", got, want) - } -} - -func TestOpenMode_EnableReAdds(t *testing.T) { - m := toollist.New() - m.SetRegistered([]string{"a", "b", "c"}) - m.Disable([]string{"b"}) - - changed := m.Enable([]string{"b"}) - if !changed { - t.Error("Enable should report change") - } - - got := m.Active() - want := []string{"a", "b", "c"} - if !slicesEqual(got, want) { - t.Errorf("Active() = %v, want %v", got, want) - } -} - -func TestAllowlistMode(t *testing.T) { - m := toollist.New() - m.SetRegistered([]string{"a", "b", "c", "d"}) - - changed := m.SetAllowed([]string{"a", "c"}) - if !changed { - t.Error("SetAllowed should report change") - } - - got := m.Active() - want := []string{"a", "c"} - if !slicesEqual(got, want) { - t.Errorf("Active() = %v, want %v", got, want) - } -} - -func TestAllowlistMode_EnableAddsToAllowlist(t *testing.T) { - m := toollist.New() - m.SetRegistered([]string{"a", "b", "c"}) - m.SetAllowed([]string{"a"}) - - m.Enable([]string{"b"}) - - got := m.Active() - want := []string{"a", "b"} - if !slicesEqual(got, want) { - t.Errorf("Active() = %v, want %v", got, want) - } -} - -func TestAllowlistMode_DisableRemovesFromAllowlist(t *testing.T) { - m := toollist.New() - m.SetRegistered([]string{"a", "b", "c"}) - m.SetAllowed([]string{"a", "b"}) - - m.Disable([]string{"a"}) - - got := m.Active() - want := []string{"b"} - if !slicesEqual(got, want) { - t.Errorf("Active() = %v, want %v", got, want) - } -} - -func TestBlocklistMode(t *testing.T) { - m := toollist.New() - m.SetRegistered([]string{"a", "b", "c", "d"}) - - changed := m.SetBlocked([]string{"b", "d"}) - if !changed { - t.Error("SetBlocked should report change") - } - - got := m.Active() - want := []string{"a", "c"} - if !slicesEqual(got, want) { - t.Errorf("Active() = %v, want %v", got, want) - } -} - -func TestBlocklistMode_DisableAddsToBlocklist(t *testing.T) { - m := toollist.New() - m.SetRegistered([]string{"a", "b", "c"}) - m.SetBlocked([]string{"c"}) - - m.Disable([]string{"a"}) - - got := m.Active() - want := []string{"b"} - if !slicesEqual(got, want) { - t.Errorf("Active() = %v, want %v", got, want) - } -} - -func TestBlocklistMode_EnableRemovesFromBlocklist(t *testing.T) { - m := toollist.New() - m.SetRegistered([]string{"a", "b", "c"}) - m.SetBlocked([]string{"b", "c"}) - - m.Enable([]string{"c"}) - - got := m.Active() - want := []string{"a", "c"} - if !slicesEqual(got, want) { - t.Errorf("Active() = %v, want %v", got, want) - } -} - -func TestSetAllowedSwitchesFromBlocklist(t *testing.T) { - m := toollist.New() - m.SetRegistered([]string{"a", "b", "c"}) - m.SetBlocked([]string{"c"}) - m.SetAllowed([]string{"a"}) - - got := m.Active() - want := []string{"a"} - if !slicesEqual(got, want) { - t.Errorf("Active() = %v, want %v", got, want) - } -} - -func TestEmptyAllowlistResetsToOpen(t *testing.T) { - m := toollist.New() - m.SetRegistered([]string{"a", "b", "c"}) - m.SetAllowed([]string{"a"}) - m.SetAllowed([]string{}) - - got := m.Active() - want := []string{"a", "b", "c"} - if !slicesEqual(got, want) { - t.Errorf("Active() = %v, want %v", got, want) - } -} - -func TestEmptyBlocklistResetsToOpen(t *testing.T) { - m := toollist.New() - m.SetRegistered([]string{"a", "b", "c"}) - m.SetBlocked([]string{"a"}) - m.SetBlocked([]string{}) - - got := m.Active() - want := []string{"a", "b", "c"} - if !slicesEqual(got, want) { - t.Errorf("Active() = %v, want %v", got, want) - } -} - -func TestBatch_EnableAndDisable(t *testing.T) { - m := toollist.New() - m.SetRegistered([]string{"a", "b", "c"}) - m.Disable([]string{"a", "b"}) - - changed, err := m.Batch([]string{"a"}, []string{"c"}, nil, nil) - if err != nil { - t.Fatalf("Batch failed: %v", err) - } - if !changed { - t.Error("Batch should report change") - } - - got := m.Active() - want := []string{"a"} - if !slicesEqual(got, want) { - t.Errorf("Active() = %v, want %v", got, want) - } -} - -func TestBatch_AllowAndBlockRejects(t *testing.T) { - m := toollist.New() - m.SetRegistered([]string{"a", "b", "c"}) - - _, err := m.Batch(nil, nil, []string{"a"}, []string{"b"}) - if err == nil { - t.Error("Batch with both allow and block should return error") - } -} - -func TestBatch_AllowThenEnableDelta(t *testing.T) { - m := toollist.New() - m.SetRegistered([]string{"a", "b", "c"}) - - // Set allow to ["a"], then enable "b" — should result in ["a", "b"] - changed, err := m.Batch([]string{"b"}, nil, []string{"a"}, nil) - if err != nil { - t.Fatalf("Batch failed: %v", err) - } - if !changed { - t.Error("Batch should report change") - } - - got := m.Active() - want := []string{"a", "b"} - if !slicesEqual(got, want) { - t.Errorf("Active() = %v, want %v", got, want) - } -} - -func TestNoChangeReturnsFalse(t *testing.T) { - m := toollist.New() - m.SetRegistered([]string{"a", "b", "c"}) - - changed := m.Enable([]string{"a"}) // already active - if changed { - t.Error("Enable of already-active tool should return false") - } -} - -func slicesEqual(a, b []string) bool { - if len(a) != len(b) { - return false - } - m := make(map[string]bool, len(a)) - for _, v := range a { - m[v] = true - } - for _, v := range b { - if !m[v] { - return false - } - } - return true -} -``` - -- [ ] **Step 2: Run tests to verify they fail** - -```bash -go test ./internal/toollist/... -v -``` - -Expected: compilation error — package `toollist` doesn't exist. - -- [ ] **Step 3: Implement tool list manager** - -Create `internal/toollist/manager.go`: - -```go -package toollist - -import ( - "fmt" - "sort" - "sync" -) - -type mode int - -const ( - modeOpen mode = iota - modeAllowlist - modeBlocklist -) - -// Manager tracks which tools are active using open/allowlist/blocklist modes. -// All methods are safe for concurrent use. -type Manager struct { - mu sync.RWMutex - registered map[string]bool // all tools the tool process has registered - current mode - set map[string]bool // allowlist entries, blocklist entries, or disabled set (open mode) -} - -func New() *Manager { - return &Manager{ - registered: make(map[string]bool), - current: modeOpen, - set: make(map[string]bool), - } -} - -// SetRegistered updates the full set of registered tools (called after reload). -func (m *Manager) SetRegistered(tools []string) { - m.mu.Lock() - defer m.mu.Unlock() - m.registered = make(map[string]bool, len(tools)) - for _, t := range tools { - m.registered[t] = true - } -} - -// Active returns the currently active tool names. -func (m *Manager) Active() []string { - m.mu.RLock() - defer m.mu.RUnlock() - return m.activeLocked() -} - -func (m *Manager) activeLocked() []string { - var result []string - switch m.current { - case modeOpen: - for t := range m.registered { - if !m.set[t] { // set contains disabled tools in open mode - result = append(result, t) - } - } - case modeAllowlist: - for t := range m.set { - if m.registered[t] { - result = append(result, t) - } - } - case modeBlocklist: - for t := range m.registered { - if !m.set[t] { - result = append(result, t) - } - } - } - sort.Strings(result) - return result -} - -// Enable adds tools. Behavior depends on mode: -// Open: re-enables disabled tools. Allowlist: adds to allowlist. Blocklist: removes from blocklist. -func (m *Manager) Enable(tools []string) bool { - m.mu.Lock() - defer m.mu.Unlock() - before := m.activeLocked() - - switch m.current { - case modeOpen: - for _, t := range tools { - delete(m.set, t) - } - case modeAllowlist: - for _, t := range tools { - m.set[t] = true - } - case modeBlocklist: - for _, t := range tools { - delete(m.set, t) - } - } - - after := m.activeLocked() - return !slicesEqual(before, after) -} - -// Disable removes tools. Behavior depends on mode: -// Open: disables tools. Allowlist: removes from allowlist. Blocklist: adds to blocklist. -func (m *Manager) Disable(tools []string) bool { - m.mu.Lock() - defer m.mu.Unlock() - before := m.activeLocked() - - switch m.current { - case modeOpen: - for _, t := range tools { - m.set[t] = true - } - case modeAllowlist: - for _, t := range tools { - delete(m.set, t) - } - case modeBlocklist: - for _, t := range tools { - m.set[t] = true - } - } - - after := m.activeLocked() - return !slicesEqual(before, after) -} - -// SetAllowed switches to allowlist mode. Empty list resets to open mode. -func (m *Manager) SetAllowed(tools []string) bool { - m.mu.Lock() - defer m.mu.Unlock() - before := m.activeLocked() - - if len(tools) == 0 { - m.current = modeOpen - m.set = make(map[string]bool) - } else { - m.current = modeAllowlist - m.set = make(map[string]bool, len(tools)) - for _, t := range tools { - m.set[t] = true - } - } - - after := m.activeLocked() - return !slicesEqual(before, after) -} - -// SetBlocked switches to blocklist mode. Empty list resets to open mode. -func (m *Manager) SetBlocked(tools []string) bool { - m.mu.Lock() - defer m.mu.Unlock() - before := m.activeLocked() - - if len(tools) == 0 { - m.current = modeOpen - m.set = make(map[string]bool) - } else { - m.current = modeBlocklist - m.set = make(map[string]bool, len(tools)) - for _, t := range tools { - m.set[t] = true - } - } - - after := m.activeLocked() - return !slicesEqual(before, after) -} - -// Batch applies multiple operations atomically. Allow and block are mutually exclusive. -// Order: mode-setting (allow/block) first, then deltas (enable/disable). -func (m *Manager) Batch(enable, disable, allow, block []string) (bool, error) { - if len(allow) > 0 && len(block) > 0 { - return false, fmt.Errorf("batch: allow and block are mutually exclusive") - } - - m.mu.Lock() - defer m.mu.Unlock() - before := m.activeLocked() - - // Mode-setting first - if len(allow) > 0 { - m.current = modeAllowlist - m.set = make(map[string]bool, len(allow)) - for _, t := range allow { - m.set[t] = true - } - } else if len(block) > 0 { - m.current = modeBlocklist - m.set = make(map[string]bool, len(block)) - for _, t := range block { - m.set[t] = true - } - } - - // Then deltas (reuse logic without locks since we hold mu) - if len(enable) > 0 { - switch m.current { - case modeOpen: - for _, t := range enable { - delete(m.set, t) - } - case modeAllowlist: - for _, t := range enable { - m.set[t] = true - } - case modeBlocklist: - for _, t := range enable { - delete(m.set, t) - } - } - } - if len(disable) > 0 { - switch m.current { - case modeOpen: - for _, t := range disable { - m.set[t] = true - } - case modeAllowlist: - for _, t := range disable { - delete(m.set, t) - } - case modeBlocklist: - for _, t := range disable { - m.set[t] = true - } - } - } - - after := m.activeLocked() - return !slicesEqual(before, after), nil -} - -func slicesEqual(a, b []string) bool { - if len(a) != len(b) { - return false - } - m := make(map[string]bool, len(a)) - for _, v := range a { - m[v] = true - } - for _, v := range b { - if !m[v] { - return false - } - } - return true -} -``` - -- [ ] **Step 4: Run tests to verify they pass** - -```bash -go test ./internal/toollist/... -v -``` - -Expected: all 16 tests PASS. - -- [ ] **Step 5: Commit** - -```bash -git add internal/toollist/ -git commit -m "feat: implement tool list state machine with open/allowlist/blocklist modes" -``` - ---- - -## Chunk 2: Go Binary Core — Process Manager, MCP Handler, Config - -### Task 5: Config — CLI Flag Parsing - -**Files:** -- Create: `internal/config/config.go` -- Create: `internal/config/config_test.go` - -- [ ] **Step 1: Write failing tests for config parsing** - -Create `internal/config/config_test.go`: - -```go -package config_test - -import ( - "reflect" - "testing" - "time" - - "github.com/msilverblatt/protomcp/internal/config" -) - -func TestParseDefaults(t *testing.T) { - cfg, err := config.Parse([]string{"dev", "server.py"}) - if err != nil { - t.Fatalf("Parse failed: %v", err) - } - if cfg.Command != "dev" { - t.Errorf("Command = %q, want %q", cfg.Command, "dev") - } - if cfg.File != "server.py" { - t.Errorf("File = %q, want %q", cfg.File, "server.py") - } - if cfg.Transport != "stdio" { - t.Errorf("Transport = %q, want %q", cfg.Transport, "stdio") - } - if cfg.CallTimeout != 5*time.Minute { - t.Errorf("CallTimeout = %v, want %v", cfg.CallTimeout, 5*time.Minute) - } - if cfg.HotReloadImmediate { - t.Error("HotReloadImmediate should default to false") - } - if cfg.LogLevel != "info" { - t.Errorf("LogLevel = %q, want %q", cfg.LogLevel, "info") - } -} - -func TestParseWithFlags(t *testing.T) { - cfg, err := config.Parse([]string{ - "run", "tools.ts", - "--transport", "grpc", - "--hot-reload", "immediate", - "--call-timeout", "30s", - "--log-level", "debug", - "--socket", "/tmp/test.sock", - "--runtime", "python3.12", - }) - if err != nil { - t.Fatalf("Parse failed: %v", err) - } - if cfg.Command != "run" { - t.Errorf("Command = %q, want %q", cfg.Command, "run") - } - if cfg.File != "tools.ts" { - t.Errorf("File = %q, want %q", cfg.File, "tools.ts") - } - if cfg.Transport != "grpc" { - t.Errorf("Transport = %q, want %q", cfg.Transport, "grpc") - } - if !cfg.HotReloadImmediate { - t.Error("HotReloadImmediate should be true") - } - if cfg.CallTimeout != 30*time.Second { - t.Errorf("CallTimeout = %v, want %v", cfg.CallTimeout, 30*time.Second) - } - if cfg.SocketPath != "/tmp/test.sock" { - t.Errorf("SocketPath = %q, want %q", cfg.SocketPath, "/tmp/test.sock") - } - if cfg.Runtime != "python3.12" { - t.Errorf("Runtime = %q, want %q", cfg.Runtime, "python3.12") - } -} - -func TestParseMissingFile(t *testing.T) { - _, err := config.Parse([]string{"dev"}) - if err == nil { - t.Error("expected error for missing file argument") - } -} - -func TestParseInvalidCommand(t *testing.T) { - _, err := config.Parse([]string{"foo", "server.py"}) - if err == nil { - t.Error("expected error for invalid command") - } -} - -func TestRuntimeCommand(t *testing.T) { - tests := []struct { - file string - wantCmd string - wantArgs []string - }{ - {"server.py", "python3", []string{"server.py"}}, - {"server.ts", "npx", []string{"tsx", "server.ts"}}, - {"server.js", "node", []string{"server.js"}}, - {"server.go", "go", []string{"run", "server.go"}}, - {"server.rs", "cargo", []string{"run", "server.rs"}}, - {"server", "server", nil}, - } - for _, tt := range tests { - cmd, args := config.RuntimeCommand(tt.file) - if cmd != tt.wantCmd { - t.Errorf("RuntimeCommand(%q) cmd = %q, want %q", tt.file, cmd, tt.wantCmd) - } - if !reflect.DeepEqual(args, tt.wantArgs) { - t.Errorf("RuntimeCommand(%q) args = %v, want %v", tt.file, args, tt.wantArgs) - } - } -} -``` - -- [ ] **Step 2: Run tests to verify they fail** - -```bash -go test ./internal/config/... -v -``` - -- [ ] **Step 3: Implement config** - -Create `internal/config/config.go`: - -```go -package config - -import ( - "fmt" - "os" - "path/filepath" - "time" -) - -type Config struct { - Command string - File string - Transport string - HotReloadImmediate bool - CallTimeout time.Duration - LogLevel string - SocketPath string - Runtime string - Host string // For network transports (default: "localhost") - Port int // For network transports (default: 8080) -} - -func Parse(args []string) (*Config, error) { - if len(args) < 1 { - return nil, fmt.Errorf("usage: protomcp [flags]") - } - - cmd := args[0] - if cmd != "dev" && cmd != "run" { - return nil, fmt.Errorf("unknown command %q: must be 'dev' or 'run'", cmd) - } - - cfg := &Config{ - Command: cmd, - Transport: "stdio", - CallTimeout: 5 * time.Minute, - LogLevel: "info", - Host: "localhost", - Port: 8080, - } - - // Find file argument (first non-flag after command) - i := 1 - if i >= len(args) || args[i] == "--" { - return nil, fmt.Errorf("missing file argument") - } - if args[i][0] != '-' { - cfg.File = args[i] - i++ - } else { - return nil, fmt.Errorf("missing file argument") - } - - // Parse flags - for i < len(args) { - switch args[i] { - case "--transport": - i++ - if i >= len(args) { - return nil, fmt.Errorf("--transport requires a value") - } - cfg.Transport = args[i] - case "--hot-reload": - i++ - if i >= len(args) { - return nil, fmt.Errorf("--hot-reload requires a value") - } - if args[i] == "immediate" { - cfg.HotReloadImmediate = true - } - case "--call-timeout": - i++ - if i >= len(args) { - return nil, fmt.Errorf("--call-timeout requires a value") - } - d, err := time.ParseDuration(args[i]) - if err != nil { - return nil, fmt.Errorf("invalid --call-timeout: %w", err) - } - cfg.CallTimeout = d - case "--log-level": - i++ - if i >= len(args) { - return nil, fmt.Errorf("--log-level requires a value") - } - cfg.LogLevel = args[i] - case "--socket": - i++ - if i >= len(args) { - return nil, fmt.Errorf("--socket requires a value") - } - cfg.SocketPath = args[i] - case "--runtime": - i++ - if i >= len(args) { - return nil, fmt.Errorf("--runtime requires a value") - } - cfg.Runtime = args[i] - case "--host": - i++ - if i >= len(args) { - return nil, fmt.Errorf("--host requires a value") - } - cfg.Host = args[i] - case "--port": - i++ - if i >= len(args) { - return nil, fmt.Errorf("--port requires a value") - } - p, err := strconv.Atoi(args[i]) - if err != nil { - return nil, fmt.Errorf("invalid --port: %w", err) - } - cfg.Port = p - default: - return nil, fmt.Errorf("unknown flag %q", args[i]) - } - i++ - } - - // Default socket path - if cfg.SocketPath == "" { - dir := os.Getenv("XDG_RUNTIME_DIR") - if dir == "" { - dir = os.TempDir() - } - cfg.SocketPath = filepath.Join(dir, "protomcp", fmt.Sprintf("%d.sock", os.Getpid())) - } - - return cfg, nil -} - -// RuntimeCommand returns the command and args to run a tool file. -// Returns (command, args) to be used with exec.Command(command, args...). -func RuntimeCommand(file string) (string, []string) { - ext := filepath.Ext(file) - switch ext { - case ".py": - cmd := "python3" - if env := os.Getenv("PROTOMCP_PYTHON"); env != "" { - cmd = env - } - return cmd, []string{file} - case ".ts": - cmd := "npx" - if env := os.Getenv("PROTOMCP_NODE"); env != "" { - return env, []string{file} - } - return cmd, []string{"tsx", file} - case ".js": - cmd := "node" - if env := os.Getenv("PROTOMCP_NODE"); env != "" { - cmd = env - } - return cmd, []string{file} - case ".go": - return "go", []string{"run", file} - case ".rs": - return "cargo", []string{"run", file} - default: - return file, nil // Treat as executable binary - } -} -``` - -- [ ] **Step 4: Run tests to verify they pass** - -```bash -go test ./internal/config/... -v -``` - -- [ ] **Step 5: Commit** - -```bash -git add internal/config/ -git commit -m "feat: implement CLI config parsing with flag defaults and runtime detection" -``` - ---- - -### Task 6: Process Manager — Spawn, Handshake, Crash Recovery - -**Files:** -- Create: `internal/process/manager.go` -- Create: `internal/process/manager_test.go` - -- [ ] **Step 1: Write failing tests for process manager** - -Create `internal/process/manager_test.go`. Tests use a mock tool process (a small Go program compiled as a test helper) that speaks the protobuf protocol over the unix socket: - -```go -package process_test - -import ( - "context" - "os" - "path/filepath" - "testing" - "time" - - "github.com/msilverblatt/protomcp/internal/process" - pb "github.com/msilverblatt/protomcp/gen/proto/protomcp" -) - -// TestStartAndHandshake verifies the process manager can spawn a tool process, -// perform the handshake, and retrieve the tool list. -func TestStartAndHandshake(t *testing.T) { - socketPath := filepath.Join(t.TempDir(), "test.sock") - - pm := process.NewManager(process.ManagerConfig{ - File: "testdata/echo_tool.py", - Runtime: "python3", - SocketPath: socketPath, - MaxRetries: 1, - }) - - ctx, cancel := context.WithTimeout(context.Background(), 10*time.Second) - defer cancel() - - tools, err := pm.Start(ctx) - if err != nil { - t.Fatalf("Start failed: %v", err) - } - defer pm.Stop() - - if len(tools) == 0 { - t.Fatal("expected at least one tool from handshake") - } -} - -// TestCallTool verifies tool call dispatch and response. -func TestCallTool(t *testing.T) { - socketPath := filepath.Join(t.TempDir(), "test.sock") - - pm := process.NewManager(process.ManagerConfig{ - File: "testdata/echo_tool.py", - Runtime: "python3", - SocketPath: socketPath, - MaxRetries: 1, - }) - - ctx, cancel := context.WithTimeout(context.Background(), 10*time.Second) - defer cancel() - - _, err := pm.Start(ctx) - if err != nil { - t.Fatalf("Start failed: %v", err) - } - defer pm.Stop() - - resp, err := pm.CallTool(ctx, "echo", `{"message": "hello"}`) - if err != nil { - t.Fatalf("CallTool failed: %v", err) - } - if resp.IsError { - t.Errorf("unexpected error: %s", resp.ResultJson) - } -} - -// TestReload verifies the reload cycle. -func TestReload(t *testing.T) { - socketPath := filepath.Join(t.TempDir(), "test.sock") - - pm := process.NewManager(process.ManagerConfig{ - File: "testdata/echo_tool.py", - Runtime: "python3", - SocketPath: socketPath, - MaxRetries: 1, - }) - - ctx, cancel := context.WithTimeout(context.Background(), 10*time.Second) - defer cancel() - - _, err := pm.Start(ctx) - if err != nil { - t.Fatalf("Start failed: %v", err) - } - defer pm.Stop() - - tools, err := pm.Reload(ctx) - if err != nil { - t.Fatalf("Reload failed: %v", err) - } - if len(tools) == 0 { - t.Fatal("expected tools after reload") - } -} -``` - -Note: `testdata/echo_tool.py` is a minimal Python tool process that speaks the protobuf protocol. It will be created as part of the Python SDK task but a simple test fixture version is needed here. Create `internal/process/testdata/echo_tool.py` — a minimal script that: -1. Reads `PROTOMCP_SOCKET` env var -2. Connects to the unix socket -3. Responds to `ListToolsRequest` with one tool ("echo") -4. Responds to `CallToolRequest` by echoing the args back -5. Responds to `ReloadRequest` with success - -- [ ] **Step 2: Implement process manager** - -Create `internal/process/manager.go`: - -The process manager must: -1. Create the unix socket and listen -2. Spawn the tool process with `PROTOMCP_SOCKET` env var -3. Accept the connection from the tool process -4. Send `ListToolsRequest`, receive `ToolListResponse` (handshake) -5. Handle `CallTool` by sending `CallToolRequest` and waiting for `CallToolResponse` (with request_id correlation) -6. Handle `Reload` by sending `ReloadRequest`, waiting for `ReloadResponse`, then re-fetching tools -7. Monitor the child process — on unexpected exit, attempt restart with exponential backoff (up to MaxRetries) -8. Handle concurrent calls using request_id for correlation -9. Provide a `Stop()` method for clean shutdown - -Key interfaces: -```go -type ManagerConfig struct { - File string - Runtime string // e.g., "python3" — if empty, detect from extension - SocketPath string - MaxRetries int // default 3 - CallTimeout time.Duration // default 5m -} - -type Manager struct { /* ... */ } - -func NewManager(cfg ManagerConfig) *Manager -func (m *Manager) Start(ctx context.Context) ([]*pb.ToolDefinition, error) -func (m *Manager) Stop() error -func (m *Manager) CallTool(ctx context.Context, name string, argsJSON string) (*pb.CallToolResponse, error) -func (m *Manager) Reload(ctx context.Context) ([]*pb.ToolDefinition, error) -func (m *Manager) HandleToolListControl(ctx context.Context, env *pb.Envelope) (*pb.ActiveToolsResponse, error) -func (m *Manager) OnCrash() <-chan error // channel that receives crash events -``` - -- [ ] **Step 3: Create test fixture echo_tool.py** - -Create `internal/process/testdata/echo_tool.py` — a minimal Python script that speaks the protobuf envelope protocol. This is a standalone test fixture (not the SDK). It: -- Reads `PROTOMCP_SOCKET` from env -- Connects to the unix socket -- Reads length-prefixed envelopes -- Responds to `ListToolsRequest` with one tool: `echo(message: str) -> str` -- Responds to `CallToolRequest` by echoing args -- Responds to `ReloadRequest` with `success=true` - -This requires the generated Python protobuf code. For the test fixture, use raw protobuf encoding with the `protobuf` pip package and the generated `protomcp_pb2.py`. - -- [ ] **Step 4: Run tests to verify they pass** - -```bash -go test ./internal/process/... -v -timeout 30s -``` - -- [ ] **Step 5: Commit** - -```bash -git add internal/process/ -git commit -m "feat: implement process manager with spawn, handshake, call dispatch, and crash recovery" -``` - ---- - -### Task 7: MCP Protocol Types and Handler - -**Files:** -- Create: `internal/mcp/types.go` -- Create: `internal/mcp/handler.go` -- Create: `internal/mcp/handler_test.go` - -- [ ] **Step 1: Write MCP JSON-RPC types** - -Create `internal/mcp/types.go` with types for: -- `JSONRPCRequest` / `JSONRPCResponse` / `JSONRPCNotification` (generic JSON-RPC 2.0) -- `InitializeRequest` / `InitializeResponse` (MCP initialization with capabilities) -- `ToolsListRequest` / `ToolsListResponse` (tools/list) -- `ToolsCallRequest` / `ToolsCallResponse` (tools/call) -- `ToolsListChangedNotification` (notifications/tools/list_changed) -- MCP capability structs (declaring `tools.listChanged: true`) - -These are standard MCP protocol types serialized as JSON, not protobuf. - -- [ ] **Step 2: Write failing tests for MCP handler** - -Create `internal/mcp/handler_test.go`: - -```go -package mcp_test - -import ( - "context" - "encoding/json" - "testing" - - "github.com/msilverblatt/protomcp/internal/mcp" - pb "github.com/msilverblatt/protomcp/gen/proto/protomcp" -) - -// mockToolBackend implements the interface the handler uses to talk to the tool process -type mockToolBackend struct { - tools []*pb.ToolDefinition -} - -func (m *mockToolBackend) ActiveTools() []*pb.ToolDefinition { - return m.tools -} - -func (m *mockToolBackend) CallTool(ctx context.Context, name, argsJSON string) (*pb.CallToolResponse, error) { - return &pb.CallToolResponse{ - ResultJson: `[{"type":"text","text":"result"}]`, - }, nil -} - -func TestHandleInitialize(t *testing.T) { - h := mcp.NewHandler(&mockToolBackend{}) - - req := mcp.JSONRPCRequest{ - JSONRPC: "2.0", - ID: json.RawMessage(`1`), - Method: "initialize", - } - - resp, err := h.Handle(context.Background(), req) - if err != nil { - t.Fatalf("Handle initialize failed: %v", err) - } - - // Should return capabilities with tools.listChanged = true - var result mcp.InitializeResult - if err := json.Unmarshal(resp.Result, &result); err != nil { - t.Fatalf("unmarshal result: %v", err) - } - if !result.Capabilities.Tools.ListChanged { - t.Error("expected tools.listChanged = true") - } -} - -func TestHandleToolsList(t *testing.T) { - backend := &mockToolBackend{ - tools: []*pb.ToolDefinition{ - {Name: "search", Description: "Search docs", InputSchemaJson: `{"type":"object","properties":{"query":{"type":"string"}}}`}, - }, - } - h := mcp.NewHandler(backend) - - req := mcp.JSONRPCRequest{ - JSONRPC: "2.0", - ID: json.RawMessage(`2`), - Method: "tools/list", - } - - resp, err := h.Handle(context.Background(), req) - if err != nil { - t.Fatalf("Handle tools/list failed: %v", err) - } - - var result mcp.ToolsListResult - if err := json.Unmarshal(resp.Result, &result); err != nil { - t.Fatalf("unmarshal result: %v", err) - } - if len(result.Tools) != 1 { - t.Fatalf("expected 1 tool, got %d", len(result.Tools)) - } - if result.Tools[0].Name != "search" { - t.Errorf("tool name = %q, want %q", result.Tools[0].Name, "search") - } -} - -func TestHandleToolsCall(t *testing.T) { - backend := &mockToolBackend{ - tools: []*pb.ToolDefinition{ - {Name: "search", Description: "Search docs", InputSchemaJson: `{}`}, - }, - } - h := mcp.NewHandler(backend) - - params, _ := json.Marshal(map[string]interface{}{ - "name": "search", - "arguments": map[string]string{"query": "hello"}, - }) - req := mcp.JSONRPCRequest{ - JSONRPC: "2.0", - ID: json.RawMessage(`3`), - Method: "tools/call", - Params: params, - } - - resp, err := h.Handle(context.Background(), req) - if err != nil { - t.Fatalf("Handle tools/call failed: %v", err) - } - - if resp.Error != nil { - t.Errorf("unexpected error: %v", resp.Error) - } -} -``` - -- [ ] **Step 3: Implement MCP handler** - -Create `internal/mcp/handler.go`: - -The handler: -- Implements `Handle(ctx, JSONRPCRequest) -> JSONRPCResponse` -- Routes by method: `initialize`, `tools/list`, `tools/call`, `notifications/initialized` -- For `initialize`: returns server info and capabilities (`tools.listChanged: true`) -- For `tools/list`: calls `backend.ActiveTools()` and formats as MCP response -- For `tools/call`: extracts tool name and args, calls `backend.CallTool()`, intercepts `enable_tools`/`disable_tools` from the response, and returns the MCP-formatted result -- Provides `ListChangedNotification() -> JSONRPCNotification` for the transport to send -- Unknown methods return JSON-RPC method-not-found error - -- [ ] **Step 4: Run tests to verify they pass** - -```bash -go test ./internal/mcp/... -v -``` - -- [ ] **Step 5: Commit** - -```bash -git add internal/mcp/ -git commit -m "feat: implement MCP JSON-RPC handler with initialize, tools/list, and tools/call" -``` - ---- - -### Task 8: Middleware — Logging and Error Handling - -**Files:** -- Create: `internal/middleware/chain.go` -- Create: `internal/middleware/logging.go` -- Create: `internal/middleware/errors.go` - -- [ ] **Step 1: Implement middleware chain** - -Create `internal/middleware/chain.go`: - -```go -package middleware - -import ( - "context" - - "github.com/msilverblatt/protomcp/internal/mcp" -) - -// Handler processes an MCP request and returns a response. -type Handler func(ctx context.Context, req mcp.JSONRPCRequest) (*mcp.JSONRPCResponse, error) - -// Middleware wraps a Handler with additional behavior. -type Middleware func(next Handler) Handler - -// Chain applies middleware in order (first middleware is outermost). -func Chain(handler Handler, middlewares ...Middleware) Handler { - for i := len(middlewares) - 1; i >= 0; i-- { - handler = middlewares[i](handler) - } - return handler -} -``` - -- [ ] **Step 2: Implement logging middleware** - -Create `internal/middleware/logging.go`: - -Logs each request (method, request ID) and response (duration, error status). Uses `log/slog` for structured logging. Log level controlled by config. - -- [ ] **Step 3: Implement error formatting middleware** - -Create `internal/middleware/errors.go`: - -Catches panics and unhandled errors from the handler, formats them as agent-friendly structured errors with `error_code`, `message`, `suggestion`, and `retryable` fields. - -- [ ] **Step 4: Commit** - -```bash -git add internal/middleware/ -git commit -m "feat: implement middleware chain with structured logging and error formatting" -``` - ---- - -## Chunk 3: Transports, File Watcher, CLI Entry Point - -### Task 9: Transport Interface and stdio Transport - -**Files:** -- Create: `internal/transport/transport.go` -- Create: `internal/transport/stdio.go` -- Create: `internal/transport/stdio_test.go` - -- [ ] **Step 1: Define transport interface** - -Create `internal/transport/transport.go`: - -```go -package transport - -import ( - "context" - - "github.com/msilverblatt/protomcp/internal/mcp" -) - -// Transport handles bidirectional MCP communication with the host/client. -type Transport interface { - // Start begins listening for MCP requests. Calls handler for each request. - // Blocks until ctx is cancelled or an error occurs. - Start(ctx context.Context, handler mcp.RequestHandler) error - - // SendNotification pushes a server-initiated notification to the client. - SendNotification(notification mcp.JSONRPCNotification) error - - // Close shuts down the transport. - Close() error -} - -// mcp.RequestHandler is: func(ctx context.Context, req JSONRPCRequest) (*JSONRPCResponse, error) -``` - -- [ ] **Step 2: Write failing tests for stdio transport** - -Create `internal/transport/stdio_test.go`: - -Test that the stdio transport reads JSON-RPC requests from stdin, passes them to the handler, and writes responses to stdout. Use `io.Pipe` to simulate stdin/stdout. - -- [ ] **Step 3: Implement stdio transport** - -Create `internal/transport/stdio.go`: - -Reads newline-delimited JSON-RPC messages from stdin, dispatches to handler, writes JSON-RPC responses to stdout. Handles `notifications/initialized` (no response needed). - -- [ ] **Step 4: Run tests** - -```bash -go test ./internal/transport/... -run TestStdio -v -``` - -- [ ] **Step 5: Commit** - -```bash -git add internal/transport/transport.go internal/transport/stdio.go internal/transport/stdio_test.go -git commit -m "feat: implement transport interface and stdio transport" -``` - ---- - -### Task 10: Streamable HTTP Transport - -**Files:** -- Create: `internal/transport/http.go` -- Create: `internal/transport/http_test.go` - -- [ ] **Step 1: Write failing tests** - -Test that: -- POST to `/mcp` with a JSON-RPC request returns a JSON-RPC response -- GET to `/mcp` opens an SSE stream for server-initiated notifications -- `SendNotification` pushes events to connected SSE clients - -- [ ] **Step 2: Implement HTTP transport** - -Uses `net/http`. Single endpoint `/mcp`. POST for requests, GET for SSE stream. Maintains a list of SSE clients for notifications. - -- [ ] **Step 3: Run tests and commit** - -```bash -go test ./internal/transport/... -run TestHTTP -v -git add internal/transport/http.go internal/transport/http_test.go -git commit -m "feat: implement streamable HTTP transport with SSE notifications" -``` - ---- - -### Task 11: SSE Transport - -**Files:** -- Create: `internal/transport/sse.go` -- Create: `internal/transport/sse_test.go` - -- [ ] **Step 1: Implement SSE transport** - -Legacy SSE transport per MCP spec: separate SSE endpoint for server-to-client, POST endpoint for client-to-server. Largely reuses HTTP transport internals. - -- [ ] **Step 2: Test and commit** - -```bash -go test ./internal/transport/... -run TestSSE -v -git add internal/transport/sse.go internal/transport/sse_test.go -git commit -m "feat: implement legacy SSE transport" -``` - ---- - -### Task 12: WebSocket Transport - -**Files:** -- Create: `internal/transport/ws.go` -- Create: `internal/transport/ws_test.go` - -- [ ] **Step 1: Implement WebSocket transport** - -Uses `nhooyr.io/websocket` (or `gorilla/websocket`). Single WebSocket connection for bidirectional JSON-RPC. Notifications are pushed as WebSocket messages. - -- [ ] **Step 2: Test and commit** - -```bash -go test ./internal/transport/... -run TestWS -v -git add internal/transport/ws.go internal/transport/ws_test.go -git commit -m "feat: implement WebSocket transport" -``` - ---- - -### Task 13: gRPC Transport - -**Files:** -- Create: `internal/transport/grpc.go` -- Create: `internal/transport/grpc_test.go` -- Modify: `proto/protomcp.proto` (add external gRPC service definition) - -- [ ] **Step 1: Add external gRPC service to proto** - -Add a separate service definition in `proto/protomcp.proto` for the external gRPC transport (client-facing, separate from the internal envelope protocol): - -```protobuf -// External gRPC service — client-facing MCP transport -service MCPService { - rpc Request(MCPRequest) returns (MCPResponse); - rpc Subscribe(MCPSubscribeRequest) returns (stream MCPNotification); -} - -message MCPRequest { - string jsonrpc_request = 1; // JSON-encoded MCP JSON-RPC request -} -message MCPResponse { - string jsonrpc_response = 1; // JSON-encoded MCP JSON-RPC response -} -message MCPSubscribeRequest {} -message MCPNotification { - string jsonrpc_notification = 1; // JSON-encoded notification -} -``` - -- [ ] **Step 2: Regenerate proto and implement gRPC transport** - -```bash -make proto -``` - -Implement gRPC server that wraps JSON-RPC messages in protobuf, dispatches to handler, and streams notifications. - -- [ ] **Step 3: Test and commit** - -```bash -go test ./internal/transport/... -run TestGRPC -v -git add internal/transport/grpc.go internal/transport/grpc_test.go proto/protomcp.proto -git commit -m "feat: implement gRPC transport with external service definition" -``` - ---- - -### Task 14: File Watcher — Reload Orchestration - -**Files:** -- Create: `internal/reload/watcher.go` -- Create: `internal/reload/watcher_test.go` - -- [ ] **Step 1: Write failing tests** - -Test that: -- Watcher detects file modifications and calls the reload callback -- Watcher debounces rapid changes (only one reload per debounce window) -- Watcher supports both single file and directory watching -- Watcher filters by relevant file extensions - -- [ ] **Step 2: Implement file watcher** - -Create `internal/reload/watcher.go`: - -Uses `github.com/fsnotify/fsnotify`. Key behavior: -- Watches the specified file or directory -- On change, debounces (100ms) then calls the reload callback -- Reload callback signature: `func() error` -- The watcher does NOT do the reload itself — it signals the main server loop -- In `run` mode (production), the watcher is never started - -```go -type Watcher struct { /* ... */ } - -func NewWatcher(path string, extensions []string, onChange func()) (*Watcher, error) -func (w *Watcher) Start(ctx context.Context) error -func (w *Watcher) Stop() error -``` - -- [ ] **Step 3: Test and commit** - -```bash -go test ./internal/reload/... -v -git add internal/reload/ -git commit -m "feat: implement file watcher with debouncing and extension filtering" -``` - ---- - -### Task 15: CLI Entry Point — Wiring It All Together - -**Files:** -- Create: `cmd/protomcp/main.go` - -- [ ] **Step 1: Implement main.go** - -Create `cmd/protomcp/main.go` that wires everything together: - -```go -package main - -import ( - "context" - "fmt" - "log/slog" - "os" - "os/signal" - - "github.com/msilverblatt/protomcp/internal/config" - "github.com/msilverblatt/protomcp/internal/mcp" - "github.com/msilverblatt/protomcp/internal/middleware" - "github.com/msilverblatt/protomcp/internal/process" - "github.com/msilverblatt/protomcp/internal/reload" - "github.com/msilverblatt/protomcp/internal/toollist" - "github.com/msilverblatt/protomcp/internal/transport" -) - -func main() { - cfg, err := config.Parse(os.Args[1:]) - if err != nil { - fmt.Fprintf(os.Stderr, "error: %v\n", err) - fmt.Fprintf(os.Stderr, "usage: protomcp [flags]\n") - os.Exit(1) - } - - // Setup structured logging - logLevel := parseLogLevel(cfg.LogLevel) - logger := slog.New(slog.NewTextHandler(os.Stderr, &slog.HandlerOptions{Level: logLevel})) - slog.SetDefault(logger) - - ctx, cancel := signal.NotifyContext(context.Background(), os.Interrupt) - defer cancel() - - // 1. Create tool list manager - tlm := toollist.New() - - // 2. Determine runtime command - var runtimeCmd string - var runtimeArgs []string - if cfg.Runtime != "" { - runtimeCmd = cfg.Runtime - runtimeArgs = []string{cfg.File} - } else { - runtimeCmd, runtimeArgs = config.RuntimeCommand(cfg.File) - } - - // 3. Start process manager - pm := process.NewManager(process.ManagerConfig{ - File: cfg.File, - RuntimeCmd: runtimeCmd, - RuntimeArgs: runtimeArgs, - SocketPath: cfg.SocketPath, - MaxRetries: 3, - CallTimeout: cfg.CallTimeout, - }) - - tools, err := pm.Start(ctx) - if err != nil { - slog.Error("failed to start tool process", "error", err) - os.Exit(1) - } - - // Register tools and log them - toolNames := make([]string, len(tools)) - for i, t := range tools { - toolNames[i] = t.Name - slog.Info("tool registered", "name", t.Name, "description", t.Description) - } - tlm.SetRegistered(toolNames) - - // 4. Create MCP handler with tool backend - backend := newToolBackend(pm, tlm, tools) - handler := mcp.NewHandler(backend) - - // 5. Apply middleware - chain := middleware.Chain( - handler.Handle, - middleware.Logging(logger), - middleware.ErrorFormatting(), - ) - - // 6. Create transport - tp, err := createTransport(cfg.Transport) - if err != nil { - slog.Error("failed to create transport", "error", err) - os.Exit(1) - } - - // 7. Handle tool list control messages from tool process - go handleToolListControl(ctx, pm, tlm, tp) - - // 8. Handle crash recovery - go handleCrashRecovery(ctx, pm, tlm, tools, tp) - - // 9. Start file watcher (dev mode only) - if cfg.Command == "dev" { - w, err := reload.NewWatcher(cfg.File, nil, func() { - // Wait for in-flight calls unless immediate mode - if !cfg.HotReloadImmediate { - pm.WaitForInFlight() - } - newTools, err := pm.Reload(ctx) - if err != nil { - slog.Error("reload failed", "error", err) - return - } - // Update tool list - newNames := make([]string, len(newTools)) - for i, t := range newTools { - newNames[i] = t.Name - } - oldActive := tlm.Active() - tlm.SetRegistered(newNames) - backend.UpdateTools(newTools) - newActive := tlm.Active() - if !slicesEqual(oldActive, newActive) { - slog.Info("tool list changed", "added", diff(newActive, oldActive), "removed", diff(oldActive, newActive)) - tp.SendNotification(mcp.ListChangedNotification()) - } - }) - if err != nil { - slog.Error("failed to create file watcher", "error", err) - os.Exit(1) - } - go w.Start(ctx) - defer w.Stop() - } - - // 10. Start transport (blocks until ctx cancelled) - slog.Info("protomcp started", "command", cfg.Command, "file", cfg.File, "transport", cfg.Transport) - if err := tp.Start(ctx, chain); err != nil { - slog.Error("transport error", "error", err) - os.Exit(1) - } -} -``` - -Note: `createTransport()`, `newToolBackend()`, `handleToolListControl()`, `handleCrashRecovery()`, `parseLogLevel()`, `slicesEqual()`, `diff()` are helper functions in the same file or a `helpers.go` file. Keep it focused — these are thin wiring functions, not business logic. - -- [ ] **Step 2: Build and verify it compiles** - -```bash -make build -``` - -Expected: `bin/protomcp` binary created. - -- [ ] **Step 3: Smoke test with a Python fixture** - -```bash -echo '{"jsonrpc":"2.0","id":1,"method":"initialize","params":{}}' | bin/protomcp dev internal/process/testdata/echo_tool.py -``` - -Expected: JSON-RPC response with capabilities. - -- [ ] **Step 4: Commit** - -```bash -git add cmd/protomcp/ -git commit -m "feat: implement CLI entry point wiring all components together" -``` - ---- - -## Chunk 4: Python SDK - -### Task 16: Python Project Setup - -**Files:** -- Create: `sdk/python/pyproject.toml` -- Create: `sdk/python/src/protomcp/__init__.py` - -- [ ] **Step 1: Create pyproject.toml** - -```toml -[build-system] -requires = ["hatchling"] -build-backend = "hatchling.build" - -[project] -name = "protomcp" -version = "0.1.0" -description = "Write MCP tools in Python. No MCP knowledge required." -requires-python = ">=3.10" -dependencies = [ - "protobuf>=4.0", -] - -[project.optional-dependencies] -dev = [ - "pytest>=7.0", - "pytest-asyncio>=0.21", -] - -[tool.hatch.build.targets.wheel] -packages = ["src/protomcp"] -``` - -- [ ] **Step 2: Create __init__.py with public API** - -```python -from protomcp.tool import tool -from protomcp.result import ToolResult -from protomcp import manager as tool_manager - -__all__ = ["tool", "ToolResult", "tool_manager"] -``` - -- [ ] **Step 3: Generate Python protobuf code** - -```bash -make proto -``` - -Verify: `sdk/python/gen/protomcp_pb2.py` exists. - -- [ ] **Step 4: Commit** - -```bash -git add sdk/python/pyproject.toml sdk/python/src/protomcp/__init__.py -git commit -m "feat: initialize Python SDK project" -``` - ---- - -### Task 17: Python — @tool Decorator and Schema Generation - -**Files:** -- Create: `sdk/python/src/protomcp/tool.py` -- Create: `sdk/python/tests/test_tool.py` - -- [ ] **Step 1: Write failing tests** - -Create `sdk/python/tests/test_tool.py`: - -```python -import json -from protomcp.tool import tool, get_registered_tools - - -def test_tool_decorator_registers(): - """@tool() should register the function as a tool.""" - clear_registry() - @tool(description="Add two numbers") - def add(a: int, b: int) -> int: - return a + b - - tools = get_registered_tools() - assert any(t.name == "add" for t in tools) - - -def test_tool_schema_from_type_hints(): - """Schema should be generated from Python type annotations.""" - clear_registry() - @tool(description="Search documents") - def search(query: str, limit: int = 10) -> list: - return [] - - tools = get_registered_tools() - t = next(t for t in tools if t.name == "search") - schema = json.loads(t.input_schema_json) - - assert schema["type"] == "object" - assert "query" in schema["properties"] - assert schema["properties"]["query"]["type"] == "string" - assert "limit" in schema["properties"] - assert schema["properties"]["limit"]["type"] == "integer" - assert schema["properties"]["limit"]["default"] == 10 - assert "query" in schema["required"] - assert "limit" not in schema["required"] # has default - - -def test_tool_callable(): - """Decorated function should still be callable normally.""" - clear_registry() - @tool(description="Double a number") - def double(n: int) -> int: - return n * 2 - - assert double(5) == 10 - - -def test_tool_optional_params(): - """Optional params should not be in required.""" - clear_registry() - from typing import Optional - - @tool(description="Greet") - def greet(name: str, greeting: Optional[str] = None) -> str: - return f"{greeting or 'Hello'}, {name}!" - - tools = get_registered_tools() - t = next(t for t in tools if t.name == "greet") - schema = json.loads(t.input_schema_json) - - assert "name" in schema["required"] - assert "greeting" not in schema["required"] - -def test_tool_optional_without_default(): - """Optional[X] without a default should NOT be required.""" - clear_registry() - from typing import Optional - - @tool(description="Maybe filter") - def search(query: str, tag: Optional[str] = None) -> list: - return [] - - tools = get_registered_tools() - t = next(t for t in tools if t.name == "search") - schema = json.loads(t.input_schema_json) - assert "tag" not in schema.get("required", []) -``` - -- [ ] **Step 2: Run tests to verify they fail** - -```bash -cd sdk/python && python -m pytest tests/test_tool.py -v -``` - -- [ ] **Step 3: Implement @tool decorator** - -Create `sdk/python/src/protomcp/tool.py`: - -```python -import inspect -import json -from dataclasses import dataclass -from typing import Any, Callable, Optional, get_type_hints - -# Global registry of tools -_registry: list["ToolDef"] = [] - - -@dataclass -class ToolDef: - name: str - description: str - input_schema_json: str - handler: Callable - - -def tool(description: str): - """Decorator that registers a function as an MCP tool. - - Schema is generated automatically from Python type annotations. - """ - def decorator(func: Callable) -> Callable: - schema = _generate_schema(func) - _registry.append(ToolDef( - name=func.__name__, - description=description, - input_schema_json=json.dumps(schema), - handler=func, - )) - return func - return decorator - - -def get_registered_tools() -> list[ToolDef]: - return list(_registry) - - -def clear_registry(): - _registry.clear() - - -_PYTHON_TYPE_TO_JSON_SCHEMA = { - str: "string", - int: "integer", - float: "number", - bool: "boolean", - list: "array", - dict: "object", -} - - -def _generate_schema(func: Callable) -> dict: - """Generate JSON Schema from function signature and type hints.""" - hints = get_type_hints(func) - sig = inspect.signature(func) - - properties = {} - required = [] - - for name, param in sig.parameters.items(): - if name in ("self", "cls", "ctx"): - continue - - hint = hints.get(name, Any) - # Skip ToolContext parameters - if hint.__name__ == "ToolContext" if hasattr(hint, "__name__") else False: - continue - json_type = _python_type_to_json(hint) - prop: dict[str, Any] = {"type": json_type} - - if param.default is not inspect.Parameter.empty: - prop["default"] = param.default - elif not _is_optional_type(hint): - required.append(name) - - properties[name] = prop - - schema: dict[str, Any] = { - "type": "object", - "properties": properties, - } - if required: - schema["required"] = required - - return schema - - -def _is_optional_type(hint) -> bool: - """Check if a type hint is Optional[X] (Union[X, None]).""" - import typing - origin = getattr(hint, "__origin__", None) - if origin is typing.Union: - return type(None) in hint.__args__ - return False - - -def _python_type_to_json(hint) -> str: - """Convert a Python type hint to a JSON Schema type string.""" - # Handle Optional[X] (Union[X, None]) - origin = getattr(hint, "__origin__", None) - if origin is type(None): - return "null" - - # Handle Optional (typing.Union with None) - import typing - if origin is typing.Union: - args = hint.__args__ - non_none = [a for a in args if a is not type(None)] - if len(non_none) == 1: - return _python_type_to_json(non_none[0]) - - return _PYTHON_TYPE_TO_JSON_SCHEMA.get(hint, "string") -``` - -- [ ] **Step 4: Run tests to verify they pass** - -```bash -cd sdk/python && python -m pytest tests/test_tool.py -v -``` - -- [ ] **Step 5: Commit** - -```bash -git add sdk/python/src/protomcp/tool.py sdk/python/tests/test_tool.py -git commit -m "feat: implement Python @tool decorator with automatic schema generation from type hints" -``` - ---- - -### Task 18: Python — ToolResult Type - -**Files:** -- Create: `sdk/python/src/protomcp/result.py` -- Create: `sdk/python/tests/test_result.py` - -- [ ] **Step 1: Write failing tests** - -```python -from protomcp.result import ToolResult - - -def test_tool_result_basic(): - r = ToolResult(result="success") - assert r.result == "success" - assert r.enable_tools is None - assert r.disable_tools is None - - -def test_tool_result_with_mutations(): - r = ToolResult( - result="connected", - enable_tools=["query_db"], - disable_tools=["connect_db"], - ) - assert r.enable_tools == ["query_db"] - assert r.disable_tools == ["connect_db"] - - -def test_tool_result_with_error(): - r = ToolResult( - is_error=True, - error_code="NOT_FOUND", - message="User not found", - suggestion="Try searching by email", - retryable=False, - ) - assert r.is_error is True - assert r.error_code == "NOT_FOUND" -``` - -- [ ] **Step 2: Implement ToolResult** - -```python -from dataclasses import dataclass, field -from typing import Optional - - -@dataclass -class ToolResult: - result: str = "" - is_error: bool = False - enable_tools: Optional[list[str]] = None - disable_tools: Optional[list[str]] = None - # Structured error fields - error_code: Optional[str] = None - message: Optional[str] = None - suggestion: Optional[str] = None - retryable: bool = False -``` - -- [ ] **Step 3: Test and commit** - -```bash -cd sdk/python && python -m pytest tests/test_result.py -v -git add sdk/python/src/protomcp/result.py sdk/python/tests/test_result.py -git commit -m "feat: implement Python ToolResult type with tool list mutations and structured errors" -``` - ---- - -### Task 19: Python — Transport Layer (Unix Socket + Envelope) - -**Files:** -- Create: `sdk/python/src/protomcp/transport.py` -- Create: `sdk/python/tests/test_transport.py` - -- [ ] **Step 1: Write failing tests** - -Test that `Transport` can: -- Connect to a unix socket -- Write a length-prefixed protobuf envelope -- Read a length-prefixed protobuf envelope -- Handle multiple sequential messages - -- [ ] **Step 2: Implement transport** - -```python -import socket -import struct -from protomcp.gen import protomcp_pb2 as pb - - -class Transport: - """Speaks length-prefixed protobuf envelopes over a unix socket.""" - - def __init__(self, socket_path: str): - self._socket_path = socket_path - self._sock: socket.socket | None = None - - def connect(self): - self._sock = socket.socket(socket.AF_UNIX, socket.SOCK_STREAM) - self._sock.connect(self._socket_path) - - def send(self, envelope: pb.Envelope): - data = envelope.SerializeToString() - length = struct.pack(">I", len(data)) - self._sock.sendall(length + data) - - def recv(self) -> pb.Envelope: - length_bytes = self._recv_exactly(4) - length = struct.unpack(">I", length_bytes)[0] - data = self._recv_exactly(length) - env = pb.Envelope() - env.ParseFromString(data) - return env - - def close(self): - if self._sock: - self._sock.close() - - def _recv_exactly(self, n: int) -> bytes: - buf = bytearray() - while len(buf) < n: - chunk = self._sock.recv(n - len(buf)) - if not chunk: - raise ConnectionError("socket closed") - buf.extend(chunk) - return bytes(buf) -``` - -- [ ] **Step 3: Test and commit** - -```bash -cd sdk/python && python -m pytest tests/test_transport.py -v -git add sdk/python/src/protomcp/transport.py sdk/python/tests/test_transport.py -git commit -m "feat: implement Python transport layer for unix socket + envelope protocol" -``` - ---- - -### Task 20: Python — tool_manager Client - -**Files:** -- Create: `sdk/python/src/protomcp/manager.py` -- Create: `sdk/python/tests/test_manager.py` - -- [ ] **Step 1: Write failing tests** - -Test that `tool_manager` methods send the correct protobuf messages over the transport and return the response. - -- [ ] **Step 2: Implement manager** - -```python -from protomcp.transport import Transport -from protomcp.gen import protomcp_pb2 as pb - -_transport: Transport | None = None - - -def _get_transport() -> Transport: - if _transport is None: - raise RuntimeError("protomcp not connected — are you running via 'protomcp dev'?") - return _transport - - -def _init(transport: Transport): - global _transport - _transport = transport - - -def enable(tool_names: list[str]) -> list[str]: - t = _get_transport() - env = pb.Envelope( - enable_tools=pb.EnableToolsRequest(tool_names=tool_names) - ) - t.send(env) - resp = t.recv() - return list(resp.active_tools.tool_names) - - -def disable(tool_names: list[str]) -> list[str]: - t = _get_transport() - env = pb.Envelope( - disable_tools=pb.DisableToolsRequest(tool_names=tool_names) - ) - t.send(env) - resp = t.recv() - return list(resp.active_tools.tool_names) - - -def set_allowed(tool_names: list[str]) -> list[str]: - t = _get_transport() - env = pb.Envelope( - set_allowed=pb.SetAllowedRequest(tool_names=tool_names) - ) - t.send(env) - resp = t.recv() - return list(resp.active_tools.tool_names) - - -def set_blocked(tool_names: list[str]) -> list[str]: - t = _get_transport() - env = pb.Envelope( - set_blocked=pb.SetBlockedRequest(tool_names=tool_names) - ) - t.send(env) - resp = t.recv() - return list(resp.active_tools.tool_names) - - -def get_active_tools() -> list[str]: - t = _get_transport() - env = pb.Envelope( - get_active_tools=pb.GetActiveToolsRequest() - ) - t.send(env) - resp = t.recv() - return list(resp.active_tools.tool_names) - - -def batch( - enable: list[str] | None = None, - disable: list[str] | None = None, - allow: list[str] | None = None, - block: list[str] | None = None, -) -> list[str]: - t = _get_transport() - env = pb.Envelope( - batch=pb.BatchUpdateRequest( - enable=enable or [], - disable=disable or [], - allow=allow or [], - block=block or [], - ) - ) - t.send(env) - resp = t.recv() - return list(resp.active_tools.tool_names) -``` - -- [ ] **Step 3: Test and commit** - -```bash -cd sdk/python && python -m pytest tests/test_manager.py -v -git add sdk/python/src/protomcp/manager.py sdk/python/tests/test_manager.py -git commit -m "feat: implement Python tool_manager client for dynamic tool list control" -``` - ---- - -### Task 21: Python — Runner (Main Loop) - -**Files:** -- Create: `sdk/python/src/protomcp/runner.py` - -- [ ] **Step 1: Implement runner** - -The runner is the main loop that the tool process runs. It: -1. Reads `PROTOMCP_SOCKET` from env -2. Connects to the unix socket via `Transport` -3. Initializes `tool_manager` with the transport -4. Listens for envelopes from the Go binary -5. Dispatches `ListToolsRequest` → returns registered tools from `tool.get_registered_tools()` -6. Dispatches `CallToolRequest` → finds the handler, calls it, returns `CallToolResponse` -7. Dispatches `ReloadRequest` → calls `importlib.reload()` on the user's module, clears and re-registers tools, returns `ReloadResponse` - -The user's tool file imports this runner at module level (or the SDK auto-runs it): - -```python -# At the bottom of a user's tool file, or auto-detected: -if __name__ == "__main__": - from protomcp.runner import run - run() -``` - -Actually, the runner should auto-start. When the tool process is spawned, the Python SDK needs to: -1. Import the user's module (which registers tools via `@tool()`) -2. Start the runner loop - -This is handled by the SDK's entry point. The Go binary runs: `python -c "import protomcp.runner; protomcp.runner.start('server')"` where `server` is the module path. Or simpler: the user's file calls `protomcp.run()` at the bottom. - -- [ ] **Step 2: Commit** - -```bash -git add sdk/python/src/protomcp/runner.py -git commit -m "feat: implement Python runner main loop with reload and tool dispatch" -``` - ---- - -## Chunk 5: TypeScript SDK - -### Task 22: TypeScript Project Setup - -**Files:** -- Create: `sdk/typescript/package.json` -- Create: `sdk/typescript/tsconfig.json` -- Create: `sdk/typescript/vitest.config.ts` -- Create: `sdk/typescript/src/index.ts` - -- [ ] **Step 1: Create package.json** - -```json -{ - "name": "protomcp", - "version": "0.1.0", - "description": "Write MCP tools in TypeScript. No MCP knowledge required.", - "type": "module", - "main": "dist/index.js", - "types": "dist/index.d.ts", - "scripts": { - "build": "tsc", - "test": "vitest run", - "test:watch": "vitest" - }, - "dependencies": { - "protobufjs": "^7.0.0", - "zod": "^3.22.0", - "zod-to-json-schema": "^3.22.0" - }, - "devDependencies": { - "typescript": "^5.3.0", - "vitest": "^1.0.0", - "@types/node": "^20.0.0" - } -} -``` - -- [ ] **Step 2: Create tsconfig.json, vitest.config.ts, index.ts** - -Standard TypeScript config with ESM output. Index exports `tool`, `ToolResult`, `toolManager`. - -- [ ] **Step 3: Install dependencies and commit** - -```bash -cd sdk/typescript && npm install -git add sdk/typescript/package.json sdk/typescript/tsconfig.json sdk/typescript/vitest.config.ts sdk/typescript/src/index.ts -git commit -m "feat: initialize TypeScript SDK project" -``` - ---- - -### Task 23: TypeScript — tool() Function with Zod Schema - -**Files:** -- Create: `sdk/typescript/src/tool.ts` -- Create: `sdk/typescript/tests/tool.test.ts` - -- [ ] **Step 1: Write failing tests** - -```typescript -import { describe, it, expect } from 'vitest'; -import { z } from 'zod'; -import { tool, getRegisteredTools, clearRegistry } from '../src/tool'; - -describe('tool()', () => { - beforeEach(() => clearRegistry()); - - it('registers a tool with Zod schema', () => { - const add = tool({ - name: 'add', - description: 'Add two numbers', - args: z.object({ - a: z.number().describe('First number'), - b: z.number().describe('Second number'), - }), - handler: (args) => args.a + args.b, - }); - - const tools = getRegisteredTools(); - expect(tools).toHaveLength(1); - expect(tools[0].name).toBe('add'); - expect(tools[0].description).toBe('Add two numbers'); - }); - - it('requires name for arrow function handlers', () => { - const t = tool({ - description: 'No name', - args: z.object({ x: z.number() }), - handler: (args) => args.x, - }); - // Arrow functions have empty .name, so falls back to tool_N - expect(t.name).toMatch(/^tool_\d+$/); - }); - - it('generates JSON Schema from Zod', () => { - tool({ - name: 'search', - description: 'Search', - args: z.object({ - query: z.string().describe('Search query'), - limit: z.number().default(10).describe('Max results'), - }), - handler: (args) => [], - }); - - const tools = getRegisteredTools(); - const schema = JSON.parse(tools[0].inputSchemaJson); - expect(schema.type).toBe('object'); - expect(schema.properties.query.type).toBe('string'); - expect(schema.properties.limit.default).toBe(10); - expect(schema.required).toContain('query'); - expect(schema.required).not.toContain('limit'); - }); - - it('handler is callable', () => { - const add = tool({ - description: 'Add', - args: z.object({ a: z.number(), b: z.number() }), - handler: (args) => args.a + args.b, - }); - - expect(add.handler({ a: 2, b: 3 })).toBe(5); - }); -}); -``` - -- [ ] **Step 2: Implement tool()** - -```typescript -import { z } from 'zod'; -import { zodToJsonSchema } from 'zod-to-json-schema'; - -interface ToolDef { - name: string; - description: string; - inputSchemaJson: string; - handler: (args: z.infer) => any; -} - -const registry: ToolDef[] = []; - -interface ToolOptions> { - description: string; - args: T; - handler: (args: z.infer) => any; - name?: string; // defaults to handler function name or export name -} - -export function tool>(options: ToolOptions): ToolDef { - const schema = zodToJsonSchema(options.args, { target: 'openApi3' }); - const def: ToolDef = { - name: options.name || options.handler.name || `tool_${registry.length}`, - description: options.description, - inputSchemaJson: JSON.stringify(schema), - handler: options.handler, - }; - registry.push(def); - return def; -} - -export function getRegisteredTools(): ToolDef[] { - return [...registry]; -} - -export function clearRegistry(): void { - registry.length = 0; -} -``` - -- [ ] **Step 3: Test and commit** - -```bash -cd sdk/typescript && npx vitest run tests/tool.test.ts -git add sdk/typescript/src/tool.ts sdk/typescript/tests/tool.test.ts -git commit -m "feat: implement TypeScript tool() function with Zod schema generation" -``` - ---- - -### Task 24: TypeScript — ToolResult, Transport, Manager, Runner - -**Files:** -- Create: `sdk/typescript/src/result.ts` -- Create: `sdk/typescript/src/transport.ts` -- Create: `sdk/typescript/src/manager.ts` -- Create: `sdk/typescript/src/runner.ts` -- Create: `sdk/typescript/tests/result.test.ts` -- Create: `sdk/typescript/tests/transport.test.ts` -- Create: `sdk/typescript/tests/manager.test.ts` - -These mirror the Python implementations exactly: - -- **result.ts**: `ToolResult` interface with `result`, `isError`, `enableTools`, `disableTools`, `errorCode`, `message`, `suggestion`, `retryable` -- **transport.ts**: Unix socket client with length-prefixed protobuf envelope read/write using `net.Socket` and `protobufjs` -- **manager.ts**: `toolManager` with `enable()`, `disable()`, `setAllowed()`, `setBlocked()`, `getActiveTools()`, `batch()` -- **runner.ts**: Main loop that connects to socket, listens for envelopes, dispatches tool calls, handles reload via Node module cache invalidation - -Follow the same TDD pattern as the Python tasks: write tests first, verify they fail, implement, verify they pass, commit. - -- [ ] **Step 1: Implement and test ToolResult** -- [ ] **Step 2: Implement and test transport** -- [ ] **Step 3: Implement and test manager** -- [ ] **Step 4: Implement runner** -- [ ] **Step 5: Commit all** - -```bash -git add sdk/typescript/src/ sdk/typescript/tests/ -git commit -m "feat: implement TypeScript SDK — ToolResult, transport, manager, runner" -``` - ---- - -## Chunk 6: Integration Tests and Documentation - -### Task 25: End-to-End Integration Tests - -**Files:** -- Create: `test/e2e/e2e_test.go` -- Create: `test/e2e/helpers.go` -- Create: `test/e2e/fixtures/simple_tool.py` -- Create: `test/e2e/fixtures/dynamic_tool.py` -- Create: `test/e2e/fixtures/simple_tool.ts` -- Create: `test/e2e/fixtures/crash_tool.py` - -- [ ] **Step 1: Create test fixtures** - -`simple_tool.py`: -```python -from protomcp import tool - -@tool(description="Echo a message back") -def echo(message: str) -> str: - return message - -@tool(description="Add two numbers") -def add(a: int, b: int) -> int: - return a + b -``` - -`dynamic_tool.py`: -```python -from protomcp import tool, ToolResult, tool_manager - -# Start with admin_action hidden — it gets enabled after auth -tool_manager.set_blocked(["admin_action"]) - -@tool(description="Authenticate") -def auth(token: str) -> ToolResult: - if token == "valid": - return ToolResult( - result="Authenticated", - enable_tools=["admin_action"], - disable_tools=["auth"], - ) - return ToolResult(is_error=True, message="Invalid token", suggestion="Use 'valid' as token") - -@tool(description="Admin action (hidden until auth)") -def admin_action() -> str: - return "admin stuff done" -``` - -`crash_tool.py`: -```python -from protomcp import tool -import sys - -@tool(description="Crash the process") -def crash() -> str: - sys.exit(1) -``` - -`simple_tool.ts`: -```typescript -import { tool } from 'protomcp'; -import { z } from 'zod'; - -export const echo = tool({ - description: 'Echo a message back', - args: z.object({ message: z.string() }), - handler: (args) => args.message, -}); -``` - -- [ ] **Step 2: Write e2e test helpers** - -Create `test/e2e/helpers.go`: - -```go -package e2e - -import ( - "bufio" - "encoding/json" - "io" - "os/exec" - "testing" - - "github.com/msilverblatt/protomcp/internal/mcp" -) - -// StartProtomcp starts the protomcp binary with the given args. -// Returns stdin writer, stdout reader, and cleanup function. -func StartProtomcp(t *testing.T, args ...string) (io.Writer, *bufio.Scanner, func()) { - t.Helper() - cmd := exec.Command("../../bin/protomcp", args...) - stdin, _ := cmd.StdinPipe() - stdout, _ := cmd.StdoutPipe() - cmd.Stderr = nil // suppress stderr in tests - - if err := cmd.Start(); err != nil { - t.Fatalf("failed to start protomcp: %v", err) - } - - scanner := bufio.NewScanner(stdout) - cleanup := func() { - stdin.Close() - cmd.Process.Kill() - cmd.Wait() - } - - return stdin, scanner, cleanup -} - -// SendRequest sends a JSON-RPC request and reads the response. -func SendRequest(t *testing.T, w io.Writer, r *bufio.Scanner, method string, params interface{}) mcp.JSONRPCResponse { - t.Helper() - id := json.RawMessage(`1`) - p, _ := json.Marshal(params) - req := mcp.JSONRPCRequest{ - JSONRPC: "2.0", - ID: id, - Method: method, - Params: p, - } - data, _ := json.Marshal(req) - w.Write(append(data, '\n')) - - if !r.Scan() { - t.Fatal("no response from protomcp") - } - - var resp mcp.JSONRPCResponse - json.Unmarshal(r.Bytes(), &resp) - return resp -} -``` - -- [ ] **Step 3: Write e2e tests** - -Create `test/e2e/e2e_test.go`: - -```go -package e2e - -import ( - "encoding/json" - "testing" - - "github.com/msilverblatt/protomcp/internal/mcp" -) - -func TestE2E_Initialize(t *testing.T) { - w, r, cleanup := StartProtomcp(t, "dev", "fixtures/simple_tool.py") - defer cleanup() - - resp := SendRequest(t, w, r, "initialize", nil) - if resp.Error != nil { - t.Fatalf("initialize error: %v", resp.Error) - } - - var result mcp.InitializeResult - json.Unmarshal(resp.Result, &result) - if !result.Capabilities.Tools.ListChanged { - t.Error("expected tools.listChanged = true") - } -} - -func TestE2E_ToolsList(t *testing.T) { - w, r, cleanup := StartProtomcp(t, "dev", "fixtures/simple_tool.py") - defer cleanup() - - SendRequest(t, w, r, "initialize", nil) - resp := SendRequest(t, w, r, "tools/list", nil) - - var result mcp.ToolsListResult - json.Unmarshal(resp.Result, &result) - if len(result.Tools) != 2 { - t.Fatalf("expected 2 tools, got %d", len(result.Tools)) - } -} - -func TestE2E_ToolsCall(t *testing.T) { - w, r, cleanup := StartProtomcp(t, "dev", "fixtures/simple_tool.py") - defer cleanup() - - SendRequest(t, w, r, "initialize", nil) - resp := SendRequest(t, w, r, "tools/call", map[string]interface{}{ - "name": "echo", - "arguments": map[string]string{"message": "hello"}, - }) - - if resp.Error != nil { - t.Fatalf("tools/call error: %v", resp.Error) - } -} - -func TestE2E_DynamicToolList(t *testing.T) { - w, r, cleanup := StartProtomcp(t, "dev", "fixtures/dynamic_tool.py") - defer cleanup() - - SendRequest(t, w, r, "initialize", nil) - - // Initially, auth should be visible but admin_action should not - // (admin_action starts disabled — depends on how the fixture sets initial state) - // Call auth with valid token - resp := SendRequest(t, w, r, "tools/call", map[string]interface{}{ - "name": "auth", - "arguments": map[string]string{"token": "valid"}, - }) - if resp.Error != nil { - t.Fatalf("auth call error: %v", resp.Error) - } - - // After auth, admin_action should now be visible in tools/list - listResp := SendRequest(t, w, r, "tools/list", nil) - var result mcp.ToolsListResult - json.Unmarshal(listResp.Result, &result) - - found := false - for _, tool := range result.Tools { - if tool.Name == "admin_action" { - found = true - } - } - if !found { - t.Error("admin_action should be visible after auth") - } -} - -func TestE2E_TypeScript(t *testing.T) { - w, r, cleanup := StartProtomcp(t, "dev", "fixtures/simple_tool.ts") - defer cleanup() - - SendRequest(t, w, r, "initialize", nil) - resp := SendRequest(t, w, r, "tools/list", nil) - - var result mcp.ToolsListResult - json.Unmarshal(resp.Result, &result) - if len(result.Tools) != 1 { - t.Fatalf("expected 1 tool, got %d", len(result.Tools)) - } - if result.Tools[0].Name != "echo" { - t.Errorf("tool name = %q, want %q", result.Tools[0].Name, "echo") - } -} -``` - -- [ ] **Step 4: Build binary and run e2e tests** - -```bash -make build -go test ./test/e2e/... -v -timeout 60s -``` - -- [ ] **Step 5: Commit** - -```bash -git add test/ -git commit -m "feat: add end-to-end integration tests for Python, TypeScript, and dynamic tool lists" -``` - ---- - -### Task 26: Documentation Site Setup - -**Files:** -- Create: `docs/package.json` -- Create: `docs/astro.config.mjs` -- Create: `docs/src/content/docs/index.mdx` - -- [ ] **Step 1: Scaffold Starlight site** - -```bash -cd docs && npm create astro@latest -- --template starlight -``` - -Or manually create `docs/package.json`: -```json -{ - "name": "protomcp-docs", - "scripts": { - "dev": "astro dev", - "build": "astro build" - }, - "dependencies": { - "@astrojs/starlight": "latest", - "astro": "latest" - } -} -``` - -- [ ] **Step 2: Configure astro.config.mjs** - -```javascript -import { defineConfig } from 'astro/config'; -import starlight from '@astrojs/starlight'; - -export default defineConfig({ - integrations: [ - starlight({ - title: 'protomcp', - description: 'Language-agnostic MCP runtime', - social: { - github: 'https://github.com/msilverblatt/protomcp', - }, - sidebar: [ - { - label: 'Getting Started', - items: [ - { label: 'Installation', slug: 'getting-started/installation' }, - { label: 'Quick Start', slug: 'getting-started/quick-start' }, - { label: 'How It Works', slug: 'getting-started/how-it-works' }, - ], - }, - { - label: 'Guides', - items: [ - { label: 'Writing Tools (Python)', slug: 'guides/writing-tools-python' }, - { label: 'Writing Tools (TypeScript)', slug: 'guides/writing-tools-typescript' }, - { label: 'Dynamic Tool Lists', slug: 'guides/dynamic-tool-lists' }, - { label: 'Hot Reload', slug: 'guides/hot-reload' }, - { label: 'Error Handling', slug: 'guides/error-handling' }, - { label: 'Production Deployment', slug: 'guides/production-deployment' }, - ], - }, - { - label: 'Reference', - items: [ - { label: 'CLI', slug: 'reference/cli' }, - { label: 'Protobuf Spec', slug: 'reference/protobuf-spec' }, - { label: 'Python API', slug: 'reference/python-api' }, - { label: 'TypeScript API', slug: 'reference/typescript-api' }, - ], - }, - { - label: 'Concepts', - items: [ - { label: 'Architecture', slug: 'concepts/architecture' }, - { label: 'Tool List Modes', slug: 'concepts/tool-list-modes' }, - { label: 'Transports', slug: 'concepts/transports' }, - ], - }, - ], - }), - ], -}); -``` - -- [ ] **Step 3: Install dependencies and verify build** - -```bash -cd docs && npm install && npm run build -``` - -- [ ] **Step 4: Commit** - -```bash -git add docs/package.json docs/astro.config.mjs docs/src/ -git commit -m "feat: scaffold Starlight documentation site with sidebar structure" -``` - ---- - -### Task 27: Documentation — Getting Started Pages - -**Files:** -- Create: `docs/src/content/docs/getting-started/installation.mdx` -- Create: `docs/src/content/docs/getting-started/quick-start.mdx` -- Create: `docs/src/content/docs/getting-started/how-it-works.mdx` - -- [ ] **Step 1: Write Installation page** - -Covers: Homebrew, binary download, building from source. Platform-specific instructions for macOS, Linux, Windows. Python and TypeScript SDK installation via pip/npm. - -- [ ] **Step 2: Write Quick Start page** - -5-minute tutorial: -1. Install protomcp -2. Write a Python tool file (3 lines) -3. Run `protomcp dev server.py` -4. Configure your MCP client -5. Use the tool — edit, save, see it update live - -Include exact terminal output at each step. - -- [ ] **Step 3: Write How It Works page** - -Architecture explainer with Mermaid diagrams: -- The three layers (host, Go binary, tool process) -- How hot-reload works (sequence diagram) -- How dynamic tool lists work (state diagram) -- Why the Go binary never needs to change - -- [ ] **Step 4: Commit** - -```bash -git add docs/src/content/docs/getting-started/ -git commit -m "docs: add Getting Started pages — installation, quick start, architecture" -``` - ---- - -### Task 28: Documentation — Guides - -**Files:** -- Create: `docs/src/content/docs/guides/writing-tools-python.mdx` -- Create: `docs/src/content/docs/guides/writing-tools-typescript.mdx` -- Create: `docs/src/content/docs/guides/dynamic-tool-lists.mdx` -- Create: `docs/src/content/docs/guides/hot-reload.mdx` -- Create: `docs/src/content/docs/guides/error-handling.mdx` -- Create: `docs/src/content/docs/guides/production-deployment.mdx` - -- [ ] **Step 1: Write Python guide** - -Covers: `@tool()` decorator, type hints for schema generation, `ToolResult`, `tool_manager`, async tools, testing tools outside protomcp. All examples pulled from test files. - -- [ ] **Step 2: Write TypeScript guide** - -Covers: `tool()` function, Zod schemas, `ToolResult`, `toolManager`. Parallel structure to Python guide. - -- [ ] **Step 3: Write Dynamic Tool Lists guide** - -Covers: inline mutations (enable_tools/disable_tools in ToolResult), programmatic control (tool_manager), modes (open/allowlist/blocklist), batch operations, event-driven examples (locks, auth). - -- [ ] **Step 4: Write Hot Reload guide** - -Covers: how it works, what triggers a reload, in-flight call handling (default vs immediate), reload logs, common gotchas (module-level side effects, import caching). - -- [ ] **Step 5: Write Error Handling guide** - -Covers: structured errors (error_code, message, suggestion, retryable), how errors are formatted by the Go binary, best practices for agent-friendly errors. - -- [ ] **Step 6: Write Production Deployment guide** - -Covers: `protomcp run` vs `protomcp dev`, transport selection for production, systemd/supervisor config, Docker (optional), monitoring via logs. - -- [ ] **Step 7: Commit** - -```bash -git add docs/src/content/docs/guides/ -git commit -m "docs: add all v1.0 guides — Python, TypeScript, dynamic tools, hot reload, errors, production" -``` - ---- - -### Task 29: Documentation — Reference Pages - -**Files:** -- Create: `docs/src/content/docs/reference/cli.mdx` -- Create: `docs/src/content/docs/reference/protobuf-spec.mdx` -- Create: `docs/src/content/docs/reference/python-api.mdx` -- Create: `docs/src/content/docs/reference/typescript-api.mdx` - -- [ ] **Step 1: Write CLI Reference** - -Exhaustive documentation of `protomcp dev`, `protomcp run`, and all flags with examples. - -- [ ] **Step 2: Write Protobuf Spec Reference** - -Full `.proto` file with annotated explanations of every message type, field, and the wire format. - -- [ ] **Step 3: Write Python API Reference** - -Complete API docs for `@tool()`, `ToolResult`, `tool_manager` — all parameters, return types, examples. - -- [ ] **Step 4: Write TypeScript API Reference** - -Same as Python reference but for TS — `tool()`, `ToolResult`, `toolManager`. - -- [ ] **Step 5: Commit** - -```bash -git add docs/src/content/docs/reference/ -git commit -m "docs: add reference pages — CLI, protobuf spec, Python API, TypeScript API" -``` - ---- - -### Task 30: Documentation — Concepts Pages - -**Files:** -- Create: `docs/src/content/docs/concepts/architecture.mdx` -- Create: `docs/src/content/docs/concepts/tool-list-modes.mdx` -- Create: `docs/src/content/docs/concepts/transports.mdx` - -- [ ] **Step 1: Write Architecture page** - -Deep dive into the three-layer architecture with Mermaid diagrams. Explains why the Go binary is language-agnostic, how protobuf enables any-language support, and the unix socket communication pattern. - -- [ ] **Step 2: Write Tool List Modes page** - -Explains open/allowlist/blocklist modes, mode transitions, interaction semantics, batch operations. Includes state diagram. - -- [ ] **Step 3: Write Transports page** - -Explains all five transports, when to use each, configuration, and client compatibility matrix. - -- [ ] **Step 4: Build docs and verify** - -```bash -cd docs && npm run build -``` - -- [ ] **Step 5: Commit** - -```bash -git add docs/src/content/docs/concepts/ -git commit -m "docs: add concept pages — architecture, tool list modes, transports" -``` - ---- - -## Chunk 7: MCP Advanced Features — Progress, Tasks, Cancellation, Logging, Structured Output, Tool Metadata - -### Task 32: Go — Progress Notification Proxy - -**Files:** -- Create: `internal/progress/progress.go` -- Create: `internal/progress/progress_test.go` - -- [ ] **Step 1: Write failing test for progress proxy** - -```go -// internal/progress/progress_test.go -package progress - -import ( - "testing" - pb "github.com/msilverblatt/protomcp/gen/proto/protomcp" -) - -func TestProgressProxy_ForwardsToMCP(t *testing.T) { - var sent []map[string]any - proxy := NewProxy(func(notification map[string]any) { - sent = append(sent, notification) - }) - proxy.HandleProgress(&pb.ProgressNotification{ - ProgressToken: "tok-1", - Progress: 5, - Total: 10, - Message: "Processing item 5", - }) - if len(sent) != 1 { - t.Fatalf("expected 1 notification, got %d", len(sent)) - } - if sent[0]["method"] != "notifications/progress" { - t.Fatalf("expected notifications/progress, got %v", sent[0]["method"]) - } -} - -func TestProgressProxy_DropsWhenNoToken(t *testing.T) { - var sent []map[string]any - proxy := NewProxy(func(notification map[string]any) { - sent = append(sent, notification) - }) - proxy.HandleProgress(&pb.ProgressNotification{ - ProgressToken: "", - Progress: 1, - }) - if len(sent) != 0 { - t.Fatalf("expected 0 notifications for empty token, got %d", len(sent)) - } -} -``` - -- [ ] **Step 2: Run test to verify it fails** - -Run: `go test ./internal/progress/ -v` -Expected: FAIL — package doesn't exist yet. - -- [ ] **Step 3: Implement progress proxy** - -```go -// internal/progress/progress.go -package progress - -import pb "github.com/msilverblatt/protomcp/gen/proto/protomcp" - -type NotifySender func(notification map[string]any) - -type Proxy struct { - send NotifySender -} - -func NewProxy(send NotifySender) *Proxy { - return &Proxy{send: send} -} - -func (p *Proxy) HandleProgress(msg *pb.ProgressNotification) { - if msg.ProgressToken == "" { - return // Client didn't request progress — silently drop - } - params := map[string]any{ - "progressToken": msg.ProgressToken, - "progress": msg.Progress, - } - if msg.Total > 0 { - params["total"] = msg.Total - } - if msg.Message != "" { - params["message"] = msg.Message - } - p.send(map[string]any{ - "method": "notifications/progress", - "params": params, - }) -} -``` - -- [ ] **Step 4: Run test to verify it passes** - -Run: `go test ./internal/progress/ -v` -Expected: PASS - -- [ ] **Step 5: Commit** - -```bash -git add internal/progress/ -git commit -m "feat: add progress notification proxy for MCP notifications/progress" -``` - ---- - -### Task 33: Go — Cancellation Tracker - -**Files:** -- Create: `internal/cancel/tracker.go` -- Create: `internal/cancel/tracker_test.go` - -- [ ] **Step 1: Write failing test for cancellation tracker** - -```go -// internal/cancel/tracker_test.go -package cancel - -import ( - "testing" -) - -func TestTracker_SetAndCheck(t *testing.T) { - tracker := NewTracker() - tracker.TrackCall("req-1") - if tracker.IsCancelled("req-1") { - t.Fatal("should not be cancelled yet") - } - tracker.Cancel("req-1") - if !tracker.IsCancelled("req-1") { - t.Fatal("should be cancelled after Cancel()") - } -} - -func TestTracker_CancelUnknownRequestIsNoop(t *testing.T) { - tracker := NewTracker() - tracker.Cancel("nonexistent") // Should not panic -} - -func TestTracker_CompleteRemovesTracking(t *testing.T) { - tracker := NewTracker() - tracker.TrackCall("req-1") - tracker.Complete("req-1") - if tracker.IsCancelled("req-1") { - t.Fatal("completed call should not report cancelled") - } -} -``` - -- [ ] **Step 2: Run test to verify it fails** - -Run: `go test ./internal/cancel/ -v` -Expected: FAIL - -- [ ] **Step 3: Implement cancellation tracker** - -```go -// internal/cancel/tracker.go -package cancel - -import "sync" - -type Tracker struct { - mu sync.RWMutex - cancelled map[string]bool -} - -func NewTracker() *Tracker { - return &Tracker{cancelled: make(map[string]bool)} -} - -func (t *Tracker) TrackCall(requestID string) { - t.mu.Lock() - defer t.mu.Unlock() - t.cancelled[requestID] = false -} - -func (t *Tracker) Cancel(requestID string) { - t.mu.Lock() - defer t.mu.Unlock() - if _, exists := t.cancelled[requestID]; exists { - t.cancelled[requestID] = true - } -} - -func (t *Tracker) IsCancelled(requestID string) bool { - t.mu.RLock() - defer t.mu.RUnlock() - return t.cancelled[requestID] -} - -func (t *Tracker) Complete(requestID string) { - t.mu.Lock() - defer t.mu.Unlock() - delete(t.cancelled, requestID) -} -``` - -- [ ] **Step 4: Run test to verify it passes** - -Run: `go test ./internal/cancel/ -v` -Expected: PASS - -- [ ] **Step 5: Commit** - -```bash -git add internal/cancel/ -git commit -m "feat: add cancellation tracker for cooperative tool call cancellation" -``` - ---- - -### Task 34: Go — Server Log Forwarder - -**Files:** -- Create: `internal/serverlog/forwarder.go` -- Create: `internal/serverlog/forwarder_test.go` - -- [ ] **Step 1: Write failing test for log forwarder** - -```go -// internal/serverlog/forwarder_test.go -package serverlog - -import ( - "testing" - pb "github.com/msilverblatt/protomcp/gen/proto/protomcp" -) - -func TestForwarder_ForwardsAboveLevel(t *testing.T) { - var sent []map[string]any - fwd := NewForwarder("info", func(n map[string]any) { sent = append(sent, n) }) - - fwd.HandleLog(&pb.LogMessage{Level: "warning", DataJson: `{"msg":"rate limit"}`}) - if len(sent) != 1 { - t.Fatalf("expected 1, got %d", len(sent)) - } - if sent[0]["method"] != "notifications/message" { - t.Fatalf("wrong method: %v", sent[0]["method"]) - } -} - -func TestForwarder_FiltersBelowLevel(t *testing.T) { - var sent []map[string]any - fwd := NewForwarder("warning", func(n map[string]any) { sent = append(sent, n) }) - - fwd.HandleLog(&pb.LogMessage{Level: "info", DataJson: `{"msg":"hello"}`}) - if len(sent) != 0 { - t.Fatalf("expected 0 (filtered), got %d", len(sent)) - } -} -``` - -- [ ] **Step 2: Run test to verify it fails** - -Run: `go test ./internal/serverlog/ -v` -Expected: FAIL - -- [ ] **Step 3: Implement log forwarder** - -```go -// internal/serverlog/forwarder.go -package serverlog - -import ( - "encoding/json" - pb "github.com/msilverblatt/protomcp/gen/proto/protomcp" -) - -var levelPriority = map[string]int{ - "debug": 0, - "info": 1, - "notice": 2, - "warning": 3, - "error": 4, - "critical": 5, - "alert": 6, - "emergency": 7, -} - -type NotifySender func(notification map[string]any) - -type Forwarder struct { - minLevel int - send NotifySender -} - -func NewForwarder(minLevelStr string, send NotifySender) *Forwarder { - return &Forwarder{ - minLevel: levelPriority[minLevelStr], - send: send, - } -} - -func (f *Forwarder) HandleLog(msg *pb.LogMessage) { - priority, ok := levelPriority[msg.Level] - if !ok { - priority = 1 // default to info for unknown levels - } - if priority < f.minLevel { - return - } - params := map[string]any{ - "level": msg.Level, - } - if msg.Logger != "" { - params["logger"] = msg.Logger - } - if msg.DataJson != "" { - var data any - if err := json.Unmarshal([]byte(msg.DataJson), &data); err == nil { - params["data"] = data - } else { - params["data"] = msg.DataJson - } - } - f.send(map[string]any{ - "method": "notifications/message", - "params": params, - }) -} -``` - -- [ ] **Step 4: Run test to verify it passes** - -Run: `go test ./internal/serverlog/ -v` -Expected: PASS - -- [ ] **Step 5: Commit** - -```bash -git add internal/serverlog/ -git commit -m "feat: add server log forwarder for MCP notifications/message" -``` - ---- - -### Task 35: Go — Async Task Manager - -**Files:** -- Create: `internal/tasks/manager.go` -- Create: `internal/tasks/manager_test.go` - -- [ ] **Step 1: Write failing test for task manager** - -```go -// internal/tasks/manager_test.go -package tasks - -import ( - "testing" -) - -func TestTaskManager_CreateAndGet(t *testing.T) { - mgr := NewManager() - mgr.Register("task-1", "req-1") - state, err := mgr.GetStatus("task-1") - if err != nil { - t.Fatal(err) - } - if state.State != "running" { - t.Fatalf("expected running, got %s", state.State) - } -} - -func TestTaskManager_FailOnCrash(t *testing.T) { - mgr := NewManager() - mgr.Register("task-1", "req-1") - mgr.FailAll("tool process crashed") - state, err := mgr.GetStatus("task-1") - if err != nil { - t.Fatal(err) - } - if state.State != "failed" { - t.Fatalf("expected failed, got %s", state.State) - } -} - -func TestTaskManager_GetUnknownTask(t *testing.T) { - mgr := NewManager() - _, err := mgr.GetStatus("nonexistent") - if err == nil { - t.Fatal("expected error for unknown task") - } -} -``` - -- [ ] **Step 2: Run test to verify it fails** - -Run: `go test ./internal/tasks/ -v` -Expected: FAIL - -- [ ] **Step 3: Implement task manager** - -```go -// internal/tasks/manager.go -package tasks - -import ( - "fmt" - "sync" -) - -type TaskState struct { - State string // running, completed, failed, cancelled - Message string -} - -type Manager struct { - mu sync.RWMutex - tasks map[string]*TaskState -} - -func NewManager() *Manager { - return &Manager{tasks: make(map[string]*TaskState)} -} - -func (m *Manager) Register(taskID, requestID string) { - m.mu.Lock() - defer m.mu.Unlock() - m.tasks[taskID] = &TaskState{State: "running"} -} - -func (m *Manager) GetStatus(taskID string) (*TaskState, error) { - m.mu.RLock() - defer m.mu.RUnlock() - state, ok := m.tasks[taskID] - if !ok { - return nil, fmt.Errorf("unknown task: %s", taskID) - } - return state, nil -} - -func (m *Manager) UpdateStatus(taskID, state, message string) error { - m.mu.Lock() - defer m.mu.Unlock() - ts, ok := m.tasks[taskID] - if !ok { - return fmt.Errorf("unknown task: %s", taskID) - } - ts.State = state - ts.Message = message - return nil -} - -func (m *Manager) FailAll(reason string) { - m.mu.Lock() - defer m.mu.Unlock() - for _, ts := range m.tasks { - if ts.State == "running" { - ts.State = "failed" - ts.Message = reason - } - } -} -``` - -- [ ] **Step 4: Run test to verify it passes** - -Run: `go test ./internal/tasks/ -v` -Expected: PASS - -- [ ] **Step 5: Commit** - -```bash -git add internal/tasks/ -git commit -m "feat: add async task manager for MCP task lifecycle" -``` - ---- - -### Task 36: Go — Integrate New Features into MCP Handler - -**Files:** -- Modify: `internal/mcp/handler.go` -- Modify: `internal/mcp/handler_test.go` - -- [ ] **Step 1: Write failing tests for new MCP handler capabilities** - -Add tests to `internal/mcp/handler_test.go`: - -```go -func TestHandler_ToolsCallWithProgressToken(t *testing.T) { - // Test that _meta.progressToken is extracted and forwarded in CallToolRequest - handler := newTestHandler() - req := makeToolCallRequest("test_tool", `{"x": 1}`, map[string]any{"progressToken": "pt-1"}) - handler.HandleRequest(req) - // Verify CallToolRequest sent to tool process includes progress_token = "pt-1" - sentMsg := handler.lastSentToToolProcess() - callReq := sentMsg.GetCallTool() - if callReq.ProgressToken != "pt-1" { - t.Fatalf("expected progress_token pt-1, got %s", callReq.ProgressToken) - } -} - -func TestHandler_NotificationsCancelled(t *testing.T) { - // Test that notifications/cancelled is forwarded as CancelRequest - handler := newTestHandler() - handler.startToolCall("req-42") - handler.HandleNotification("notifications/cancelled", map[string]any{"requestId": "req-42"}) - if !handler.cancelTracker.IsCancelled("req-42") { - t.Fatal("expected req-42 to be cancelled") - } -} - -func TestHandler_ToolsCallReturnsStructuredContent(t *testing.T) { - // Test that structuredContent is included when outputSchema is defined - handler := newTestHandler() - handler.setToolOutputSchema("search", `{"type":"array"}`) - resp := handler.callTool("search", `{"q":"test"}`) - if resp["structuredContent"] == nil { - t.Fatal("expected structuredContent in response") - } -} - -func TestHandler_ToolDefinitionIncludesMetadata(t *testing.T) { - // Test that tool metadata (title, hints, task_support) is included in tools/list - handler := newTestHandler() - tools := handler.handleToolsList() - for _, tool := range tools { - toolMap := tool.(map[string]any) - if toolMap["name"] == "delete_doc" { - annotations := toolMap["annotations"].(map[string]any) - if annotations["destructiveHint"] != true { - t.Fatal("expected destructiveHint on delete_doc") - } - } - } -} -``` - -- [ ] **Step 2: Run tests to verify they fail** - -Run: `go test ./internal/mcp/ -v -run "ProgressToken|Cancelled|StructuredContent|Metadata"` -Expected: FAIL - -- [ ] **Step 3: Update MCP handler implementation** - -Update `internal/mcp/handler.go` to: -1. Extract `_meta.progressToken` from `tools/call` requests and include it in `CallToolRequest.progress_token` -2. Handle `notifications/cancelled` by forwarding `CancelRequest` to tool process and marking in cancel tracker -3. Include `outputSchema` in tool definitions from `ToolDefinition.output_schema_json` -4. Include `structuredContent` in `tools/call` responses from `CallToolResponse.structured_content_json` -5. Include `annotations` (title, hints) in tool definitions from `ToolDefinition` metadata fields -6. Include `execution.taskSupport` in tool definitions when `ToolDefinition.task_support` is true -7. Handle `tasks/get`, `tasks/result`, `tasks/cancel` by forwarding to tool process -8. Advertise `tasks` capability in `initialize` response when any tool has `task_support` -9. Handle `notifications/message` from tool process by forwarding via server log forwarder - -- [ ] **Step 4: Run tests to verify they pass** - -Run: `go test ./internal/mcp/ -v` -Expected: PASS - -- [ ] **Step 5: Commit** - -```bash -git add internal/mcp/ -git commit -m "feat: integrate progress, cancellation, tasks, logging, structured output, and tool metadata into MCP handler" -``` - ---- - -### Task 37: Python SDK — ToolContext (Progress + Cancellation) - -**Files:** -- Create: `sdk/python/src/protomcp/context.py` -- Create: `sdk/python/tests/test_context.py` - -- [ ] **Step 1: Write failing test** - -```python -# sdk/python/tests/test_context.py -from protomcp.context import ToolContext - -def test_report_progress_sends_notification(): - sent = [] - ctx = ToolContext(progress_token="pt-1", send_fn=lambda msg: sent.append(msg)) - ctx.report_progress(progress=5, total=10, message="Working") - assert len(sent) == 1 - assert sent[0].progress.progress_token == "pt-1" - assert sent[0].progress.progress == 5 - assert sent[0].progress.total == 10 - -def test_report_progress_noop_without_token(): - sent = [] - ctx = ToolContext(progress_token="", send_fn=lambda msg: sent.append(msg)) - ctx.report_progress(progress=1) - assert len(sent) == 0 - -def test_is_cancelled(): - ctx = ToolContext(progress_token="", send_fn=lambda msg: None) - assert not ctx.is_cancelled() - ctx._cancelled = True - assert ctx.is_cancelled() -``` - -- [ ] **Step 2: Run test to verify it fails** - -Run: `cd sdk/python && python -m pytest tests/test_context.py -v` -Expected: FAIL - -- [ ] **Step 3: Implement ToolContext** - -```python -# sdk/python/src/protomcp/context.py -import threading -from protomcp.gen import protomcp_pb2 as pb - -class ToolContext: - def __init__(self, progress_token: str, send_fn): - self._progress_token = progress_token - self._send_fn = send_fn - self._cancelled = False - self._lock = threading.Lock() - - def report_progress(self, progress: int, total: int = 0, message: str = ""): - if not self._progress_token: - return - envelope = pb.Envelope( - progress=pb.ProgressNotification( - progress_token=self._progress_token, - progress=progress, - total=total, - message=message, - ) - ) - self._send_fn(envelope) - - def is_cancelled(self) -> bool: - with self._lock: - return self._cancelled - - def _set_cancelled(self): - with self._lock: - self._cancelled = True -``` - -- [ ] **Step 4: Run test to verify it passes** - -Run: `cd sdk/python && python -m pytest tests/test_context.py -v` -Expected: PASS - -- [ ] **Step 5: Commit** - -```bash -git add sdk/python/src/protomcp/context.py sdk/python/tests/test_context.py -git commit -m "feat(python): add ToolContext with progress reporting and cancellation" -``` - ---- - -### Task 38: Python SDK — Server Logging API - -**Files:** -- Create: `sdk/python/src/protomcp/log.py` -- Create: `sdk/python/tests/test_log.py` - -- [ ] **Step 1: Write failing test** - -```python -# sdk/python/tests/test_log.py -from protomcp.log import ServerLogger - -def test_log_info(): - sent = [] - logger = ServerLogger(send_fn=lambda msg: sent.append(msg)) - logger.info("hello", data={"count": 5}) - assert len(sent) == 1 - assert sent[0].log.level == "info" - assert '"count": 5' in sent[0].log.data_json or '"count":5' in sent[0].log.data_json - -def test_log_with_logger_name(): - sent = [] - logger = ServerLogger(send_fn=lambda msg: sent.append(msg), name="cache") - logger.debug("hit") - assert sent[0].log.logger == "cache" - -def test_all_levels(): - sent = [] - logger = ServerLogger(send_fn=lambda msg: sent.append(msg)) - for level in ["debug", "info", "notice", "warning", "error", "critical", "alert", "emergency"]: - getattr(logger, level)("test") - assert len(sent) == 8 -``` - -- [ ] **Step 2: Run test to verify it fails** - -Run: `cd sdk/python && python -m pytest tests/test_log.py -v` -Expected: FAIL - -- [ ] **Step 3: Implement ServerLogger** - -```python -# sdk/python/src/protomcp/log.py -import json -from protomcp.gen import protomcp_pb2 as pb - -class ServerLogger: - def __init__(self, send_fn, name: str = ""): - self._send_fn = send_fn - self._name = name - - def _log(self, level: str, message: str, data: dict | None = None): - data_json = json.dumps(data) if data else json.dumps({"message": message}) - envelope = pb.Envelope( - log=pb.LogMessage( - level=level, - logger=self._name, - data_json=data_json, - ) - ) - self._send_fn(envelope) - - def debug(self, message: str, **kwargs): self._log("debug", message, kwargs.get("data")) - def info(self, message: str, **kwargs): self._log("info", message, kwargs.get("data")) - def notice(self, message: str, **kwargs): self._log("notice", message, kwargs.get("data")) - def warning(self, message: str, **kwargs): self._log("warning", message, kwargs.get("data")) - def error(self, message: str, **kwargs): self._log("error", message, kwargs.get("data")) - def critical(self, message: str, **kwargs): self._log("critical", message, kwargs.get("data")) - def alert(self, message: str, **kwargs): self._log("alert", message, kwargs.get("data")) - def emergency(self, message: str, **kwargs): self._log("emergency", message, kwargs.get("data")) -``` - -- [ ] **Step 4: Run test to verify it passes** - -Run: `cd sdk/python && python -m pytest tests/test_log.py -v` -Expected: PASS - -- [ ] **Step 5: Commit** - -```bash -git add sdk/python/src/protomcp/log.py sdk/python/tests/test_log.py -git commit -m "feat(python): add server logging API for MCP notifications/message" -``` - ---- - -### Task 39: Python SDK — Structured Output and Tool Metadata - -**Files:** -- Modify: `sdk/python/src/protomcp/tool.py` -- Modify: `sdk/python/tests/test_tool.py` - -- [ ] **Step 1: Write failing tests for output_type and metadata** - -Add to `sdk/python/tests/test_tool.py`: - -```python -from dataclasses import dataclass - -def test_tool_with_output_type(): - clear_registry() - - @dataclass - class SearchResult: - title: str - score: float - - @tool(description="Search", output_type=SearchResult) - def search(query: str) -> SearchResult: - return SearchResult(title="test", score=0.9) - - tools = get_registered_tools() - assert tools[0].output_schema_json != "" - schema = json.loads(tools[0].output_schema_json) - assert "title" in schema["properties"] - assert "score" in schema["properties"] - -def test_tool_with_metadata(): - clear_registry() - - @tool( - description="Delete doc", - title="Delete Document", - destructive=True, - idempotent=True, - ) - def delete_doc(doc_id: str) -> str: - return "deleted" - - tools = get_registered_tools() - assert tools[0].title == "Delete Document" - assert tools[0].destructive_hint is True - assert tools[0].idempotent_hint is True - -def test_tool_with_task_support(): - clear_registry() - - @tool(description="Long task", task_support=True) - async def long_task(data: str) -> str: - return "done" - - tools = get_registered_tools() - assert tools[0].task_support is True -``` - -- [ ] **Step 2: Run tests to verify they fail** - -Run: `cd sdk/python && python -m pytest tests/test_tool.py -v -k "output_type or metadata or task_support"` -Expected: FAIL - -- [ ] **Step 3: Update @tool decorator** - -Update `sdk/python/src/protomcp/tool.py` to accept `output_type`, `title`, `destructive`, `idempotent`, `read_only`, `open_world`, and `task_support` parameters. Generate `output_schema_json` from the output type's fields using the same schema generation logic. Set the corresponding fields on `ToolDefinition`. - -- [ ] **Step 4: Run tests to verify they pass** - -Run: `cd sdk/python && python -m pytest tests/test_tool.py -v` -Expected: PASS - -- [ ] **Step 5: Commit** - -```bash -git add sdk/python/src/protomcp/tool.py sdk/python/tests/test_tool.py -git commit -m "feat(python): add output_type, tool metadata, and task_support to @tool decorator" -``` - ---- - -### Task 40: Python SDK — Update Runner for New Features - -**Files:** -- Modify: `sdk/python/src/protomcp/runner.py` -- Modify: `sdk/python/src/protomcp/__init__.py` - -- [ ] **Step 1: Update runner to handle new message types** - -Update `sdk/python/src/protomcp/runner.py` to: -1. Create `ToolContext` with `progress_token` from `CallToolRequest` and pass it to tool handlers that accept a `ctx` parameter (detected via `inspect.signature`) -2. Handle incoming `CancelRequest` by calling `ctx._set_cancelled()` on the matching in-flight call -3. Initialize `ServerLogger` and expose as module-level `log` in `__init__.py` -4. For tools with `task_support=True`, run handler in a background thread, immediately return `CreateTaskResponse`, and handle `TaskStatusRequest`/`TaskResultRequest`/`TaskCancelRequest` by querying the running task - -- [ ] **Step 2: Update __init__.py exports** - -```python -# sdk/python/src/protomcp/__init__.py -from protomcp.tool import tool -from protomcp.result import ToolResult -from protomcp.manager import tool_manager -from protomcp.context import ToolContext -from protomcp.log import ServerLogger - -log = ServerLogger(send_fn=None) # Initialized by runner on connect -``` - -- [ ] **Step 3: Run all Python tests** - -Run: `cd sdk/python && python -m pytest tests/ -v` -Expected: PASS - -- [ ] **Step 4: Commit** - -```bash -git add sdk/python/src/protomcp/ -git commit -m "feat(python): integrate progress, cancellation, logging, and async tasks into runner" -``` - ---- - -### Task 41: TypeScript SDK — ToolContext, Logging, Structured Output, Metadata - -**Files:** -- Create: `sdk/typescript/src/context.ts` -- Create: `sdk/typescript/src/log.ts` -- Create: `sdk/typescript/tests/context.test.ts` -- Create: `sdk/typescript/tests/log.test.ts` -- Modify: `sdk/typescript/src/tool.ts` -- Modify: `sdk/typescript/tests/tool.test.ts` -- Modify: `sdk/typescript/src/runner.ts` -- Modify: `sdk/typescript/src/index.ts` - -- [ ] **Step 1: Write failing tests for ToolContext** - -```typescript -// sdk/typescript/tests/context.test.ts -import { describe, it, expect } from 'vitest'; -import { ToolContext } from '../src/context'; - -describe('ToolContext', () => { - it('sends progress notification', () => { - const sent: any[] = []; - const ctx = new ToolContext('pt-1', (msg) => sent.push(msg)); - ctx.reportProgress(5, 10, 'Working'); - expect(sent).toHaveLength(1); - expect(sent[0].progress.progressToken).toBe('pt-1'); - }); - - it('is noop without progress token', () => { - const sent: any[] = []; - const ctx = new ToolContext('', (msg) => sent.push(msg)); - ctx.reportProgress(1); - expect(sent).toHaveLength(0); - }); - - it('tracks cancellation', () => { - const ctx = new ToolContext('', () => {}); - expect(ctx.isCancelled()).toBe(false); - ctx.setCancelled(); - expect(ctx.isCancelled()).toBe(true); - }); -}); -``` - -- [ ] **Step 2: Write failing tests for ServerLogger** - -```typescript -// sdk/typescript/tests/log.test.ts -import { describe, it, expect } from 'vitest'; -import { ServerLogger } from '../src/log'; - -describe('ServerLogger', () => { - it('sends log message', () => { - const sent: any[] = []; - const logger = new ServerLogger((msg) => sent.push(msg)); - logger.info('hello', { count: 5 }); - expect(sent).toHaveLength(1); - expect(sent[0].log.level).toBe('info'); - }); - - it('supports all RFC 5424 levels', () => { - const sent: any[] = []; - const logger = new ServerLogger((msg) => sent.push(msg)); - const levels = ['debug', 'info', 'notice', 'warning', 'error', 'critical', 'alert', 'emergency'] as const; - for (const level of levels) { - logger[level]('test'); - } - expect(sent).toHaveLength(8); - }); -}); -``` - -- [ ] **Step 3: Run tests to verify they fail** - -Run: `cd sdk/typescript && npx vitest run --reporter=verbose` -Expected: FAIL - -- [ ] **Step 4: Implement ToolContext** - -```typescript -// sdk/typescript/src/context.ts -export class ToolContext { - private _cancelled = false; - constructor( - private readonly progressToken: string, - private readonly sendFn: (msg: any) => void, - ) {} - - reportProgress(progress: number, total?: number, message?: string) { - if (!this.progressToken) return; - this.sendFn({ - progress: { - progressToken: this.progressToken, - progress, - ...(total !== undefined && { total }), - ...(message !== undefined && { message }), - }, - }); - } - - isCancelled(): boolean { return this._cancelled; } - setCancelled() { this._cancelled = true; } -} -``` - -- [ ] **Step 5: Implement ServerLogger** - -```typescript -// sdk/typescript/src/log.ts -const LEVELS = ['debug', 'info', 'notice', 'warning', 'error', 'critical', 'alert', 'emergency'] as const; -type LogLevel = typeof LEVELS[number]; - -export class ServerLogger { - constructor( - private sendFn: (msg: any) => void, - private name?: string, - ) {} - - private _log(level: LogLevel, message: string, data?: Record) { - this.sendFn({ - log: { - level, - logger: this.name ?? '', - dataJson: JSON.stringify(data ?? { message }), - }, - }); - } - - debug(msg: string, data?: Record) { this._log('debug', msg, data); } - info(msg: string, data?: Record) { this._log('info', msg, data); } - notice(msg: string, data?: Record) { this._log('notice', msg, data); } - warning(msg: string, data?: Record) { this._log('warning', msg, data); } - error(msg: string, data?: Record) { this._log('error', msg, data); } - critical(msg: string, data?: Record) { this._log('critical', msg, data); } - alert(msg: string, data?: Record) { this._log('alert', msg, data); } - emergency(msg: string, data?: Record) { this._log('emergency', msg, data); } -} -``` - -- [ ] **Step 6: Update tool() to support output, metadata, and task_support** - -Update `sdk/typescript/src/tool.ts` to accept `output` (Zod schema), `title`, `destructiveHint`, `idempotentHint`, `readOnlyHint`, `openWorldHint`, and `taskSupport` options. Generate `output_schema_json` from the Zod output schema. Set corresponding fields on the `ToolDefinition`. - -Add tests to `sdk/typescript/tests/tool.test.ts`: - -```typescript -it('generates output schema from Zod', () => { - const OutputSchema = z.object({ title: z.string(), score: z.number() }); - const t = tool({ - name: 'search', - description: 'Search', - args: z.object({ q: z.string() }), - output: OutputSchema, - handler: (args) => ({ title: 'test', score: 0.9 }), - }); - expect(t.definition.outputSchemaJson).toBeTruthy(); - const schema = JSON.parse(t.definition.outputSchemaJson); - expect(schema.properties.title).toBeTruthy(); -}); - -it('includes tool metadata', () => { - const t = tool({ - name: 'delete_doc', - description: 'Delete', - args: z.object({ id: z.string() }), - title: 'Delete Document', - destructiveHint: true, - handler: (args) => 'deleted', - }); - expect(t.definition.title).toBe('Delete Document'); - expect(t.definition.destructiveHint).toBe(true); -}); -``` - -- [ ] **Step 7: Update runner and index exports** - -Update `sdk/typescript/src/runner.ts` to handle new message types (same as Python runner — ToolContext injection, cancel handling, async task lifecycle, log initialization). - -Update `sdk/typescript/src/index.ts`: - -```typescript -export { tool, ToolResult } from './tool'; -export { toolManager } from './manager'; -export { ToolContext } from './context'; -export { ServerLogger } from './log'; -``` - -- [ ] **Step 8: Run all TypeScript tests** - -Run: `cd sdk/typescript && npx vitest run --reporter=verbose` -Expected: PASS - -- [ ] **Step 9: Commit** - -```bash -git add sdk/typescript/ -git commit -m "feat(typescript): add progress, cancellation, logging, structured output, and tool metadata" -``` - ---- - -### Task 42: E2E Tests for New Features - -**Files:** -- Create: `test/e2e/fixtures/progress_tool.py` -- Create: `test/e2e/fixtures/async_tool.py` -- Create: `test/e2e/fixtures/logging_tool.py` -- Create: `test/e2e/fixtures/structured_output_tool.py` -- Modify: `test/e2e/e2e_test.go` - -- [ ] **Step 1: Create test fixtures** - -`progress_tool.py`: -```python -from protomcp import tool, ToolContext - -@tool(description="Count with progress") -def count_with_progress(n: int, ctx: ToolContext) -> str: - for i in range(n): - ctx.report_progress(i + 1, n, f"Step {i + 1}") - return f"Counted to {n}" -``` - -`async_tool.py`: -```python -import asyncio -from protomcp import tool - -@tool(description="Async task", task_support=True) -async def slow_task(duration_ms: int) -> str: - await asyncio.sleep(duration_ms / 1000) - return f"Completed after {duration_ms}ms" -``` - -`logging_tool.py`: -```python -from protomcp import tool, log - -@tool(description="Tool that logs") -def logging_tool(message: str) -> str: - log.info("Tool called", data={"message": message}) - log.debug("Debug detail") - return f"Logged: {message}" -``` - -`structured_output_tool.py`: -```python -from dataclasses import dataclass -from protomcp import tool - -@dataclass -class SearchResult: - title: str - score: float - -@tool(description="Structured search", output_type=SearchResult) -def search(query: str) -> SearchResult: - return SearchResult(title=f"Result for {query}", score=0.95) -``` - -- [ ] **Step 2: Write e2e tests** - -Add to `test/e2e/e2e_test.go`: - -```go -func TestE2E_ProgressNotifications(t *testing.T) { - binary, cleanup := buildBinary(t) - defer cleanup() - client := startWithFixture(t, binary, "progress_tool.py") - defer client.Close() - - notifications := client.CollectNotifications() - result := client.CallTool("count_with_progress", map[string]any{ - "n": 3, - }, map[string]any{"progressToken": "pt-test"}) - - assertNoError(t, result) - progressNotifs := filterNotifications(notifications, "notifications/progress") - if len(progressNotifs) != 3 { - t.Fatalf("expected 3 progress notifications, got %d", len(progressNotifs)) - } -} - -func TestE2E_AsyncTask(t *testing.T) { - binary, cleanup := buildBinary(t) - defer cleanup() - client := startWithFixture(t, binary, "async_tool.py") - defer client.Close() - - result := client.CallTool("slow_task", map[string]any{"duration_ms": 100}) - // Should get a CreateTaskResult with taskId - taskID := result["taskId"].(string) - if taskID == "" { - t.Fatal("expected taskId in response") - } - - // Poll until complete - var status map[string]any - for i := 0; i < 20; i++ { - status = client.TaskGet(taskID) - if status["state"] == "completed" { - break - } - time.Sleep(50 * time.Millisecond) - } - if status["state"] != "completed" { - t.Fatalf("expected completed, got %s", status["state"]) - } - - // Get result - taskResult := client.TaskResult(taskID) - assertNoError(t, taskResult) -} - -func TestE2E_ServerLogging(t *testing.T) { - binary, cleanup := buildBinary(t) - defer cleanup() - client := startWithFixture(t, binary, "logging_tool.py") - defer client.Close() - - notifications := client.CollectNotifications() - client.CallTool("logging_tool", map[string]any{"message": "test"}) - - logNotifs := filterNotifications(notifications, "notifications/message") - if len(logNotifs) == 0 { - t.Fatal("expected at least one log notification") - } -} - -func TestE2E_StructuredOutput(t *testing.T) { - binary, cleanup := buildBinary(t) - defer cleanup() - client := startWithFixture(t, binary, "structured_output_tool.py") - defer client.Close() - - // Verify tools/list includes outputSchema - tools := client.ListTools() - searchTool := findTool(tools, "search") - if searchTool["outputSchema"] == nil { - t.Fatal("expected outputSchema on search tool") - } - - // Verify call result includes structuredContent - result := client.CallTool("search", map[string]any{"query": "test"}) - if result["structuredContent"] == nil { - t.Fatal("expected structuredContent in result") - } -} -``` - -- [ ] **Step 3: Run e2e tests** - -Run: `go test ./test/e2e/... -v -timeout 60s -run "Progress|Async|Logging|Structured"` -Expected: PASS - -- [ ] **Step 4: Commit** - -```bash -git add test/e2e/ -git commit -m "test: add e2e tests for progress, async tasks, server logging, and structured output" -``` - ---- - -### Task 43: Documentation — New Feature Guides - -**Files:** -- Create: `docs/src/content/docs/guides/progress-notifications.mdx` -- Create: `docs/src/content/docs/guides/async-tasks.mdx` -- Create: `docs/src/content/docs/guides/cancellation.mdx` -- Create: `docs/src/content/docs/guides/server-logging.mdx` -- Create: `docs/src/content/docs/guides/structured-output.mdx` - -- [ ] **Step 1: Write progress notifications guide** - -Cover: how to use `ToolContext.report_progress()`, what happens with/without progressToken, Python and TS examples. - -- [ ] **Step 2: Write async tasks guide** - -Cover: `task_support=True`, how async tools work, task lifecycle (create → poll → result), cancellation of tasks, Python and TS examples. - -- [ ] **Step 3: Write cancellation guide** - -Cover: `ctx.is_cancelled()`, cooperative cancellation pattern, how it interacts with async tasks, Python and TS examples. - -- [ ] **Step 4: Write server logging guide** - -Cover: log levels (RFC 5424), `log.info()`/`log.debug()` etc., how `--log-level` filters, named loggers, Python and TS examples. - -- [ ] **Step 5: Write structured output guide** - -Cover: `output_type`/`output` parameter, how `outputSchema` appears in tool definitions, `structuredContent` in results, Python and TS examples. - -- [ ] **Step 6: Update Starlight sidebar config** - -Add new guide pages to the sidebar in `docs/astro.config.mjs`. - -- [ ] **Step 7: Build docs to verify** - -Run: `cd docs && npm run build` -Expected: builds successfully. - -- [ ] **Step 8: Commit** - -```bash -git add docs/ -git commit -m "docs: add guides for progress, async tasks, cancellation, logging, and structured output" -``` - ---- - -### Task 31: Final Verification - -- [ ] **Step 1: Run all Go tests** - -```bash -make test -``` - -Expected: all tests pass. - -- [ ] **Step 2: Run Python SDK tests** - -```bash -make test-python -``` - -Expected: all tests pass. - -- [ ] **Step 3: Run TypeScript SDK tests** - -```bash -make test-ts -``` - -Expected: all tests pass. - -- [ ] **Step 4: Run e2e tests** - -```bash -go test ./test/e2e/... -v -timeout 60s -``` - -Expected: all tests pass. - -- [ ] **Step 5: Build documentation** - -```bash -cd docs && npm run build -``` - -Expected: builds successfully. - -- [ ] **Step 6: Build release binary** - -```bash -goreleaser build --snapshot --clean -``` - -Expected: binaries for linux/darwin/windows amd64/arm64. - -- [ ] **Step 7: Final commit** - -```bash -git add -A -git commit -m "chore: final v1.0 verification — all tests pass, docs build, release builds" -``` diff --git a/docs/superpowers/plans/2026-03-12-readme-demos-interactive.md b/docs/superpowers/plans/2026-03-12-readme-demos-interactive.md deleted file mode 100644 index 71874a1..0000000 --- a/docs/superpowers/plans/2026-03-12-readme-demos-interactive.md +++ /dev/null @@ -1,1695 +0,0 @@ -# README, Working Demos & Interactive Demo Page — Implementation Plan - -> **For agentic workers:** REQUIRED: Use superpowers:subagent-driven-development (if subagents available) or superpowers:executing-plans to implement this plan. Steps use checkbox (`- [ ]`) syntax for tracking. - -**Goal:** Create a README, runnable examples at three tiers in Python+TypeScript, and an animated interactive demo page in the Starlight docs site. - -**Architecture:** Three independent deliverables. Examples use the protomcp Python/TypeScript SDKs directly. The demo page uses Astro components embedded in a Starlight MDX page with CSS animations and minimal JS for tab toggling. - -**Tech Stack:** Markdown (README), Python/TypeScript (examples), Astro/Starlight (demo page), CSS @keyframes (animations), bash (demo runner) - ---- - -## Chunk 1: Working Code Examples - -### Task 1: Basic Python Example - -**Files:** -- Create: `examples/python/basic.py` - -- [ ] **Step 1: Create the basic Python example** - -```python -# examples/python/basic.py -# A minimal protomcp tool — adds two numbers. -# Run: pmcp dev examples/python/basic.py - -from protomcp import tool, ToolResult - -@tool("Add two numbers") -def add(a: int, b: int) -> ToolResult: - return ToolResult(result=str(a + b)) - -@tool("Multiply two numbers") -def multiply(a: int, b: int) -> ToolResult: - return ToolResult(result=str(a * b)) -``` - -- [ ] **Step 2: Verify it parses** - -Run: `python -c "import ast; ast.parse(open('examples/python/basic.py').read()); print('OK')"` -Expected: `OK` - -- [ ] **Step 3: Commit** - -```bash -git add examples/python/basic.py -git commit -m "examples: add basic Python example" -``` - ---- - -### Task 2: Basic TypeScript Example - -**Files:** -- Create: `examples/typescript/basic.ts` - -- [ ] **Step 1: Create the basic TypeScript example** - -```typescript -// examples/typescript/basic.ts -// A minimal protomcp tool — adds two numbers. -// Run: pmcp dev examples/typescript/basic.ts - -import { tool, ToolResult } from 'protomcp'; -import { z } from 'zod'; - -tool({ - name: 'add', - description: 'Add two numbers', - args: z.object({ a: z.number(), b: z.number() }), - handler({ a, b }) { - return new ToolResult({ result: String(a + b) }); - }, -}); - -tool({ - name: 'multiply', - description: 'Multiply two numbers', - args: z.object({ a: z.number(), b: z.number() }), - handler({ a, b }) { - return new ToolResult({ result: String(a * b) }); - }, -}); -``` - -- [ ] **Step 2: Commit** - -```bash -git add examples/typescript/basic.ts -git commit -m "examples: add basic TypeScript example" -``` - ---- - -### Task 3: Real-World Python Example - -**Files:** -- Create: `examples/python/real_world.py` - -- [ ] **Step 1: Create the real-world Python example** - -This is a file search tool that demonstrates progress reporting, logging, and cancellation. - -```python -# examples/python/real_world.py -# A file search tool demonstrating progress, logging, and cancellation. -# Run: pmcp dev examples/python/real_world.py - -import os -import fnmatch -from protomcp import tool, ToolResult, ToolContext, log - -@tool("Search files in a directory by glob pattern", read_only=True) -def search_files(ctx: ToolContext, directory: str, pattern: str, max_results: int = 50) -> ToolResult: - log.info(f"Searching {directory} for '{pattern}'") - - if not os.path.isdir(directory): - return ToolResult( - result=f"Directory not found: {directory}", - is_error=True, - error_code="INVALID_PATH", - message="The specified directory does not exist", - suggestion="Check the path and try again", - ) - - matches = [] - all_files = [] - for root, dirs, files in os.walk(directory): - for f in files: - all_files.append(os.path.join(root, f)) - - total = len(all_files) - log.debug(f"Found {total} files to scan") - - for i, filepath in enumerate(all_files): - if ctx.is_cancelled(): - log.warning("Search cancelled by client") - return ToolResult( - result=f"Cancelled after scanning {i}/{total} files. Found {len(matches)} matches so far.", - is_error=True, - error_code="CANCELLED", - retryable=True, - ) - - if i % 100 == 0: - ctx.report_progress(i, total, f"Scanning... {i}/{total}") - - if fnmatch.fnmatch(os.path.basename(filepath), pattern): - matches.append(filepath) - if len(matches) >= max_results: - log.info(f"Hit max_results={max_results}, stopping early") - break - - ctx.report_progress(total, total, "Complete") - log.info(f"Search complete: {len(matches)} matches") - return ToolResult(result="\n".join(matches) if matches else "No files found") -``` - -- [ ] **Step 2: Verify it parses** - -Run: `python -c "import ast; ast.parse(open('examples/python/real_world.py').read()); print('OK')"` -Expected: `OK` - -- [ ] **Step 3: Commit** - -```bash -git add examples/python/real_world.py -git commit -m "examples: add real-world Python example with progress and cancellation" -``` - ---- - -### Task 4: Real-World TypeScript Example - -**Files:** -- Create: `examples/typescript/real-world.ts` - -- [ ] **Step 1: Create the real-world TypeScript example** - -```typescript -// examples/typescript/real-world.ts -// A file search tool demonstrating progress, logging, and cancellation. -// Run: pmcp dev examples/typescript/real-world.ts - -import { tool, ToolResult, ToolContext, ServerLogger } from 'protomcp'; -import { z } from 'zod'; -import * as fs from 'fs'; -import * as path from 'path'; - -// Note: ServerLogger requires a transport send function. In a real pmcp process, -// this is wired up automatically by the runner. For demonstration purposes, -// we show the API shape — logging calls are forwarded to the MCP host. - -tool({ - name: 'search_files', - description: 'Search files in a directory by glob pattern', - readOnlyHint: true, - args: z.object({ - directory: z.string(), - pattern: z.string(), - max_results: z.number().default(50), - }), - handler({ directory, pattern, max_results }, ctx: ToolContext) { - if (!fs.existsSync(directory)) { - return new ToolResult({ - result: `Directory not found: ${directory}`, - isError: true, - errorCode: 'INVALID_PATH', - message: 'The specified directory does not exist', - suggestion: 'Check the path and try again', - }); - } - - const matches: string[] = []; - const allFiles: string[] = []; - - function walk(dir: string) { - for (const entry of fs.readdirSync(dir, { withFileTypes: true })) { - const full = path.join(dir, entry.name); - if (entry.isDirectory()) walk(full); - else allFiles.push(full); - } - } - walk(directory); - - const total = allFiles.length; - - for (let i = 0; i < total; i++) { - if (ctx.isCancelled()) { - return new ToolResult({ - result: `Cancelled after scanning ${i}/${total} files. Found ${matches.length} matches so far.`, - isError: true, - errorCode: 'CANCELLED', - retryable: true, - }); - } - - if (i % 100 === 0) { - ctx.reportProgress(i, total, `Scanning... ${i}/${total}`); - } - - if (matchGlob(path.basename(allFiles[i]), pattern)) { - matches.push(allFiles[i]); - if (matches.length >= max_results) break; - } - } - - ctx.reportProgress(total, total, 'Complete'); - return new ToolResult({ - result: matches.length > 0 ? matches.join('\n') : 'No files found', - }); - }, -}); - -function matchGlob(filename: string, pattern: string): boolean { - const regex = new RegExp( - '^' + pattern.replace(/\*/g, '.*').replace(/\?/g, '.') + '$' - ); - return regex.test(filename); -} -``` - -- [ ] **Step 2: Commit** - -```bash -git add examples/typescript/real-world.ts -git commit -m "examples: add real-world TypeScript example with progress and cancellation" -``` - ---- - -### Task 5: Full Showcase Python Example - -**Files:** -- Create: `examples/python/full_showcase.py` - -- [ ] **Step 1: Create the full showcase Python example** - -Multi-tool server demonstrating structured output, dynamic tool lists, metadata, progress, cancellation, and logging. - -```python -# examples/python/full_showcase.py -# Full-featured protomcp demo — multiple tools showcasing the complete API. -# Run: pmcp dev examples/python/full_showcase.py - -import json -import time -from dataclasses import dataclass -from protomcp import tool, ToolResult, ToolContext, log -from protomcp import tool_manager - -# --- Tool 1: Structured output with output schema --- - -@dataclass -class WeatherData: - location: str - temperature_f: float - conditions: str - humidity: int - -@tool( - "Get current weather for a location", - output_type=WeatherData, - read_only=True, - title="Weather Lookup", -) -def get_weather(location: str) -> ToolResult: - log.info(f"Weather lookup for {location}") - # Simulated weather data - data = WeatherData( - location=location, - temperature_f=72.5, - conditions="Partly cloudy", - humidity=45, - ) - return ToolResult(result=json.dumps({ - "location": data.location, - "temperature_f": data.temperature_f, - "conditions": data.conditions, - "humidity": data.humidity, - })) - -# --- Tool 2: Long-running operation with progress --- - -@tool( - "Analyze a dataset (simulated long-running task)", - title="Dataset Analyzer", - idempotent=True, - task_support=True, -) -def analyze_dataset(ctx: ToolContext, dataset_name: str, depth: str = "basic") -> ToolResult: - log.info(f"Starting analysis of {dataset_name} at depth={depth}") - steps = 10 - - for i in range(steps): - if ctx.is_cancelled(): - log.warning(f"Analysis cancelled at step {i}/{steps}") - return ToolResult( - result=f"Analysis cancelled at step {i}/{steps}", - is_error=True, - error_code="CANCELLED", - retryable=True, - ) - ctx.report_progress(i, steps, f"Analyzing step {i+1}/{steps}...") - time.sleep(0.1) # Simulate work - - ctx.report_progress(steps, steps, "Analysis complete") - log.info("Analysis finished successfully") - return ToolResult(result=json.dumps({ - "dataset": dataset_name, - "depth": depth, - "rows_analyzed": 15000, - "anomalies_found": 3, - "summary": "Dataset is healthy with 3 minor anomalies detected.", - })) - -# --- Tool 3: Dynamic tool list management --- - -@tool( - "Enable or disable tools at runtime", - title="Tool Manager", - destructive=True, -) -def manage_tools(action: str, tool_names: str) -> ToolResult: - names = [n.strip() for n in tool_names.split(",")] - log.info(f"manage_tools: action={action}, names={names}") - - if action == "enable": - active = tool_manager.enable(names) - elif action == "disable": - active = tool_manager.disable(names) - elif action == "list": - active = tool_manager.get_active_tools() - else: - return ToolResult( - result=f"Unknown action: {action}", - is_error=True, - error_code="INVALID_ACTION", - suggestion="Use 'enable', 'disable', or 'list'", - ) - - return ToolResult(result=json.dumps({"active_tools": active})) - -# --- Tool 4: Demonstrates error handling and logging levels --- - -@tool( - "Validate data against a schema (demonstrates error handling)", - title="Data Validator", - read_only=True, - idempotent=True, -) -def validate_data(data_json: str, strict: bool = False) -> ToolResult: - log.debug("Starting validation") - - try: - data = json.loads(data_json) - except json.JSONDecodeError as e: - log.error(f"Invalid JSON: {e}") - return ToolResult( - result=f"Invalid JSON: {e}", - is_error=True, - error_code="PARSE_ERROR", - message="The input is not valid JSON", - suggestion="Check for syntax errors and try again", - retryable=True, - ) - - issues = [] - if not isinstance(data, dict): - issues.append("Root must be an object") - elif "name" not in data: - issues.append("Missing required field: name") - - if strict and isinstance(data, dict): - allowed = {"name", "value", "tags"} - extra = set(data.keys()) - allowed - if extra: - issues.append(f"Unknown fields: {', '.join(extra)}") - - if issues: - log.warning(f"Validation failed: {issues}") - return ToolResult( - result=json.dumps({"valid": False, "issues": issues}), - is_error=True, - error_code="VALIDATION_FAILED", - ) - - log.info("Validation passed") - return ToolResult(result=json.dumps({"valid": True, "issues": []})) -``` - -- [ ] **Step 2: Verify it parses** - -Run: `python -c "import ast; ast.parse(open('examples/python/full_showcase.py').read()); print('OK')"` -Expected: `OK` - -- [ ] **Step 3: Commit** - -```bash -git add examples/python/full_showcase.py -git commit -m "examples: add full showcase Python example with structured output, dynamic tools, and error handling" -``` - ---- - -### Task 6: Full Showcase TypeScript Example - -**Files:** -- Create: `examples/typescript/full-showcase.ts` - -- [ ] **Step 1: Create the full showcase TypeScript example** - -```typescript -// examples/typescript/full-showcase.ts -// Full-featured protomcp demo — multiple tools showcasing the complete API. -// Run: pmcp dev examples/typescript/full-showcase.ts - -import { tool, ToolResult, ToolContext, toolManager, ServerLogger } from 'protomcp'; -import { z } from 'zod'; - -// Note: ServerLogger is wired to the MCP host transport automatically by the runner. -// For demonstration, we show how to create one — in practice, use the runner-provided instance. - -// --- Tool 1: Structured output with output schema --- - -const WeatherOutput = z.object({ - location: z.string(), - temperature_f: z.number(), - conditions: z.string(), - humidity: z.number(), -}); - -tool({ - name: 'get_weather', - description: 'Get current weather for a location', - title: 'Weather Lookup', - readOnlyHint: true, - output: WeatherOutput, - args: z.object({ location: z.string() }), - handler({ location }) { - const data = { - location, - temperature_f: 72.5, - conditions: 'Partly cloudy', - humidity: 45, - }; - return new ToolResult({ result: JSON.stringify(data) }); - }, -}); - -// --- Tool 2: Long-running operation with progress + task support --- - -tool({ - name: 'analyze_dataset', - description: 'Analyze a dataset (simulated long-running task)', - title: 'Dataset Analyzer', - idempotentHint: true, - taskSupport: true, - args: z.object({ - dataset_name: z.string(), - depth: z.enum(['basic', 'deep']).default('basic'), - }), - async handler({ dataset_name, depth }, ctx: ToolContext) { - const steps = 10; - for (let i = 0; i < steps; i++) { - if (ctx.isCancelled()) { - return new ToolResult({ - result: `Analysis cancelled at step ${i}/${steps}`, - isError: true, - errorCode: 'CANCELLED', - retryable: true, - }); - } - ctx.reportProgress(i, steps, `Analyzing step ${i + 1}/${steps}...`); - await new Promise(r => setTimeout(r, 100)); // Simulate work - } - ctx.reportProgress(steps, steps, 'Analysis complete'); - return new ToolResult({ - result: JSON.stringify({ - dataset: dataset_name, - depth, - rows_analyzed: 15000, - anomalies_found: 3, - summary: 'Dataset is healthy with 3 minor anomalies detected.', - }), - }); - }, -}); - -// --- Tool 3: Dynamic tool list management --- - -tool({ - name: 'manage_tools', - description: 'Enable or disable tools at runtime', - title: 'Tool Manager', - destructiveHint: true, - args: z.object({ - action: z.enum(['enable', 'disable', 'list']), - tool_names: z.string().describe('Comma-separated tool names'), - }), - async handler({ action, tool_names }) { - const names = tool_names.split(',').map(n => n.trim()); - let active: string[]; - switch (action) { - case 'enable': - active = await toolManager.enable(names); - break; - case 'disable': - active = await toolManager.disable(names); - break; - case 'list': - active = await toolManager.getActiveTools(); - break; - } - return new ToolResult({ result: JSON.stringify({ active_tools: active }) }); - }, -}); - -// --- Tool 4: Error handling, validation, and logging --- - -tool({ - name: 'validate_data', - description: 'Validate data against a schema (demonstrates error handling)', - title: 'Data Validator', - readOnlyHint: true, - idempotentHint: true, - args: z.object({ - data_json: z.string(), - strict: z.boolean().default(false), - }), - handler({ data_json, strict }) { - let data: unknown; - try { - data = JSON.parse(data_json); - } catch (e) { - return new ToolResult({ - result: `Invalid JSON: ${e}`, - isError: true, - errorCode: 'PARSE_ERROR', - message: 'The input is not valid JSON', - suggestion: 'Check for syntax errors and try again', - retryable: true, - }); - } - - const issues: string[] = []; - if (typeof data !== 'object' || data === null || Array.isArray(data)) { - issues.push('Root must be an object'); - } else { - if (!('name' in data)) issues.push('Missing required field: name'); - if (strict) { - const allowed = new Set(['name', 'value', 'tags']); - const extra = Object.keys(data).filter(k => !allowed.has(k)); - if (extra.length) issues.push(`Unknown fields: ${extra.join(', ')}`); - } - } - - if (issues.length) { - return new ToolResult({ - result: JSON.stringify({ valid: false, issues }), - isError: true, - errorCode: 'VALIDATION_FAILED', - }); - } - - return new ToolResult({ result: JSON.stringify({ valid: true, issues: [] }) }); - }, -}); -``` - -- [ ] **Step 2: Commit** - -```bash -git add examples/typescript/full-showcase.ts -git commit -m "examples: add full showcase TypeScript example with structured output, dynamic tools, and error handling" -``` - ---- - -### Task 7: Example Dependency Files - -**Files:** -- Create: `examples/python/requirements.txt` -- Create: `examples/typescript/package.json` -- Create: `examples/typescript/tsconfig.json` - -- [ ] **Step 1: Create Python requirements.txt** - -``` -protomcp -``` - -- [ ] **Step 2: Create TypeScript package.json** - -```json -{ - "name": "protomcp-examples", - "private": true, - "type": "module", - "dependencies": { - "protomcp": "file:../../sdk/typescript", - "zod": "^3.22.0" - } -} -``` - -Note: Uses `file:` reference to the local SDK so examples work without publishing to npm. - -- [ ] **Step 3: Create TypeScript tsconfig.json** - -```json -{ - "compilerOptions": { - "target": "ES2022", - "module": "ES2022", - "moduleResolution": "node", - "esModuleInterop": true, - "strict": true, - "outDir": "dist" - }, - "include": ["*.ts"] -} -``` - -- [ ] **Step 4: Commit** - -```bash -git add examples/python/requirements.txt examples/typescript/package.json examples/typescript/tsconfig.json -git commit -m "examples: add dependency files for Python and TypeScript examples" -``` - ---- - -### Task 8: Demo Runner Script - -**Files:** -- Create: `examples/run-demo.sh` - -- [ ] **Step 1: Create the demo runner script** - -This script starts `pmcp dev` for each example, sends JSON-RPC messages over stdio, and prints human-readable output. It performs the full MCP handshake: `initialize` → `initialized` notification → `tools/list` → `tools/call`. - -```bash -#!/usr/bin/env bash -set -euo pipefail - -# examples/run-demo.sh -# Runs each example through pmcp, demonstrating the MCP protocol interaction. -# Requires: pmcp installed and on PATH - -RED='\033[0;31m' -GREEN='\033[0;32m' -CYAN='\033[0;36m' -YELLOW='\033[1;33m' -NC='\033[0m' - -SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)" -PASSED=0 -FAILED=0 - -if ! command -v pmcp &> /dev/null; then - echo -e "${RED}Error: pmcp not found. Install it first: brew install protomcp/tap/protomcp${NC}" - exit 1 -fi - -# Send a JSON-RPC request to a pmcp process and capture the response. -# Usage: run_example