From 6f0d95c6da232624418e46f2fd8ba457900745fd Mon Sep 17 00:00:00 2001 From: Nikita Nemirovsky Date: Mon, 18 May 2026 13:04:28 +0800 Subject: [PATCH] docs: condense CLAUDE.md without dropping facts Reduce CLAUDE.md from 56k to 40k chars by tightening prose, merging short paragraphs, de-duplicating repeated invariants, and collapsing illustrative tables into compact form. No technical facts, CLI commands, key-file references, or invariants removed. --- CLAUDE.md | 311 +++++++++++++++++++----------------------------------- 1 file changed, 110 insertions(+), 201 deletions(-) diff --git a/CLAUDE.md b/CLAUDE.md index f66ec25..47be431 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -1,6 +1,6 @@ # Sluice - CLAUDE.md -Credential-injecting approval proxy for AI agents. Two layers of governance: MCP-level (semantic tool control) and network-level (all-protocol interception). Asks for human approval via Telegram, injects credentials, and forwards. +Credential-injecting approval proxy for AI agents. Two governance layers: MCP-level (semantic tool control) and network-level (all-protocol interception). Asks for human approval via Telegram, injects credentials, forwards. ## Build and Test @@ -11,69 +11,50 @@ go test ./... -v -timeout 30s ## E2e Tests -End-to-end tests live in `e2e/` and use build tags. They start a real sluice binary, configure policies, make connections through the proxy, and verify credential injection, MCP gateway flows, and audit log integrity. Protocol coverage: HTTP/HTTPS, SSH, MCP, WebSocket, gRPC, QUIC/HTTP3, DNS, and IMAP/SMTP. +E2e tests in `e2e/` use build tags. They start a real sluice binary, configure policies, connect through the proxy, and verify credential injection, MCP gateway flows, and audit-log integrity across HTTP/HTTPS, SSH, MCP, WebSocket, gRPC, QUIC/HTTP3, DNS, IMAP/SMTP. -Build tags: -- `e2e` -- required for all e2e tests -- `e2e && linux` -- Docker compose integration tests -- `e2e && darwin` -- Apple Container tests (macOS only) +Build tags: `e2e` (all), `e2e && linux` (Docker compose integration), `e2e && darwin` (Apple Container, macOS only). ```bash -make test-e2e # run all e2e tests locally -make test-e2e-docker # run Linux e2e tests via Docker Compose -make test-e2e-macos # run macOS e2e tests (Apple Container) +make test-e2e # all e2e tests locally +make test-e2e-docker # Linux e2e via Docker Compose +make test-e2e-macos # macOS e2e (Apple Container) ``` -Or directly: -```bash -go test -tags=e2e ./e2e/ -v -count=1 -timeout=300s -go test -tags="e2e linux" ./e2e/ -v -count=1 -timeout=300s -go test -tags="e2e darwin" ./e2e/ -v -count=1 -timeout=300s -``` +Direct: `go test -tags=e2e ./e2e/ -v -count=1 -timeout=300s` (add `linux` or `darwin` to the tag list for the platform-specific suites). -CI runs e2e tests via `.github/workflows/e2e-linux.yml` and `.github/workflows/e2e-macos.yml`. +CI runs e2e via `.github/workflows/e2e-linux.yml` and `e2e-macos.yml`. ## Releases -Use the `release-tools:new` skill (`/release-tools:new`) to cut a new release. The skill handles version calculation, tag push, and description prompt. - -**Naming:** -- Tag: `vX.Y.Z` (e.g. `v0.10.0`) -- Release title: same as tag (`v0.10.0`), NOT `Version X.Y.Z` +Use the `release-tools:new` skill; it handles version calc, tag push, description prompt. **Naming:** tag `vX.Y.Z` (e.g. `v0.10.0`); release title same as tag, NOT `Version X.Y.Z`. **Version selection:** -- **Minor** (`v0.9.0` -> `v0.10.0`): default for most releases. Use when a merged PR adds `feat` commits, new CLI flags, new protocol support, or user-visible behavior changes. -- **Hotfix** (`v0.10.0` -> `v0.10.1`): only when a PR contains `fix` commits exclusively (no feat, no breaking changes). Common case: CI-only regressions, test fixes shipped alongside a real bug fix, flakiness fixes. -- **Major** (`v0.10.0` -> `v1.0.0`): breaking changes to CLI flags, SQLite schema, policy TOML format, or MCP gateway API. **Always discuss with the user before a major bump.** Never pick `major` autonomously. +- **Minor** (`v0.9.0`->`v0.10.0`): default. PR adds `feat`, new CLI flags, new protocol, or user-visible behavior change. +- **Hotfix** (`v0.10.0`->`v0.10.1`): PR has `fix` commits exclusively (no feat/breaking) — CI-only regressions, flakiness fixes, test fixes alongside a real bug fix. +- **Major** (`v0.10.0`->`v1.0.0`): breaking changes to CLI flags, SQLite schema, policy TOML, or MCP gateway API. **Always discuss with the user; never pick `major` autonomously.** -**Skip releases for:** `chore`, `docs`, `ci`, `test` only PRs (see `feedback_tag_policy.md` memory). +**Skip releases for** `chore`/`docs`/`ci`/`test`-only PRs (see `feedback_tag_policy.md`). -**Release workflow (goreleaser):** Pushing a `v*` tag triggers `.github/workflows/release.yml` which uses goreleaser to build Linux/darwin binaries and upload them to the release. The workflow uploads binaries even if the release was pre-created with a description, so write the description first via `gh release edit` (or let the skill do it), then push the tag. +**Goreleaser workflow:** pushing a `v*` tag triggers `.github/workflows/release.yml` (goreleaser builds + uploads Linux/darwin binaries). Binaries upload even if the release was pre-created, so write the description first via `gh release edit`, then push the tag. -**Cross-repo references in release notes:** Bare `#number` gets auto-linked to this repo's issue tracker, which is wrong for PRs in other repos. Use an explicit markdown link with `owner/repo#number` as the link text: `[lqqyt2423/go-mitmproxy#100](https://github.com/lqqyt2423/go-mitmproxy/pull/100)`. Don't add any prose prefix like "go-mitmproxy PR" before it. +**Cross-repo refs in release notes:** bare `#number` auto-links to this repo's tracker (wrong for other repos' PRs). Use an explicit markdown link with `owner/repo#number` as link text: `[lqqyt2423/go-mitmproxy#100](https://github.com/lqqyt2423/go-mitmproxy/pull/100)`, no prose prefix. ## System Architecture -Two governance layers work together: +**Layer 1: MCP Gateway** — between agent and MCP tool servers. Sees tool names, args, responses; catches local tools (filesystem, exec) that never hit the network. -**Layer 1: MCP Gateway** sits between agent and MCP tool servers. Sees tool names, arguments, responses. Catches local tools (filesystem, exec) that never hit the network. +**Layer 2: SOCKS5 Proxy** — between container and internet. Sees every TCP/UDP connection, injects credentials at network level, catches anything bypassing MCP. -**Layer 2: SOCKS5 Proxy** sits between container and internet. Sees every TCP/UDP connection. Injects credentials at the network level. Catches anything that bypasses MCP. - -| Scenario | MCP Gateway | SOCKS5 Proxy | -|----------|-------------|--------------| -| `github__delete_repository` | Sees tool name + args, can block | Sees `api.github.com:443` only | -| `filesystem__write_file` | Sees path + content, can block | Invisible (no network) | -| Raw `fetch("https://evil.com")` | Bypasses MCP entirely | Catches it | -| SSH connection | Not an MCP call | Sees `github.com:22` | +Complementary coverage: `filesystem__write_file` — MCP sees path+content (can block), invisible to SOCKS5; raw `fetch("https://evil.com")` — bypasses MCP, SOCKS5 catches it; `github__delete_repository` — MCP sees tool+args, SOCKS5 sees only `api.github.com:443`. ### Components | Component | Role | |-----------|------| | **OpenClaw** | AI agent, no real credentials (Docker/Apple Container/tart VM) | -| **tun2proxy** | Routes ALL TCP + UDP through TUN to SOCKS5 | -| **Sluice SOCKS5 Proxy** | Network-level policy + MITM + credential injection | +| **tun2proxy** | Routes ALL TCP+UDP through TUN to SOCKS5 | +| **Sluice SOCKS5 Proxy** | Network policy + MITM + credential injection | | **Sluice MCP Gateway** | Semantic tool governance (stdio + HTTP + WebSocket) | | **Sluice Telegram Bot** | Approval UX + credential/policy management | | **Vault** | Encrypted credential storage + phantom token mapping | @@ -108,155 +89,121 @@ sluice cert generate # generate CA certificate for HTTPS MITM sluice audit verify # verify audit log hash chain integrity ``` -When `--destination` is provided, `sluice cred add` also creates an allow rule and binding in the store. The flag may be repeated to create multiple bindings that share the same `--ports`, `--header`, and `--template` values (use `sluice binding add` afterwards for per-destination customization). When `--env-var` is provided, the phantom token is injected into the agent container as that environment variable via `docker exec` (no shared volume needed). For HTTP/WebSocket upstreams, `--command` holds the URL. Env values prefixed with `vault:` are resolved from the credential vault at upstream spawn time. - -Two credential types: `static` (default) for API keys and `oauth` for OAuth access/refresh token pairs. OAuth credentials prompt for tokens via stdin (not CLI flags) to avoid shell history exposure. +`sluice cred add --destination` also creates an allow rule and binding; repeat the flag for multiple bindings sharing `--ports`/`--header`/`--template` (use `sluice binding add` for per-destination customization). `--env-var` injects the phantom into the agent container as that env var via `docker exec`. For HTTP/WebSocket upstreams `--command` holds the URL; env values prefixed `vault:` resolve from the vault at upstream spawn time. Two credential types: `static` (default, API keys), `oauth` (access/refresh pairs); OAuth tokens prompt via stdin (not flags) to avoid shell-history exposure. -`sluice cred update` uses PATCH partial-update semantics for OAuth credentials. Pressing Enter at the refresh token prompt (or omitting the second line when piping via stdin) preserves the currently stored refresh token. To explicitly clear the refresh token, update the credential again with the desired empty value through the REST API (`PATCH /api/credentials/` with `"refresh_token": ""`). This prevents an access-token rotation from silently destroying the stored refresh token. +`sluice cred update` is PATCH partial-update for OAuth: Enter at the refresh-token prompt (or omit the second piped line) preserves the stored refresh token (prevents an access-token rotation silently destroying it). To clear it, `PATCH /api/credentials/` with `"refresh_token": ""`. -`sluice binding update --destination` also updates the paired auto-created allow rule (tagged `binding-add:` or `cred-add:`) so the new destination is not orphaned. If no paired rule exists (e.g. because it was manually removed), the binding destination is still updated and a warning is printed. No fallback rule is created so an operator's intentional removal is not silently reverted. `--env-var` on binding update can be used to change or clear the env var name after the initial binding was created. +`sluice binding update --destination` also updates the paired auto-created allow rule (tagged `binding-add:` or `cred-add:`) so the new destination isn't orphaned. If no paired rule exists, the binding still updates with a warning and no fallback rule is created (don't silently revert an operator's removal). `--env-var` here changes/clears the env var name. -Runtime flags: `--mcp-base-url` sets the external URL the agent uses to reach sluice's MCP gateway (e.g. `http://sluice:3000`). This is added to `SelfBypass` so sluice does not policy-check its own MCP traffic. Defaults to deriving from `--health-addr`. `--agent ` selects an agent profile (`openclaw`, `hermes`); the profile controls the env file path inside the container, the secrets-reload mechanism, and the MCP wiring command. The default is `openclaw`. May also be set via `SLUICE_AGENT_PROFILE`. +Runtime flags: `--mcp-base-url` sets the external URL the agent uses to reach sluice's MCP gateway (e.g. `http://sluice:3000`); added to `SelfBypass`; defaults to deriving from `--health-addr`. `--agent ` selects the agent profile (`openclaw`/`hermes`) controlling env-file path, secrets-reload, MCP wiring; default `openclaw`; also `SLUICE_AGENT_PROFILE`. ## Channel Feature Parity -Sluice exposes management surfaces over multiple channels: the **CLI**, the **HTTP/REST API** (`api/openapi.yaml`), the **Telegram bot**, and any future channel. **Every store-backed management feature must be reachable from all channels.** When you add or change a feature that reads or writes the SQLite store (policy rules, credentials, bindings, MCP upstreams, credential pools, config/default-verdict, …), implement it on **CLI and REST and Telegram**, not just one. +Management surfaces: **CLI**, **HTTP/REST API** (`api/openapi.yaml`), **Telegram bot**, future channels. **Every store-backed management feature must be reachable from all channels** (policy rules, credentials, bindings, MCP upstreams, pools, config/default-verdict — on CLI **and** REST **and** Telegram). Mechanism: put operation logic in a **channel-agnostic package** (the `store` methods, or a small `*ops` package like `internal/poolops`); each channel is a thin adapter. Logic written inline in one channel (the historical pools-CLI-only gap) is the anti-pattern. -The mechanism for keeping them honest: put the operation logic in a **channel-agnostic package** (e.g. the `store` methods, or a small `*ops` package like `internal/poolops`) and have each channel be a thin adapter over it. Logic written inline in one channel (the historical cause of the pools-are-CLI-only gap — rotate/status lived in `cmd/sluice/pool.go`) is the anti-pattern; lift it so the channels cannot drift. - -The **only** acceptable single-channel features are those with a documented rationale that makes them meaningless elsewhere — e.g. `sluice cert generate` / `sluice audit verify` are local-filesystem/operator tools with no remote-management semantics; OAuth token entry is stdin-only on the CLI specifically to keep secrets out of shell history and out of the REST/Telegram surfaces. State the rationale in the code and docs when you deliberately scope a feature to one channel; absent that rationale, parity is required and a reviewer should block the PR. +The only acceptable single-channel features have a documented rationale making them meaningless elsewhere — `sluice cert generate` / `sluice audit verify` are local-filesystem operator tools; OAuth token entry is stdin-only to keep secrets out of shell history and the REST/Telegram surfaces. State the rationale in code and docs; absent it, parity is required and a reviewer should block the PR. ## Agent Profiles -Profiles abstract per-agent runtime conventions so sluice's container managers stay agent-agnostic. Each profile carries `EnvFileRelPath` (where to write phantom-token env vars), `ReloadCmd` (argv to exec for in-place secret reload, or nil), and `WireMCPCmd` (argv to register sluice as an MCP server in the agent's config). +Profiles abstract per-agent runtime conventions so container managers stay agent-agnostic. Each carries `EnvFileRelPath` (phantom env-var path), `ReloadCmd` (argv for in-place secret reload, or nil), `WireMCPCmd` (argv to register sluice as an MCP server). | Profile | Env file | Reload | MCP wiring | |---------|----------|--------|------------| | `openclaw` (default) | `~/.openclaw/.env` | `node -e secrets.reload` over the agent's WebSocket gateway | `node -e wire-mcp ` patches `mcp.servers.` | -| `hermes` | `~/.hermes/.env` | None — Hermes has no documented in-place reload; new env values take effect on next message / restart | `sh -c '[ -f /opt/hermes/.venv/bin/activate ] && . /opt/hermes/.venv/bin/activate; exec python3 -c