CatchAttack v2: AI-native MCP-centric detection platform rewrite#108
Merged
Conversation
…eleton
Per BUILD_BRIEF.md Phase 0 + BUILD_BRIEF_ADDENDUM.md.
- Move services/, contracts/, supabase/, ops/, backend/, docker/,
docker-compose.dev.yml, the Vite/React SPA (src/, public/, vite/tailwind
configs, package.json), the standalone frontend/, and the legacy
docs/test artefacts into legacy/. Quarantined verbatim via git mv so
history follows. legacy/ is excluded from ruff, biome, mypy, and CI.
- Scaffold the v2 tree: apps/{web,conductor}, mcp/, mcp-proxy/, agent/,
packages/{schemas,ui}, infra/, detections/, docs/adr/. Each directory
has a README explaining its phase boundary.
- Workspace managers: pnpm (TS) + uv (Python). Root pyproject and
package.json set up minimal workspaces; members added as their
pyproject.toml files land per phase.
- Add Makefile, ruff.toml, mypy.ini, biome.json, .editorconfig, .env.example.
- Add mcp-proxy/ Phase 0 skeleton: ProxyConfig + PolicyEngine
(dry-run + target-allowlist + approval-token enforcement) + append-only
JSONL AuditLog with secret redaction. FastAPI app exposes /health and
/policy/preview; /mcp passthrough is 501 until Phase 1 wires the
Streamable HTTP transport. 13 tests pass; mypy --strict clean.
- Replace legacy GitHub Actions (renamed *.disabled) with ci.yml running
ruff + biome + pytest + mypy on push/PR.
- Add ADR-0001 capturing quarantine + monorepo + proxy decisions.
make verify = uv sync + pnpm install + ruff format --check + biome check.
Phase 0 contract satisfied locally.
Phase: 0
Closes Phase 1 of BUILD_BRIEF.md (sigma MCP + Claude Desktop hookup) and
the Phase 0/1 portion of the addendum's mandatory proxy routing.
mcp/sigma — new in-house MCP server
- Four strict-schema tools: parse_sigma, lint_sigma, convert_sigma,
dedupe_against_corpus. All inputs use additionalProperties=false.
- Five conversion targets: splunk (SPL), sentinel (KQL), chronicle
(Google SecOps UDM-search via pysigma-backend-secops), elastic
(Lucene), falcon (CrowdStrike LogScale). Lazy backend imports so a
missing wheel doesn't sink the whole module.
- Linter combines pySigma schema validity with style checks (UUID/id,
ATT&CK technique tag presence, allowed level/status, title length).
- Dedupe combines AST jaccard overlap with a pluggable text embedder:
HashEmbedder (zero-dep, deterministic blake2s) by default;
SentenceTransformersEmbedder (all-MiniLM-L6-v2) via the embeddings
extra. DedupeReport surfaces which embedder produced each score.
- sigma://corpus/{rule_id} read-only resource + sigma_review prompt.
- stdio (Claude Desktop) and streamable-http (proxy) transports.
- 35 tests cover each tool's happy path + validation failures + the
end-to-end MCP surface via fastmcp's in-memory client.
- sample rule: detections/enterprise/windows/execution/
win_susp_powershell_encoded_b64.yml (the Phase 1 demo target).
mcp-proxy — routing layer wired up
- New router.py builds a FastMCP server that mounts each non-stub
upstream under its namespace via fastmcp's mount(). Translates
fastmcp's underscore prefix (`sigma_lint_sigma`) back to the brief's
dotted form (`sigma.lint_sigma`) for policy + audit alignment.
- PolicyMiddleware enforces the Phase 0 dry-run + target-allowlist
+ approval-token rules before forwarding any tools/call, and
records every decision (allow/deny) in the JSONL audit log.
- The router's streamable-HTTP app is mounted at /mcp inside the
existing FastAPI app. /health and /policy/preview keep working.
- 3 integration tests drive the router with a mock upstream FastMCP:
non-destructive happy path, denied destructive call, allowlisted
destructive call with dry_run forwards through.
- upstreams.example.yaml registers sigma in stdio mode pointing at
`uv run sigma-mcp`.
infra
- Add mcp/sigma to uv workspace members.
- Makefile verify now runs install + fmt-check + lint + pytest +
mypy across mcp-proxy and mcp/sigma; pytest is run from each
project directory so our `mcp/` folder doesn't shadow the
installed mcp SDK package on sys.path.
- CI workflow syncs --all-packages and runs each project's pytest +
mypy.
- mypy overrides for sigma.*, fastmcp.*, sentence_transformers
(no stubs / heavyweight optional extra).
docs
- ADR-0002 records the FastMCP-mount routing decision and the
underscore↔dot namespace translation.
- ADR-0003 records the pluggable-embedder decision and rationale
for keeping MiniLM optional.
- mcp/sigma/README.md includes the Claude Desktop config snippet
for the Phase 1 demo.
Phase 1 contract met:
- make verify → 51 tests pass (16 mcp-proxy + 35 mcp/sigma);
mypy --strict clean on 14 source files; ruff clean; biome clean.
Phase: 1
Closes Phase 2 of BUILD_BRIEF.md plus the addendum §B.2 mock-or-real
strategy and the §B.1 no-license Wazuh fallback.
mcp/mocks/splunk — first concrete instance of the mock-upstream pattern
- FastMCP server exposing the brief's four tools: search,
list_saved_searches, deploy_rule, estimate_fp_rate. Tool shapes mirror
Splunk REST responses so flipping mock→real (Splunkbase App 7931 inside
Splunk's mgmt:8089) requires no Conductor or UI changes.
- Deterministic synthetic event store: 12 hosts, ~3.5k events/week, 50
seeded saved searches. Tiny SPL→regex translator covers bare tokens
and field=value forms.
- 14 tests: store unit tests + 5 end-to-end MCP tests via fastmcp Client.
mcp/wazuh — in-house Wazuh MCP (no upstream exists)
- WazuhClient wraps Manager API (port 55000, JWT) and Indexer search
(port 9200, OpenSearch query DSL). Pluggable httpx transports so tests
run offline.
- Tools: search (indexer), list_rules (manager), deploy_rule (renders
XML, PUTs to /rules/files/{filename} when dry_run=false; reports
requires_manager_restart=true), estimate_fp_rate.
- WazuhRuleSpec restricts rule_id to the 100000–199999 local range and
level 0–15, with match_text/pcre2/if_sid options. XML escaping is
applied to all user-supplied strings.
- 18 tests: client unit tests (httpx.MockTransport for both ports),
XML rendering tests, end-to-end MCP tests.
mcp-proxy
- upstreams.example.yaml registers splunk (stdio→splunk-mock) and wazuh
(stdio→wazuh-mcp), both with mock_url/real_cmd switching ready for
flip mock→real. wazuh.deploy_rule added to destructive_tools with a
filename allowlist (local_rules.xml, lab_rules.xml).
infra
- compose.yaml: wazuh-indexer + wazuh-manager (single-node), mcp-proxy
service via Dockerfile.proxy, optional splunk service behind a
`splunk` profile (operator opt-in for EULA).
- Dockerfile.proxy: python:3.12-slim with all in-tree MCP servers
installed via `uv sync --all-packages --frozen`.
apps/conductor/prompts/system_v1.md
- v1 prompt skeleton mirroring addendum §D plus a Phase 2 fragment
defining the sigma_to_siem workflow (parse → lint → dedupe gate →
convert → estimate_fp → dry-run deploy → report). The full Phase 4
closed-loop workflow is stubbed but unwired.
scripts/sigma_pre_commit.py + .pre-commit-config.yaml
- Pre-commit hook runs mcp/sigma's lint_sigma over any changed
detections/**/*.y*ml. Schema errors block the commit; style warnings
surface but pass.
infra/Makefile + CI
- PY_PACKAGES variable iterates over all four Python projects for
pytest + mypy. CI adds steps for splunk-mock and wazuh plus the
pre-commit smoke check.
- Makefile fmt-check/lint now properly fail when ruff finds issues
(previously the `||` swallowed failures behind a misleading "ruff
not installed" message).
docs
- ADR-0004 records the mock-or-real strategy: per-vendor mocks at
mcp/mocks/<vendor>/ with REST-shaped pydantic models, fastmcp tool
surfaces matching the official server, deterministic synthetic data;
flip via upstreams.<vendor>.mode.
- mcp/mocks/splunk/README.md and mcp/wazuh/README.md document tool
surfaces, env vars, and the mock→real flip path.
Phase 2 contract met:
- make verify → 83 tests pass (16 mcp-proxy + 35 mcp/sigma + 14
mcp/mocks/splunk + 18 mcp/wazuh); mypy --strict clean on 23 source
files; ruff clean; biome clean; sigma pre-commit hook green.
Phase: 2
Closes Phase 3 of BUILD_BRIEF.md + addendum §E (capture-bundle schema and
evidence MCP surface).
agent/ — cross-platform Go endpoint agent (Go 1.24)
- cobra CLI: enroll, run, inventory, atomic, capture-spawn.
- internal/inventory: gopsutil-based cross-platform host facts +
conservative EDR detection heuristic (Falcon, Defender, Elastic
Defend, S1) over process names.
- internal/capture: pure FFmpegArgs() builds HLS-output ffmpeg argv for
gdigrab (Windows), x11grab (Linux), avfoundation (macOS). Spawn()
shells out via exec.CommandContext.
- internal/atomic: pure Argv() builds Invoke-AtomicTest (Windows) or
atomic-runner (Linux/macOS) command lines. Run() honours --dry-run
and accepts an OverrideRun injectable for tests.
- internal/enroll: persists ca/cert/key/agent_id/server under
--cert-dir; exchange function is injectable. Real gRPC exchange
lands in Phase 4.
- proto/agent.proto + internal/gen/agentv1: AgentBridge service with
Connect bidi stream + Enroll one-shot. Generated Go bindings
committed so CI doesn't need protoc.
- .goreleaser.yaml: windows/amd64, linux/amd64, darwin/arm64.
- 13 Go tests; all pure, no shell-out, run on any platform.
packages/schemas/capture_bundle.schema.json — addendum §E.1 verbatim
- Single source of truth for capture-bundle shape.
- Pydantic models in mcp/evidence/src/evidence_mcp/models.py mirror it.
mcp/evidence — addendum §E.2 implementation
- 7 strict-schema tools: get_capture, list_captures, summarize_capture,
query_events, get_artifact_url, add_marker, count_detection_hits.
- FilesystemStorage rooted at $EVIDENCE_MCP_ROOT. Layout mirrors §E.4
S3 layout so Phase 5 S3Storage drops in behind the same interface.
- get_artifact_url returns file:// in Phase 3, presigned S3 later.
- query_events hard-capped at 500 rows server-side per the addendum.
- summarize_capture computed at put_bundle time; cached on disk.
- 17 tests covering storage round-trips and the full MCP surface.
mcp/agents — bridge between proxy and Go agent fleet
- 5 tools: list_agents, get_inventory, run_atomic, start_capture,
stop_capture.
- AgentTransport Protocol with two implementations:
InMemoryAgentTransport.with_seed() — lab-linux-01 + lab-win-01,
used in tests and Phase 3 demos.
GRPCAgentTransport — Phase 4 stub, returns NotImplementedError.
- Toggle via --mode memory|grpc or AGENTS_TRANSPORT env.
- 13 tests covering transport behaviour + the full MCP surface.
mcp-proxy upstreams
- evidence and agents flipped from "stub" to stdio routing
(uv run evidence-mcp / uv run agents-mcp).
- evidence.add_marker added to destructive_tools (proxy enforces
dry-run + approval policy).
infra/compose.yaml
- Added MinIO service (ports 9000 + 9001) for capture-bundle object
storage. Phase 3 evidence MCP defaults to filesystem; Phase 5 flips
to S3 with the same credentials.
Tooling / docs
- Makefile: PY_PACKAGES now includes mcp/evidence + mcp/agents; new
test-go target runs `go test ./agent/...`; verify orchestrates all
three.
- CI: pytest + mypy steps for mcp/evidence and mcp/agents, plus a new
Go job (vet + test + golangci-lint).
- ADR-0005 records all Phase 3 architecture decisions: pure-argv
separation, pluggable transports/storage, agent-initiated bidi
stream, committed protoc output.
- READMEs for agent/, mcp/evidence/, mcp/agents/.
Phase 3 contract met:
- make verify → 113 tests pass (16 mcp-proxy + 35 mcp/sigma +
14 mcp/mocks/splunk + 18 mcp/wazuh + 17 mcp/evidence + 13 mcp/agents);
mypy --strict clean on 31 source files; 13 Go tests pass; ruff +
golangci-lint clean.
Phase: 3
Closes Phase 4 of BUILD_BRIEF.md — the flagship closed-loop workflow plus
the FastAPI surface to drive it.
apps/conductor/ — server-side AI Conductor
- FastAPI app with the brief's Phase 4 endpoints:
POST /workflows/{name}/run queue a workflow
GET /workflows/runs/{id} full run state
GET /workflows/runs/{id}/sse live StepEvent stream
GET /workflows /health
- In-memory RunRegistry; each Run has an asyncio.Queue of StepEvents that
the SSE endpoint streams via sse-starlette. Phase 5 swaps the registry
for Postgres without changing the contract.
closed_loop_rule_synthesis (11-step workflow)
- 1. agents.list_agents → confirm agent connected
2. agents.run_atomic → first emulation (capture_id_1)
3. evidence.summarize_capture → must have ≥1 notable marker
4. llm.draft_sigma_rule → drafts Sigma YAML
5. sigma.lint_sigma + sigma.dedupe_against_corpus
6. sigma.convert_sigma(target=splunk|wazuh)
7. {splunk|wazuh}.estimate_fp_rate → p95 < fp_threshold
8. {splunk|wazuh}.deploy_rule(dry_run=true)
9. agents.run_atomic → second emulation (capture_id_2)
10. {splunk|wazuh}.search → must return ≥1 hit
11. pr.open → branch + report + (optional) GH PR
- Each gate failure raises GateFailedError with a named code
(preflight.unknown_agent, evidence.empty, lint.errors,
dedupe.near_duplicate, fp.too_high, validation.no_hits, …) so the run
record and UI can map to remediation hints.
- The Conductor NEVER calls deploy_rule(dry_run=false). Promotion is
human-only.
Clients
- FastMCPClient: dotted ↔ underscored namespace translation
(sigma.lint_sigma ↔ sigma_lint_sigma); payload normalisation across
fastmcp's response shapes.
- AnthropicLLM: claude-opus-4-7, prompt-cached system message, narrow
draft_sigma_rule() surface.
- StaticMCPClient + StaticLLM: deterministic test fakes with
respond_to(tool, params_subset, result=…) routing.
- LocalBranchPROpener: writes rule + report + spl + fp_report +
reasoning under detections/_meta/conductor_runs/<short_id>/, commits
to conductor/<slug>-<short_id> branch.
- GitHubMCPPROpener: layers github.create_pull_request via the MCP
proxy on top of the local opener. Falls back to local-only on failure.
Tests (18 conductor tests)
- test_clients: namespace translation, params-subset dispatch, YAML
extraction.
- test_workflow_closed_loop: happy path + every gate failure
(preflight.unknown_agent, evidence.empty, lint.errors,
dedupe.near_duplicate, fp.too_high, validation.no_hits).
- test_workflow_integration: composes the in-tree sigma + splunk-mock +
agents + evidence servers into a single FastMCP router, drives the
full workflow against it without any mocks at the MCP boundary. The
agents transport's run_atomic is hooked to materialise real
CaptureBundle manifests on each call so the downstream tools have
data.
- test_api: FastAPI surface smoke (queue, fetch, 404 on unknown
workflow).
Tooling
- pyproject.toml: apps/conductor added to uv workspace members.
- Makefile: PY_PACKAGES includes apps/conductor; verify message updated.
- CI: pytest + mypy steps for apps/conductor.
- ADR-0006: documents the 11-step linear shape, named gate failures,
pluggable LLM + PR clients, and the in-process integration test.
Phase 4 contract met:
- make verify → 131 Python tests pass (16 + 35 + 14 + 18 + 17 + 13 + 18);
13 Go tests pass; mypy --strict clean on 42 source files;
ruff + golangci-lint + biome clean.
Phase: 4
Closes Phase 5 of BUILD_BRIEF.md — the operator-facing UI.
apps/web/ — Next.js 15 / App Router / React 19 / Tailwind v4 / Auth.js v5
- Six brief-mandated routes:
/coverage MITRE matrix coloured by rule count × validation
/captures list capture bundles (evidence.list_captures)
/captures/[id] HLS player + SVG timeline with marker lanes
/rules DAC browser walking detections/
/rules/prs Conductor PR queue with reasoning + SPL + FP report
/agents fleet view (agents.list_agents)
/runs workflow registry
/runs/[id] per-run view with live SSE event tail
- Server components by default; two client components:
CapturePlayer (lazy hls.js + native Safari fallback)
RunTail (EventSource subscriber)
- /api/runs/[id]/sse proxies the Conductor's SSE stream so the browser
never knows the Conductor URL.
- Auth.js v5 with three modes via AUTH_MODE:
dev — fixed operator identity, no real auth (default)
github — GitHub OIDC
email — magic-link (Phase 6 wiring)
- Design tokens in @theme {}: attacker oklch(60% 0.18 25), defender
oklch(60% 0.15 240), warn, ok. Dark by default.
Server-side data layer (src/lib/)
- conductor.ts: typed fetch around /workflows/* + SSE proxy.
- mcp.ts: 80-line JSON-RPC client for /mcp; translates dotted ↔
underscored tool names. Avoids bundling @modelcontextprotocol/sdk.
- mitre.ts: 14 enterprise tactics + ~120 representative techniques as a
seeded table; filesystem coverage lookup derived from detections/.
- rules.ts: Sigma-rule walker for the DAC browser.
- auth.ts: NextAuth config with the three modes.
Capture timeline — SVG, server-rendered
- Five lanes: atomic steps, detection hits, process spawns, sysmon/
network, operator notes.
- Red 5-point stars for attack markers; blue circles for detection hits.
filled vs hollow encodes validation status per BUILD_BRIEF.md §5.
Tests
- tests/routes.spec.ts: Playwright suite asserting every brief-mandated
route renders + landmark text. Runs against `pnpm start` (production
build). Conductor and MCP proxy NOT running during the test; empty
states are part of what's tested.
- 6 Playwright tests pass in ~5s.
Tooling
- pnpm-workspace already covered apps/*.
- Makefile: test-ts now runs typecheck + build + Playwright in apps/web.
verify message bumped to Phase 5.
- CI: typescript job adds web typecheck, build, and Playwright
(chromium-only, --with-deps install).
- biome ignores .next, next-env.d.ts, playwright-report, test-results.
- .gitignore: web build artefacts.
ADR-0007 documents the stack pin (Next 15.5 + React 19 + Tailwind v4 +
Auth.js v5), server-component-by-default policy, SSE proxy decision,
graceful empty states, design tokens via @theme, and rejected
alternatives (full MCP SDK, client-side SSE, create-next-app,
cytoscape in this phase).
Phase 5 contract met:
- make verify → 131 Python + 13 Go + 6 Playwright = 150 tests pass;
mypy --strict clean on 42 source files; ruff + golangci-lint + biome
clean; Next.js production build green; route chunks 137-744 B, shared
102 KiB.
Phase: 5
Closes Phase 6 of BUILD_BRIEF.md — watch an emulation live, then auto-
redirect to the recorded view.
agent/internal/livekit/ — Go LiveKit publisher
- RoomName(run_id) = "run-<id>"; TrackName(agent_id) = "screen-<id>" —
identical formulas to the Conductor's Python side.
- PublisherToken mints a JWT (roomJoin + canPublish, canSubscribe=false)
via livekit/protocol/auth — the agent can publish its screen track and
nothing else.
- Publisher.Connect joins the room; PublishH264 is a clearly-marked
operator integration point (real WebRTC needs a display + media
server, so CI exercises naming + token claims only).
- `agent live-token` sub-command mints a publisher token from flags or
$LIVEKIT_* env.
- agent/config.toml adds the [livekit] feature-flag block.
- 7 Go tests: room/track naming, config validation, JWT claim structure.
apps/conductor — LiveKit token minting + live-marker hub
- livekit.py: mint_viewer_token issues a subscribe-only JWT (browsers
only watch); MarkerHub is a per-run in-process pub/sub.
- New endpoints:
GET /live/{run_id}/token mint a viewer token (BFF proxies it)
WS /live/{run_id}/markers marker stream (brief's literal ask)
GET /live/{run_id}/markers/sse SSE form — Next can't proxy a WS
- closed_loop_rule_synthesis publishes live markers: atomic_step_start
on each emulation, detection_hit on a validated rule fire. Marker
shape matches the capture-bundle Marker so live + recorded timelines
render identically.
- The runner closes the hub on run completion → subscribers get a
done sentinel.
- 12 new tests (token minting, hub fan-out/isolation/close, the live
token endpoint, the marker WebSocket).
apps/web — /captures/live/[run_id]
- LiveView client component: subscribes to the LiveKit room via
livekit-client for sub-second video, consumes the marker SSE stream,
renders an append-only marker list (red stars for attacks, blue
circles for detections), and auto-redirects to the run view 2.5s
after the run completes.
- /api/livekit/[run_id]/token proxies the Conductor's token endpoint;
/api/runs/[id]/markers proxies the marker SSE — the LiveKit secret and
Conductor URL stay server-side.
- 1 new Playwright test for the live route shell.
infra
- compose.yaml: livekit-server + livekit-egress services. Egress
records the room to HLS in the same MinIO bucket the evidence MCP
reads ("one pipeline, both outputs").
- livekit.yaml + egress.yaml dev configs.
ADR-0008 documents: one-room-per-run, the publisher/viewer token trust
split, the WebSocket-for-the-brief / SSE-for-the-browser marker decision
(Next can't proxy a WS upgrade), the agent frame-pump as an operator
integration point, and Egress recording as config not code.
Phase 6 contract met:
- make verify → 143 Python + 20 Go + 7 Playwright = 170 tests pass;
mypy --strict clean on 43 source files; ruff + golangci-lint + biome
clean; Next.js build green.
Phase: 6
First Phase 7 increment per BUILD_BRIEF.md §4 Phase 7+ and the addendum's
integrate-don't-rebuild rule (§A): the one genuine in-house build item
(Stratus, no upstream exists) plus the next-priority vendor mock (Falcon).
mcp/stratus — in-house Stratus Red Team MCP (cloud TTP emulation)
- 5 tools: list_techniques, get_status, detonate, revert, cleanup —
for AWS / Azure / GCP / Kubernetes attack-technique emulation.
- StratusRunner Protocol with two implementations:
InMemoryStratusRunner — deterministic, binary-free, seeded with a
representative catalogue slice; tracks COLD/WARM/DETONATED state.
CLIStratusRunner — shells out to the `stratus` binary; command
mapping shipped, output parsing is a documented integration point.
- detonate/revert/cleanup default dry_run=true and are registered in
the proxy's destructive_tools.
- 12 tests: catalogue, lifecycle, platform filtering, ATT&CK mapping,
full MCP surface.
mcp/mocks/falcon — mock CrowdStrike Falcon MCP
- Second instance of ADR-0004's mock-or-real pattern (after Splunk).
- 4 tools matching the official falcon-mcp detections/hosts/intel
modules: search_detections, search_hosts, search_intel, push_ioa_rule.
- FalconStore seeds 12 hosts + 50 detections (Falcon 1-100 severity
scale, 8 ATT&CK techniques) + 3 intel indicators. Field shapes follow
the Falcon REST API, commented per model.
- push_ioa_rule renders the real Custom-IOA rule-group payload;
dry_run=true previews, the proxy enforces approval policy.
- 13 tests: store determinism + filtering, MCP surface, IOA dry-run vs
real path.
proxy + workspace
- upstreams.example.yaml registers stratus (stdio→stratus-mcp) and
falcon (stdio→falcon-mock, real_cmd flips to falcon-mcp).
stratus.detonate/revert/cleanup added to destructive_tools.
- pyproject + Makefile: both packages added to the uv workspace and
PY_PACKAGES.
- CI: pytest + mypy steps for mcp/stratus and mcp/mocks/falcon.
ADR-0009 records the Phase 7 scoping: most vendors are
integrate-not-build per addendum §A.1 (incl. MITRE ATT&CK — the brief's
"build mcp/mitre" is superseded); Stratus is the genuine build item;
remaining vendor mocks (Sentinel, Chronicle, S1, Elastic, Caldera,
MISP) are deferred so each lands as its own reviewable increment.
Phase 7 contract met:
- make verify → 168 Python + 20 Go + 7 Playwright = 195 tests pass;
mypy --strict clean on 45 source files; ruff + golangci-lint + biome
clean; Next.js build green.
Phase: 7
…elastic/caldera) Extends Phase 7 — after mcp/stratus + mcp/mocks/falcon, this finishes the addendum §B.2 vendor mock fleet so every official-MCP vendor is routable through the proxy without commercial credentials. mcp/mocks/sentinel — Microsoft Sentinel - list_incidents, run_kql_hunt, list_analytics_rules, deploy_analytics_rule. - deploy_analytics_rule renders the scheduled-rule ARM template; dry_run=true previews. Destructive, proxy-enforced. mcp/mocks/chronicle — Google SecOps (Chronicle) - udm_search, list_detections, list_rules, deploy_yaral_rule. - deploy_yaral_rule renders YARA-L 2.0; dry_run=true previews. mcp/mocks/sentinelone — SentinelOne (Purple AI) - powerquery, list_alerts, list_threats, get_inventory. - READ-ONLY — matches the real purple-mcp, which exposes no deploy/destructive tools. A test asserts no tool name contains deploy/push/isolate. mcp/mocks/elastic — Elastic Security - esql_query, list_detection_rules, deploy_detection_rule. - Conforms to the Elastic Agent Builder MCP (Kibana 9.2+); the standalone mcp-server-elasticsearch repo is deprecated per addendum. mcp/mocks/caldera — MITRE CALDERA - list_abilities, list_operations, create_operation, run_ability. - Emulation source, the counterpart to mcp/agents and mcp/stratus. create_operation + run_ability default dry_run=true. Each mock: - Deterministic synthetic data seeded per-seed; strict additionalProperties=false schemas; renders the real vendor's deploy payload shape. - Query tools (run_kql_hunt, udm_search, powerquery, esql_query) seed a local RNG from the query string so the same query is reproducible — needed for the closed-loop validation step. - 5-6 end-to-end MCP tests via the fastmcp in-memory client. proxy + workspace - upstreams.example.yaml: all five registered (stdio→mock, real_cmd/ real_url ready to flip). destructive_tools updated with sentinel.deploy_analytics_rule, chronicle.deploy_yaral_rule, elastic.deploy_detection_rule, caldera.create_operation, caldera.run_ability. - pyproject + Makefile: all five added to the uv workspace and PY_PACKAGES. - CI: a loop runs pytest + mypy for the five new mocks. ADR-0009 extended (§3) with the vendor-mock-fleet table and the read-only-purple-mcp + query-determinism rationale. The MISP/CTI integration (brief Phase 7 item #10) is the one remaining item, deferred per the addendum's larger-design note. mcp/mocks/README.md status table: all 7 mocks now implemented. make verify → 197 Python + 20 Go + 7 Playwright = 224 tests pass; mypy --strict clean on 63 source files; ruff + golangci-lint + biome clean; Next.js build green. Phase: 7
Three-lens review (reuse / quality / efficiency) of the 8-phase branch.
Fixed the findings worth fixing; left intentional parallelism alone.
Quality
- closed_loop_rule_synthesis: replace raw marker kind/colour strings with
named module constants (a typo previously failed silently — the live
timeline just wouldn't match a lane).
- closed_loop_rule_synthesis: collapse the duplicated splunk-vs-other
`{"spl": …}` / `{"query": …}` branching (steps 7 and 10) into one
`_siem_query_params()` helper.
- closed_loop_rule_synthesis: name the ±1-minute validation-search
window (`_VALIDATION_WINDOW`) instead of an inline magic timedelta.
- runs: introduce an `EventLevel` StrEnum (info/warn/error) — StepEvent
and `_emit` were stringly-typed despite a fixed 3-value set and a
typed web consumer (RunTail keys row styling off it).
- StaticMCPClient: add a public `override()` method. Five workflow tests
were reaching into the private `_handlers` list to drop+replace a
handler; they now use the supported method.
- agents/page.tsx: flatten a 3-arm nested ternary into early returns
with a shared `Shell` wrapper (matches the pattern CapturePage
already uses).
Efficiency
- evidence FilesystemStorage: add an in-memory `capture_id → Path`
index built once at construction and kept current by put_bundle.
`_find_dir` was doing an O(total-captures) `rglob` on every call —
and it is hit ~4× per closed-loop run. Now O(1) with an rglob
fallback only on a genuine miss. `list_bundles` iterates the index
instead of re-walking.
- evidence query_events: replace exists()-then-open with
open-and-handle-FileNotFoundError (drops a TOCTOU stat race).
- web rules.ts: wrap `listRules()` in React `cache()` so `/rules` and
`/coverage` share one `detections/` tree walk per render; hoist the
per-key metadata regexes to module scope (were `new RegExp`'d per
key per file).
- web mitre.ts: derive `loadCoverage()` from `listRules()` instead of
running a second independent recursive `detections/` walk + regex
pass — deletes the duplicate walker entirely.
Also fixed a latent bug in mcp/sigma's test `_payload` helper
(`if data:` → `if data is not None:` — a falsy-but-present structured
result was silently skipped).
Left intentionally unchanged: the per-package `_payload` test helper
(13 isolated uv packages; a shared test package would be the premature
abstraction the brief warns against), the per-domain `ServerError`
models (they genuinely diverge), and the parallel vendor-mock shapes
(deliberate, per the addendum).
make verify → 224 tests pass; mypy --strict clean; ruff +
golangci-lint + biome clean; Next.js build green.
Replace process-salted hash() ids with deterministic zlib.crc32 across the falcon, sentinel, chronicle, sentinelone, and elastic mocks so rule and query ids are stable between runs. Hoist the chronicle detection sort to construction time. Give cleanup its own CleanupResult model instead of reusing RevertResult, and drop a misleading "warms first" comment from the in-memory stratus runner. Remove the unused os import hack from the live-api test. https://claude.ai/code/session_014mWDmRKwSXEXBjbQpRLjyj
Code fixes from a full-tree review: - proxy: audit records now reflect upstream failures (was always "forwarded": true); warn at startup when no approval token is set. - wazuh: reject path-traversal rule filenames (basename .xml only). - sigma: lint no longer crashes (IndexError) on an empty title; correct the stale signed-hash embedder comments. - evidence: drop dead _epoch_ms / unused empty_stats arg; query_events goes through a public storage.events_dir() instead of a private call. - agents/splunk: remove dead _kwargs / _seed_for; fix the splunk SPL noise-token skip to test the field name, not the value. - conductor: Run.close() evicts when the SSE queue is full so the done sentinel always lands; cancel in-flight workflow tasks on shutdown via an app lifespan; reuse one MCP connection per run; require the run to exist before minting a LiveKit token; pick the present hit-count key explicitly. - web: validate run/capture ids in the BFF proxy routes against a safe pattern; wrap the SSE proxy fetch so a down Conductor returns 502. - agent: validate the ATT&CK technique inside Argv (injection-safe at the construction site); create the capture output dir before ffmpeg; log inventory enumeration failures; drop a dead context shim. - falcon mock: move the HTTP port off 9103 (collided with agents-mcp). Tests: add coverage for wazuh filename rejection, empty-title lint, the caldera run_ability real path, the splunk invalid-time-range error, and a 404 for an unknown live-token run. Docs: refresh the root README, CONTRIBUTING, .env.example, and the docs/, infra/, mcp/, mcp-proxy/, conductor, web, and mocks READMEs to match the shipped Phase 7 state; wire `make dev` to docker compose; stop tracking the stale .env (it is git-ignored). https://claude.ai/code/session_014mWDmRKwSXEXBjbQpRLjyj
pnpm/action-setup got `version: 9` while package.json already pins `packageManager: pnpm@9.12.0`; the action errors on the conflicting declarations before any step runs. Drop the action input and let it read the version from packageManager. https://claude.ai/code/session_014mWDmRKwSXEXBjbQpRLjyj
The LiveKit dependencies pin `go 1.25.0` in agent/go.mod, but the Go job pinned setup-go to 1.24 and golangci-lint to v1.62 — v1.62 predates Go 1.25 and its bundled type-checker cannot analyse the module, failing with exit 3 before any lint runs. - setup-go now reads the version from agent/go.mod (go-version-file) so CI tracks the module's requirement automatically. - golangci-lint-action bumped v6 -> v8 with golangci-lint v2 (latest); verified clean (0 issues) against the agent locally. https://claude.ai/code/session_014mWDmRKwSXEXBjbQpRLjyj
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
This is a complete architectural rewrite of CatchAttack from a Kafka/Avro/5-microservice monolith to a lean, AI-native, MCP-centric platform. The legacy implementation is preserved under
legacy/while the new v2 tree introduces a modular, gRPC-based agent architecture with an autonomous Conductor workflow engine.Key Changes
Core Architecture
Agent (Go): Cross-platform endpoint agent with gRPC bidirectional streaming to the MCP bridge
agent/proto/agent.proto: Defines the agent ↔ bridge contract (mTLS authenticated)agent/cmd/agent/main.go: CLI withenroll,run,inventory,atomicsub-commandsagent/internal/: Modular packages for enrollment, capture (ffmpeg), atomic test execution, LiveKit publishing, and host inventory collectionConductor (Python): Autonomous detection-engineering workflow runtime
apps/conductor/src/conductor/workflows/closed_loop_rule_synthesis.py: Phase 4 flagship workflow (emulate → summarize → draft Sigma → lint → dedupe → convert to SPL → FP estimate → dry-run → re-emulate → validate → open PR)apps/conductor/src/conductor/api.py: FastAPI surface for workflow queuing and SSE event streamingapps/conductor/src/conductor/runner.py: In-process workflow execution with step-level event trackingapps/conductor/src/conductor/clients/: Pluggable clients for MCP, Anthropic LLM, and PR openingWeb UI (Next.js 15): Clean, server-rendered interface
apps/web/src/app/: Six brief-mandated routes (agents, captures, rules, coverage, runs, live view)apps/web/src/lib/: Server-side clients for Conductor, MCP proxy, and Auth.js v5apps/web/src/app/captures/[id]/Timeline.tsx: SVG timeline with atomic steps and detection hitsapps/web/src/app/captures/live/[run_id]/LiveView.tsx: Real-time WebRTC video + marker streamingInfrastructure & Deployment
infra/compose.yaml: Local dev stack (mcp-proxy, wazuh-indexer, wazuh-manager, livekit, conductor, web)infra/Dockerfile.proxy: Bundled in-tree MCP servers (sigma, splunk-mock, wazuh)infra/livekit.yaml&infra/egress.yaml: LiveKit server config and HLS recording pipelineagent/.goreleaser.yaml: Cross-platform agent binary releasesTesting & CI/CD
apps/conductor/tests/test_workflow_closed_loop.py: Workflow contract tests with StaticMCPClientapps/conductor/tests/test_workflow_integration.py: End-to-end integration against in-tree MCP serversapps/web/tests/routes.spec.ts: Playwright smoke tests for six routes.github/workflows/ci.yml: Unified Python (ruff + mypy + pytest) and TypeScript (biome + playwright) CIMakefile: Root-level targets for fmt, verify, testDocumentation & ADRs
docs/adr/: Nine Architecture Decision Records covering agent design, MCP proxy routing, Sigma deduplication, mock-or-real vendor MCPs, web UI, live mode, and Phase 7 Stratus/FalconBUILD_BRIEF.md(referenced): Seven-phase delivery plan with explicit gates and human review requirementsREADME.md: Lean v2 overview with legacy quarantine noticeCONTRIBUTING.md: Streamlined contributor guidelinesDetection-as-Code
detections/: Sigma YAML rules with CODEOWNERS gatingapps/conductor/prompts/system_v1.md: Conductor system prompt (versioned, loaded at startup)Configuration
https://claude.ai/code/session_014mWDmRKwSXEXBjbQpRLjyj