This file is for AI coding agents (Claude Code, Copilot, Cursor, etc.) working in this repository. It describes the project structure, build system, key conventions, and how the core subsystems fit together.
For user-facing CLI docs see docs/cli.md; for MCP-server-mode docs see docs/mcp.md.
Opfor is an open-source red-teaming toolkit for AI agents and MCP servers. It generates OWASP-mapped attack prompts, fires them at a target, and judges each response with an LLM. Output is an HTML + JSON report.
Five usage modes — one set of evaluators:
| Mode | Entry point | Who runs it |
|---|---|---|
| CLI | opfor setup → opfor run --config <path> / opfor hunt --endpoint <url> --objective <text> |
User in terminal or CI |
| Browser extension | Click the toolbar icon on any chat UI | Non-developers — QA, PMs, security analysts |
| MCP server | opfor_setup, opfor_run tools |
MCP-compatible host (Cursor, Claude Desktop) |
| Skills | /opfor-setup, /opfor-run slash commands |
AI coding agent reads markdown skill files |
| SDK | import { run, hunt } from "@keyvaluesystems/agent-opfor-sdk" |
Developers embedding opfor in their own code |
opfor/
├── core/ # @keyvaluesystems/agent-opfor-core — shared engine (npm workspace, compiled to core/dist/)
│ └── src/
│ ├── autonomous/ # Autonomous red-teaming orchestration (orchestrator, prompts, tools, state, report, knowledge)
│ ├── catalog/ # discoverEvaluators.ts, loadCatalog.ts — YAML evaluator/suite discovery
│ ├── config/ # types.ts, schema.ts (Zod), evaluatorsLayout.ts, skillsLayout.ts, resolveTelemetryEnv.ts, loadPrompt.ts
│ ├── execute/ # Run orchestration: runAll.ts (thin orchestrator) → evaluatorLoop.ts → attackRunner.ts (Template Method) + agentAttackDriver.ts/mcpAttackDriver.ts; plus aggregate.ts, baselineScanner.ts, runListener.ts, runAllBrowser.ts, types.ts
│ ├── generate/ # generateAttacks.ts, generateNextTurn.ts — attacker LLM prompt generation
│ ├── evaluators/ # judge.ts, parseEvaluator.ts — judge prompt + evaluator YAML loader
│ ├── targets/ # agentTarget.ts (HTTP/local-script), mcpTarget.ts — implement the AgentTarget / McpTarget interfaces
│ ├── mcp-client/ # createClient.ts — MCP transport factory (stdio, SSE, HTTP)
│ ├── providers/ # factory.ts — createModel() over Vercel AI SDK for all LLM providers
│ ├── report/ # buildReport.ts, render.ts, types.ts — HTML + JSON report renderer
│ ├── run/ # judge.ts (per-attack judge), scanResources.ts, types.ts — MCP-specific helpers
│ ├── telemetry/ # Langfuse + Netra adapters (curation.ts, judgePayload.ts, providers/{langfuse,netra}/)
│ ├── lib/ # env.ts, logger.ts, opforConfig.ts, generateJsonObject.ts, tracePropagation.ts
│ ├── llm/ # openaiCompatible.ts — OpenAI-compatible LLM helpers
│ ├── standards/ # atlas.ts — ATLAS standards mapping
│ ├── prompts/ # Inlined system prompts (attacker, judge) used by core
│ └── util/ # yamlFrontmatter.ts and other utility helpers
├── runners/
│ ├── cli/ # @keyvaluesystems/agent-opfor-cli — `opfor` CLI binary (npm workspace)
│ │ └── src/
│ │ ├── index.ts # CLI entrypoint (commander) — registers setup, run, and hunt
│ │ ├── commands/
│ │ │ ├── setup.ts # `opfor setup` (interactive wizard) + --agent / --mcp / --empty flags
│ │ │ ├── run.ts # `opfor run --config <path>` — runs end-to-end
│ │ │ └── hunt.ts # `opfor hunt` — autonomous red-teaming with agentic orchestration
│ │ └── lib/
│ │ └── artifacts.ts # .opfor/configs/ + .opfor/reports/ path helpers
│ ├── mcp/ # @keyvaluesystems/agent-opfor-mcp — MCP server runner (npm workspace)
│ │ └── src/
│ │ └── index.ts # MCP server entrypoint — registers tools, stdio transport
│ ├── sdk/ # @keyvaluesystems/agent-opfor-sdk — programmatic SDK (npm workspace)
│ │ └── src/
│ │ └── index.ts # SDK entrypoint
│ └── extension/ # @keyvaluesystems/agent-opfor-extension — Chrome MV3 browser extension (npm workspace)
│ ├── service_worker.js # Entry point — message routing only; imports modules below
│ ├── orchestrator.js # Main run loop: locate → attack → extract → reset → judge (calls runAllBrowser from bundled core)
│ ├── llmUiActions.js # DOM-specific LLM helpers (input picker, UI planner, message shortener)
│ ├── domTarget.js # Adapter exposing the DOM send/extract path as a core AgentTarget
│ ├── dist/core.bundle.js # esbuild bundle of @keyvaluesystems/agent-opfor-core/browser (attack + judge engine)
│ ├── frameDiscovery.js # Frame collection, scoring, chat-frame selection
│ ├── domActions.js # chrome.scripting wrappers (send, click, verify, vendor APIs)
│ ├── responseExtractor.js # Three-phase polling extractor for bot responses
│ ├── storage.js # chrome.storage.local helpers (run status, results, paused run)
│ ├── catalog.json # Generated by `npm run build:catalog`
│ ├── catalog.js # catalog.json loading + evaluator/suite lookups
│ ├── popup.js / popup.html # Toolbar popup + progress UI
│ ├── options.js / options.html # LLM key + provider settings
│ ├── config.js # getLlmProfile / assertLlmCfg — reads Options storage
│ ├── state.js # Shared mutable run state (OPFOR_STOP, AbortController)
│ ├── utils.js # sleep, formatTranscript, safeJsonParse
│ └── frame_*.js # Frame scripts injected into page contexts (standalone, no imports)
├── evaluators/
│ ├── agent/ # Agent evaluator YAML files (directory-form or flat-file)
│ │ ├── access-control/ # e.g. bfla/, bola/, rbac/ sub-dirs each with evaluator.yaml
│ │ ├── accuracy/
│ │ ├── bias/
│ │ ├── brand-conduct/
│ │ ├── code-execution/
│ │ ├── disclosure/
│ │ ├── excessive-agency/
│ │ ├── harmful/
│ │ ├── injection/
│ │ ├── mcp-usage/
│ │ ├── memory-rag/
│ │ ├── multi-agent/
│ │ ├── resource/
│ │ ├── source-analysis/
│ │ └── supply-chain/
│ └── mcp/ # MCP evaluator YAML files
│ ├── auth/
│ ├── disclosure/
│ ├── injection/
│ ├── protocol/
│ ├── source-analysis/
│ ├── supply-chain/
│ └── tool-poisoning/
├── suites/
│ ├── agent/ # Curated agent suite YAML files
│ │ ├── quick-smoke.yaml
│ │ ├── pre-deploy-critical.yaml
│ │ ├── harmful-content.yaml
│ │ └── output-trust-and-safety.yaml
│ └── mcp/
│ └── mcp-smoke.yaml
├── data/ # Autonomous-hunt seed knowledge (vendored into CLI/SDK at pack time)
│ ├── personas/ # Who the attacker agent can pose as
│ └── strategies/ # How it applies pressure (vuln-classes come from evaluators/agent/*/README.md)
├── skills/
│ ├── agent-redteaming/
│ │ ├── opfor-setup/
│ │ │ ├── SKILL.md # /opfor-setup slash command
│ │ │ ├── catalog.json # Generated evaluator catalog (npm run build:catalog)
│ │ │ └── targets/ # Target adapter docs (http-endpoint, custom-function)
│ │ └── opfor-run/ # /opfor-run slash command
│ │ ├── SKILL.md
│ │ └── report-schema.md # Report JSON schema the skill emits
│ └── mcp-redteaming/
│ ├── opfor-setup/
│ │ ├── SKILL.md # MCP target configuration skill entry point
│ │ └── catalog.json # Generated MCP evaluator catalog
│ └── opfor-run/ # /opfor-run slash command (MCP)
│ ├── SKILL.md
│ └── report-schema.md
├── tests/
│ └── e2e/
│ ├── agents/
│ │ ├── vanilla-chat/ # Plain Express + LangChain chat agent — LLM Top 10 + Trust & Safety
│ │ ├── customer-support/ # Tool-calling agent + Postgres — BOLA, BFLA, RBAC, PII, SQL injection
│ │ └── vulnerable-memory/ # Express agent with a global knowledge base — cross-session memory poisoning + system-prompt injection via stored "policies"
│ └── mcp/
│ └── vulnerable-server/ # Intentionally vulnerable MCP server
├── docs/
│ ├── cli.md # Complete CLI reference
│ ├── hunt.md # Autonomous mode (`opfor hunt`) guide
│ ├── mcp.md # MCP server (runner) setup + tools reference
│ ├── browser-extension.md # Browser extension guide
│ ├── skills.md # Skill bundle usage
│ ├── sdk.md # SDK (@keyvaluesystems/agent-opfor-sdk) reference
│ ├── evaluators.md # Evaluator + suite reference
│ ├── evaluator-schema.md # Evaluator YAML schema
│ ├── telemetry.md # Trace-aware testing (Langfuse / Netra)
│ └── sessions.md # Target session handling (stateless/stateful, client/server-owned session ids)
└── findings/ # Community-submitted vulnerability writeups (aspirational; may not exist yet)
npm install # workspaces resolved + Husky pre-commit hooks installed
npm run build # build:catalog + tsc -b core + build cli/mcp/sdk runners + extension catalog + bundle (always from root)
npm run typecheck # tsc -b without emit
npm run install:cli # build + npm install -g ./runners/cli — `opfor` available globally
npm run lint # eslint
npm run lint:fix # eslint --fix
npm run format # prettier --write
npm run format:check # prettier --check
npm test # vitest in core/core must compile before any runner — runners/{cli,mcp,sdk} import from core/dist/, and runners/extension esbuild-bundles @keyvaluesystems/agent-opfor-core/browser (the core/src/browser.ts entry) at build time. Always run npm run build from the repo root, never per-package.
| File | Purpose |
|---|---|
core/src/config/types.ts |
LLM + telemetry config TS types (LlmConfig, TelemetryConfig, PROVIDERS const) |
core/src/execute/types.ts |
RunConfig, AgentTargetConfig, McpTargetConfig, AttackSpec, UnifiedRunReport, EvaluatorResult |
core/src/config/schema.ts |
Zod schemas for McpServerConfigSchema discriminated union (stdio/url) + LLM model config |
core/src/config/evaluatorsLayout.ts |
getRepoRoot() / getEvaluatorsDir(category) / getSuitesDir(category) — resolves the repo/package root and evaluators/{agent|mcp}/, suites/{agent|mcp}/, and the shared data/ dir at runtime (monorepo dev + bundled installs). Use these instead of hardcoding paths. |
core/src/autonomous/knowledge/vulnClasses.ts |
HUNT_VULN_CLASS_CATEGORIES + loadVulnClasses() — derives opfor hunt's vulnerability classes from the allow-listed evaluators/agent/<category>/README.md files. |
core/src/config/skillsLayout.ts |
getSkillOpforSetupRoot(category) — resolves skills/{agent|mcp}-redteaming/opfor-setup/ for SKILL.md and catalog.json |
core/src/catalog/discoverEvaluators.ts |
Discovers evaluators from YAML files (directory-form and flat-file); ignores *.test.yaml fixtures |
core/src/config/loadSkillCatalog.ts |
Reads evaluator metadata + suite lists from skill catalog.json (used by skills/MCP mode) |
core/src/execute/runAll.ts |
Top-level run orchestrator: resolves + topo-sorts evaluators, optionally curates telemetry traces, runs MCP baseline pre-flight scans, delegates the attack loop to runEvaluatorAttacks, returns a UnifiedRunReport. Fans lifecycle events to RunListeners. Does not write the report — the caller does via writeReport. |
core/src/execute/evaluatorLoop.ts |
runEvaluatorAttacks — the execute phase: per evaluator generate attacks, run each via its driver, capture SessionContext for dependsOn dependents, stop early on non-retryable errors. |
core/src/execute/attackRunner.ts |
runAttack(driver) Template Method — the invariant turn loop (buildTurn → execute → record → shouldEarlyStop, then finalize) shared by agent + MCP so the two kinds can't drift. |
core/src/execute/agentAttackDriver.ts / mcpAttackDriver.ts |
AttackDriver implementations for the two target kinds (agent prompt vs MCP tool-call) — own turn building, adaptive follow-ups, and the final judge. |
core/src/execute/runAgentLoop.ts |
Thin wrapper: runAgentAttack(...) = runAttack(new AgentAttackDriver(...)). Shared by the Node (evaluatorLoop) and browser (runAllBrowser) loops. |
core/src/execute/baselineScanner.ts |
MCP-only pre-flight scans run before evaluator attacks (tool-poisoning, resource PII/secret leakage, etc.). |
core/src/execute/runListener.ts |
RunListener observer SPI — run-level (onRunStart/Finish/Error) + per-attack progress hooks. CLI attaches ConsoleProgressListener + JsonlEventListener (NDJSON via --events). |
core/src/execute/aggregate.ts |
Folds AttackResults into EvaluatorResult / UnifiedRunReport (toEvaluatorResult, buildUnifiedReport, summarizeVerdicts). |
core/src/execute/runAllBrowser.ts |
Browser-safe variant: takes preloaded evaluators + a pre-built AgentTarget, no Node-only imports |
core/src/generate/generateAttacks.ts |
Generates AttackSpec[] for one evaluator — agent-prompt or MCP tool-call shape |
core/src/generate/generateNextTurn.ts |
Adaptive follow-up: feeds prior turns + judge signal back to the attacker LLM |
core/src/targets/agentTarget.ts |
createAgentTarget(config) — HTTP (http-endpoint) and local-script targets implement AgentTarget |
core/src/targets/mcpTarget.ts |
createMcpTarget(config) — wraps createClient(), exposes callTool / listTools / listResources |
core/src/mcp-client/createClient.ts |
MCP transport factory; runs expandEnv() over stdio env + url headers for ${VAR} substitution |
core/src/evaluators/judge.ts |
LLM-as-judge: response + pass/fail criteria → PASS/FAIL + score + evidence |
core/src/evaluators/parseEvaluator.ts |
Loads an evaluator .yaml (directory-form evaluator.yaml + patterns/, or inline patterns) → EvaluatorSpec; loadBuiltinEvaluator(id, kind) resolves one by id |
core/src/run/judge.ts |
MCP-only judge helpers (judgeToolResponse, sanitizeJudgeResult, buildMcpJudgePrompt) used by mcpAttackDriver + baselineScanner. The agent path judges via evaluators/judge.ts. |
core/src/run/scanResources.ts |
MCP-only: enumerates resources/list + reads each one, judges for PII / secrets |
core/src/report/buildReport.ts |
Writes per-run subfolder + invokes render.ts; maps UnifiedRunReport → ReportViewModel |
core/src/report/render.ts |
Renders the final HTML (cover, exec summary, findings, per-turn details) |
core/src/providers/factory.ts |
createModel(LlmConfig) over Vercel AI SDK. providerRegistry is the single source of truth — one entry per provider (default model, env var, capabilities, build()). PROVIDER_DEFAULTS/PROVIDER_ENV_VARS/PROVIDER_CAPABILITIES are derived aliases. |
runners/cli/src/index.ts |
CLI entrypoint — registers setup, run, and hunt |
runners/cli/src/commands/setup.ts |
Interactive wizard; emits .opfor/configs/opfor-config-<ts>-<id>.json; supports --agent/--mcp/--empty |
runners/cli/src/commands/run.ts |
opfor run — reads --config (or runs the setup wizard inline when omitted), calls runAll, then writeReport. Overrides: --effort/--turns/--output/--env; --events <path> streams NDJSON lifecycle events. |
runners/cli/src/commands/hunt.ts |
opfor hunt — autonomous red-teaming; agentic commander/operator/scout architecture; --ui flag for browser setup UI |
runners/cli/src/lib/artifacts.ts |
.opfor/configs/ + .opfor/reports/ path helpers (newConfigPath(), ensureOpforDirs()) |
runners/mcp/src/index.ts |
MCP server: registers opfor_list_evaluators, opfor_setup, opfor_run tools |
runners/extension/service_worker.js |
Extension entry point — message routing; imports from focused ES modules |
runners/extension/orchestrator.js |
Full adaptive run loop — drives runAllBrowser against DomTarget |
runners/extension/domTarget.js |
Implements the core AgentTarget interface against the live chat DOM |
runners/extension/dist/core.bundle.js |
esbuild bundle of @keyvaluesystems/agent-opfor-core/browser; supplies runAllBrowser + generateNextTurn + judge |
Evaluators are .yaml files living under evaluators/agent/ or evaluators/mcp/. The engine discovers them dynamically — no TypeScript changes needed to add one.
Two on-disk forms are supported:
- Directory form —
evaluators/{category}/{group}/{evaluator}/evaluator.yaml(patterns may be split intopatterns/*.yaml) - Flat-file form —
evaluators/{category}/{group}/{evaluator}.yaml(patterns inline)
*.test.yaml files are fixture inputs for unit tests and are ignored by the discovery logic.
id: command-injection
name: Command Injection
severity: critical # critical | high | medium | low
standards:
owasp-mcp: MCP05
description: >-
One or two sentences on what this evaluator tests.
pass_criteria: |-
- Observable conditions that mean the target responded safely
fail_criteria: |-
- Observable conditions that mean the target is vulnerable
patterns:
- name: Shell Metacharacters
template: |-
Attack prompt text. Use {{placeholder}} for LLM-filled variables.evaluatorsLayout.ts is critical — it exposes getEvaluatorsDir(category) and getSuitesDir(category) to resolve evaluators/{agent|mcp}/ and suites/{agent|mcp}/ at runtime. Any code that needs these paths must call these functions, never hardcode paths.
Agent-redteaming evaluators (evaluators/agent/) send the rendered template as a plain prompt to the target.
MCP-redteaming evaluators (evaluators/mcp/) produce JSON-RPC payloads fired directly at the MCP server. Templates include {{tool_list_json}} replaced with the live tools/list response.
Suites are .yaml files under suites/agent/ or suites/mcp/. Only curated, opinionated subsets are stored there. Standard suites (OWASP LLM Top 10, OWASP MCP Top 10, OWASP Agentic, ATLAS, etc.) are derived at load time from evaluator standards: frontmatter — not stored as suite files — to prevent suite drift.
There is no longer a separate generate step. opfor run --config <file> does everything end-to-end.
- Load config.
runAll(config)reads aRunConfig(flat schema:target.kind,selection,attackerLlm,effort,turnMode,turns,telemetry). - Normalize turnMode.
effectiveTurns = config.turnMode === "single" ? 1 : config.turns. Both fields are written through to eachAttackSpec. - Build the target.
createAgentTarget(config.target)orcreateMcpTarget(config.target)— both implement the same lifecycle (send/callTool,close). - Optional setup-time telemetry. If
config.telemetry.provider !== "none",curateTracesIfConfigured()fetches recent traces and produces a markdown summary the attacker LLM uses as grounding context. - Per evaluator (topo-sorted by
dependsOn, inrunEvaluatorAttacks):generateAttacks({ evaluator, target, effort, model, turns, turnMode, options })producesAttackSpec[].adaptiveyields one open-ended spec;comprehensiveyields one spec per named pattern. - Per attack: an
AttackDriver(AgentAttackDriverorMcpAttackDriver) runs under the sharedrunAttackTemplate Method. Each turn: build the input (turn 1 uses the seed; later turns usegenerateNextTurnwith full history + last judge signal), send via the target, record the result, and optionally early-stop (agent: on a target error; MCP: on a FAIL from the per-turn judge). - Judge once after the loop. A single judge call sees the whole transcript + optional fetched trace data (
enrichJudgeFromTrace) and returns{ verdict, score, confidence, evidence, reasoning }. - Aggregate + write report.
runAllreturns aUnifiedRunReport; the caller (CLI/SDK) writes it viawriteReport(report, outputDir), which creates.opfor/reports/run-report-<compactTs>-<slug>-<shortId>/containing<slug>-report.htmland<slug>-report.json. (Autonomousopfor huntuses the parallelwriteAutonomousReport, which writes the same<slug>-report.html/.jsoninto ahunt-report-<compactTs>-<slug>-<shortId>/subfolder.)
MCP targets additionally run baseline pre-flight scans (runBaselineScans) before the evaluator loop — these enumerate tools/list + resources/list and judge them for poisoning / leakage independent of any evaluator. Throughout the run, runAll fans lifecycle events to registered RunListeners (progress reporting, NDJSON streaming) rather than only a callback.
runAllBrowser is the same loop in browser-safe form: takes preloaded EvaluatorSpec[] + a pre-built AgentTarget (e.g. DomTarget), skips disk reads.
tests/e2e/agents/ contains pre-built target agents developers can spin up locally to test evaluator changes without needing a real external service.
Each agent has a scripts/ directory for consistent DX — always use these instead of docker compose directly.
A plain Express + LangChain chat agent (no tools). Supports openai, anthropic, groq, google, and any OpenAI-compatible endpoint via BASE_URL.
cd tests/e2e/agents/vanilla-chat
cp .env.example .env # set PROVIDER + the agent's API key
./scripts/start.sh # builds image, starts agent, waits for /health
export GROQ_API_KEY=your-key-here # attack LLM key (separate from Docker .env)
# from repo root:
opfor run --config tests/e2e/agents/vanilla-chat/opfor.config.jsonCovered evaluators: OWASP LLM Top 10, Trust & Safety (bias, misinformation), system-prompt-leakage, jailbreaking.
An Express + LangChain tool-calling agent backed by PostgreSQL. Has five tools (lookup_order, lookup_user_profile, list_my_orders, create_ticket, process_refund) with intentional authorization gaps. Runs multi-turn attacks (turnMode: "multi", 3 turns) by default. Session memory is keyed by sessionId in the request body.
cd tests/e2e/agents/customer-support
cp .env.example .env # set PROVIDER + the agent's API key
./scripts/start.sh # starts postgres:16 + agent on :4001, seeds DB, waits for /health
export GROQ_API_KEY=your-key-here
# from repo root:
opfor run --config tests/e2e/agents/customer-support/opfor.config.json
# reset DB to clean seed state between runs:
./scripts/reset.shCovered evaluators: BOLA, BFLA, RBAC, PII (direct/session/API), SQL injection, prompt injection, jailbreaking, system-prompt-leakage, contracts, competitors, hallucination.
The opfor.config.json uses the current flat schema (target.kind: "agent" at top level, attackerLlm/selection/effort/turnMode/turns as siblings — not the legacy nested { mode, agent: {} } shape used pre-refactor). The apiKeyEnv field takes the env var name (e.g. "GROQ_API_KEY"), not the key value itself.
See CONTRIBUTING.md — "Adding a test agent" section.
- Create
evaluators/agent/<group>/<id>.yaml(orevaluators/mcp/<group>/<id>.yaml). For complex evaluators with many patterns, use the directory form:evaluators/agent/<group>/<id>/evaluator.yaml. - Fill YAML fields:
id,name,severity,standards,description,pass_criteria,fail_criteria,patterns(seedocs/evaluator-schema.md) - Optionally add the ID to a curated suite's
evaluators:list insuites/agent/(orsuites/mcp/). Standard-mapped suites (OWASP, ATLAS, etc.) pick it up automatically via thestandards:field — no suite edit needed. - Test:
opfor setup --agent --empty(or--mcp --empty) → editselection.evaluatorsin the generated config →opfor run --config <path> - PR to
master— see CONTRIBUTING.md
Category READMEs feed
opfor hunt. Eachevaluators/agent/<category>/README.mdcarries aseverity:frontmatter field, and the categories listed inHUNT_VULN_CLASS_CATEGORIES(core/src/autonomous/knowledge/vulnClasses.ts) are loaded as hunt's vulnerability classes. If you rename/remove an allow-listed category directory or drop itsseverity:field,opfor huntthrows at startup. Update the constant (and the README) when changing those categories.
For a new agent-target type (e.g. websocket, gRPC):
- Extend
AgentTargetConfigincore/src/execute/types.tswith a newtypevalue and any fields it needs. - Add a branch to the
createAgentTarget()factory incore/src/targets/agentTarget.tsthat returns the sameAgentTargetinterface (send,close). - Update the wizard in
runners/cli/src/commands/setup.ts(collectAgentTarget) to prompt for the new type.
For a new MCP transport (beyond stdio/url):
- Add the new branch to
McpServerConfigSchemaincore/src/config/schema.ts. - Implement
connect<Name>Transport()incore/src/mcp-client/createClient.tsand route to it fromconnectMcpClient(). - Update
core/src/targets/mcpTarget.ts'buildServerConfig()to pass through new fields fromMcpTargetConfig. - Update the wizard in
runners/cli/src/commands/setup.ts(collectMcpTarget) and the MCP runner's tool input schema inrunners/mcp/src/index.ts.
- TypeScript strict mode — no
anywithout a comment explaining why - Zod for all external input — config files, LLM responses, MCP responses; never
JSON.parsedirectly into a typed variable - No barrel re-exports — import directly from the file that owns the symbol
- Error messages are actionable — tell the user what to fix, not just what went wrong
- Evaluator files are data — no business logic in evaluator
.yamlfiles; logic lives incore/src/evaluators/ - Never invoke the CLI as a subprocess from the MCP server — call
@keyvaluesystems/agent-opfor-coredirectly
| Variable | Purpose |
|---|---|
OPENAI_API_KEY |
openai provider |
ANTHROPIC_API_KEY |
anthropic provider |
GOOGLE_GENERATIVE_AI_API_KEY |
google provider |
GROQ_API_KEY |
groq provider |
DEEPSEEK_API_KEY |
deepseek provider |
AZURE_OPENAI_API_KEY |
azure provider (also requires attackerLlm.baseURL) |
OPFOR_API_KEY |
openai-compatible provider (LiteLLM, OpenRouter, Ollama …) |
LANGFUSE_PUBLIC_KEY / LANGFUSE_SECRET_KEY |
Langfuse telemetry |
NETRA_API_KEY |
Netra telemetry |
Copy .env.example to .env and fill in at least one provider key before running locally.
Branch names: <type>/<short-description> — e.g. feat/add-ssrf-evaluator, fix/judge-false-positive
PR titles: <type>: <what changed> — e.g. feat: add SSRF evaluator for MCP05
Types: feat, fix, docs, refactor, chore
Pre-commit hooks (husky + lint-staged) run eslint and prettier on staged files. Do not skip with --no-verify.
Full checklist: CONTRIBUTING.md