Black-box contract tests for A2A agents.
a2a-proof discovers a deployed agent, sends real A2A requests, and checks its observable
behavior. It needs no access to the agent's source code, framework, prompts, or model provider.
The official A2A TCK checks protocol conformance,
A2A ITK checks interoperability between SDKs, and the
A2A Inspector supports interactive debugging.
a2a-proof adds repeatable contracts for the behavior of your deployed agent.
It targets A2A 1.0 over JSON-RPC, HTTP+JSON, and gRPC. The SDK compatibility layer also supports AP2 v0.2.0 agents that expose A2A 0.3 JSON-RPC, including signed mandate-to-receipt payment flows.
| Risk | Contract |
|---|---|
| A prompt, model, or backend change alters an answer | Text, structured data, JSON Schema, and file integrity assertions |
| An LLM succeeds only some of the time or gets slower | Repeated trials, statistically bounded pass rates, parallel runs, and p50/p95 latency |
| A long-running task breaks after acceptance | State trajectories, stream resumption, cancellation, persistence, and push delivery |
| Staging no longer behaves like production | Agent Card preflight and deployment diff |
| A contract change needs review without a live agent | Recorded cassettes replayed offline |
| Agent text leaks a secret or system prompt | Global invariants and bounded failure evidence |
| An orchestrator stops calling its specialist, or forwards credentials to it | Delegation contracts against a recording downstream agent |
| An agent produces an invalid payment proof | Signed AP2 mandate-chain and receipt verification |
With uv installed, try a real loopback A2A exchange without an API key or external service:
uvx a2a-proof demo
uvx a2a-proof demo --failThe second command deliberately shows a contract failure and exits 1.
Point the runner at your agent:
uvx a2a-proof init https://agent.example.cominit reads the Agent Card and creates a2a-proof.yaml. Replace its smoke scenario with behavior
your users depend on:
version: 1
agent:
url: https://agent.example.com
defaults:
trials: 5
pass_rate: 0.8
scenarios:
- name: billing dispute routing
message: A customer says their card was charged twice for order 4815.
latency:
p95_seconds: 15
expect:
state: completed
data:
- path: /queue
equals: billing-disputes
- path: /priority
matches: "(?i)^high$"$ uvx a2a-proof run
┏━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━┳━━━━━━┓
┃ Result ┃ Scenario ┃ Trials ┃ Time ┃
┡━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━╇━━━━━━┩
│ PASS │ billing dispute routing │ 4/5 │ 3.2s │
└────────┴─────────────────────────┴────────┴──────┘
1 scenario passed in 3.2sRun one scenario, save failure evidence, or compare the same contract against another deployment:
uvx a2a-proof run --scenario "billing dispute routing"
uvx a2a-proof run --format junit --output a2a-proof.xml --evidence evidence
uvx a2a-proof diff --against https://candidate-agent.example.com
uvx a2a-proof record --output cassette.json && uvx a2a-proof run --replay cassette.jsonAfter checking out the repository, one step runs its default contract as a CI check:
- uses: aspix2k/a2a-proof@v0.16.0Set config only when the contract is not a2a-proof.yaml.
- Writing contracts
- Assertions
- Task lifecycle
- Push notifications
- Delegation contracts
- Recording and replay
- AP2 contracts
- External agent showcases
- Running in development and CI
- Configuration schema
The schema provides completion and inline validation in YAML-aware editors. init links it
automatically.
Agent responses and file metadata are treated as untrusted input. Requests, response parts, regular expressions, embedded schemas, evidence, and local file access are bounded. Redirects, credential-bearing URLs, external schema references, and cross-origin interfaces are rejected by default. Remote file URLs are never fetched or written to reports. Requests recorded by the delegation agent stay in memory and are never written to reports or evidence.
See SECURITY.md for private vulnerability reports.
Contributions follow CONTRIBUTING.md. Licensed under the MIT License.