feat(audit): add optional external_execution_evidence on AuditEntry (#301)#314
feat(audit): add optional external_execution_evidence on AuditEntry (#301)#314carloshvp wants to merge 3 commits into
Conversation
…#301) Introduce an optional, independently-signed execution receipt bound to an audit entry, distinct from response_payload_hash. response_payload_hash is what the gateway forwarded; external_execution_evidence is what an independent authority (for example a safety controller) attested. Confirmed direction: Option A. - chain.py: add the optional external_execution_evidence field to AuditEntry and an append() keyword. Serialized uniformly via asdict (null when absent), so receipt-less entries hash exactly as before and existing evidence keeps verifying. - schemas/audit-entry.schema.json: add the optional receipt object (issuer, issuer_key_id, signature, evidence_hash, evidence_type, linked_call_id), not in required so entries that predate the field still validate. - cmcp_verify: opt-in receipt verification. When external_evidence_keys is supplied, check linked_call_id == call_id and the issuer Ed25519 signature over the canonical receipt. Receipt-less entries and callers without keys are unaffected. - LIMITATIONS.md: state what the receipt does and does not prove. - conformance tests: absent verifies and keeps old hashing, populated verifies, tampered fails, linked_call_id mismatch fails, unknown issuer key fails. Scope note: this lands the data model, schema, verification, and tests. Proxy ingestion (how a controller receipt rides in the upstream response) and the industrial-embodied-ai example follow next; the transport convention is flagged for maintainer input. Pre-existing audit-entry.schema.json drift (detail, workflow_id, extra entry_type enum values) is noted and left out of scope. Signed-off-by: Carlos Hernandez <carloshvp@gmail.com>
imran-siddique
left a comment
imran-siddique
left a comment
There was a problem hiding this comment.
Thanks for this - attaching controller-signed external execution evidence to individual audit entries is exactly the kind of receipt binding that compliance use cases need. The overall approach makes sense. A few things to work through before this is ready:
Schema gaps
evidence_hash has no pattern constraint. Every other hash field in the schema uses "pattern": "^sha(256|384):[0-9a-f]+". Add one here so schema validation can reject malformed hashes before they reach the verifier.
issuer_key_id has no documented format. Is it a hex-encoded SHA-256 of the public key (matching agent_manifest.py's _key_id())? A DID key identifier? Something else? Pick a format and enforce it in the schema.
Specification
What is the evidence_hash pre-image exactly? The field is described as a "hash of the evidence" but there is no canonical definition of what bytes are hashed, in what encoding, with what canonicalization. An implementer reading the schema today cannot produce a verifiable evidence_hash without guessing. This needs a computation spec in docs/spec/verification-library.md (or a new doc).
evidence_type is a free-form string. Even an initial enumeration of documented values (e.g., tee-signed-receipt, controller-jwt, opaque-receipt) would help both implementers and verifiers. Free-form types make it impossible to write a strict verifier.
docs/spec/verification-library.md is not updated to document the new external_evidence_keys parameter in verify_audit_bundle(). The spec doc should describe what keys are expected, what the verification flow is, and what EXTERNAL_EVIDENCE_VERIFICATION_FAILED would look like as an error code.
Type inconsistency
verify.py declares external_evidence_keys: dict[str, str] | None. The sibling PR #315 uses trusted_agent_manifest_keys: dict[str, bytes] (raw public key bytes). If these two verification paths are going to coexist in the same library, the key type should be consistent. String-encoded keys require a documented encoding convention that currently doesn't exist.
Testing
The conformance tests cover present/absent/tampered/call_id-mismatch cases - that is a solid starting point. But there is no end-to-end test that runs the full path: chain.append(..., external_execution_evidence=...) → export bundle → verify_audit_bundle(external_evidence_keys=...). The conformance suite exercises the schema and mismatch detection, not the cryptographic verification path. Add an integration test that uses a real key pair, produces a valid signature, and confirms it passes.
TRACE Claim
When a session's audit chain contains entries with external_execution_evidence, can a verifier tell from the TRACE Claim alone that external evidence was bound? Currently the Claim carries audit_chain_tip but no evidence-presence flag. Is that intentional? If so, call it out in LIMITATIONS.md.
Signed-off-by: Carlos Hernandez <carloshvp@gmail.com>
|
Addressed the review feedback in What changed:
Validation run locally:
GitHub currently shows the PR mergeable with the visible |
Signed-off-by: Carlos Hernandez <carloshvp@gmail.com>
|
Follow-up pushed for the #301 path:
Validation rerun before push:
|
|
@imran-siddique this should be ready for re-review when you have a chance. The original review feedback was addressed in Once this lands, I can re-pin and undraft the embodied-AI example follow-up in agentrust-io/examples#29. |
2 participants
Summary
Implements Option A for #301 (confirmed direction). Adds an optional,
independently-signed execution receipt bound to an audit entry, kept distinct
from
response_payload_hash:response_payload_hashis what the gatewayforwarded,
external_execution_evidenceis what an independent authority (forexample a safety controller) attested.
This PR lands the data model, schema, verification, and tests. Proxy ingestion
and the example follow next (see Scope and follow-ups).
What changed
audit/chain.py: optionalexternal_execution_evidencefield onAuditEntryand an
append()keyword. Serialized uniformly viaasdict(null whenabsent), so receipt-less entries hash exactly as before and existing evidence
keeps verifying. No change to the canonical body rule.
schemas/audit-entry.schema.json: optional receipt object (issuer,issuer_key_id,signature,evidence_hash,evidence_type,linked_call_id), not inrequiredso entries that predate the field stillvalidate.
cmcp_verify.verify_audit_bundle: opt-in receipt verification via anexternal_evidence_keysparameter (issuer_key_id to hex Ed25519 public key).When supplied, it checks
linked_call_id == call_idand the issuer signatureover the canonical receipt (all fields except
signature). Receipt-lessentries and callers without keys are unaffected.
LIMITATIONS.md: what the receipt does and does not prove.TestExternalExecutionEvidence301): absent-verifies andkeeps the old hashing, populated-verifies, tampered-fails,
linked_call_id-mismatch-fails, unknown-issuer-key-fails.
Validation
pytest tests/unit tests/conformance: 701 passedruff,mypy,bandit: cleanEvery existing audit and verify test passes unchanged, so the change is
backward compatible.
Scope and follow-ups
Kept out of this PR on purpose, happy to do them next:
default is to let the upstream MCP server include the receipt in its tool
response and have the proxy bind it at the existing post-scan audit append
(the Populate response_payload_hash in the forwarding path audit entries #293 path). Flagging the transport convention for your input before I
wire it.
reject receipt (examples repo).
Aside:
audit-entry.schema.jsonalready drifts from the dataclass (detail,workflow_id, and a fewentry_type/policy_decisionenum values aremissing). Left out of scope here, happy to reconcile separately.
@imran-siddique tagging you as requested.