feat(audit): privacy-preserving tool_transcript entries (#126)#316
feat(audit): privacy-preserving tool_transcript entries (#126)#316aryanputta wants to merge 1 commit into
Conversation
Add an entries array to the cMCP TRACE Claim tool_transcript: one entry per tool call carrying tool_name, the catalog data_class, and the policy decision, derived from the audit chain. No request/response payloads, so the per-call view a regulator reads leaks no PII. tool_transcript.hash still binds the full transcript to the audit-chain tip; transcript_entries_hash() lets a verifier recompute the entries digest offline. - trace_claim.py: ToolTranscriptEntry, cmcp-local ToolTranscriptOut with entries, builder/verify helpers, generate_trace_claim transcript_entries param - session/manager.py: derive entries from tool-call audit entries + catalog - schemas/trace-claim.schema.json: entries array under trace.tool_transcript - tests: ordering, hash==chain tip, no-payload privacy, hash roundtrip, optional - CHANGELOG Signed-off-by: Aryan Putta <aryansputta@gmail.com>
|
🟡 Contributor Check: MEDIUM
Automated check by AGT Contributor Check. |
imran-siddique
left a comment
There was a problem hiding this comment.
The overall design is right: deriving entries from the audit chain (not from raw payloads) keeps the transcript privacy-preserving, and keeping tool_transcript.hash as the audit-chain tip preserves the existing binding semantics. A few things to address before this can merge.
1. audit.db committed
The diff includes a binary SQLite file at repo root. This is a test-run artifact and must not be committed. Strip it from the branch and add *.db (or at minimum audit.db) to .gitignore. Committing binary database files bloats history, causes spurious conflicts, and leaks local run state.
2. transcript_entries_hash() is a utility with nothing to verify against
The PR adds transcript_entries_hash(entries) and the docstring says "a verifier recomputes this from the entries and checks it against the digest the profile carries." But the profile does not carry that digest anywhere. tool_transcript.hash is the audit-chain tip, not the entries digest. There is no second field in the schema, the model, or the claim output to carry the entries hash.
This needs a decision: either add a transcript_entries_hash field to ToolTranscriptOut and trace-claim.schema.json so a verifier actually has something to compare against, or remove transcript_entries_hash() entirely (it currently promises offline verification but provides no anchor for it). Leaving an exported utility function whose docstring describes a verification step that is not implementable will confuse integrators.
You flagged this as an open question in the PR body — it needs to be resolved before merge.
3. data_class is a free-form string; decision is properly enumerated
decision has a clean Literal enum. data_class is a bare str that falls back to "unknown". The catalog's sensitivity_level values presumably have a bounded set — if so, add a Literal or Enum constraint here too, or at minimum document the expected values so a verifier knows what to expect. Otherwise a typo in a catalog entry silently passes schema validation.
Items 1 and 2 need to be resolved before merge. Item 3 is worth addressing but not a hard blocker if the catalog sensitivity level is intentionally open-ended.
|
Adding one small follow-up suggestion to Imran’s review: would it be worth adding a session-level regression test for the actual The current tests cover passing Small semantics question too: the schema allows |
Partially addresses #126.
What
Adds a per-call
entriesarray to the cMCP TRACE Claimtrace.tool_transcript, so a regulator can read the tool-call decision trail offline:Each entry is built from the audit chain, not from raw tool-call parameters or response bodies, so the transcript stays privacy-preserving (no PII).
Acceptance criteria
tool_transcript.hash == audit_chain.chain_tip(criterion 1, unchanged binding).tool_name,data_class(from the catalogsensitivity_level),decision(allow/deny/advisory_deny/redact/fault/n/a).test_transcript_entries_carry_no_payloads.call_count == 3; entries match; hash roundtrip.Design notes / decisions for review
tool_transcript.hash = audit_chain.chain_tip; one acceptance bullet instead reads "SHA-256 of canonical JSON of the entries". These conflict. I kepthash = chain_tip(binds the full transcript via the chain) and added a separatetranscript_entries_hash()so a verifier can recompute the entries digest independently. Happy to fold this into a dedicated field if maintainers prefer the second reading.data_classsource. Taken from the catalog entrysensitivity_levelfor the tool, falling back to"unknown"when the tool is not in the catalog.tool_transcriptis defined in the cMCP claim schema (not pulled fromagentrust-trace), soentriesis added cleanly without a cross-repo model change. A matchingtrace-spectool_transcript.entriesfield can follow if the canonical spec wants to carry the same array.Tests
tests/unit/test_trace_claim.py: 30 pass locally (5 new). Pre-existing failures elsewhere in the suite are unrelated (missingagent_os/Cedar backend in a bare checkout; fail identically on main). Ruff clean. DCO signed-off.