Skip to content

Empirical schema audit: verify UserPromptSubmit + SessionStart discriminator predicates against captured stdin #812

@michael-wojcik

Description

@michael-wojcik

Context

PR #808 empirically falsified the agent_id is None discriminator for TaskCompleted hook predicates. Captures from the in-repo TaskCompleted shim showed agent_id never appears on TaskCompleted stdin; the actual discriminator is teammate_name presence. PR #808 fixes is_lead_at_task_completed accordingly.

The fix raises a sibling question: are the OTHER per-event discriminator predicates (is_lead_emit_authorized for PostToolUse, is_lead_drain_authorized for UserPromptSubmit, is_lead_at_session_start for SessionStart) using empirically-correct fields?

Empirical evidence collected during PR #808 capture experiment

PostToolUse stdin (3 captures from this machine, 2026-05-20):

// Teammate-frame (backend-coder session)
{
  "hook_event_name": "PostToolUse",
  "agent_id": "a1287d9e2db8f12d0",     // PRESENT
  "agent_type": "backend-coder",        // teammate role
  // teammate_name: ABSENT, team_name: ABSENT
}

// Lead-frame (orchestrator session)
{
  "hook_event_name": "PostToolUse",
  // agent_id: ABSENT
  "agent_type": "PACT:pact-orchestrator",  // lead role
  // teammate_name: ABSENT, team_name: ABSENT
}

Verdict for PostToolUse: is_lead_emit_authorized body agent_id is None is empirically CORRECT (PR #783's design holds).

TaskCompleted stdin (verified in PR #808): teammate_name is the discriminator. NOT agent_id.

UserPromptSubmit stdin (1 capture, lead-only):

{
  "hook_event_name": "UserPromptSubmit",
  // agent_id: ABSENT
  "agent_type": "PACT:pact-orchestrator",
  "prompt": "...",
  "permission_mode": "...",
  // teammate_name: ABSENT, team_name: ABSENT
}

Verdict for UserPromptSubmit: UNVERIFIED. Need a teammate-fired UserPromptSubmit capture to confirm the discriminator. UserPromptSubmit may not have a teammate-fire path at all (only lead types prompts).

SessionStart stdin: NOT CAPTURED yet (the shim installs on TaskCompleted + PostToolUse hooks only; SessionStart fires once at session start and was missed).

Proposed work

  1. Verify PostToolUse — pin the empirical schema as the canonical fixture under pact-plugin/tests/fixtures/wake_lifecycle/ with _meta.capture_method: \"logging-shim\" provenance. (Already done in PR fix(#781, #760, #738): actor-discriminator capture-shape HARD GATE bundle #808 follow-up if it lands the fixtures; otherwise file as in-scope here.)

  2. UserPromptSubmit empirical verification — Does UserPromptSubmit ever fire from a teammate session? If yes, capture the stdin. If no (i.e., only the lead types prompts; teammates communicate via SendMessage), document that is_lead_drain_authorized body is structurally always-True (no teammate-fire path exists). Either way, update the predicate docstring with empirical provenance.

  3. SessionStart empirical verification — Per upstream docs (code.claude.com/docs/en/hooks.md), SubagentStart receives agent_type: type/name of the agent. SessionStart may use the same field. Extend the shim or write a dedicated SessionStart shim to capture the stdin shape from both lead and teammate session-starts.

  4. Per-event discriminator truth table — Compile the verified-empirically truth table into pact-plugin/docs/hook-discriminator-schemas.md (or similar) so future predicate authors don't repeat the over-extrapolation that PR fix(#781, #760, #738): actor-discriminator capture-shape HARD GATE bundle #808 caught. Pin: discriminator FIELD differs PER EVENT; do not assume agent_id is the universal lead-vs-teammate signal.

Acceptance criteria

  • Each of the 4 per-event predicate helpers (is_lead_emit_authorized, is_lead_at_task_completed, is_lead_drain_authorized, is_lead_at_session_start) has empirical-capture provenance documented in its helper docstring.
  • If any predicate body is empirically wrong, fix it in this issue's PR.
  • If any predicate body is empirically vacuous (no teammate-fire path exists for that event), document the always-True behavior explicitly + remove the misleading docstring claim that suggests it discriminates.
  • A captured-from-production fixture for each event lands in pact-plugin/tests/fixtures/wake_lifecycle/ with proper _meta.capture_method provenance.

Cross-refs

Metadata

Metadata

Assignees

No one assigned

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions