Skip to content

auditor_reminder hook: false-positive race on same-turn coder+auditor dispatch + asymmetric agent_dispatch emission in orchestrator templates #495

@michael-wojcik

Description

@michael-wojcik

Problem

pact-plugin/hooks/auditor_reminder.py (PostToolUse on Task) emits a false-positive "Coder dispatched without a concurrent auditor" warning when coder + auditor are dispatched in the same orchestrator turn — which is the mandated default per comPACT.md Auditor Dispatch and the "DEFAULT TO CONCURRENT" rule in orchestration.

Empirically observed in session pact-2aebf1e4 (2026-04-20).

Root cause — synchronous detector on async state

The hook checks ~/.claude/teams/{team}/config.json members[] at PostToolUse time:

```
auditor_reminder.py:39-68 _team_has_auditor()
-> reads members[*].agentType for "pact-auditor"
```

But members[] is platform-owned. It is not written by PACT plugin code (verified: `grep -rn 'joinedAt' pact-plugin/` returns only test fixtures). The platform appends a member row after the spawned agent joins — 9 to 22+ seconds after `Task()` returns.

Live evidence from session `pact-2aebf1e4`:

Event UTC Δ
team-lead created 20:08:01.639
secretary `joinedAt` 20:08:24.049 +22.41 s
preparer `agent_dispatch` journal write 2026-04-21 05:08:10
preparer `joinedAt` 2026-04-21 05:08:19.590 +9.0 s

For a sibling `Task()` call in the same orchestrator response, `members[]` has not been updated yet. So the hook's `_team_has_auditor` returns False even though the auditor was just dispatched.

Why this matters

Issue #312 converted auditor dispatch from conditional to opt-out default precisely because conditional decision-making failed under load. A false-positive warning that fires every time the orchestrator correctly follows the protocol trains the orchestrator to ignore it — which erodes the guardrail PR #313 was built to provide.

Timing sequence

```
t0 TaskCreate(coder-task) -> ~/.claude/tasks/{team}/1.json written (sync)
t1 TaskUpdate(1, owner=coder-1) -> 1.json.metadata.subagent_type="pact-backend-coder"
t2 Bash heredoc: session_journal.py write --type agent_dispatch "coder-1"
-> session-journal.jsonl appended (O_APPEND + flock, sync)
t3 TaskCreate(auditor-task) -> 2.json written
t4 TaskUpdate(2, owner=auditor) -> 2.json.metadata.subagent_type="pact-auditor"
t5 [orchestrate.md:649-662 has NO Bash heredoc here] <-- gap
t6 Task(name="coder-1", subagent_type="pact-backend-coder", ...) returns
-> PostToolUse(Task) fires: auditor_reminder.py ★
- reads config.json: only team-lead in members[], NO auditor yet
- emits false-positive warning
t7 Task(name="auditor", subagent_type="pact-auditor", ...) returns
-> PostToolUse(Task) fires: auditor_reminder.py (for auditor itself)
- subagent_type=="pact-auditor" -> skipped, no warning emitted
t8+ platform appends coder-1 and auditor to members[] (~9-22s after t6/t7)
```

Ranked detection signals (from investigation)

Full analysis in `docs/preparation/auditor-reminder-timing.md`.

Rank Signal Reliability Implementation False-negative risk
#1 Journal `agent_dispatch` events (scan session-journal.jsonl tail) High for canonical dispatch — synchronous CLI write gated by `set -e` before every Task() Low — JSONL tail scan Low IF emission sites are symmetric
#2 Task-file scan `~/.claude/tasks/{team}/*.json` for `metadata.subagent_type=="pact-auditor"` Medium-High — TaskCreate/TaskUpdate are sync Medium — glob + parse Medium — `metadata.subagent_type` is convention, not platform-enforced
#3 Task-file subject prefix regex (`^auditor:`) Medium — naming convention only Low Medium-High
#4 Hook `tool_input` (current call only) Useful only as pre-filter Trivial Cannot detect siblings
#5 (current) `members[]` scan Low for same-turn dispatch (already implemented) This is the bug

Expanded scope — the fix is NOT hook-only

The investigation surfaced two asymmetric-emission gaps that would convert a naive hook fix (Signal #1) into a silent false-negative. Per PR #454's symmetric detector-integrity audit rule, every `agent_dispatch` emission site must be audited before routing detection through journal events.

Gap 1 — orchestrate.md:649-662 auditor dispatch block does NOT write `agent_dispatch`

Coder block at orchestrate.md:617-625 includes:
```

  1. TaskCreate(...)
  2. TaskUpdate(owner=...)
  3. Bash heredoc: agent_dispatch journal write <-- present
  4. Task(...)
    ```

Auditor block at orchestrate.md:649-662:
```

  1. TaskCreate(subject="auditor: ...", metadata={completion_type: signal})
  2. TaskUpdate(owner="auditor")
  3. Task(...) <-- agent_dispatch missing
    ```

If the hook fix routes detection through `agent_dispatch`, this asymmetry creates a permanent false-negative specifically for the auditor it's trying to detect.

Gap 2 — comPACT.md has no explicit auditor dispatch block

`grep auditor pact-plugin/commands/comPACT.md` returns only protocol description (lines 259, 273) — no TaskCreate + Task() template. Orchestrators must extrapolate from the generic multi-specialist block, meaning `agent_dispatch` emission is discretionary rather than structurally guaranteed.

Open questions (from the investigation)

  1. Recency filter scope — a coder dispatched in a later CODE phase should not be exempted because an auditor was dispatched earlier. Should Signal Claude Code: not strictly follow framework #1 filter events after the last `phase_transition` to CODE? Within N seconds? Per-turn?
  2. Fallback when Signal Claude Code: not strictly follow framework #1 is absent — if an orchestrator skips the `agent_dispatch` write (e.g., ad-hoc Task() outside `/PACT:orchestrate` / `/PACT:comPACT`), should the hook fall back to Signal feat: Add PACT slash commands and update PACT_Prompt with explicit agent and PR workflows #2 (task-file scan), fail-conservative (emit the warning), or fail-silent?
  3. Test enforcement — can a test statically verify that every `Task()` dispatch block in commands/*.md is preceded by an `agent_dispatch` heredoc? Would lock the symmetry in at CI time.
  4. Non-PACT dispatches — what's the expected behavior when a user invokes `Task()` directly (not via `/PACT:orchestrate` or `/PACT:comPACT`)? Should the hook warn or stay silent?

Suggested fix shape (subject to plan-mode review)

  1. Hook: consult Signal Claude Code: not strictly follow framework #1 (journal `agent_dispatch` events) with a recency filter to be designed, fall back to Signal feat: Add PACT slash commands and update PACT_Prompt with explicit agent and PR workflows #2 (task-file scan) when Claude Code: not strictly follow framework #1 is absent, fall back to current Signal Create pact-observability-patterns skill #5 (members[]) only for backward compatibility.
  2. Dispatch-template symmetry: add `agent_dispatch` heredoc to orchestrate.md:649-662 auditor block; add explicit auditor dispatch block to comPACT.md.
  3. CI test (optional): static check over commands/*.md — every `Task(` invocation must be preceded by an `agent_dispatch` heredoc within the same dispatch block.

Variety & workflow recommendation

References



REVISION (2026-04-21, post-audit) — Recommendation INVERTED, prior addendum SUPERSEDED

After the original deliverable was filed, the preparer completed the full symmetric-emission audit deferred from plan-mode. The audit produced findings that materially change the recommended fix shape and supersede the "Preparer addendum" section that previously appeared here.

Audit results — agent_dispatch is NOT symmetric across PACT command files

Independent grep verification (counts of agent_dispatch string occurrences):

Command file agent_dispatch writes Status
commands/orchestrate.md 8 occurrences across multiple specialist blocks Partial — auditor block at lines 649-662 missing it
commands/comPACT.md 4 occurrences Partial — auditor section (lines 257-274) has no Task() block at all
commands/rePACT.md 0 Zero coverage
commands/peer-review.md 0 Zero coverage
commands/plan-mode.md 0 Zero coverage

3 of 5 command files emit zero agent_dispatch events. Specialists dispatched under rePACT, peer-review, or plan-mode never produce the journal signal at all. A hook fix routed through agent_dispatch would silently false-negative for every dispatch under those workflows.

task_assignment claim — FALSIFIED

The original "Preparer addendum" speculated that a platform-emitted task_assignment journal event might exist and outrank Signal #1. It does not.

  • grep -rn 'task_assignment' pact-plugin/zero matches
  • Live session journal contains only three event types: session_start, agent_dispatch, agent_handoff — no task_assignment
  • _REQUIRED_FIELDS_BY_TYPE schema in session_journal.py:81-161 declares 15 event types; task_assignment is not among them
  • Platform DOES have a TaskCompleted hook (subscribed by handoff_gate.py) and a documented TaskCreated hook event (NOT subscribed by PACT) — but these are in-memory hook events, not journal events

The intent of memory b5b281c1af67... is correct (a platform-emitted, harder-to-omit signal earlier than orchestrator-authored agent_dispatch does exist), but the mechanism is wrong. The actual signal is the task file on disk at ~/.claude/tasks/{team}/{id}.json with platform-written owner (via TaskUpdate) and metadata.subagent_type (via Task() dispatch). It's a file, not a journal event.

Revised ranked signal table — Signal #2 PROMOTED to primary

Rank Signal Symmetric coverage Notes
#1 (NEW PRIMARY) Task-file scan for metadata.subagent_type=="pact-auditor" in ~/.claude/tasks/{team}/*.json YesTaskCreate is universal across every workflow path Glob + N JSON parses at PostToolUse. Cost: single-digit ms typical. Requires recency filter (see Open Question #3)
#2 (demoted from #1) Journal agent_dispatch events No — asymmetric across 5 command files Strong when emitted; load-bearing only for orchestrate.md's coder+auditor canonical path
#3 Hook tool_input self-filter Yes (stateless) Pre-filter only: current Task IS auditor → exit early. Already handled by CODER_TYPES filter
#4 members[] (current hook signal) Yes (eventually) 9-22s delay — unreliable for same-turn — keep demoted

Revised fix scope and variety

The fix splits into two scope alternatives — choose during plan-mode:

Scope Files touched Test updates Doc updates Raw variety Calibrated (+6 hook_infra floor)
(b) Task-file scan only (lighter) auditor_reminder.py 1 test file 0 3-4 9-10 (Medium-High → comPACT)
(a) Close all emission gaps + journal signal (full symmetry) auditor_reminder.py + 5 command files (orchestrate, comPACT, rePACT, peer-review, plan-mode) + session_state.py (verify feature-subject fallback) test_session_state.py, test_session_journal.py pact-protocols.md + pact-state-recovery.md 8-9 14-15 (High, ATOMIZE-worthy → orchestrate)

Scope (b) is the recommended path: lighter footprint, no command-file edits, signal #2 is structurally symmetric because TaskCreate is called by every workflow. Scope (a) is justified only if agent_dispatch symmetry has independent value for other consumers (session_state.py:299-302 feature-subject disk fallback, session_resume._build_journal_resume, test suites).

Updated open questions (was 4, now 7)

  1. Preferred fix scope: (a) close emission-site gaps or (b) task-file scan as primary?
  2. Correct the task_assignment memory — who owns the correction (secretary ad-hoc save now, or wait until fix PR ships)?
  3. Recency filter design — options: filter by status != "completed", filter by mtime within N seconds, filter by phase_transition boundary, or no filter (rely on same-turn proximity).
  4. Pre-filter self-check — keep existing CODER_TYPES guard at auditor_reminder.py:81-83 as invariant.
  5. Non-PACT Task() dispatches (user-direct, non-PACT commands) — Signal feat: Add PACT slash commands and update PACT_Prompt with explicit agent and PR workflows #2 handles these naturally because TaskCreate is platform-called; Signal Claude Code: not strictly follow framework #1 does not. Another point favoring feat: Add PACT slash commands and update PACT_Prompt with explicit agent and PR workflows #2.
  6. Variety re-estimate — scope (a) = 14-15 (orchestrate + ATOMIZE), scope (b) = 9-10 (comPACT). Workflow choice follows scope choice.
  7. Future-proofing — if PACT subscribes to platform TaskCreated hook (orthogonal future work), would provide an even earlier signal than task-file scan. Out of scope here, worth tracking as follow-up.

Source

Full analysis at docs/preparation/auditor-reminder-timing.md (now 564 lines, was 390). Addenda A/B/C contain the audit, falsification, and calibration details.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions