From 138a9d0e46d648cf7dfc99542f19becfe79184a2 Mon Sep 17 00:00:00 2001 From: Kun Chen <3233006+kunchenguid@users.noreply.github.com> Date: Mon, 6 Jul 2026 19:35:57 -0700 Subject: [PATCH 01/63] feat(bin): encode project AGENTS.md authoring bar with canonical self-governance section (#307) * Encode project AGENTS authoring bar * no-mistakes(review): Captain, centralize CLAUDE promotion governance * no-mistakes(review): make ensure_maintenance_section idempotent-success, drop || true guards * no-mistakes(review): separate appended maintenance section on newline-less CLAUDE.md promotion * no-mistakes(review): assert maintenance heading present before separator check in test * no-mistakes(document): sync docs with AGENTS.md authoring bar and self-governance --------- Co-authored-by: fmtest --- AGENTS.md | 3 ++ bin/fm-brief.sh | 9 +++- bin/fm-ensure-agents-md.sh | 34 +++++++++++++++ docs/architecture.md | 1 + docs/scripts.md | 2 +- tests/fm-brief.test.sh | 18 ++++++++ tests/fm-ensure-agents-md.test.sh | 71 +++++++++++++++++++++++++++++++ 7 files changed, 135 insertions(+), 3 deletions(-) create mode 100755 tests/fm-ensure-agents-md.test.sh diff --git a/AGENTS.md b/AGENTS.md index ad4610270..d0cd1890a 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -354,6 +354,9 @@ Firstmate keeps project knowledge split by ownership. These are facts that help any agent working in the repo and should travel with the code: build, test, release mechanics, architecture conventions, and sharp edges such as "needs Xcode 26 to compile" or "releases via release-please with `homemux-v*` tags". This knowledge lives in the project's committed `AGENTS.md`. A project's `AGENTS.md` is the real file; `CLAUDE.md` is a symlink to it. +A project's `AGENTS.md` is only for knowledge useful to almost every future session in that repo. +Prefer a pointer to the authoritative file, command, or doc over repeating what the codebase already shows, and rewrite or prune stale entries instead of appending by default. +The canonical self-governance wording for project `AGENTS.md` files lives in `bin/fm-ensure-agents-md.sh`; this section states the principle and points there. **Fleet and captain-private knowledge** belongs to firstmate. Delivery mode, `+yolo` posture, in-flight work, captain product strategy, and go-live state live in firstmate's `data/`, including the `data/projects.md` registry line and any planning docs. diff --git a/bin/fm-brief.sh b/bin/fm-brief.sh index 70658fcaa..9d1a2f1d9 100755 --- a/bin/fm-brief.sh +++ b/bin/fm-brief.sh @@ -27,7 +27,10 @@ # Ship briefs begin with a worktree-isolation assertion before the branch step. # Scout tasks ignore mode - their deliverable is a report, not a merge. # Ship tasks include a project-memory section so durable project-intrinsic -# learnings can be committed to AGENTS.md through the project's delivery path. +# learnings can be committed to AGENTS.md through the project's delivery path; +# it carries the AGENTS.md authoring bar (widely useful knowledge only, pointers +# over copied detail) and has the crewmate add the fm-ensure-agents-md.sh +# self-governance section when a touched project AGENTS.md lacks it. # Refuses to overwrite an existing brief. set -eu @@ -254,7 +257,9 @@ $RULE1 # Project memory If \`AGENTS.md\` or \`CLAUDE.md\` already exists, or if this task produced durable project-intrinsic knowledge, run \`$FM_ROOT/bin/fm-ensure-agents-md.sh .\` in the worktree. -If this task produced durable project-intrinsic knowledge, record it in \`AGENTS.md\` as part of your change. +Record only project knowledge useful to almost every future session. +For anything the codebase already shows, prefer a pointer to the authoritative file, command, or doc over copying the detail. +If you touch a project \`AGENTS.md\` that lacks \`## Maintaining this file\`, add that short self-governance section from \`$FM_ROOT/bin/fm-ensure-agents-md.sh\` in the same pass. Keep it proportionate: skip \`AGENTS.md\` edits for trivial tasks that produced no durable project knowledge. $DOD diff --git a/bin/fm-ensure-agents-md.sh b/bin/fm-ensure-agents-md.sh index 0ed2c5a67..1a2effd54 100755 --- a/bin/fm-ensure-agents-md.sh +++ b/bin/fm-ensure-agents-md.sh @@ -4,6 +4,9 @@ # relative symlink to it for compatibility. Creates a minimal AGENTS.md skeleton # when neither file exists, promotes a real CLAUDE.md file when it is the only # file present, and refuses to clobber distinct real files or wrong symlinks. +# Owns the canonical "## Maintaining this file" self-governance wording for +# project AGENTS.md files, appending it to created skeletons and promoted +# CLAUDE.md files that lack it. # This is a worktree utility for crewmates, not a supervision script, so it does # not call fm-guard.sh. # Usage: fm-ensure-agents-md.sh [repo-or-worktree-dir] @@ -29,6 +32,35 @@ cd "$DIR" AGENTS=AGENTS.md CLAUDE=CLAUDE.md +write_maintenance_section() { + cat <<'EOF' +## Maintaining this file + +Keep this file for knowledge useful to almost every future agent session in this project. +Do not repeat what the codebase already shows; point to the authoritative file or command instead. +Prefer rewriting or pruning existing entries over appending new ones. +When updating this file, preserve this bar for all agents and keep entries concise. +EOF +} + +ensure_maintenance_section() { + if grep -Fqx '## Maintaining this file' "$AGENTS"; then + return 0 + fi + sep='' + if [ -s "$AGENTS" ]; then + if [ -n "$(tail -c 1 "$AGENTS")" ]; then + sep=$'\n\n' + else + sep=$'\n' + fi + fi + { + printf '%s' "$sep" + write_maintenance_section + } >> "$AGENTS" +} + write_skeleton() { cat > "$AGENTS" <<'EOF' # Project agent memory @@ -37,6 +69,7 @@ This file is the project's committed home for project-intrinsic agent knowledge: - Add durable project-specific notes here as they are discovered through real work. EOF + ensure_maintenance_section } is_correct_claude_symlink() { @@ -101,6 +134,7 @@ fi if [ -e "$CLAUDE" ]; then if [ -f "$CLAUDE" ]; then mv "$CLAUDE" "$AGENTS" + ensure_maintenance_section ln -s "$AGENTS" "$CLAUDE" echo "promoted: moved CLAUDE.md to AGENTS.md and symlinked CLAUDE.md -> AGENTS.md in $DIR" exit 0 diff --git a/docs/architecture.md b/docs/architecture.md index be8b6f527..b2b298345 100644 --- a/docs/architecture.md +++ b/docs/architecture.md @@ -180,6 +180,7 @@ The watcher, wake queue, arm wrapper, and afk daemon are unchanged; X mode is la Durable project-intrinsic agent knowledge lives in each project's committed `AGENTS.md`, with `CLAUDE.md` as a symlink. Ship briefs prompt crewmates to create or update those files through the normal delivery path; `data/projects.md` stays a thin private registry. +Each project `AGENTS.md` carries a short `## Maintaining this file` self-governance section; `bin/fm-ensure-agents-md.sh` owns the canonical wording and appends it when creating the skeleton or promoting an existing `CLAUDE.md`. The full ownership rule - what is project-intrinsic versus fleet-private, and how firstmate keeps the two apart without writing into project clones - is owned by firstmate's operating manual in [`AGENTS.md`](../AGENTS.md) (project memory ownership). ## Operational memory routing diff --git a/docs/scripts.md b/docs/scripts.md index 40e9ffbad..77a68195d 100644 --- a/docs/scripts.md +++ b/docs/scripts.md @@ -12,7 +12,7 @@ If you have changed away from the firstmate home in an interactive shell, invoke | `fm-update.sh` | Self-update the running firstmate repo and registered secondmate homes with fast-forward-only pulls from origin | | `fm-backlog-handoff.sh` | Move already-judged in-scope queued backlog items from the main home into a seeded secondmate home | | `fm-brief.sh` | Scaffold a ship brief with a worktree-isolation assertion, a report-only scout brief with `--scout`, or a secondmate charter with `--secondmate` | -| `fm-ensure-agents-md.sh` | Ensure project `AGENTS.md` is the real memory file and `CLAUDE.md` symlinks to it | +| `fm-ensure-agents-md.sh` | Ensure project `AGENTS.md` is the real memory file and `CLAUDE.md` symlinks to it; owns the canonical `## Maintaining this file` self-governance wording and appends it when creating the skeleton or promoting a real `CLAUDE.md` | | `fm-guard.sh` | Warn when the primary checkout is tangled, when queued wakes are pending, or when a stale or missing watcher needs a prominent banner; `FM_GUARD_READ_ONLY=1` keeps the alarms but suppresses drain, arm, and checkout repair commands | | `fm-turnend-guard.sh` | Claude Code Stop hook, primary-scoped only: blocks (exit 2, exact reason) a primary turn end when work is in flight without a live identity-matched watcher lock and fresh beacon, using Claude Code's own `stop_hook_active` field so it never blocks twice in one turn (docs/turnend-guard.md) | | `fm-home-seed.sh` | Lease/provision a secondmate home transactionally, clone projects, initialize gates, and maintain `data/secondmates.md` | diff --git a/tests/fm-brief.test.sh b/tests/fm-brief.test.sh index be1df2be9..c127c708e 100755 --- a/tests/fm-brief.test.sh +++ b/tests/fm-brief.test.sh @@ -76,6 +76,24 @@ test_no_mistakes_dod_wording() { pass "fm-brief.sh: no-mistakes DOD wording avoids the apostrophe regression" } +test_ship_project_memory_wording() { + local home id brief + home="$TMP_ROOT/project-memory-home" + mkdir -p "$home/data" + id="brief-memory-c1" + FM_HOME="$home" "$ROOT/bin/fm-brief.sh" "$id" some-proj >/dev/null 2>&1 + brief="$home/data/$id/brief.md" + assert_present "$brief" "brief was not scaffolded" + assert_grep "Record only project knowledge useful to almost every future session." "$brief" \ + "project-memory contract lost the durable-knowledge bar" + assert_grep "prefer a pointer to the authoritative file, command, or doc over copying the detail" "$brief" \ + "project-memory contract lost pointer-over-copy guidance" + assert_grep "lacks \`## Maintaining this file\`, add that short self-governance section" "$brief" \ + "project-memory contract lost the self-governance add-in-same-pass rule" + pass "fm-brief.sh: ship project-memory wording carries the AGENTS.md authoring bar" +} + test_script_parses test_ship_modes_generate_clean_briefs test_no_mistakes_dod_wording +test_ship_project_memory_wording diff --git a/tests/fm-ensure-agents-md.test.sh b/tests/fm-ensure-agents-md.test.sh new file mode 100755 index 000000000..85b4013e5 --- /dev/null +++ b/tests/fm-ensure-agents-md.test.sh @@ -0,0 +1,71 @@ +#!/usr/bin/env bash +# Behavior tests for bin/fm-ensure-agents-md.sh. +set -u + +# shellcheck source=tests/lib.sh +. "$(dirname "${BASH_SOURCE[0]}")/lib.sh" + +TMP_ROOT=$(fm_test_tmproot fm-ensure-agents-md) + +test_created_agents_md_includes_self_governance() { + local repo agents + repo="$TMP_ROOT/new-project" + mkdir -p "$repo" + "$ROOT/bin/fm-ensure-agents-md.sh" "$repo" >/dev/null 2>&1 || fail "fm-ensure-agents-md.sh failed for empty project" + agents="$repo/AGENTS.md" + assert_present "$agents" "AGENTS.md was not created" + assert_present "$repo/CLAUDE.md" "CLAUDE.md symlink was not created" + [ -L "$repo/CLAUDE.md" ] || fail "CLAUDE.md is not a symlink" + assert_grep "## Maintaining this file" "$agents" "self-governance section heading missing" + assert_grep "Keep this file for knowledge useful to almost every future agent session in this project." "$agents" \ + "self-governance section lost the future-session bar" + assert_grep "Do not repeat what the codebase already shows; point to the authoritative file or command instead." "$agents" \ + "self-governance section lost pointer-over-copy guidance" + assert_grep "Prefer rewriting or pruning existing entries over appending new ones." "$agents" \ + "self-governance section lost rewrite-or-prune guidance" + assert_grep "When updating this file, preserve this bar for all agents and keep entries concise." "$agents" \ + "self-governance section lost all-agents maintenance guidance" + pass "fm-ensure-agents-md.sh: created AGENTS.md includes self-governance section" +} + +test_promoted_claude_md_includes_self_governance() { + local repo agents count + repo="$TMP_ROOT/claude-project" + mkdir -p "$repo" + cat > "$repo/CLAUDE.md" <<'EOF' +# Existing agent memory + +Run tests with `make test`. +EOF + "$ROOT/bin/fm-ensure-agents-md.sh" "$repo" >/dev/null 2>&1 || fail "fm-ensure-agents-md.sh failed for CLAUDE.md promotion" + agents="$repo/AGENTS.md" + assert_present "$agents" "AGENTS.md was not created during promotion" + [ -L "$repo/CLAUDE.md" ] || fail "CLAUDE.md is not a symlink after promotion" + assert_grep "Run tests with \`make test\`." "$agents" \ + "promotion lost existing CLAUDE.md content" + count=$(grep -Fc "## Maintaining this file" "$agents") + [ "$count" -eq 1 ] || fail "promotion wrote $count self-governance sections" + assert_grep "Keep this file for knowledge useful to almost every future agent session in this project." "$agents" \ + "promoted AGENTS.md missing self-governance wording" + pass "fm-ensure-agents-md.sh: promoted CLAUDE.md includes self-governance section" +} + +test_promoted_claude_md_without_trailing_newline_keeps_blank_separator() { + local repo agents before + repo="$TMP_ROOT/no-trailing-newline-project" + mkdir -p "$repo" + printf '# Existing agent memory\n\nRun tests with make test.' > "$repo/CLAUDE.md" + "$ROOT/bin/fm-ensure-agents-md.sh" "$repo" >/dev/null 2>&1 || fail "fm-ensure-agents-md.sh failed for newline-less CLAUDE.md promotion" + agents="$repo/AGENTS.md" + assert_grep "Run tests with make test." "$agents" \ + "newline-less promotion lost or mangled the last content line" + assert_grep "## Maintaining this file" "$agents" \ + "newline-less promotion did not append the self-governance section" + before=$(grep -B1 -Fx '## Maintaining this file' "$agents" | head -n 1) + [ -z "$before" ] || fail "self-governance heading not preceded by a blank line (got: $before)" + pass "fm-ensure-agents-md.sh: newline-less promotion keeps a blank separator line" +} + +test_created_agents_md_includes_self_governance +test_promoted_claude_md_includes_self_governance +test_promoted_claude_md_without_trailing_newline_keeps_blank_separator From 463cb51df699004e247dda6c15232d83a780d299 Mon Sep 17 00:00:00 2001 From: Johans Ballestar <47162770+Ballestar@users.noreply.github.com> Date: Mon, 6 Jul 2026 20:30:53 -0700 Subject: [PATCH 02/63] feat(skills): add captain-invocable bearings status-report skill (#300) * Add captain-invocable bearings skill Generates a pick-up-where-I-left-off status report from live fleet state to data/status-report-.md plus a concise chat summary. Read-mostly procedure: reads backlog, per-task crew state via bin/fm-crew-state.sh, open PRs via gh-axi, scout reports, pending decisions, and date-gated queued work; composes the exemplar's sections (TL;DR, Check first, Landed, In flight, Plans, Decisions pending, Date-gated/queued); never tears down, merges, or mutates task state as a side effect. * no-mistakes(document): docs: list new /bearings skill in README built-in skills table --- .agents/skills/bearings/SKILL.md | 58 ++++++++++++++++++++++++++++++++ README.md | 1 + 2 files changed, 59 insertions(+) create mode 100644 .agents/skills/bearings/SKILL.md diff --git a/.agents/skills/bearings/SKILL.md b/.agents/skills/bearings/SKILL.md new file mode 100644 index 000000000..bd38167b2 --- /dev/null +++ b/.agents/skills/bearings/SKILL.md @@ -0,0 +1,58 @@ +--- +name: bearings +description: Generate a "pick up where I left off" status report from firstmate's live fleet state. Use when the captain invokes /bearings or asks for a bearings report, morning brief, status report, catch-up, "where did I leave off", or "what's in the works". Reads live fleet state cheaply (backlog, per-task crew state, open PRs, scout reports, pending decisions, date-gated queued work), composes a scannable dated report to data/status-report-.md, and surfaces a concise version in chat; it is read-mostly and must not tear down, merge, or mutate task state as a side effect of producing the brief. +user-invocable: true +metadata: + internal: true +--- + +# bearings + +Generate a "pick up where I left off" report from the fleet's live state, so the captain can resume in one read after a break, a night, or a context reset. +The deliverable is a dated markdown file plus a concise chat summary; this is the reusable version of the worked example at `data/status-report-2026-07-06.md` when that file is present in this home. +This skill is read-mostly. +It reads fleet state and writes exactly one report file. +It never tears down a task, merges a PR, dispatches new work, or mutates any task state as a side effect of producing the brief - those belong to the captain's explicit word and the normal task lifecycle. + +## What it does + +1. **Gather live fleet state, cheaply and in this order.** + Read each source once and do not re-derive what a script already reports. + - `data/backlog.md` - the In flight / Queued / Done sections are the spine of the report; the active backend edits this file in place, so reading it directly is always correct. + - Each `state/.meta` for the current direct-report set, then each task's live state via `bin/fm-crew-state.sh `. + Never infer current state from a raw `tail` of `state/.status`: that log is an append-only wake-event history and its last line goes stale the moment a resolved gate lets a run resume, while `bin/fm-crew-state.sh` reconciles the authoritative run-step over the stale log line - which is exactly what a catch-up report needs. + - Open PRs awaiting the captain's merge, via `gh-axi` per repo touched by in-flight or recent tasks. + A PR-based ship task records `pr=` in `state/.meta`; collect those repos, list their open PRs, and cross-reference each task's recorded PR. + - Recent scout deliverables at `data//report.md` and any planning docs the captain should pick up from. + - Pending human decisions (needs-decision findings), needed credentials or logins (such as an SSO re-auth), and date-gated queued items. + A queued item only becomes "next work" when its blocker is gone and its time/date gate has arrived; until then it stays queued with the reason. + +2. **Compose the report with these sections, matching the worked example's structure, tone, and level of detail.** + The exemplar is `data/status-report-2026-07-06.md` in this home's `data/` when present; match its scannability, not a raw state dump. + - **Title** - `# Bearings - ` (the exemplar used "Morning status" for a morning brief; use that phrasing when the captain specifically asks for a morning brief). + - **TL;DR** - two or three sentences framing where things stand. + - **Check first** - anything likely waiting on the captain: a PR to merge, a needed credential or login, or anything blocking pickup, each PR with the full `https://...` URL, never a bare `#number`. + - **Landed** since the captain last worked - merged PRs, completed scouts, and finished local-only work, drawn from the Done section and recent merges. + - **In flight** now - each live direct report with its current state in one line; if nothing is in flight, say so plainly rather than omitting the section. + - **Plans / main pickup points** - pointers to the relevant `data//report.md` files and any Lavish boards (`.lavish/*.html`) the captain should reopen. + - **Decisions pending** from the captain - relayed verbatim from needs-decision findings, with options where the crewmate offered them. + - **Date-gated / queued** next work - queued items whose blocker is gone and whose gate has arrived, plus anything still blocked with the reason. + +3. **Write the report to a dated file so it persists, and surface a concise version in chat.** + - Write the full report to `data/status-report-.md` using today's date. + This is the required artifact; it lives in gitignored `data/` alongside the worked example. + If the file already exists for today, overwrite it with the fresh snapshot rather than appending. + - Surface a concise version to the captain in chat - the TL;DR plus the "Check first" list - and point to the file for the full picture. + - For a richer review surface, optionally offer a Lavish board with `lavish-axi` when the report has enough structure to deserve one, but the markdown file is the required artifact and the chat summary is the required minimum. + +## Tone and content rules + +- This report is a private, captain-facing internal artifact that lives in gitignored `data/`, so unlike normal captain chat it MAY reference task ids, PR URLs, and repo names - the captain works with these directly and needs them to resume; keep it organized and scannable, not a raw dump. +- Every PR reference is a full `https://...` URL, never a bare `#number`; a shorthand `#number` is fine only as a back-reference after the full URL has already appeared in the same report. +- Never include PHI or secret values; the report is an operational artifact, but it is still subject to the same security and compliance rules that govern everything else in this fleet. + +## Supervision discipline + +This skill is read-mostly and changes no fleet state. +Do not tear down a task, merge a PR, dispatch queued work, or mutate any `state/` or `data/` file other than the single report file as a side effect of generating the brief. +If the state you read suggests an action - a PR ready to merge, a queued item whose gate has arrived, a needs-decision finding - name it in the report under "Check first" or "Date-gated / queued" and let the captain decide, rather than taking the action from inside this skill. diff --git a/README.md b/README.md index d6d31cd00..018b189df 100644 --- a/README.md +++ b/README.md @@ -120,6 +120,7 @@ Claude and grok use the slash form shown here; codex uses the same names with `$ | Skill | What it does | | ------------------ | -------------------------------------------------------------------------------------------------------------------------------------------- | | `/afk` | Enter away-mode supervision: the sub-supervisor self-handles routine wakes in bash and escalates only captain-relevant events as one batched digest, cutting supervision cost while you step away | +| `/bearings` | Generate a "pick up where I left off" status report from live fleet state - backlog, per-task crew state, open PRs, scout reports, pending decisions, and date-gated queued work - written to a dated file in `data/` and surfaced concisely in chat; read-mostly, mutates no task state | | `/updatefirstmate` | Self-update the running firstmate and its secondmates to the latest from origin with fast-forward-only pulls, then re-read instructions and nudge secondmates | | `/stow` | Sweep the session for uncaptured durable knowledge, route each finding to its disk home per AGENTS.md, file undone next steps to the backlog, and report what is now safe to reset | From 49ffe5dcf85a5a002f9885eaf620c9ecea92ab2c Mon Sep 17 00:00:00 2001 From: pchuri Date: Tue, 7 Jul 2026 12:32:27 +0900 Subject: [PATCH 03/63] fix(watcher): make PID identity locale-invariant (#285) * fix(watcher): pin LC_ALL=C in fm_pid_identity for locale-invariant identity ps's lstart date format follows the caller's LC_TIME/LC_ALL. The watcher records its process identity under one locale, but arm/guard/turn-end re-read it under the machine's ambient locale. On a non-C locale (e.g. ko_KR) the two strings differ only in the date portion, so fm_watcher_lock_matches_pid / fm_watcher_healthy reject a genuinely live watcher - breaking fm-watch-arm.sh, fm-guard.sh, and fm-turnend-guard.sh on every non-C-locale machine. Pin LC_ALL=C on that one ps call so the write and read sides agree regardless of machine locale, matching the LC_ALL=C determinism the file already uses elsewhere. Add a colocated regression test asserting fm_pid_identity is locale-invariant across exported LC_ALL/LC_TIME. * no-mistakes(document): Document watcher PID identity coverage --- CONTRIBUTING.md | 2 +- bin/fm-wake-lib.sh | 5 ++++- tests/fm-watcher-lock.test.sh | 30 +++++++++++++++++++++++++++--- 3 files changed, 32 insertions(+), 5 deletions(-) diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index f0fb5d648..a11e8613b 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -71,7 +71,7 @@ for script in bin/*.sh bin/backends/*.sh; do bash -n "$script"; done # syntax- shellcheck bin/*.sh bin/backends/*.sh tests/*.sh # lint the toolbelt and behavior tests; CI enforces this for test_script in tests/*.test.sh; do bash "$test_script"; done # behavior tests, matching CI and no-mistakes commands.test tests/fm-wake-queue.test.sh # durable wake queue losslessness, catch-up, double-drain, duplicate-collapse, and drain liveness guard tests -tests/fm-watcher-lock.test.sh # watcher singleton, lock-race, watch-arm liveness, and guard-warning tests +tests/fm-watcher-lock.test.sh # watcher singleton, lock-race, PID identity stability, watch-arm liveness, and guard-warning tests tests/fm-turnend-guard.test.sh # shared supervision predicate plus Claude Stop-hook scoping, loop guard, fail-open, and live watcher health tests tests/fm-watch-triage.test.sh # always-on watcher triage: benign absorb, actionable surface, stale status-log override, wedge threshold, repeated wedge demand marker, heartbeat backstop, and afk one-shot coherence tests/fm-daemon.test.sh # sub-supervisor classifier, /afk presence-gating, max-defer, composer, and fm-send submit tests diff --git a/bin/fm-wake-lib.sh b/bin/fm-wake-lib.sh index 56395fd08..c0e27b1e2 100755 --- a/bin/fm-wake-lib.sh +++ b/bin/fm-wake-lib.sh @@ -28,7 +28,10 @@ fm_pid_identity() { case "$pid" in ''|*[!0-9]*) return 1 ;; esac - out=$(ps -p "$pid" -o lstart= -o command= 2>/dev/null) || return 1 + # Pin LC_ALL=C so lstart's date format is locale-invariant: the identity is + # written under one locale but re-read under the machine's ambient locale, which + # would otherwise mismatch on a non-C locale (e.g. ko_KR) and reject a live watcher. + out=$(LC_ALL=C ps -p "$pid" -o lstart= -o command= 2>/dev/null) || return 1 [ -n "$out" ] || return 1 printf '%s\n' "$out" | sed 's/^[[:space:]]*//' } diff --git a/tests/fm-watcher-lock.test.sh b/tests/fm-watcher-lock.test.sh index 1e9c6420a..563bfee04 100755 --- a/tests/fm-watcher-lock.test.sh +++ b/tests/fm-watcher-lock.test.sh @@ -1,8 +1,8 @@ #!/usr/bin/env bash # tests/fm-watcher-lock.test.sh - watcher singleton + lock-primitive races + -# watch-arm liveness + guard warnings. These are safety-critical concurrency -# invariants (a race bug may not reproduce through an e2e), so they stay as -# focused real-process units. +# PID identity stability + watch-arm liveness + guard warnings. These are +# safety-critical process invariants (a race bug may not reproduce through an +# e2e), so they stay as focused real-process units. set -u # shellcheck source=tests/wake-helpers.sh @@ -619,7 +619,31 @@ test_arm_fails_loud_when_no_fresh_watcher_confirmable() { pass "arm reports FAILED and exits non-zero when no fresh watcher can be confirmed" } +test_pid_identity_is_locale_invariant() { + # The watcher records its process identity under one locale; arm/guard/turn-end + # re-read it under the machine's ambient locale. ps's lstart date format follows + # LC_TIME, so an unpinned read on a non-C locale (e.g. ko_KR) would differ only + # in the date portion and reject a genuinely live watcher. The fix pins LC_ALL=C + # inside fm_pid_identity, so its output must be byte-identical regardless of the + # caller's exported LC_ALL/LC_TIME. That invariant holds on any host because the + # pin is internal, so this stays deterministic on CI even where an alternate + # locale like ko_KR.UTF-8 is not installed (the equality then holds trivially). + local live baseline via_lc_all via_lc_time + sleep 300 & + live=$! + baseline=$(LC_ALL=C bash -c '. "$1"; fm_pid_identity "$2"' _ "$LIB" "$live" 2>/dev/null) + via_lc_all=$(LC_ALL=ko_KR.UTF-8 bash -c '. "$1"; fm_pid_identity "$2"' _ "$LIB" "$live" 2>/dev/null) + via_lc_time=$(LC_TIME=ko_KR.UTF-8 bash -c 'unset LC_ALL; . "$1"; fm_pid_identity "$2"' _ "$LIB" "$live" 2>/dev/null) + kill "$live" 2>/dev/null || true + wait "$live" 2>/dev/null || true + [ -n "$baseline" ] || fail "fm_pid_identity produced no baseline identity under LC_ALL=C" + [ "$via_lc_all" = "$baseline" ] || fail "fm_pid_identity varied with exported LC_ALL (got '$via_lc_all', want '$baseline')" + [ "$via_lc_time" = "$baseline" ] || fail "fm_pid_identity varied with exported LC_TIME (got '$via_lc_time', want '$baseline')" + pass "fm_pid_identity is locale-invariant across LC_ALL/LC_TIME" +} + test_singleton_start +test_pid_identity_is_locale_invariant test_stale_watch_lock_reclaimed test_live_stale_watch_lock_is_actionable test_guard_warnings From 0a7cb88c9d190771c1e11d86bedc7866c266f187 Mon Sep 17 00:00:00 2001 From: "Stephen S. Brouhard" Date: Tue, 7 Jul 2026 00:09:18 -0400 Subject: [PATCH 04/63] docs: document codex app backend contract (#222) * docs: reconcile Codex App backend contract * no-mistakes(document): Sync backend docs * docs: clarify Codex Desktop bridge blocker * no-mistakes(document): Align Codex App backend docs * no-mistakes(test): Captain, stabilize watcher self-eviction test cadence * no-mistakes(document): Document Codex App backend contract * no-mistakes(document): Captain, document blocked codex-app coverage * docs: make Codex App contract doc authoritative * no-mistakes(document): Align Codex App backend docs * docs: redact local Codex App smoke paths --------- Co-authored-by: Stephen Brouhard --- AGENTS.md | 8 +- CONTRIBUTING.md | 6 +- README.md | 2 + bin/fm-backend.sh | 3 + bin/fm-spawn.sh | 7 +- docs/architecture.md | 1 + docs/codex-app-backend.md | 211 ++++++++++++++++++++++++++++++++++ docs/configuration.md | 3 +- docs/scripts.md | 4 +- tests/fm-backend.test.sh | 27 ++++- tests/fm-watcher-lock.test.sh | 2 +- 11 files changed, 256 insertions(+), 18 deletions(-) create mode 100644 docs/codex-app-backend.md diff --git a/AGENTS.md b/AGENTS.md index d0cd1890a..f023d989c 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -77,7 +77,7 @@ config/crew-harness crewmate harness override; LOCAL, gitignored; absent or "de config/crew-dispatch.json optional crewmate dispatch profiles; LOCAL, gitignored; firstmate-maintained but human-editable natural-language rules that choose a per-task harness/model/effort profile (section 4). Inherited by secondmate homes config/secondmate-harness harness the PRIMARY uses to launch SECONDMATE agents, optionally followed by a model and effort token on the same line (" [] []"; section 4); LOCAL, gitignored; absent or "default" harness falls back to config/crew-harness then firstmate's own. The primary's own setting; NOT inherited into secondmate homes (secondmates do not spawn secondmates) config/backlog-backend backlog backend override; LOCAL, gitignored; absent or "tasks-axi" = default tasks-axi backend, "manual" = force hand-editing; inherited by secondmate homes (section 10) -config/backend runtime session-provider backend override for new tasks; LOCAL, gitignored; absent = falls through to runtime auto-detection (the runtime firstmate itself is executing inside), then tmux; tmux is the verified reference backend (docs/tmux-backend.md), while herdr, zellij, orca, and cmux are experimental spawn backends (docs/herdr-backend.md, docs/zellij-backend.md, docs/orca-backend.md, docs/cmux-backend.md) - herdr and cmux can also be selected by runtime auto-detection, zellij and orca never are (always explicit); not inherited into secondmate homes +config/backend runtime session-provider backend override for new tasks; LOCAL, gitignored; absent = falls through to runtime auto-detection (the runtime firstmate itself is executing inside), then tmux; tmux is the verified reference backend (docs/tmux-backend.md), while herdr, zellij, orca, and cmux are experimental spawn backends (docs/herdr-backend.md, docs/zellij-backend.md, docs/orca-backend.md, docs/cmux-backend.md) - herdr and cmux can also be selected by runtime auto-detection, zellij and orca never are (always explicit), and codex-app is not accepted; see docs/codex-app-backend.md; not inherited into secondmate homes config/cmux-socket-password optional cmux control-socket password; LOCAL, gitignored; read fresh on every cmux CLI call and passed through without ever overriding an operator's own ambient CMUX_SOCKET_PASSWORD when absent (docs/cmux-backend.md "Setup") config/x-mode.env generated X-mode watcher cadence; LOCAL, gitignored; source before arming watcher when present data/ personal fleet records; LOCAL, gitignored as a whole @@ -478,6 +478,7 @@ bin/fm-spawn.sh projects/ --backend herdr # experimental herdr backe bin/fm-spawn.sh projects/ --backend zellij # experimental zellij backend (docs/zellij-backend.md); version-gates at spawn bin/fm-spawn.sh projects/ --backend orca # experimental Orca backend (docs/orca-backend.md); Orca owns worktree + terminal; Escape unsupported bin/fm-spawn.sh projects/ --backend cmux # experimental cmux backend (docs/cmux-backend.md); GUI-first macOS-only, treehouse still owns worktree; requires a one-time socket-access setup (docs/cmux-backend.md "Setup") +# backend=codex-app is not accepted yet; see docs/codex-app-backend.md. bin/fm-spawn.sh projects/ --scout # scout task; records kind=scout in meta bin/fm-spawn.sh --secondmate # launch a registered persistent secondmate in its home bin/fm-spawn.sh --secondmate # launch or recover an explicit secondmate home @@ -488,7 +489,7 @@ Dispatch several tasks in one call by passing `id=repo` pairs instead of a singl If one pair fails, the rest still run and the batch exits non-zero. When `config/crew-dispatch.json` exists, include a shared `--harness` for every crewmate or scout batch after consulting the dispatch rules. -The script resolves the harness (`fm-harness.sh crew` for crewmate/scout tasks only when `config/crew-dispatch.json` is absent, `fm-harness.sh secondmate` for `kind=secondmate`; section 4), resolves the runtime backend (`--backend`, then `FM_BACKEND`, then `config/backend`, then runtime auto-detection - the runtime firstmate itself is executing inside, from `$TMUX`/`HERDR_ENV=1`/cmux runtime signals, nesting resolved innermost-first (`$TMUX`, then `HERDR_ENV=1`, then cmux's primary `CMUX_WORKSPACE_ID` marker and documented macOS-only fallbacks last, since cmux is a terminal application rather than a nestable multiplexer; docs/cmux-backend.md "Runtime auto-detection") - then `tmux`; an auto-detected herdr or cmux spawn prints a loud stderr notice, auto-detected tmux stays silent; zellij and orca are never auto-detected, only explicit `--backend `/`FM_BACKEND=`/`config/backend`), validates the requested backend against spawn-capable adapters, owns the verified launch templates, resolves the project's delivery mode (`fm-project-mode.sh`) for ship/scout tasks, and records `harness=`, `model=`, `effort=`, `kind=`, `mode=`, and `yolo=` in the task's meta; only a non-default runtime backend is recorded as `backend=` because absent means tmux. +The script resolves the harness (`fm-harness.sh crew` for crewmate/scout tasks only when `config/crew-dispatch.json` is absent, `fm-harness.sh secondmate` for `kind=secondmate`; section 4), resolves the runtime backend (`--backend`, then `FM_BACKEND`, then `config/backend`, then runtime auto-detection - the runtime firstmate itself is executing inside, from `$TMUX`/`HERDR_ENV=1`/cmux runtime signals, nesting resolved innermost-first (`$TMUX`, then `HERDR_ENV=1`, then cmux's primary `CMUX_WORKSPACE_ID` marker and documented macOS-only fallbacks last, since cmux is a terminal application rather than a nestable multiplexer; docs/cmux-backend.md "Runtime auto-detection") - then `tmux`; an auto-detected herdr or cmux spawn prints a loud stderr notice, auto-detected tmux stays silent; zellij and orca are never auto-detected, only explicit `--backend `/`FM_BACKEND=`/`config/backend`), validates the requested backend against spawn-capable adapters, rejects `codex-app` as unknown, owns the verified launch templates, resolves the project's delivery mode (`fm-project-mode.sh`) for ship/scout tasks, and records `harness=`, `model=`, `effort=`, `kind=`, `mode=`, and `yolo=` in the task's meta; only a non-default runtime backend is recorded as `backend=` because absent means tmux. A backend spawn refusal - a missing dependency, an unauthenticated socket, or a version gate - must be surfaced to the captain as a blocker; never silently retry the spawn on a different backend to work around it. A non-flag third argument containing whitespace is treated as a raw launch command (only for verifying new adapters). When `config/crew-dispatch.json` exists, the script refuses crewmate or scout launches without an explicit harness because firstmate must have already resolved the profile choice at intake. @@ -496,6 +497,7 @@ When `--model` or `--effort` is omitted, the corresponding meta value is `defaul For `kind=secondmate`, the same script launches in the registered or explicit firstmate home instead of running `treehouse get` for a project, records `home=` and `projects=`, and uses the charter brief as the launch prompt. For ship and scout tasks, tmux/herdr/zellij/cmux create a runtime endpoint and run `treehouse get`; Orca creates an Orca-owned worktree, validates it, then creates the terminal. In all cases, the script asserts the resolved worktree is a genuine isolated worktree distinct from the primary checkout (aborting the spawn otherwise, to prevent the worktree tangle of section 8), installs the turn-end hook, records `state/.meta`, and launches the agent with the brief. +Selecting `backend=codex-app` fails as an unknown backend; see `docs/codex-app-backend.md`. For grok, the turn-end hook is one firstmate-owned global hook under `$GROK_HOME/hooks/`, or `~/.grok/hooks/` when `GROK_HOME` is unset, activated only when the worktree holds the per-task `.fm-grok-turnend` token pointer that matches `state/.grok-turnend-token`; teardown removes the pointer and token. For `kind=secondmate`, the script creates the same kind of runtime endpoint but starts directly in the persistent home. With herdr, ordinary crewmate and scout spawns use the current `FM_HOME` workspace; a primary `--secondmate` spawn uses the secondmate target home's workspace, so secondmate-owned tabs do not mix into the primary `firstmate` space. @@ -694,7 +696,7 @@ Heartbeats back off exponentially while they are the only wakes firing (600s dou Due per-task checks run before signal scanning so chatty crewmate status updates cannot starve slow polls like merge detection. Never rely on hooks or status files alone; when a heartbeat wake does reach you, the review of every window is mandatory and unconditional. -Each task's backend live-task inventory is the ground truth (tmux when `backend=` is absent; a task's meta may record a different `backend=` - herdr, zellij, orca, and cmux are the other implemented, spawn-capable, experimental backends today, docs/herdr-backend.md, docs/zellij-backend.md, docs/orca-backend.md, and docs/cmux-backend.md). +Each task's backend live-task inventory is the ground truth (tmux when `backend=` is absent; a task's meta may record a different `backend=` - herdr, zellij, orca, and cmux are the other implemented, spawn-capable, experimental backends today, docs/herdr-backend.md, docs/zellij-backend.md, docs/orca-backend.md, and docs/cmux-backend.md; codex-app is not selectable, see docs/codex-app-backend.md). For `kind=secondmate`, an idle pane is healthy. A secondmate may be sitting on its own watcher with no visible pane changes, so parent supervision uses status writes plus heartbeat review, not pane-staleness. `fm-watch.sh` therefore skips stale-pane wakes for windows whose meta records `kind=secondmate`. diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index a11e8613b..7e6229950 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -39,14 +39,14 @@ See the [no-mistakes quick start](https://kunchenguid.github.io/no-mistakes/star Everything personal to one captain's fleet (`.env`, `data/`, `state/`, `config/`, `projects/`, `.no-mistakes/`) is gitignored; never commit it. The root `.tasks.toml` is tracked `tasks-axi` config for `data/backlog.md`; compatible `tasks-axi` is the default backend for routine backlog mutations. A local `config/backlog-backend=manual` opt-out forces hand-editing and stays gitignored. - A local `config/backend` file explicitly overrides runtime auto-detection for new task endpoints and stays gitignored; spawn-supported values are `tmux` plus experimental `herdr`, `zellij`, `orca`, and `cmux`. + A local `config/backend` file explicitly overrides runtime auto-detection for new task endpoints and stays gitignored; spawn-supported values are `tmux` plus experimental `herdr`, `zellij`, `orca`, and `cmux`, while `codex-app` is documented only in `docs/codex-app-backend.md`. It does not make `data/` tracked. - Helper scripts in `bin/` are plain bash. Each starts with a usage header comment; keep it accurate when you change behavior. Test scripts and helpers in `tests/` are plain bash too. `shellcheck bin/*.sh bin/backends/*.sh tests/*.sh` must pass, and CI enforces it. - Changes to harness adapters (detection in `bin/fm-harness.sh`, launch and hook mechanics in `bin/fm-spawn.sh`, busy signatures in `bin/fm-watch.sh` and `bin/fm-tmux-lib.sh`, cleanup in `bin/fm-teardown.sh`, and facts in `.agents/skills/harness-adapters/SKILL.md`) must be verified empirically against the real harness, never written from documentation alone. -- Changes to runtime session backends (`bin/fm-backend.sh`, `bin/backends/`, and the scripts that dispatch through them) need empirical adapter notes in the relevant backend guide: `docs/tmux-backend.md`, `docs/herdr-backend.md`, `docs/zellij-backend.md`, `docs/orca-backend.md`, or `docs/cmux-backend.md`. +- Changes to runtime session backends (`bin/fm-backend.sh`, `bin/backends/`, and the scripts that dispatch through them) need empirical adapter notes in the relevant backend guide: `docs/tmux-backend.md`, `docs/herdr-backend.md`, `docs/zellij-backend.md`, `docs/orca-backend.md`, `docs/cmux-backend.md`, or `docs/codex-app-backend.md` for blocked Codex App transport work. - In Markdown, put each full sentence on its own line. - `README.md` stays a concise overview plus pointers: it never carries a wall of inline detail. Route detail to the most specific `docs/` file (architecture, configuration, or a backend guide) and link to it instead. @@ -100,7 +100,7 @@ tests/fm-teardown.test.sh # fm-teardown.sh landed-work safety an tests/fm-review-diff.test.sh # fm-review-diff.sh authoritative review diff coverage: recorded pr_head=, fetched refs/pull//head, no-pr local branch behavior, and warning fallback tests/fm-pr-merge.test.sh # fm-pr-merge.sh records pr= and available pr_head= before merging, parses PR URLs into gh-axi number/--repo calls, defaults to squash, preserves explicit merge methods, rejects malformed URLs and repo overrides, and propagates real merge failures tests/fm-crew-state.test.sh # fm-crew-state.sh current-state reconciliation: run-step authority including closed panes and ci log-tail checks-green detection, stale checks-green and needs-decision/blocked superseded by resumed work, genuine-parked, cross-branch runs-list attribution, pane/status-log fallback, scout skip, torn-down/missing-meta graceful -tests/fm-backend.test.sh # runtime-backend abstraction: fm-backend.sh selection/meta/dispatch helpers, shell-portable sourced backend matching, and old-vs-new fake-tool command-log conformance for fm-send/fm-peek/fm-spawn/fm-teardown +tests/fm-backend.test.sh # runtime-backend abstraction: fm-backend.sh selection/meta/dispatch helpers, shell-portable sourced backend matching, blocked codex-app refusal, and old-vs-new fake-tool command-log conformance for fm-send/fm-peek/fm-spawn/fm-teardown tests/fm-backend-tmux-smoke.test.sh # real (private-socket) tmux smoke test for the tmux adapter: create/duplicate-refuse, send text + Enter, send literal + key, bounded capture, live-window resolve, kill tests/fm-backend-herdr.test.sh # fake herdr CLI unit tests for the experimental herdr adapter, including version/tool gates, target parsing, send/capture, structural composer-state verification, slash-submit retry regression coverage, native busy state, per-home workspace-label resolution, default-tab prune safety, restored-layout husk replacement, and verified CLI bug workarounds tests/fm-backend-herdr-smoke.test.sh # real herdr adapter smoke test, skipped when herdr or jq is unavailable, using an isolated throwaway HERDR_SESSION and guarded session cleanup, including live-agent duplicate refusal and no-agent husk replacement diff --git a/README.md b/README.md index 018b189df..c3e32e550 100644 --- a/README.md +++ b/README.md @@ -109,6 +109,7 @@ Setup guides for tmux (the default) and every other supported backend (herdr, ze You chat with the first mate. It routes each request to a crewmate in its own session endpoint and git worktree, supervises the fleet with a zero-token event-driven watcher, and brings you finished PRs, approved local merges, or investigation reports. Optional secondmates extend this to persistent domain supervisors, dispatch profiles let you steer which harness handles which task, and an opt-in X mode lets the same fleet answer public mentions. +`codex-app` is not a runtime backend yet; [docs/codex-app-backend.md](docs/codex-app-backend.md) owns the Codex App boundary. Full architecture - the supervision engine, worktree isolation, secondmates, dispatch profiles, project modes, optional X mode, fleet sync, and self-update - is in [docs/architecture.md](docs/architecture.md). @@ -145,6 +146,7 @@ Firstmate's skills live in two separate places with different audiences: - [docs/zellij-backend.md](docs/zellij-backend.md) - setup guide for the experimental zellij backend, plus its verification notes and known gaps. - [docs/orca-backend.md](docs/orca-backend.md) - setup guide for the experimental Orca backend, plus its lifecycle notes and known gaps. - [docs/cmux-backend.md](docs/cmux-backend.md) - setup guide for the experimental cmux backend, plus its verification notes and known gaps. +- [docs/codex-app-backend.md](docs/codex-app-backend.md) - Codex App backend boundary, evidence, and rollout contract. - [docs/turnend-guard.md](docs/turnend-guard.md) - the primary session's structural "no turn ends blind" backstop: verified Claude Code Stop-hook mechanism, scoping, and known gaps. - [docs/scripts.md](docs/scripts.md) - the `bin/` toolbelt reference. - [`AGENTS.md`](AGENTS.md) - firstmate's full operating manual for the orchestrator agent. diff --git a/bin/fm-backend.sh b/bin/fm-backend.sh index ff0a93434..6aa0d439d 100644 --- a/bin/fm-backend.sh +++ b/bin/fm-backend.sh @@ -26,6 +26,8 @@ # marker) with no explicit backend setting - unlike Orca, which stays # never-auto-detected because it also owns the task worktree; see # docs/cmux-backend.md for its empirical basis. +# Codex App is intentionally not in the known set yet. +# docs/codex-app-backend.md owns that blocked backend contract. # # Compatibility contract: a task's meta may omit `backend=`; every reader here # treats that as `tmux` (fm_backend_of_meta), and fm-spawn.sh does not write @@ -63,6 +65,7 @@ FM_BACKEND_CONFIG_DIR="${FM_CONFIG_OVERRIDE:-$FM_HOME/config}" # spawn-capable; unlike tmux/herdr/zellij it is also the worktree provider. # cmux is EXPERIMENTAL and spawn-capable, session-provider-only like # herdr/zellij - verified against the real 0.64.17 binary (docs/cmux-backend.md). +# codex-app remains deliberately absent; see docs/codex-app-backend.md. FM_BACKEND_KNOWN="tmux herdr zellij orca cmux" FM_BACKEND_SPAWN="tmux herdr zellij orca cmux" diff --git a/bin/fm-spawn.sh b/bin/fm-spawn.sh index ea59454ab..0a6c903f0 100755 --- a/bin/fm-spawn.sh +++ b/bin/fm-spawn.sh @@ -19,9 +19,10 @@ # herdr, zellij, orca, and cmux. Orca owns both the task worktree and # terminal, so ship/scout Orca spawns do not run treehouse get; cmux is a # session provider only, exactly like herdr/zellij, so it does. An -# auto-detected herdr or cmux spawns print a loud stderr notice; -# auto-detected tmux stays silent; zellij and orca are never auto-detected -# (always explicit). Default tmux spawns do not write backend= to meta; +# auto-detected herdr or cmux spawn prints a loud stderr notice; +# auto-detected tmux stays silent; zellij and orca are never auto-detected. +# codex-app is not a known backend yet; docs/codex-app-backend.md owns that +# blocked backend contract. Default tmux spawns do not write backend= to meta; # absent backend= means tmux. cmux does not support --secondmate spawns yet. # A backend spawn refusal (missing dependency, version gate, unauthenticated # socket, or unsupported secondmate mode) is terminal for that selected backend; diff --git a/docs/architecture.md b/docs/architecture.md index b2b298345..4e2d7fc8b 100644 --- a/docs/architecture.md +++ b/docs/architecture.md @@ -62,6 +62,7 @@ Zellij's container shape is simpler than herdr's: one shared `firstmate` session Orca is experimental and selected only explicitly: Orca owns both worktree and terminal lifecycle, records `orca_worktree_id=` and `terminal=`, and removes worktrees through `orca worktree rm` only after the usual firstmate teardown checks pass. Its current behavior and limitations are recorded in `docs/orca-backend.md`. cmux is experimental, GUI-first, macOS-only, and can be selected explicitly or by runtime auto-detection from its primary `CMUX_WORKSPACE_ID` marker plus documented fallback signals: treehouse remains its worktree provider (cmux is a session provider only, like herdr/zellij), and its full verification - the socket access setup requirement with Automation mode recommended, the read-screen-fails-on-a-fresh-surface finding, the close-surface-refuses-on-the-last-surface finding, the source-verified runtime marker and fallback behavior, and known gaps - is recorded in `docs/cmux-backend.md`. cmux's container shape is one workspace per task with one surface, no per-home container split; workspace titles are scoped by the active home label plus a short hash of the resolved `FM_ROOT` path, and `--secondmate` spawns are refused, mirroring Orca. +Codex App support is recorded in `docs/codex-app-backend.md`; it is not selectable as a runtime backend. ## Worktrees, not branches in your checkout diff --git a/docs/codex-app-backend.md b/docs/codex-app-backend.md new file mode 100644 index 000000000..6a2bbd8eb --- /dev/null +++ b/docs/codex-app-backend.md @@ -0,0 +1,211 @@ +# Codex App backend contract + +Status: blocked for Firstmate as a selectable shell backend. +The Codex Desktop host-tool loop works, including status-file writes, but Firstmate does not yet have a supported shell-callable bridge to those host tools. + +This document replaces the earlier passive visible-thread ledger shape. +A manual ledger is not a backend. + +## Backend acceptance contract + +A Codex App backend must satisfy the same lifecycle contract as the terminal-backed adapters: + +1. Firstmate creates the task endpoint and receives a durable thread id. +2. Firstmate sends the initial prompt and later operator messages to that endpoint. +3. Firstmate observes enough live thread state or transcript to supervise the task. +4. Firstmate can archive, kill, or otherwise stop supervising the endpoint. +5. The Codex thread can report back through Firstmate's normal `state/.status` lifecycle. + +The final point is mandatory. +If a Desktop-owned thread cannot write Firstmate status files, the backend cannot be treated as complete. + +## Verified Desktop host-tool smoke + +Latest verified host-tool smoke date: 2026-07-06. +Environment: Codex Desktop host tools, local host, saved project `/projects/sift`, Desktop-owned worktree ``, Firstmate home ``. +Local absolute path prefixes are redacted as `` and ``; file names, host-tool ids, thread ids, status lines, and report values are otherwise exact. + +Codex Desktop/OpenAI local bundle metadata from the smoke machine: + +```text +$ /usr/libexec/PlistBuddy -c 'Print :CFBundleShortVersionString' /Applications/Codex.app/Contents/Info.plist +26.623.101652 + +$ /usr/libexec/PlistBuddy -c 'Print :CFBundleVersion' /Applications/Codex.app/Contents/Info.plist +4674 + +$ /usr/libexec/PlistBuddy -c 'Print :CFBundleIdentifier' /Applications/Codex.app/Contents/Info.plist +com.openai.codex + +$ stat -f '%Sm %N' -t '%Y-%m-%d %H:%M:%S %z' /Applications/Codex.app/Contents/Info.plist +2026-07-02 21:55:53 -0400 /Applications/Codex.app/Contents/Info.plist +``` + +Smoke target files: + +```text +/state/codex-app-host-smoke-20260706-live.status +/data/codex-app-host-smoke-20260706-live/report.md +``` + +Host-tool operation sequence: + +1. `list_projects` confirmed the saved project target. +2. `create_thread` requested a new Codex Desktop project worktree thread. +3. `list_threads` recovered the created thread id after queued worktree setup. +4. `read_thread` observed the active and completed initial turn. +5. Shell reads verified the status/report files under the Firstmate home. +6. `send_message_to_thread` delivered a follow-up to the same thread. +7. `read_thread` observed the completed follow-up turn. +8. `set_thread_archived` archived the thread. +9. A final `read_thread` still returned the transcript and showed `status.type=notLoaded`. + +Exact host-tool requests and relevant output: + +```text +list_projects: + projectId=/projects/sift + projectKind=local + label=sift + path=/projects/sift + +create_thread request: + target.type=project + target.projectId=/projects/sift + target.environment.type=worktree + prompt smoke_id=codex-app-host-smoke-20260706-live + prompt status_file=/state/codex-app-host-smoke-20260706-live.status + prompt report_file=/data/codex-app-host-smoke-20260706-live/report.md + prompt required status line: working: Codex Desktop thread started + prompt required sentinel: FM_CODEX_APP_HOST_TOOL_SMOKE_20260706_LIVE_OK + +create_thread response: + pendingWorktreeId=local:a4a96438-a0ed-4305-b83c-5a47336f5abf + +list_threads query=codex-app-host-smoke-20260706-live: + id=019f39ea-5cca-7031-bfb0-f8054a2b253a + hostId=local + status=active + cwd= + +read_thread initial turn while active: + thread.id=019f39ea-5cca-7031-bfb0-f8054a2b253a + thread.status.type=active + cwd= + agentMessage: Running the smoke exactly as delegated: repo identity first, then the Firstmate status/report writes, then the requested `sed` checks. + +read_thread initial turn after completion: + thread.status.type=idle + turn.status=completed + durationMs=54923 + +$ pwd + + +$ git rev-parse --show-toplevel + + +$ git branch --show-current + +$ sed -n '1,20p' /state/codex-app-host-smoke-20260706-live.status +working: Codex Desktop thread started + +$ sed -n '1,40p' /data/codex-app-host-smoke-20260706-live/report.md +smoke_id=codex-app-host-smoke-20260706-live +cwd= +git_root= +branch= +status_file=/state/codex-app-host-smoke-20260706-live.status +status_file_write=ok +sentinel=FM_CODEX_APP_HOST_TOOL_SMOKE_20260706_LIVE_OK + +send_message_to_thread request: + threadId=019f39ea-5cca-7031-bfb0-f8054a2b253a + prompt required status line: done: follow-up delivered through send_message_to_thread + +send_message_to_thread response: + threadId=019f39ea-5cca-7031-bfb0-f8054a2b253a + +read_thread follow-up turn: + turn.status=completed + durationMs=7118 + +$ sed -n '1,20p' /state/codex-app-host-smoke-20260706-live.status +working: Codex Desktop thread started +done: follow-up delivered through send_message_to_thread + +set_thread_archived request: + threadId=019f39ea-5cca-7031-bfb0-f8054a2b253a + archived=true + +set_thread_archived response: + threadId=019f39ea-5cca-7031-bfb0-f8054a2b253a + archived=true + +read_thread after archive: + thread.id=019f39ea-5cca-7031-bfb0-f8054a2b253a + thread.status.type=notLoaded + thread.cwd= + transcript still included the initial and follow-up completed turns. +``` + +Result: a Desktop-owned Codex thread can write Firstmate status files when the prompt gives it the absolute status path and the Desktop permission context can write that checkout. +The return channel is real at the Codex Desktop host-tool layer. + +## Codex Desktop API blocker + +Firstmate's backend scripts are Bash entry points. +They can call `tmux`, `herdr`, `zellij`, primitive Orca CLI surfaces, and `cmux` directly. +The Codex Desktop host tools verified above are available to the Codex Desktop conversation, not to arbitrary Firstmate subprocesses. +The missing piece is therefore a supported Codex Desktop transport that a Bash backend can call, not another Firstmate-local ledger. + +The available Codex CLI and app-server probes found useful pieces but not a supported visible-thread backend transport: + +- `codex app-server --stdio` exposes JSON-RPC methods such as `thread/start`, `turn/start`, `thread/read`, and `thread/archive`. +- A one-shot stdio probe could create a thread record, and `thread/archive` worked through that same stdio process. +- The managed daemon path was unavailable in this Desktop install. +- A raw proxy attempt against the Desktop control socket did not accept plain JSON-RPC framing. + +That is not enough to add `codex-app` to `FM_BACKEND_KNOWN` or `FM_BACKEND_SPAWN`. +A Firstmate backend must be able to create a thread, start or continue turns, read live state while turns run, and archive/stop the same endpoint through a Codex Desktop-supported shell-callable API. +Shipping a local ledger would only record intentions; it would not supervise the actual Desktop thread. + +## Required Codex Desktop bridge + +Firstmate should implement a Codex App adapter only after Codex Desktop exposes one of these supported interfaces: + +- A supported CLI wrapper around the Desktop host tools: create thread, send message, read transcript/state, archive thread. +- A documented JSON-RPC or MCP transport that Firstmate can call from Bash with stable request/response framing. +- A small maintained helper binary/script that speaks the supported transport and returns plain JSON to `bin/backends/codex-app.sh`. + +Minimum command semantics: + +```text +create: + input: task id, cwd/worktree request, initial prompt + output: thread id, Desktop-owned cwd if different, initial status + +send: + input: thread id, text + output: accepted/rejected delivery result + +capture/read: + input: thread id, bounded transcript or status cursor + output: enough text/state for fm-peek.sh, fm-watch.sh, and fm-crew-state.sh + +archive/kill: + input: thread id + output: archived/stopped result + +status return channel: + the thread must be able to append Firstmate status lines to state/.status +``` + +Once that bridge exists, the implementation should add a real `bin/backends/codex-app.sh`, persist `backend=codex-app` and `codex_app_thread_id=` in `state/.meta`, and wire spawn/send/peek/watch/teardown through the same dispatcher paths used by the existing adapters. + +## Rollout clause + +After a supported shell-callable Codex Desktop/OpenAI bridge exists, Firstmate should implement Codex App for ship and scout tasks first. +Secondmate support remains out of scope until ship/scout supervision, status return, send/read, and archive/teardown are proven through the normal backend dispatcher. + +Until then, Codex App support remains a verified host-tool smoke plus this blocked backend contract, not a selectable backend. diff --git a/docs/configuration.md b/docs/configuration.md index 44552b7bf..6485e5a12 100644 --- a/docs/configuration.md +++ b/docs/configuration.md @@ -28,6 +28,7 @@ Auto-detected herdr or cmux prints a stderr notice naming `config/backend` and ` Zellij and Orca are never auto-detected; select them by putting the name in a local `config/backend` file, by exporting `FM_BACKEND=`, or by telling the first mate in chat. Any value other than `tmux`, `herdr`, `zellij`, `orca`, or `cmux` is rejected until another adapter is implemented and verified. `fm-spawn.sh` accepts `tmux`, `herdr`, `zellij`, `orca`, and `cmux` for ship and scout tasks; `backend=orca` and `backend=cmux` both still refuse `--secondmate` until secondmate launch semantics are designed for each. +`codex-app` is not an accepted runtime backend yet; [`docs/codex-app-backend.md`](codex-app-backend.md) owns the Codex App boundary. A herdr spawn additionally version-gates against the installed `herdr` binary's protocol and requires `jq`, refusing loudly on an incompatible or missing installation. A zellij spawn additionally version-gates against the installed `zellij` binary's version and requires `jq`, refusing loudly when either is missing or the version is older than 0.44. A cmux spawn additionally version-gates against the installed `cmux` binary's version, requires `jq`, and requires the control socket to be reachable and accessible (see [`docs/cmux-backend.md`](cmux-backend.md) "Setup" for the one-time socket-access configuration this needs; Automation mode is the recommended socket control mode, with Password mode supported via `config/cmux-socket-password`), refusing loudly and non-retryably on a `cmuxOnly`/unauthenticated socket. @@ -220,7 +221,7 @@ FM_STATE_OVERRIDE= # alternate state dir, mainly for tests FM_DATA_OVERRIDE= # alternate data dir, mainly for tests FM_PROJECTS_OVERRIDE= # alternate projects dir, mainly for tests FM_CONFIG_OVERRIDE= # alternate config dir, mainly for tests -FM_BACKEND= # optional runtime backend override for new spawns; tmux/herdr/zellij/orca/cmux support ship/scout spawns +FM_BACKEND= # optional runtime backend override for new spawns; tmux/herdr/zellij/orca/cmux support ship/scout spawns, codex-app is not accepted HERDR_SESSION=default # herdr-only: named session for normal backend ops; not enough for destructive cleanup (docs/herdr-backend.md) FM_BACKEND_HERDR_COMPOSER_LINES=20 # herdr-only: tail lines scanned to locate the composer row for submit verification FM_BACKEND_HERDR_IDLE_RE='^Type a message\.\.\.$' # herdr-only: empty-composer placeholder regex after border/prompt stripping diff --git a/docs/scripts.md b/docs/scripts.md index 77a68195d..d0a1dc15d 100644 --- a/docs/scripts.md +++ b/docs/scripts.md @@ -16,8 +16,8 @@ If you have changed away from the firstmate home in an interactive shell, invoke | `fm-guard.sh` | Warn when the primary checkout is tangled, when queued wakes are pending, or when a stale or missing watcher needs a prominent banner; `FM_GUARD_READ_ONLY=1` keeps the alarms but suppresses drain, arm, and checkout repair commands | | `fm-turnend-guard.sh` | Claude Code Stop hook, primary-scoped only: blocks (exit 2, exact reason) a primary turn end when work is in flight without a live identity-matched watcher lock and fresh beacon, using Claude Code's own `stop_hook_active` field so it never blocks twice in one turn (docs/turnend-guard.md) | | `fm-home-seed.sh` | Lease/provision a secondmate home transactionally, clone projects, initialize gates, and maintain `data/secondmates.md` | -| `fm-spawn.sh` | Spawn one task, several `id=repo` pairs, or a persistent secondmate with `--secondmate`; accepts concrete `--harness`, `--model`, `--effort`, and `--backend` axes; ship/scout spawns require an explicit resolved harness when dispatch profiles are active and an isolated worktree, install per-harness turn-end signaling, and secondmate spawns resolve the secondmate harness plus optional `config/secondmate-harness` model/effort tokens, locally sync the home, propagate declared inheritable config, land herdr tabs in the target home's workspace, land home-scoped zellij tabs in the selected shared zellij session, land cmux workspaces in the shared cmux app, or create Orca worktrees/terminals before launch | -| `fm-backend.sh` | Runtime session-provider backend selector with explicit/env/config/runtime auto-detection precedence, meta helper, selector resolver, spawn-capability validation, operation dispatcher, and shell-portable backend-name membership for bash-sourced scripts or zsh-sourced diagnostics; defaults absent `backend=` meta to `tmux`; `fm_backend_target_exists` is a cheap read-only alive/dead endpoint check that never starts a server or session; `fm_backend_composer_state` exposes backend composer checks for guarded submit paths | +| `fm-spawn.sh` | Spawn one task, several `id=repo` pairs, or a persistent secondmate with `--secondmate`; accepts concrete `--harness`, `--model`, `--effort`, and `--backend` axes; rejects `backend=codex-app`; ship/scout spawns require an explicit resolved harness when dispatch profiles are active and an isolated worktree, install per-harness turn-end signaling, and secondmate spawns resolve the secondmate harness plus optional `config/secondmate-harness` model/effort tokens, locally sync the home, propagate declared inheritable config, land herdr tabs in the target home's workspace, land home-scoped zellij tabs in the selected shared zellij session, land cmux workspaces in the shared cmux app, or create Orca worktrees/terminals before launch | +| `fm-backend.sh` | Runtime session-provider backend selector with explicit/env/config/runtime auto-detection precedence, meta helper, selector resolver, spawn-capability validation, operation dispatcher, and shell-portable backend-name membership for bash-sourced scripts or zsh-sourced diagnostics; deliberately keeps `codex-app` out of known/spawn-capable backends; defaults absent `backend=` meta to `tmux`; `fm_backend_target_exists` is a cheap read-only alive/dead endpoint check that never starts a server or session; `fm_backend_composer_state` exposes backend composer checks for guarded submit paths | | `fm-backend-hometag-lib.sh` | Shared home-tag derivation for zellij tab titles and cmux workspace titles, using the active `FM_HOME` label plus a short hash of the resolved `FM_ROOT` path | | `backends/tmux.sh` | Verified tmux session-provider adapter used by `fm-backend.sh`; owns create, send, capture, current-path, live-window, and kill primitives | | `backends/herdr.sh` | Experimental herdr session-provider adapter used by `fm-backend.sh`; owns version/tool gating, per-home workspace/tab creation, created-vs-adopted default-tab prune safety, restored-layout husk respawn replacement, session-scoped CLI calls, send with structural composer-state verification, capture, native busy-state, current-path, label-based live discovery, and kill primitives | diff --git a/tests/fm-backend.test.sh b/tests/fm-backend.test.sh index ef1f62003..9ea34fb55 100755 --- a/tests/fm-backend.test.sh +++ b/tests/fm-backend.test.sh @@ -13,8 +13,8 @@ # diffs the two command logs byte-for-byte - the report's P1 checklist # item "run current main scripts and refactored scripts against the same # fake tools and compare command logs". -# 3. Asserts the new `--backend`/`FM_BACKEND` selection refuses an unknown -# backend loudly (tmux is the only verified adapter in P1). +# 3. Asserts the `--backend`/`FM_BACKEND` selection refuses unknown backends +# and the blocked `codex-app` backend loudly. # # fm-watch.sh's signal/stale/check/heartbeat wake-string contract is already # exercised end-to-end against this refactor by tests/fm-watch-triage.test.sh @@ -445,13 +445,15 @@ test_backend_validate_refuses_unknown() { fm_backend_validate tmux 2>/dev/null || fail "fm_backend_validate should accept tmux" fm_backend_validate orca 2>/dev/null || fail "fm_backend_validate should accept orca" local out - # bogus names a backend with no adapter at all; tmux, herdr, zellij, and - # orca are all known adapters, and all four are spawn-supported. + # bogus names a backend with no adapter at all; tmux, herdr, zellij, orca, + # and cmux are all known adapters and spawn-supported. out=$(fm_backend_validate bogus 2>&1) && fail "fm_backend_validate should refuse bogus (no such adapter)" assert_contains "$out" "unknown backend 'bogus'" "fm_backend_validate did not name the rejected backend" + out=$(fm_backend_validate codex-app 2>&1) && fail "fm_backend_validate should refuse codex-app" + assert_contains "$out" "unknown backend 'codex-app'" "fm_backend_validate accepted codex-app" out=$(fm_backend_validate "tmux herdr" 2>&1) && fail "fm_backend_validate should refuse a multi-token backend name" assert_contains "$out" "unknown backend 'tmux herdr'" "fm_backend_validate accepted a multi-token backend name" - pass "fm_backend_validate: implemented adapters accepted, an unknown backend refused loudly" + pass "fm_backend_validate: implemented adapters accepted, unknown and blocked codex-app backends refused loudly" } test_backend_source_shell_portable() { @@ -485,8 +487,11 @@ test_backend_validate_spawn_accepts_orca() { fm_backend_validate_spawn herdr 2>/dev/null || fail "fm_backend_validate_spawn should accept herdr" fm_backend_validate_spawn zellij 2>/dev/null || fail "fm_backend_validate_spawn should accept zellij" fm_backend_validate_spawn orca 2>/dev/null || fail "fm_backend_validate_spawn should accept orca" + fm_backend_validate_spawn cmux 2>/dev/null || fail "fm_backend_validate_spawn should accept cmux" out=$(fm_backend_validate_spawn bogus 2>&1) && fail "fm_backend_validate_spawn should still refuse unknown backends" assert_contains "$out" "unknown backend 'bogus'" "fm_backend_validate_spawn did not preserve unknown-backend validation" + out=$(fm_backend_validate_spawn codex-app 2>&1) && fail "fm_backend_validate_spawn should refuse codex-app" + assert_contains "$out" "unknown backend 'codex-app'" "fm_backend_validate_spawn accepted codex-app" out=$(fm_backend_validate_spawn "tmux herdr" 2>&1) && fail "fm_backend_validate_spawn should refuse a multi-token backend name" assert_contains "$out" "unknown backend 'tmux herdr'" "fm_backend_validate_spawn accepted a multi-token backend name" pass "fm_backend_validate_spawn: all implemented lifecycle backends are spawn-supported" @@ -942,6 +947,17 @@ test_spawn_refuses_unknown_backend_flag() { pass "fm-spawn.sh --backend bogus is refused loudly" } +test_spawn_refuses_codex_app_backend_flag() { + local out status + out=$(FM_ROOT_OVERRIDE='' FM_HOME='' FM_STATE_OVERRIDE='' FM_DATA_OVERRIDE='' \ + FM_PROJECTS_OVERRIDE='' FM_CONFIG_OVERRIDE='' FM_SPAWN_NO_GUARD=1 \ + "$ROOT/bin/fm-spawn.sh" nope-codex-app-z1 projects/none claude --backend codex-app 2>&1) + status=$? + [ "$status" -ne 0 ] || fail "fm-spawn --backend codex-app should refuse" + assert_contains "$out" "unknown backend 'codex-app'" "fm-spawn did not preserve the blocked codex-app contract" + pass "fm-spawn.sh --backend codex-app is refused" +} + test_spawn_refuses_unknown_fm_backend_env() { local out status out=$(FM_ROOT_OVERRIDE='' FM_HOME='' FM_STATE_OVERRIDE='' FM_DATA_OVERRIDE='' \ @@ -1053,6 +1069,7 @@ test_spawn_conformance_old_vs_new test_spawn_symlinked_project_prefix_avoids_false_refusal test_teardown_conformance_old_vs_new test_spawn_refuses_unknown_backend_flag +test_spawn_refuses_codex_app_backend_flag test_spawn_refuses_unknown_fm_backend_env test_spawn_default_backend_writes_no_meta_field test_spawn_explicit_backend_flag_beats_autodetect_herdr_env diff --git a/tests/fm-watcher-lock.test.sh b/tests/fm-watcher-lock.test.sh index 563bfee04..26d847df7 100755 --- a/tests/fm-watcher-lock.test.sh +++ b/tests/fm-watcher-lock.test.sh @@ -415,7 +415,7 @@ test_watcher_self_evicts_on_lock_takeover() { state="$dir/state" fakebin="$dir/fakebin" out="$dir/watch.out" - PATH="$fakebin:$PATH" FM_STATE_OVERRIDE="$state" FM_POLL=1 FM_SIGNAL_GRACE=1 FM_CHECK_INTERVAL=999999 FM_HEARTBEAT=999999 "$WATCH" > "$out" & + PATH="$fakebin:$PATH" FM_STATE_OVERRIDE="$state" FM_POLL=0.2 FM_SIGNAL_GRACE=1 FM_CHECK_INTERVAL=999999 FM_HEARTBEAT=999999 "$WATCH" > "$out" & pid=$! i=0 while [ "$i" -lt 50 ]; do From a790a207bab0ba7485dbef93278a7721d10cf421 Mon Sep 17 00:00:00 2001 From: "Stephen S. Brouhard" Date: Tue, 7 Jul 2026 00:10:28 -0400 Subject: [PATCH 05/63] docs: add Codex Desktop coordination skill (#275) * docs: add Codex App coordination skill * no-mistakes(review): Captain, mark Codex App skill agent-only * no-mistakes(document): Document Codex App backend boundary * no-mistakes(document): Captain, document Codex Desktop backend boundary * no-mistakes(lint): Captain, lint clean * no-mistakes(document): Document Codex Desktop boundaries * docs: narrow Codex App skill playbook --- .agents/skills/firstmate-codexapp/SKILL.md | 108 ++++++++++++++++++ .../firstmate-codexapp/agents/openai.yaml | 4 + AGENTS.md | 1 + 3 files changed, 113 insertions(+) create mode 100644 .agents/skills/firstmate-codexapp/SKILL.md create mode 100644 .agents/skills/firstmate-codexapp/agents/openai.yaml diff --git a/.agents/skills/firstmate-codexapp/SKILL.md b/.agents/skills/firstmate-codexapp/SKILL.md new file mode 100644 index 000000000..a8eccfde8 --- /dev/null +++ b/.agents/skills/firstmate-codexapp/SKILL.md @@ -0,0 +1,108 @@ +--- +name: firstmate-codexapp +description: >- + Agent-only playbook for coordinating visible Codex Desktop threads alongside Firstmate without pretending they are a selectable shell backend. + Use before creating, reading, steering, archiving, debugging, or reviewing a Codex App visible thread for Firstmate work, and before responding to requests to make Codex App native to Firstmate. +user-invocable: false +metadata: + internal: true +--- + +# firstmate-codexapp + +## Overview + +Use this playbook when Firstmate work needs a visible Codex Desktop thread. +The current supported shape is Desktop host-tool choreography plus an explicit status-file return-channel check, not a `codex-app` value in `FM_BACKEND`. + +## Boundary + +Codex Desktop visible threads are companion host-tool workflows, not a selectable Firstmate backend. +Read `docs/codex-app-backend.md` when it exists in this checkout; that document owns the acceptance contract, bridge requirement, status-return requirement, and staged rollout. + +If local helper scripts exist for Codex App work, use only helpers explicitly provided by the operator or maintained by Firstmate. +For helpers outside `bin/`, inspect the source or header before running `--help`. + +## Preflight + +1. Confirm this session is running inside Codex Desktop and that the host tools are exposed. + Search exact names when needed: `create_thread`, `list_threads`, `read_thread`, `send_message_to_thread`, `archive`, and `set_thread_archived`. +2. Confirm the target repository is already saved as a Codex Desktop project. + No host tool currently creates Codex App projects for an agent, so the human must add the project in Desktop before a created thread can reliably land there. +3. Do not create projectless threads for repo work. + If the project is absent, stop and ask for the project to be added or use a normal Firstmate backend instead. +4. Decide whether this is a real Firstmate-managed task or a visible companion thread. + A real task needs a task id, an isolated worktree or Desktop-owned cwd, a branch plan, and a writable `state/.status` path. + +## Create And Send + +When creating a visible thread, use the Desktop host tool, not shell imitation. +Target the saved project and ask the worker to start by reporting: + +```text +pwd +git rev-parse --show-toplevel +git branch --show-current +git log --oneline --max-count=3 +``` + +For writable repo work, instruct the worker to use the Codex-created current directory. +Do not tell it to `cd` into the saved project checkout for edits, commits, no-mistakes, pushes, or PR work. + +When sending follow-up instructions, use `send_message_to_thread`. +If the user types directly into the visible thread, treat that as authoritative and reconcile from `read_thread` instead of undoing it. + +## Status Return Channel + +A Desktop-owned Codex thread can append to Firstmate status files only when the prompt gives an absolute path and the Desktop permission context can write that checkout. +That makes status writes a verified return-channel requirement, not a fact to assume. + +For a Firstmate-managed task, include an explicit status instruction: + +```text +Append supervisor-visible status lines to /state/.status. +Use only these prefixes for supervisor-actionable changes: working:, needs-decision:, blocked:, done:, failed:. +Before doing substantive work, append "working: Codex Desktop thread started". +``` + +Verify the return channel before treating the thread as supervised: + +- `read_thread` shows the worker attempted the status write. +- The local `state/.status` file contains the expected line. +- If available, the transcript includes a file-change entry for that status file. + +If the thread cannot write the status file, keep it as a visible companion thread only. +Do not claim it is a complete Firstmate backend. + +## Observe And Reconcile + +Use `read_thread` for thread truth. +Use `list_threads` only to find or recover a visible thread id, not as a replacement for reading the transcript. + +For Firstmate reconciliation, prefer concrete evidence: + +- thread id and project +- current Desktop-owned cwd +- branch name +- last meaningful thread state +- latest status file line +- PR URL when one exists + +Avoid repeating long transcripts into Firstmate docs or PR bodies. +Summarize only the host-tool calls, the status-file result, and the archive result. + +## Archive + +Archive through the Desktop host tool: `archive` when that is the exposed primitive, or `set_thread_archived(threadId=, archived=true)` when that is the exposed tool name. +Archiving can remove the thread from normal sidebar/project views, but it should not erase the transcript or landed work. + +For companion threads, archive the thread and report where the durable work landed. +If there is a real Firstmate task record, leave teardown decisions to the normal Firstmate task flow instead of this skill. + +## Failure Signals + +- Missing Desktop project: ask the human to add the target project in Codex Desktop, or use a normal backend. +- Missing host tools: do not simulate them with shell files; use a terminal backend instead. +- Status file not updated: treat the thread as unsupervised until the return channel is proven. +- Worker editing the saved project checkout instead of its Desktop cwd: stop and decide whether to salvage the branch before continuing. +- Production `codex-app` backend request: read `docs/codex-app-backend.md` and do not invent a local adapter. diff --git a/.agents/skills/firstmate-codexapp/agents/openai.yaml b/.agents/skills/firstmate-codexapp/agents/openai.yaml new file mode 100644 index 000000000..81263294d --- /dev/null +++ b/.agents/skills/firstmate-codexapp/agents/openai.yaml @@ -0,0 +1,4 @@ +interface: + display_name: "Firstmate Codex App" + short_description: "Operate visible Codex Desktop threads" + default_prompt: "Use $firstmate-codexapp to coordinate visible Codex Desktop threads without pretending they are a shell backend." diff --git a/AGENTS.md b/AGENTS.md index f023d989c..c7638a390 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -865,6 +865,7 @@ These skills are not captain-invocable; they are conditional operating reference - `stuck-crewmate-recovery` - load after a stale wake, looping pane, repeated confusion, an answered-by-brief question, an unresponsive crewmate, or a failed steer. - `secondmate-provisioning` - load before creating, seeding, validating, launching, handing backlog to, recovering, pushing inherited config into, or retiring a secondmate home, and before editing `data/secondmates.md`. - `fmx-respond` - load on an `x-mention ` `check:` wake to handle the mention, on an `x-mode-error ...` `check:` wake to report the X-mode configuration blocker, and on any milestone or terminal wake for an X-linked task before posting its completion follow-up; relevant only when X mode is on. +- `firstmate-codexapp` - load before coordinating a visible Codex Desktop thread, evaluating a Codex App backend request, or reconciling Codex Desktop host-tool smoke evidence for Firstmate work. - `firstmate-coding-guidelines` - load before changing firstmate's shared, tracked material, as defined by section 1's list, whether editing directly or briefing a crewmate for a firstmate-repo task. ## 14. X mode From f5e89b9aa8e4092f02b615777d20a80999f82fb8 Mon Sep 17 00:00:00 2001 From: Kun Chen <3233006+kunchenguid@users.noreply.github.com> Date: Tue, 7 Jul 2026 11:41:42 -0700 Subject: [PATCH 06/63] fix(afk): stop herdr escalation redelivery loop (#317) * fix(afk): recognize unbordered herdr composer rows to stop escalation redelivery loop fm_backend_herdr_composer_state only recognized bordered composer rows (the grok shape). Real claude and codex render their live input row with no border at all, so once a harness's own startup banner scrolled out of the capture window the classifier read the composer as unknown forever. fm_backend_herdr_send_text_submit never confirmed "empty", so escalate_flush never cleared state/.subsuper-escalations, and the away-mode daemon retyped and resubmitted the same buffered digest every housekeeping cycle - reproduced live against a real herdr+claude pane (5+ identical deliveries in 40s). The classifier now recognizes an unbordered (bare) composer row led by a known prompt glyph alongside the existing bordered shape, keeping whichever match is bottom-most so a stale decorative box never outranks the live composer. * no-mistakes(review): Narrow herdr bare prompt matcher * no-mistakes(document): Sync herdr composer docs --- CONTRIBUTING.md | 2 +- bin/backends/cmux.sh | 20 ++--- bin/backends/herdr.sh | 102 ++++++++++++++++++-------- docs/configuration.md | 1 + docs/herdr-backend.md | 47 ++++++++++-- tests/fm-afk-inject-herdr-e2e.test.sh | 12 +-- tests/fm-backend-herdr.test.sh | 86 ++++++++++++++++++++-- 7 files changed, 211 insertions(+), 59 deletions(-) diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index 7e6229950..9ccad4f24 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -102,7 +102,7 @@ tests/fm-pr-merge.test.sh # fm-pr-merge.sh records pr= and avail tests/fm-crew-state.test.sh # fm-crew-state.sh current-state reconciliation: run-step authority including closed panes and ci log-tail checks-green detection, stale checks-green and needs-decision/blocked superseded by resumed work, genuine-parked, cross-branch runs-list attribution, pane/status-log fallback, scout skip, torn-down/missing-meta graceful tests/fm-backend.test.sh # runtime-backend abstraction: fm-backend.sh selection/meta/dispatch helpers, shell-portable sourced backend matching, blocked codex-app refusal, and old-vs-new fake-tool command-log conformance for fm-send/fm-peek/fm-spawn/fm-teardown tests/fm-backend-tmux-smoke.test.sh # real (private-socket) tmux smoke test for the tmux adapter: create/duplicate-refuse, send text + Enter, send literal + key, bounded capture, live-window resolve, kill -tests/fm-backend-herdr.test.sh # fake herdr CLI unit tests for the experimental herdr adapter, including version/tool gates, target parsing, send/capture, structural composer-state verification, slash-submit retry regression coverage, native busy state, per-home workspace-label resolution, default-tab prune safety, restored-layout husk replacement, and verified CLI bug workarounds +tests/fm-backend-herdr.test.sh # fake herdr CLI unit tests for the experimental herdr adapter, including version/tool gates, target parsing, send/capture, structural composer-state verification with bordered and unbordered prompt fixtures, slash-submit retry regression coverage, native busy state, per-home workspace-label resolution, default-tab prune safety, restored-layout husk replacement, and verified CLI bug workarounds tests/fm-backend-herdr-smoke.test.sh # real herdr adapter smoke test, skipped when herdr or jq is unavailable, using an isolated throwaway HERDR_SESSION and guarded session cleanup, including live-agent duplicate refusal and no-agent husk replacement tests/fm-backend-autodetect-smoke.test.sh # real herdr auto-detection smoke test, skipped when herdr, jq, or treehouse is unavailable, using the same guarded session cleanup tests/fm-backend-herdr-workspace-per-home-e2e.test.sh # mandatory isolated E2E for workspace-per-home: primary and secondmate-shaped homes, a crewmate spawned from a secondmate home, teardown, list-live recovery diff --git a/bin/backends/cmux.sh b/bin/backends/cmux.sh index d2a27e29f..5f74c0283 100644 --- a/bin/backends/cmux.sh +++ b/bin/backends/cmux.sh @@ -521,16 +521,16 @@ fm_backend_cmux_capture() { # [expected-label] } # fm_backend_cmux_composer_state: classify the composer's own row as -# empty|pending|unknown. Adapted directly from herdr's structural border-row -# classifier (fm_backend_herdr_composer_state, bin/backends/herdr.sh:598-665) -# per the build task's explicit direction - this is the highest-risk piece of -# a new backend's send-and-verify logic, and cmux's `read-screen` gives the -# same kind of plain-text capture with no cursor-row primitive that herdr's -# `pane read` does, so the same structural approach applies unchanged: locate -# the composer row as the only captured line whose TRIMMED content both -# STARTS and ENDS with the same border glyph (│, ┃, or a plain ASCII |), -# scanning forward and keeping the LAST match so an earlier border-shaped line -# (scrollback, a popup) never outranks the real bottom-anchored composer row. +# empty|pending|unknown. Adapted from the bordered-row branch of herdr's +# structural classifier (fm_backend_herdr_composer_state) per the build task's +# explicit direction - this is the highest-risk piece of a new backend's +# send-and-verify logic, and cmux's `read-screen` gives the same kind of +# plain-text capture with no cursor-row primitive that herdr's `pane read` +# does. The cmux classifier intentionally remains border-row based: locate the +# composer row as the only captured line whose TRIMMED content both STARTS and +# ENDS with the same border glyph (│, ┃, or a plain ASCII |), scanning forward +# and keeping the LAST match so an earlier border-shaped line (scrollback, a +# popup) never outranks the real bottom-anchored composer row. FM_BACKEND_CMUX_COMPOSER_LINES=${FM_BACKEND_CMUX_COMPOSER_LINES:-20} FM_BACKEND_CMUX_IDLE_RE=${FM_BACKEND_CMUX_IDLE_RE:-'^Type a message\.\.\.$'} diff --git a/bin/backends/herdr.sh b/bin/backends/herdr.sh index 51835ca9a..67c0d70c8 100644 --- a/bin/backends/herdr.sh +++ b/bin/backends/herdr.sh @@ -595,19 +595,38 @@ fm_backend_herdr_capture() { # printf '%s' "$out" | tail -n "$lines" } -# fm_backend_herdr_composer_state: classify the composer's own row - the -# interior line of its rounded-corner box - as empty|pending|unknown, scanning -# a generous tail-window capture of . herdr's CLI exposes no -# cursor-row primitive (unlike tmux's #{cursor_y}), so this locates the -# composer row structurally: it is the only captured line whose TRIMMED -# content both STARTS and ENDS with the same border glyph (│, ┃, or a plain -# ASCII |). The box's own top/bottom rows use rounded corners (╭─…─╮ / ╰─…─╯), -# which never match; popup item rows and horizontal separator rows carry no -# border glyph at all; the footer help line ("Enter:send │ … │ …", verified -# grok 0.2.82) uses │ only as an INTERIOR separator and does not start with -# one, so it never matches either. Scans forward and keeps the LAST match, so -# a border-shaped line earlier in scrollback/a popup can never outrank the -# real (bottom-anchored) composer row. +# fm_backend_herdr_composer_state: classify the composer's own row as +# empty|pending|unknown, scanning a generous tail-window capture of . +# herdr's CLI exposes no cursor-row primitive (unlike tmux's #{cursor_y}), so +# this locates the composer row structurally, recognizing TWO row shapes and +# keeping whichever match comes LAST (scanning forward), so a shape earlier in +# scrollback/a popup can never outrank the real (bottom-anchored) composer row: +# +# bordered - a boxed composer (verified grok 0.2.82): the row's TRIMMED +# content both STARTS and ENDS with the same border glyph (│, ┃, +# or a plain ASCII |). The box's own top/bottom rows use rounded +# corners (╭─…─╮ / ╰─…─╯), which never match; popup item rows and +# horizontal separator rows carry no border glyph at all; the +# footer help line ("Enter:send │ … │ …") uses │ only as an +# INTERIOR separator and does not start with one, so it never +# matches either. +# bare - an UNBORDERED composer (verified real claude 2.x and codex +# 0.142.x, both under herdr 0.7.1, docs/herdr-backend.md +# "Incident (2026-07-07)"): the row's TRIMMED content starts with +# one of the verified agent-specific prompt glyphs but carries no +# closing border at all - claude's own live input row is a bare +# "❯ …" with no surrounding │, and codex's is a bare "› …". Both +# harnesses ALSO render bordered decorative boxes elsewhere (a +# startup welcome banner, an update-available notice) that +# satisfy the bordered shape above; requiring a match on EITHER +# shape and keeping the last (bottom-most) one is what keeps the +# live composer winning over a stale decorative box still sitting +# in the same capture window - a bordered box is only ever +# followed later on screen by the actual live composer, never the +# reverse, in every harness observed so far. The bare shape is +# deliberately narrower than the bordered content classifier so a +# no-agent shell fallback prompt (`>`, `$`, `%`, or `#`) falls +# through to `unknown` instead of being misread as delivered. # # empty - blank, a bare prompt glyph, or known ghost/placeholder text # ("Type a message...", verified grok 0.2.82's empty-composer @@ -618,42 +637,67 @@ fm_backend_herdr_capture() { # # composer (e.g. "/compact" -> "/compact compaction # instructions", verified live against real grok 0.2.82) - that # first Enter is a SELECTION, not a submission. -# unknown - the pane could not be read, or no composer row was found in the -# captured window. +# unknown - the pane could not be read, or no composer row (of either shape) +# was found in the captured window. +# +# KNOWN REMAINING GAP (docs/herdr-backend.md "Incident (2026-07-07)"): codex's +# idle composer shows dynamic tip/hint text ("Use /skills to list available +# skills") rather than blank or a fixed placeholder string, so it cannot be +# told apart from real pending input by pattern matching alone - a genuinely +# idle codex composer under herdr classifies as "pending", not "empty". This +# makes injection defer forever rather than redeliver, which is a narrower, +# already-safe failure mode (the buffer is preserved, never silently lost, and +# the max-defer wedge alarm still fires) - not fixed here; a real fix needs +# either an upstream herdr cursor-row/style primitive or a codex-specific +# signal, neither available today. FM_BACKEND_HERDR_COMPOSER_LINES=${FM_BACKEND_HERDR_COMPOSER_LINES:-20} # Known ghost/placeholder composer text. Extend this if another # herdr-verified harness needs its own idle placeholder recognized. FM_BACKEND_HERDR_IDLE_RE=${FM_BACKEND_HERDR_IDLE_RE:-'^Type a message\.\.\.$'} +# Known bare (unbordered) prompt glyphs a composer row may start with: ❯ +# (claude) and › (codex) only. Generic shell-style glyphs > $ % # are still +# recognized after a bordered composer row has already been structurally found. +FM_BACKEND_HERDR_BARE_PROMPT_RE=${FM_BACKEND_HERDR_BARE_PROMPT_RE:-'^[❯›]'} fm_backend_herdr_composer_state() { # -> empty|pending|unknown - local target=$1 cap line trimmed stripped="" found=0 + local target=$1 cap line trimmed stripped="" found=0 shape="" cap=$(fm_backend_herdr_capture "$target" "$FM_BACKEND_HERDR_COMPOSER_LINES") || { printf 'unknown'; return 0; } while IFS= read -r line; do trimmed="${line#"${line%%[![:space:]]*}"}" trimmed="${trimmed%"${trimmed##*[![:space:]]}"}" [ -n "$trimmed" ] || continue case "$trimmed" in - '│'*'│'|'┃'*'┃'|'|'*'|') : ;; - *) continue ;; + '│'*'│'|'┃'*'┃'|'|'*'|') + stripped=$trimmed + shape=bordered + found=1 + ;; + *) + if printf '%s' "$trimmed" | grep -qE "$FM_BACKEND_HERDR_BARE_PROMPT_RE"; then + stripped=$trimmed + shape=bare + found=1 + fi + ;; esac - stripped=$trimmed - found=1 done < <(printf '%s\n' "$cap") [ "$found" -eq 1 ] || { printf 'unknown'; return 0; } - # Strip the border glyphs, then trim again. - stripped=${stripped//│/} - stripped=${stripped//┃/} - stripped=${stripped//|/} - stripped="${stripped#"${stripped%%[![:space:]]*}"}" - stripped="${stripped%"${stripped##*[![:space:]]}"}" + if [ "$shape" = bordered ]; then + # Strip the border glyphs, then trim again. + stripped=${stripped//│/} + stripped=${stripped//┃/} + stripped=${stripped//|/} + stripped="${stripped#"${stripped%%[![:space:]]*}"}" + stripped="${stripped%"${stripped##*[![:space:]]}"}" + fi # A bare prompt glyph = empty composer. case "$stripped" in - '❯'|'>'|'$'|'%'|'#') printf 'empty'; return 0 ;; + '❯'|'›'|'>'|'$'|'%'|'#') printf 'empty'; return 0 ;; esac # Strip a leading prompt glyph before judging what remains. case "$stripped" in - '❯ '*|'> '*|'$ '*|'% '*|'# '*) stripped=${stripped#??} ;; - '❯'*|'>'*|'$'*|'%'*|'#'*) stripped=${stripped#?} ;; + '❯ '*|'› '*|'> '*|'$ '*|'% '*|'# '*) stripped=${stripped#??} ;; + '❯'*|'›'*|'>'*|'$'*|'%'*|'#'*) stripped=${stripped#?} ;; esac stripped="${stripped#"${stripped%%[![:space:]]*}"}" stripped="${stripped%"${stripped##*[![:space:]]}"}" diff --git a/docs/configuration.md b/docs/configuration.md index 6485e5a12..c3d75dcba 100644 --- a/docs/configuration.md +++ b/docs/configuration.md @@ -225,6 +225,7 @@ FM_BACKEND= # optional runtime backend override for new spawns; tmux HERDR_SESSION=default # herdr-only: named session for normal backend ops; not enough for destructive cleanup (docs/herdr-backend.md) FM_BACKEND_HERDR_COMPOSER_LINES=20 # herdr-only: tail lines scanned to locate the composer row for submit verification FM_BACKEND_HERDR_IDLE_RE='^Type a message\.\.\.$' # herdr-only: empty-composer placeholder regex after border/prompt stripping +FM_BACKEND_HERDR_BARE_PROMPT_RE='^[❯›]' # herdr-only: verified agent glyphs recognized as an UNBORDERED (bare) composer row, e.g. claude's ❯ or codex's › (docs/herdr-backend.md "Incident (2026-07-07)") FM_BACKEND_ORCA_COMPOSER_LINES=200 # orca-only: terminal-read lines scanned to locate the composer row for submit verification FM_BACKEND_ORCA_IDLE_RE='^Type a message\.\.\.$' # orca-only: empty-composer placeholder regex after border/prompt stripping FM_ZELLIJ_SESSION=firstmate # zellij-only: named session for normal backend ops and test isolation (docs/zellij-backend.md) diff --git a/docs/herdr-backend.md b/docs/herdr-backend.md index 30b7f3067..18af84031 100644 --- a/docs/herdr-backend.md +++ b/docs/herdr-backend.md @@ -242,16 +242,18 @@ Plain (non-argument) commands like `/new` did submit on the first Enter in the s The tmux backend was NOT affected by this incident: `fm_tmux_composer_state` reads the actual cursor row and classifies it as pending whenever real text remains, so its retry loop correctly issued the second Enter and landed the same live repro; this was confirmed side-by-side against the same real grok pane. **Fix:** `fm_backend_herdr_composer_state` replaces the delta-based check with a structural read of the composer's OWN row, mirroring what the cursor-row read gives tmux. -Herdr's CLI exposes no cursor-row primitive, so the composer row is located by shape instead of position: it is the only line in a generous tail capture whose trimmed content both starts and ends with the same border glyph (`│`, `┃`, or a plain `|`) - the box's own top/bottom rows use rounded corners and never match, popup item rows and separator rows carry no border glyph at all, and the footer help line uses `│` only as an interior separator (never as the first/last character), so none of those can be mistaken for the composer. +Herdr's CLI exposes no cursor-row primitive, so the composer row is located by shape instead of position. +For bordered composers, the row is the only line in a generous tail capture whose trimmed content both starts and ends with the same border glyph (`│`, `┃`, or a plain `|`) - the box's own top/bottom rows use rounded corners and never match, popup item rows and separator rows carry no border glyph at all, and the footer help line uses `│` only as an interior separator (never as the first/last character), so none of those can be mistaken for the composer. +For unbordered live composers, added after the 2026-07-07 incident below, the row is a bottom-most trimmed line starting with a verified agent prompt glyph (`❯` for claude or `›` for codex); decorative bordered boxes above it lose to that bottom-most match. A popup-close-with-placeholder-fill still reads as real content on that row, so it correctly classifies as pending and the retry loop sends the required second Enter, instead of stopping early. Known ghost/placeholder composer text (`Type a message...`, verified grok 0.2.82's empty-composer hint) is recognized and still reads as empty. -`FM_BACKEND_HERDR_IDLE_RE` extends that placeholder match, and `FM_BACKEND_HERDR_COMPOSER_LINES` controls the tail-window scan depth; both are documented in [`docs/configuration.md`](configuration.md). -See `fm_backend_herdr_composer_state` and `fm_backend_herdr_send_text_submit` in `bin/backends/herdr.sh` for the implementation, and `tests/fm-backend-herdr.test.sh`'s composer-state and send-text-submit sections (including a dedicated regression test asserting the second Enter is actually sent) for the fake-harness coverage. +`FM_BACKEND_HERDR_IDLE_RE` extends that placeholder match, `FM_BACKEND_HERDR_BARE_PROMPT_RE` controls the recognized unbordered prompt glyphs, and `FM_BACKEND_HERDR_COMPOSER_LINES` controls the tail-window scan depth; all three are documented in [`docs/configuration.md`](configuration.md). +See `fm_backend_herdr_composer_state` and `fm_backend_herdr_send_text_submit` in `bin/backends/herdr.sh` for the implementation, and `tests/fm-backend-herdr.test.sh`'s composer-state and send-text-submit sections (including the bordered slash-submit retry and unbordered real claude/codex prompt fixtures) for the fake-harness coverage. -## Composer verification: structural border-row read, not delta-based +## Composer verification: structural composer-row read, not delta-based The herdr adapter's submit-verification no longer diffs raw pane content before/after Enter (see the incident above for why that was unsafe). -It instead classifies the composer's own row - located structurally, as described above - as empty or pending after each Enter attempt, retried (Enter only, never retyped) until it reads empty or retries are exhausted. +It instead classifies the composer's own row - located structurally as the bottom-most bordered composer row or verified bare prompt row described above - as empty or pending after each Enter attempt, retried (Enter only, never retyped) until it reads empty or retries are exhausted. This mirrors tmux's cursor-row classification in spirit, without needing an equivalent cursor-row read primitive from herdr's CLI. A dedicated composer-state or cursor-row read primitive is still a candidate upstream Herdr feature request; it would let this backend eventually verify with the same precision as tmux's native cursor-row read, rather than a structural approximation over a plain-text capture. @@ -364,12 +366,37 @@ Fixed by routing through `fm_backend_herdr_cli` (which appends `--session` on to This fix is backend-plumbing, not daemon-specific: it also corrects the same liveness check other callers use (`bin/fm-session-start.sh`'s per-task endpoint-liveness digest read). **Empirical verification (real herdr, isolated session only).** `tests/fm-afk-inject-herdr-e2e.test.sh` mirrors `tests/fm-afk-inject-e2e.test.sh`'s three scenarios (human-partial-input deferral, swallowed-Enter retry, a normal single digest) plus a fourth (a persistently pending composer that never clears must alarm via `state/.subsuper-inject-wedged`, preserve the buffer, and never crash the daemon) against a real, throwaway, NEVER-default `HERDR_SESSION`, torn down with `herdr_safe_stop_and_delete` exactly like `tests/fm-backend-herdr-smoke.test.sh`. -The "supervisor pane" is a tiny deterministic bash loop drawing a bordered composer row (not a real harness), matching the structural classifier `fm_backend_herdr_composer_state` expects; a thin `herdr` PATH shim swallows exactly one `pane send-keys enter` call to simulate the swallowed-Enter scenario, since herdr's real CLI has no built-in way to drop a keystroke. +The "supervisor pane" is a tiny deterministic bash loop drawing a bordered composer row (not a real harness), exercising the bordered branch of `fm_backend_herdr_composer_state`; a thin `herdr` PATH shim swallows exactly one `pane send-keys enter` call to simulate the swallowed-Enter scenario, since herdr's real CLI has no built-in way to drop a keystroke. +Real claude/codex unbordered prompt coverage lives in `tests/fm-backend-herdr.test.sh`'s captured-fixture regression tests described in the 2026-07-07 incident below. Building that test surfaced one more real finding worth recording for anyone writing a similar herdr-driven composer script: `tput cols`, called from WITHIN a script launched into a herdr pane via `pane run`/`send-text`, reported a stale/default `80` regardless of the pane's actual width, while an interactively-typed one-off `tput cols` in the same pane correctly reported its real width (54, in the environment this was verified in). A composer redraw that trusts `tput cols` for its own line-wrapping math can therefore silently overflow the pane's real width and wrap across two terminal rows - breaking the structural single-row border classifier's assumption (the digest looked "concatenated with itself" because the guard never fired: the composer read `unknown` instead of `pending`, so the busy/composer guard did not defer a second attempt). The test's composer script works around this with a hardcoded conservative width rather than trusting `tput cols` in this execution context. -This is a test-harness-only concern - `fm_backend_herdr_composer_state` and `fm_backend_herdr_send_text_submit` themselves are unchanged and were reverified correct once the test's own composer script stayed within the pane's real width - but it is a sharp edge for any future herdr-launched interactive script that computes its own layout from `tput`. +This `tput` issue is a test-harness-only concern: once the test's own composer script stayed within the pane's real width, `fm_backend_herdr_composer_state` and `fm_backend_herdr_send_text_submit` behaved as expected, but it remains a sharp edge for any future herdr-launched interactive script that computes its own layout from `tput`. + +## Incident (2026-07-07): away-mode escalation redelivery loop on herdr + +While `state/.afk` was set on a herdr-backed fleet, `bin/fm-supervise-daemon.sh` re-injected the SAME buffered escalation digest into the primary's own supervisor pane every housekeeping cycle instead of clearing `state/.subsuper-escalations` once delivery landed. +Observed twice: 2026-07-06 (three byte-identical digests in a row) and 2026-07-07 (two byte-identical catch-all-scan digests), each redelivery waking the primary's LLM turn for an escalation it had already handled - defeating away-mode's whole point. + +Reproduced live against a real, isolated `HERDR_SESSION`, a real `claude` process (the primary's own harness) as the supervisor pane, and the real `fm-supervise-daemon.sh` (not a synthetic composer script): with one buffered `stale persisted 241s` escalation and `FM_HOUSEKEEPING_TICK=1`, the daemon delivered the identical digest to the live pane at least 5 times in 40 seconds, and the agent itself eventually replied "The message is identical again and my position hasn't changed... I'll treat further identical escalations as noise" - an exact live match for the reported symptom. + +Root cause: `fm_backend_herdr_composer_state`'s structural composer-row read (added for the 2026-07-03 incident above) recognizes only BORDERED composer rows - a line whose trimmed content both starts and ends with the same border glyph (`│`, `┃`, `|`). +Real `claude`'s live input row is a BARE, unbordered `❯ …` - no border glyph anywhere around it - flanked by plain horizontal-rule separator lines, not a box. +Claude's own startup welcome banner IS bordered, so immediately after launch the classifier's "last bordered row wins" scan locks onto the banner's own blank interior spacer row and misreads it as the composer (a coincidental, and wrong, "empty"). +Once ordinary conversation scrolls that banner out of the 20-line capture window - true of any real supervisor pane with any history at all, which is every production case - NO bordered row exists anywhere in view, so the classifier reports `unknown` for a genuinely empty composer, forever. +`fm_backend_herdr_send_text_submit` treats only `empty` as a confirmed submit; `unknown` counts as failure, so `escalate_flush` never clears the buffer even though the real Enter genuinely submitted the digest to the real pane. +Because `pane_input_pending`'s pre-type guard only defers on `pending` (never `unknown`), the next housekeeping tick's flush attempt retypes and resubmits the SAME unmodified buffer content - the redelivery loop. + +Also discovered while reproducing: real `codex` (0.142.x) has the identical unbordered-live-row shape, using `›` instead of claude's `❯`, confirming this is not claude-specific. +Codex additionally shows dynamic tip/hint text in its idle composer (e.g. "Use /skills to list available skills") rather than a fixed placeholder string or a blank row, so no static regex can safely tell a genuinely idle codex composer apart from real pending text. +This narrower gap is NOT fixed here: an idle real-codex supervisor pane under herdr now classifies as `pending` rather than `unknown`, which makes injection defer indefinitely instead of redelivering - a strictly safer failure mode (the buffer is preserved, never silently dropped, and the existing max-defer wedge alarm still fires) but still not a correct read of the composer. +A real fix needs either an upstream herdr cursor-row/style primitive or a codex-specific signal; neither exists today. + +**Fix:** `fm_backend_herdr_composer_state` now recognizes TWO composer-row shapes in one scan - the existing bordered shape, and a new bare (unbordered) shape: a trimmed line that STARTS with a verified agent-specific prompt glyph (`❯` claude or `›` codex) with no closing border required at all. +The bare-row default is deliberately limited to `❯` and `›`, while generic shell-style glyphs (`>`, `$`, `%`, `#`) stay recognized only after the bordered shape has already identified a composer row, so a no-agent shell fallback cannot be misread as a delivered escalation. +Both shapes are checked in the SAME forward scan, keeping whichever match comes LAST (bottom-most on screen), rather than trying bordered-only first and falling back to bare-only when nothing bordered is found: a bordered decorative box (a welcome banner, an update notice) is always rendered ABOVE the live composer, never below it, in every harness observed, so "last match of either shape wins" always resolves to the genuinely live, bottom-most row instead of a stale decorative box still sitting in the capture window. +See `fm_backend_herdr_composer_state` in `bin/backends/herdr.sh` for the implementation, and `tests/fm-backend-herdr.test.sh`'s "unbordered (bare) composer rows" section (fixtures captured verbatim from real `claude`/`codex` panes) for the regression coverage - each of those tests read `unknown` before this fix and reads the correct verdict after. ## Known gaps and follow-up notes @@ -389,3 +416,9 @@ This is a test-harness-only concern - `fm_backend_herdr_composer_state` and `fm_ - **RESOLVED: a restart's restored-layout husk no longer needs a manual pane close before respawn.** See "Respawn idempotency: a restored task tab is a husk, not a duplicate" above for the fix (`fm_backend_herdr_pane_agent_state`, `fm_backend_herdr_create_task`'s close-and-replace). Left over from that fix: the `dead` (`pane_not_found`) husk classification is exercised only at the unit level, never against the real binary - killing a pane's process on a live server was observed to make herdr reap the whole tab immediately (never leaving a dead-but-still-listed pane for the duplicate check to find), and a real session restart was never observed to produce one either. It remains a conservative, defensively-coded path for a herdr failure mode (e.g. a restored process that fails to start) nobody has reproduced against the real binary yet. +- **Codex's dynamic idle-composer tip text still misreads as `pending`.** See "Incident (2026-07-07)" above. + A real, genuinely idle codex composer under herdr shows rotating hint text instead of a blank row or a fixed placeholder, so `fm_backend_herdr_composer_state` cannot tell it apart from real unsubmitted input by pattern matching. + The practical effect is a safe-but-wrong deferral (the max-defer wedge alarm still fires, the buffer is never lost), not a redelivery loop, so it was left as a follow-up rather than folded into that fix. + A real fix needs either an upstream herdr cursor-row/style primitive or a codex-specific idle signal. +- **Not implemented: a "paused / awaiting-external" crew state for the stale-wedge escalation.** Raised alongside the 2026-07-07 incident: an in-flight crew intentionally idling on a known external wait (a vendor rate limit, say) still trips `bin/fm-supervise-daemon.sh`'s "stale persisted ... (possible wedge)" escalation exactly like a genuinely wedged crew, with no way to mark the wait as expected. + Deferred as a separate item - it changes the stale-classification/status vocabulary shared with `bin/fm-watch.sh` and `bin/fm-classify-lib.sh`, which is a bigger surface than this redelivery-loop fix should carry. diff --git a/tests/fm-afk-inject-herdr-e2e.test.sh b/tests/fm-afk-inject-herdr-e2e.test.sh index aa41b3d76..565fd90bb 100755 --- a/tests/fm-afk-inject-herdr-e2e.test.sh +++ b/tests/fm-afk-inject-herdr-e2e.test.sh @@ -18,9 +18,9 @@ # binary untouched. # # The "supervisor pane" is a tiny deterministic bash loop (not a real harness): -# it draws a bordered composer row ("│ > │") matching the structural -# classifier fm_backend_herdr_composer_state expects, and logs every submitted -# line (hex + text + injection/user classification) - the same technique +# it draws a bordered composer row ("│ > │") that exercises the bordered +# branch of fm_backend_herdr_composer_state, and logs every submitted line (hex +# + text + injection/user classification) - the same technique # tests/fm-afk-inject-e2e.test.sh uses for its tmux supervisor pane, so this # test asserts on submitted CONTENT, not pane appearance. set -u @@ -96,9 +96,9 @@ EOF # --- deterministic bordered-composer loop, drawn in the scratch pane --------- # Mirrors tests/fm-afk-inject-e2e.test.sh's supervisor-loop.sh, but draws a -# "│ > │" border so fm_backend_herdr_composer_state's structural -# classifier (a row whose trimmed content starts AND ends with the same border -# glyph) recognizes it, exactly like a real bordered-TUI harness composer. +# "│ > │" border so the bordered branch of +# fm_backend_herdr_composer_state recognizes it, exactly like a bordered-TUI +# harness composer. LOOP_SCRIPT="$STATE_DIR/supervisor-loop.sh" cat > "$LOOP_SCRIPT" <<'LOOP' #!/usr/bin/env bash diff --git a/tests/fm-backend-herdr.test.sh b/tests/fm-backend-herdr.test.sh index 84be04698..878a3f20f 100755 --- a/tests/fm-backend-herdr.test.sh +++ b/tests/fm-backend-herdr.test.sh @@ -877,16 +877,86 @@ test_composer_state_unknown_on_capture_failure() { } test_composer_state_unknown_when_no_composer_row_found() { - local dir log resp fb out + local dir log resp fb out glyph idx=1 dir="$TMP_ROOT/composer-no-row"; mkdir -p "$dir/responses"; log="$dir/log"; resp="$dir/responses"; : > "$log" - # A capture with no border-delimited row at all (e.g. a bare shell prompt, - # never a firstmate-recognized composer box). - printf 'plain-shell-prompt$ \n' > "$resp/1.out" + for glyph in '>' '$' '%' '#'; do + printf '%s \n' "$glyph" > "$resp/$idx.out" + idx=$((idx + 1)) + done + fb=$(make_herdr_fakebin "$dir") + for glyph in '>' '$' '%' '#'; do + out=$( PATH="$fb:$PATH" FM_HERDR_LOG="$log" FM_HERDR_RESPONSES="$resp" \ + bash -c '. "$0/bin/backends/herdr.sh"; fm_backend_herdr_composer_state default:w1:p2' "$ROOT" ) + [ "$out" = unknown ] || fail "a bare shell prompt '$glyph' should read as unknown, got '$out'" + done + pass "fm_backend_herdr_composer_state: reports unknown for bare shell prompts with no composer row" +} + +# --- composer_state: unbordered (bare) composer rows ------------------------- +# Regression coverage for the away-mode redelivery-loop incident +# (docs/herdr-backend.md "Incident (2026-07-07)"): real claude and codex +# composer rows carry NO border glyph at all - the fixtures below are captured +# verbatim (character-for-character) from a real herdr session running real +# `claude`/`codex` (see the dated evidence entry). Before the fix these all +# read "unknown" (claude/codex fixtures) or produced a false "empty" from a +# stale decorative box (the banner-priority fixture) - none of them correctly +# tracked the live composer, which is exactly what caused +# bin/fm-supervise-daemon.sh's fm_backend_herdr_send_text_submit to never +# confirm a landed injection, so escalate_flush never cleared +# state/.subsuper-escalations and the same digest was redelivered every cycle. + +test_composer_state_claude_unbordered_prompt_is_empty() { + local dir log resp fb out + dir="$TMP_ROOT/composer-claude-bare-empty"; mkdir -p "$dir/responses"; log="$dir/log"; resp="$dir/responses"; : > "$log" + printf ' 20\n 21\n\n\xe2\x9c\xbb Worked for 2s\n\n\xe2\x94\x80\xe2\x94\x80\xe2\x94\x80\xe2\x94\x80\xe2\x94\x80\xe2\x94\x80\n\xe2\x9d\xaf\n\xe2\x94\x80\xe2\x94\x80\xe2\x94\x80\xe2\x94\x80\xe2\x94\x80\xe2\x94\x80\n Opus 4.8 (1M context) \xe2\x96\x8d 3%%\n \xe2\x86\x90 for agents\n' > "$resp/1.out" + fb=$(make_herdr_fakebin "$dir") + out=$( PATH="$fb:$PATH" FM_HERDR_LOG="$log" FM_HERDR_RESPONSES="$resp" \ + bash -c '. "$0/bin/backends/herdr.sh"; fm_backend_herdr_composer_state default:w1:p2' "$ROOT" ) + [ "$out" = empty ] || fail "a genuinely idle, unbordered real-claude '❯' prompt row (no border glyph anywhere in view) should read empty, got '$out' (regression: this used to read 'unknown' forever, which is exactly what broke escalate_flush's buffer-clear)" + pass "fm_backend_herdr_composer_state: a real-claude unbordered '❯' prompt row (no border box in view) reads empty" +} + +test_composer_state_claude_unbordered_prompt_is_pending() { + local dir log resp fb out + dir="$TMP_ROOT/composer-claude-bare-pending"; mkdir -p "$dir/responses"; log="$dir/log"; resp="$dir/responses"; : > "$log" + printf ' 20\n 21\n\n\xe2\x9c\xbb Worked for 2s\n\n\xe2\x94\x80\xe2\x94\x80\xe2\x94\x80\xe2\x94\x80\xe2\x94\x80\xe2\x94\x80\n\xe2\x9d\xaf hello there this is a test message\n\xe2\x94\x80\xe2\x94\x80\xe2\x94\x80\xe2\x94\x80\xe2\x94\x80\xe2\x94\x80\n' > "$resp/1.out" + fb=$(make_herdr_fakebin "$dir") + out=$( PATH="$fb:$PATH" FM_HERDR_LOG="$log" FM_HERDR_RESPONSES="$resp" \ + bash -c '. "$0/bin/backends/herdr.sh"; fm_backend_herdr_composer_state default:w1:p2' "$ROOT" ) + [ "$out" = pending ] || fail "real unsubmitted text in an unbordered real-claude prompt row should read pending, got '$out'" + pass "fm_backend_herdr_composer_state: a real-claude unbordered '❯ ' prompt row reads pending" +} + +# The exact incident shape: a bordered decorative box (claude's own startup +# welcome banner) is STILL in the capture window, sitting ABOVE the live, +# unbordered "❯" prompt. Before the fix, the bordered branch was the ONLY one +# ever consulted, so the LAST bordered row (the banner's own blank interior +# spacer row, immediately above its closing ╰──╯) won by construction and was +# misread as the live composer - which happened to strip to empty here, but +# for the same reason never tracks the REAL composer once real text is typed +# below the banner (see the daemon-level E2E evidence in +# docs/herdr-backend.md). The live, bottom-most row must win regardless of +# shape. +test_composer_state_bare_prompt_below_stale_bordered_banner_wins() { + local dir log resp fb out + dir="$TMP_ROOT/composer-banner-priority"; mkdir -p "$dir/responses"; log="$dir/log"; resp="$dir/responses"; : > "$log" + printf '\xe2\x95\xad\xe2\x94\x80 Claude Code \xe2\x94\x80\xe2\x94\x80\xe2\x94\x80\xe2\x94\x80\xe2\x94\x80\xe2\x94\x80\xe2\x95\xae\n\xe2\x94\x82 Welcome back Kun! \xe2\x94\x82\n\xe2\x94\x82 \xe2\x94\x82\n\xe2\x95\xb0\xe2\x94\x80\xe2\x94\x80\xe2\x94\x80\xe2\x94\x80\xe2\x94\x80\xe2\x94\x80\xe2\x94\x80\xe2\x94\x80\xe2\x94\x80\xe2\x94\x80\xe2\x94\x80\xe2\x94\x80\xe2\x94\x80\xe2\x94\x80\xe2\x94\x80\xe2\x94\x80\xe2\x94\x80\xe2\x94\x80\xe2\x94\x80\xe2\x95\xaf\n\n\xe2\x94\x80\xe2\x94\x80\xe2\x94\x80\xe2\x94\x80\xe2\x94\x80\xe2\x94\x80\n\xe2\x9d\xaf still typing captain\n\xe2\x94\x80\xe2\x94\x80\xe2\x94\x80\xe2\x94\x80\xe2\x94\x80\xe2\x94\x80\n' > "$resp/1.out" + fb=$(make_herdr_fakebin "$dir") + out=$( PATH="$fb:$PATH" FM_HERDR_LOG="$log" FM_HERDR_RESPONSES="$resp" \ + bash -c '. "$0/bin/backends/herdr.sh"; fm_backend_herdr_composer_state default:w1:p2' "$ROOT" ) + [ "$out" = pending ] || fail "the live unbordered prompt row below a stale bordered banner must win (pending, real text present), got '$out'" + pass "fm_backend_herdr_composer_state: a live unbordered prompt row below a stale bordered decorative box still wins (not misread as the box's own row)" +} + +test_composer_state_codex_bare_prompt_glyph_is_empty() { + local dir log resp fb out + dir="$TMP_ROOT/composer-codex-bare"; mkdir -p "$dir/responses"; log="$dir/log"; resp="$dir/responses"; : > "$log" + printf '\xe2\x80\xa2 You have 2 usage limit resets available.\n\n\xe2\x80\xba\n\n gpt-5.5 xhigh \xc2\xb7 Context 100%% left\n' > "$resp/1.out" fb=$(make_herdr_fakebin "$dir") out=$( PATH="$fb:$PATH" FM_HERDR_LOG="$log" FM_HERDR_RESPONSES="$resp" \ bash -c '. "$0/bin/backends/herdr.sh"; fm_backend_herdr_composer_state default:w1:p2' "$ROOT" ) - [ "$out" = unknown ] || fail "a capture with no recognizable composer row should read as unknown, got '$out'" - pass "fm_backend_herdr_composer_state: reports unknown when no border-delimited composer row is found" + [ "$out" = empty ] || fail "a bare '›' (codex) prompt glyph with no trailing text should read empty, got '$out'" + pass "fm_backend_herdr_composer_state: a real-codex unbordered '›' prompt row reads empty" } # --- send_text_submit: structural composer-row verify-and-retry -------------- @@ -1323,6 +1393,10 @@ test_composer_state_real_text_is_pending test_composer_state_popup_placeholder_fill_is_pending test_composer_state_unknown_on_capture_failure test_composer_state_unknown_when_no_composer_row_found +test_composer_state_claude_unbordered_prompt_is_empty +test_composer_state_claude_unbordered_prompt_is_pending +test_composer_state_bare_prompt_below_stale_bordered_banner_wins +test_composer_state_codex_bare_prompt_glyph_is_empty test_send_text_submit_detects_landed_send test_send_text_submit_detects_swallowed_enter test_send_text_submit_popup_autocomplete_requires_second_enter From 01dca334dc1f57978262f1d0a70c476e86c7407a Mon Sep 17 00:00:00 2001 From: Kun Chen <3233006+kunchenguid@users.noreply.github.com> Date: Tue, 7 Jul 2026 16:39:59 -0700 Subject: [PATCH 07/63] fix(backends): confirm Herdr submits with native agent state (#323) * fix(herdr): confirm message submit via native agent-state, not composer text fm_backend_herdr_send_text_submit now confirms a landed submit by polling herdr's own agent-state (agent get) for the idle->working transition instead of reading composer content. Composer scraping remains, unchanged, for the away-mode daemon's pre-injection empty-box guard only. This fixes the practical effect of the codex idle-tip gap from the 2026-07-07 incident: codex's dynamic idle-composer hint text can no longer misread as pending and block/mis-confirm a send, since confirmation no longer looks at composer text at all. Verified empirically against real claude and codex agents (timing, swallowed-Enter, unreadable-target, and already-busy-target scenarios), and against the real away-mode daemon end-to-end after updating its synthetic supervisor-pane test fixture to register itself as a real herdr agent (herdr's own report-agent primitive) so it can still exercise the new confirmation path. * no-mistakes(review): Captain, harden herdr submit confirmation * no-mistakes(review): Captain, harden herdr submit confirmation * no-mistakes(document): Sync Herdr submit docs * no-mistakes: apply CI fixes --- CONTRIBUTING.md | 2 +- bin/backends/cmux.sh | 16 +- bin/backends/herdr.sh | 237 ++++++++++++++---- bin/backends/zellij.sh | 5 +- bin/fm-backend.sh | 4 +- bin/fm-send.sh | 19 +- bin/fm-supervise-daemon.sh | 19 +- docs/architecture.md | 2 +- docs/configuration.md | 4 +- docs/herdr-backend.md | 101 ++++++-- docs/scripts.md | 4 +- docs/zellij-backend.md | 4 +- tests/fm-afk-inject-e2e.test.sh | 24 +- tests/fm-afk-inject-herdr-e2e.test.sh | 47 +++- tests/fm-backend-herdr.test.sh | 331 +++++++++++++++++++++++--- 15 files changed, 676 insertions(+), 143 deletions(-) diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index 9ccad4f24..164b3ab42 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -102,7 +102,7 @@ tests/fm-pr-merge.test.sh # fm-pr-merge.sh records pr= and avail tests/fm-crew-state.test.sh # fm-crew-state.sh current-state reconciliation: run-step authority including closed panes and ci log-tail checks-green detection, stale checks-green and needs-decision/blocked superseded by resumed work, genuine-parked, cross-branch runs-list attribution, pane/status-log fallback, scout skip, torn-down/missing-meta graceful tests/fm-backend.test.sh # runtime-backend abstraction: fm-backend.sh selection/meta/dispatch helpers, shell-portable sourced backend matching, blocked codex-app refusal, and old-vs-new fake-tool command-log conformance for fm-send/fm-peek/fm-spawn/fm-teardown tests/fm-backend-tmux-smoke.test.sh # real (private-socket) tmux smoke test for the tmux adapter: create/duplicate-refuse, send text + Enter, send literal + key, bounded capture, live-window resolve, kill -tests/fm-backend-herdr.test.sh # fake herdr CLI unit tests for the experimental herdr adapter, including version/tool gates, target parsing, send/capture, structural composer-state verification with bordered and unbordered prompt fixtures, slash-submit retry regression coverage, native busy state, per-home workspace-label resolution, default-tab prune safety, restored-layout husk replacement, and verified CLI bug workarounds +tests/fm-backend-herdr.test.sh # fake herdr CLI unit tests for the experimental herdr adapter, including version/tool gates, target parsing, send/capture, native agent-state submit confirmation, structural composer-state guard/fallback coverage with bordered and unbordered prompt fixtures, slash-submit retry regression coverage, native busy state, per-home workspace-label resolution, default-tab prune safety, restored-layout husk replacement, and verified CLI bug workarounds tests/fm-backend-herdr-smoke.test.sh # real herdr adapter smoke test, skipped when herdr or jq is unavailable, using an isolated throwaway HERDR_SESSION and guarded session cleanup, including live-agent duplicate refusal and no-agent husk replacement tests/fm-backend-autodetect-smoke.test.sh # real herdr auto-detection smoke test, skipped when herdr, jq, or treehouse is unavailable, using the same guarded session cleanup tests/fm-backend-herdr-workspace-per-home-e2e.test.sh # mandatory isolated E2E for workspace-per-home: primary and secondmate-shaped homes, a crewmate spawned from a secondmate home, teardown, list-live recovery diff --git a/bin/backends/cmux.sh b/bin/backends/cmux.sh index 5f74c0283..567b071cf 100644 --- a/bin/backends/cmux.sh +++ b/bin/backends/cmux.sh @@ -573,12 +573,16 @@ fm_backend_cmux_composer_state() { # [expected-label] -> empty|pending # fm_backend_cmux_send_text_submit: type into once (raw, # unsubmitted, via send_literal), then submit with a named Enter key, retried # (Enter only, never retyped) until the composer's own row reads empty. -# Mirrors fm_backend_herdr_send_text_submit's verification strategy exactly: -# a slash-command popup's first Enter can close the popup and fill an -# argument-hint placeholder into the composer rather than submitting, which a -# raw-diff check would misread as "submitted" - classifying the composer row -# specifically avoids that false positive, so the retry loop correctly sends -# a second Enter when needed. Echoes empty|pending|unknown|send-failed, the +# Mirrors fm_backend_herdr_send_text_submit's ORIGINAL (composer-row) +# verification strategy: a slash-command popup's first Enter can close the +# popup and fill an argument-hint placeholder into the composer rather than +# submitting, which a raw-diff check would misread as "submitted" - +# classifying the composer row specifically avoids that false positive, so +# the retry loop correctly sends a second Enter when needed. Herdr's adapter +# has since moved its own confirmation to a native agent-state read instead +# (docs/herdr-backend.md "Native agent-state submit confirmation"); cmux has +# no analogous native primitive, so this composer-row approach remains +# cmux's own confirmation strategy. Echoes empty|pending|unknown|send-failed, the # SAME vocabulary every existing backend already speaks. fm_backend_cmux_send_text_submit() { # [expected-label] local target=$1 text=$2 retries=$3 sleep_s=$4 settle=$5 expected_label=${6:-} i=0 state diff --git a/bin/backends/herdr.sh b/bin/backends/herdr.sh index 67c0d70c8..712f77d3b 100644 --- a/bin/backends/herdr.sh +++ b/bin/backends/herdr.sh @@ -582,7 +582,7 @@ fm_backend_herdr_send_key() { # # rows for a default-sized pane), instead of clamping to the last N lines - it # does not merely ignore the bound, it drops the read entirely. This silently # broke exactly the small bounded reads this adapter relies on most (including -# the composer-state verification read used by send_text_submit). Workaround: +# the composer-state guard/fallback reads around submit and injection). Workaround: # always request a generous fetch far above any realistic viewport height, then # trim to the caller's requested bound ourselves with `tail`. fm_backend_herdr_capture() { # @@ -710,40 +710,88 @@ fm_backend_herdr_composer_state() { # -> empty|pending|unknown # fm_backend_herdr_send_text_submit: type into once (raw, # unsubmitted, via send_literal), then submit with a named Enter key, retried -# (Enter only, never retyped) until the composer's own row reads empty. -# Verified hazard (herdr-verification-p2.md "slash/$ autocomplete popup"): a -# `/`- or `$`-prefixed send opens a completion popup within ~0.1s, exactly -# like tmux's claude/codex popups, so the caller's before the first -# Enter matters here the same way it does for tmux. +# (Enter only, never retyped) until herdr's NATIVE agent-state (agent get) +# confirms a real turn started. Verified hazard (herdr-verification-p2.md +# "slash/$ autocomplete popup"): a `/`- or `$`-prefixed send opens a +# completion popup within ~0.1s, exactly like tmux's claude/codex popups, so +# the caller's before the first Enter matters here the same way it +# does for tmux. # -# Verification strategy (incident 2026-07-03: two grok/herdr crewmates left a -# fully-typed `/no-mistakes` sitting unsubmitted for minutes, footer still -# reading "Enter:send", while fm-send exited 0): a prior version of this -# function verified submission by diffing raw pane content before/after -# Enter - ANY change counted as "submitted". Live-verified against real grok -# 0.2.82: a slash command's first Enter closes the completion popup and, for -# an argument-taking command, EXPANDS the composer text into an argument-hint -# placeholder ("/compact" -> "/compact compaction instructions") rather than -# submitting - the raw pane content visibly changes (popup gone, text -# different) even though nothing was sent, so the old diff-based check -# false-positived "empty" (submitted) after exactly one Enter, precisely -# matching the incident. A genuine second Enter was required to actually -# submit. fm_backend_herdr_composer_state avoids this by classifying the -# composer's own row specifically: a popup-close-with-placeholder-fill still -# reads as "pending" (real text remains), so the retry loop below correctly -# sends the second Enter instead of stopping early. Echoes -# empty|pending|unknown|send-failed, the SAME vocabulary fm-send.sh already -# branches on for tmux. +# Confirmation signal (rewritten for the 2026-07-07 incident below; +# superseded a composer-content read that itself replaced a delta-based check +# for the 2026-07-03 incident): when the target is legibly idle before Enter, +# submission is confirmed by fm_backend_herdr_wait_for_working observing a +# submit-active agent_status after Enter, NOT by reading the composer's own +# row. This makes the normal confirmation path cross-agent: it is the same +# semantic signal regardless of what text a harness's idle composer happens +# to display. +# +# Incident (2026-07-07): a redelivery loop in the away-mode daemon. Root +# cause: fm_backend_herdr_composer_state's structural row classifier +# recognizes only bordered and two hardcoded bare prompt glyphs; real codex's +# IDLE composer shows dynamic tip/hint text ("Use /skills to list available +# skills") that no static pattern can tell apart from genuinely unsubmitted +# input, so a codex confirmation built on composer content classified a +# landed submit as still-"pending" forever - see docs/herdr-backend.md for +# the full account. Composer content is retained for other callers (the +# away-mode daemon's PRE-injection empty-box guard, still dispatched via +# fm_backend_composer_state / fm_backend_herdr_composer_state, unchanged) and +# for submit attempts whose pre-Enter agent-state baseline is not legibly +# idle. +# +# This also still correctly handles the earlier 2026-07-03 incident (a +# slash-command popup selection/placeholder-fill on the FIRST Enter is not a +# genuine submission) without any popup-specific logic at all: filling a +# composer placeholder never starts a turn, so agent_status simply never +# reports "working" for that Enter, and the retry loop below sends a second +# Enter exactly as it did before - the fix generalizes instead of special- +# casing the popup shape. +# +# Failure-mode analysis (the two directions the caller-facing contract must +# not get wrong - see docs/herdr-backend.md "Native agent-state submit +# confirmation" for the empirical timing behind this): +# - Slow transition: fm_backend_herdr_wait_for_working samples repeatedly +# across herdr's per-attempt confirmation budget (not once at the end), so a +# transition landing partway through a window is still caught before this +# loop gives up and sends a needless extra Enter. +# - Instant round-trip (a turn starts AND returns to idle between two +# polls): unavoidable in the absolute, but bounded by how tightly polls +# are packed into the budget; real claude/codex measured first-working +# at 90-490ms, comfortably inside a several-hundred-ms, multiply-sampled +# window, so this has not been observed in practice. On the (unobserved) +# residual chance it happens, the verdict is "pending" and the caller +# never retypes - only re-sends Enter, which lands on an already-empty +# composer and is a no-op, not a duplicate delivery of (see +# fm-send.sh/fm-supervise-daemon.sh: retyping only happens if a caller +# re-invokes this function from scratch with the same text after seeing +# an error, which is a human/escalation decision, not an automatic +# retry). +# Echoes empty|pending|unknown|send-failed, the SAME vocabulary fm-send.sh +# already branches on for tmux ("empty" means "confirmed submitted" for every +# backend; how each backend confirms it is an internal decision - herdr's is +# no longer literally "the composer read empty"). fm_backend_herdr_send_text_submit() { # - local target=$1 text=$2 retries=$3 sleep_s=$4 settle=$5 i=0 state + local target=$1 text=$2 retries=$3 sleep_s=$4 settle=$5 i=0 verdict baseline confirm_sleep fm_backend_herdr_parse_target "$target" || { printf 'unknown'; return 0; } fm_backend_herdr_send_literal "$target" "$text" || { printf 'send-failed'; return 0; } sleep "$settle" + baseline=$(fm_backend_herdr_classify_submit_agent_status \ + "$(fm_backend_herdr_agent_status_raw "$FM_BACKEND_HERDR_SESSION" "$FM_BACKEND_HERDR_PANE")") + confirm_sleep=$(fm_backend_herdr_submit_confirm_budget "$sleep_s") while :; do fm_backend_herdr_send_key "$target" Enter || true - sleep "$sleep_s" - state=$(fm_backend_herdr_composer_state "$target") - [ "$state" = pending ] || { printf '%s' "$state"; return 0; } + if [ "$baseline" = idle ]; then + verdict=$(fm_backend_herdr_wait_for_working "$FM_BACKEND_HERDR_SESSION" "$FM_BACKEND_HERDR_PANE" \ + "$confirm_sleep" "$FM_BACKEND_HERDR_SUBMIT_POLLS") + else + sleep "$sleep_s" + verdict=$(fm_backend_herdr_composer_state "$target") + fi + case "$verdict" in + busy) printf 'empty'; return 0 ;; + empty) printf 'empty'; return 0 ;; + unknown) printf 'unknown'; return 0 ;; + esac i=$((i + 1)) [ "$i" -lt "$retries" ] || { printf 'pending'; return 0; } done @@ -757,19 +805,15 @@ fm_backend_herdr_kill() { # fm_backend_herdr_cli "$FM_BACKEND_HERDR_SESSION" pane close "$FM_BACKEND_HERDR_PANE" >/dev/null 2>&1 || true } -# fm_backend_herdr_busy_state: semantic busy state from herdr's native -# agent-state detection (agent.get), the "first backend where fm_session_busy_state -# gets real semantics" per the design report. working -> busy (actively -# generating); idle/done -> idle; blocked -> idle (a blocked agent is stuck -# waiting on the human, not grinding - the watcher should treat it like a -# stale pane needing attention, not suppress it as busy); unknown/unparseable -# -> unknown, the caller's cue to fall back to pane-regex detection. -fm_backend_herdr_busy_state() { # - fm_backend_herdr_target_ready "$1" || { printf 'unknown'; return 0; } - local out status - out=$(fm_backend_herdr_cli "$FM_BACKEND_HERDR_SESSION" agent get "$FM_BACKEND_HERDR_PANE" 2>/dev/null) || { printf 'unknown'; return 0; } - status=$(printf '%s' "$out" | jq -r '.result.agent.agent_status // empty' 2>/dev/null) - case "$status" in +# fm_backend_herdr_classify_agent_status: map a raw `agent get` agent_status +# value to the adapter's watcher busy|idle|unknown vocabulary. working -> +# busy (actively generating); idle/done -> idle; blocked -> idle (a blocked +# agent is stuck waiting on the human, not grinding - the watcher should +# treat it like a stale pane needing attention, not suppress it as busy); +# unknown/unparseable/empty -> unknown, the caller's cue to fall back to +# pane-regex detection. +fm_backend_herdr_classify_agent_status() { # + case "$1" in working) printf 'busy' ;; idle|done) printf 'idle' ;; blocked) printf 'idle' ;; @@ -777,6 +821,117 @@ fm_backend_herdr_busy_state() { # esac } +fm_backend_herdr_classify_submit_agent_status() { # + case "$1" in + working|blocked) printf 'busy' ;; + idle|done) printf 'idle' ;; + *) printf 'unknown' ;; + esac +} + +# fm_backend_herdr_agent_status_raw: one `agent get` read, echoing the raw +# agent_status string (working/idle/done/blocked/...), or empty on any +# failure. Deliberately skips fm_backend_herdr_target_ready's server-ensure +# round trip (an extra `status --json` call) that fm_backend_herdr_busy_state +# pays on every call: fm_backend_herdr_wait_for_working polls this in a tight +# loop right after a caller has already parsed the target and confirmed the +# server is live (e.g. fm_backend_herdr_send_text_submit, immediately after a +# successful send-text), so re-checking server liveness on every poll would +# only add latency without adding safety. +fm_backend_herdr_agent_status_raw() { # + local session=$1 pane_id=$2 out + out=$(fm_backend_herdr_cli "$session" agent get "$pane_id" 2>/dev/null) || { printf ''; return 0; } + printf '%s' "$out" | jq -r '.result.agent.agent_status // empty' 2>/dev/null +} + +# fm_backend_herdr_busy_state: semantic busy state from herdr's native +# agent-state detection (agent.get), the "first backend where fm_session_busy_state +# gets real semantics" per the design report. See +# fm_backend_herdr_classify_agent_status for the status->busy/idle/unknown +# mapping. +fm_backend_herdr_busy_state() { # + fm_backend_herdr_target_ready "$1" || { printf 'unknown'; return 0; } + fm_backend_herdr_classify_agent_status \ + "$(fm_backend_herdr_agent_status_raw "$FM_BACKEND_HERDR_SESSION" "$FM_BACKEND_HERDR_PANE")" +} + +# fm_backend_herdr_wait_for_working: poll :'s NATIVE +# agent-state (agent get) up to times spread evenly across +# , returning on stdout the STRONGEST signal observed: +# +# busy - a submit-active status was observed at least once. This is +# confirmation that a real turn started or reached a prompt - +# the submit landed - independent of +# whatever the composer's own text happens to show (docs/ +# herdr-backend.md "Incident (2026-07-07)": composer content is +# what fooled the OLD confirmation on codex's dynamic idle-tip +# text). Returned the INSTANT it is seen, without waiting out the +# rest of the budget. +# idle - the target was legibly read at least once and never reported +# "busy" across the whole window - a genuine "not (yet) +# submitted" signal, not a read failure. The caller retries +# Enter on this verdict. +# unknown - EVERY poll in the window failed to read the target at all (a +# hard I/O failure - pane gone, socket error - not a timing +# race). The caller must not keep retrying Enter against a target +# it cannot even read. +# +# spread across (rather than one check at the end) +# is what makes this robust against a SLOW transition: a caller now gets +# several samples across that window instead of a single one, so a transition +# that lands partway through is not missed just because it had not landed by +# the FIRST sample. +# Empirical evidence (docs/herdr-backend.md "Native agent-state submit +# confirmation"): real claude and codex observed first-working at 90-490ms +# after Enter, so a several-hundred-ms budget sampled repeatedly reliably +# catches it. The remaining, inherent gap - a turn so fast it starts AND +# returns to idle between two samples - is bounded by how tightly is +# packed into ; nothing observed in real testing has come +# close to that, but it is a residual risk, not a mathematical impossibility +# (see the doc section for the full characterization and the failure-mode +# analysis for both directions this must guard). +# FM_BACKEND_HERDR_SUBMIT_POLLS (default 6): how many samples +# fm_backend_herdr_send_text_submit spreads across each Enter attempt's +# confirmation budget. Overridable for tests (a value of 1 +# reproduces the old single-check-at-the-end timing exactly, for byte-for-byte +# call-count assertions). +FM_BACKEND_HERDR_SUBMIT_POLLS=${FM_BACKEND_HERDR_SUBMIT_POLLS:-6} +FM_BACKEND_HERDR_SUBMIT_MIN_SLEEP=${FM_BACKEND_HERDR_SUBMIT_MIN_SLEEP:-0.6} + +fm_backend_herdr_submit_confirm_budget() { # + awk -v b="${1:-0}" -v m="$FM_BACKEND_HERDR_SUBMIT_MIN_SLEEP" 'BEGIN { + b += 0 + m += 0 + if (b < 0) b = 0 + if (m < 0) m = 0 + if (m > b) b = m + printf "%.4f", b + }' 2>/dev/null || printf '%s' "${1:-0}" +} + +fm_backend_herdr_wait_for_working() { # + local session=$1 pane_id=$2 budget=$3 polls=${4:-1} i interval raw bs saw_idle=0 + case "$polls" in ''|*[!0-9]*|0) polls=1 ;; esac + interval=$(awk -v b="$budget" -v p="$polls" 'BEGIN { d = p - 1; if (d < 1) d = 1; v = b / d; if (v < 0) v = 0; printf "%.4f", v }' 2>/dev/null) + case "$interval" in ''|*[!0-9.]*) interval=0 ;; esac + for ((i = 0; i < polls; i++)); do + if [ "$polls" -eq 1 ] || [ "$i" -gt 0 ]; then + sleep "$interval" + fi + raw=$(fm_backend_herdr_agent_status_raw "$session" "$pane_id") + bs=$(fm_backend_herdr_classify_submit_agent_status "$raw") + case "$bs" in + busy) printf 'busy'; return 0 ;; + idle) saw_idle=1 ;; + esac + done + if [ "$saw_idle" -eq 1 ]; then + printf 'idle' + else + printf 'unknown' + fi +} + # fm_backend_herdr_pane_for_tab: the root pane id for in # of , via one pane list call filtered by tab_id (never assumes a # tab-number/pane-number correspondence - herdr numbers them independently). diff --git a/bin/backends/zellij.sh b/bin/backends/zellij.sh index 0fdd9f79b..4df2ac214 100644 --- a/bin/backends/zellij.sh +++ b/bin/backends/zellij.sh @@ -491,8 +491,9 @@ fm_backend_zellij_capture() { # [expected-label] # fm_backend_zellij_send_text_submit: type into once (raw, # unsubmitted, via send_literal), then submit with a named Enter key, retried # (Enter only, never retyped) until the pane visibly changes. Unlike herdr's -# current structural composer-row verifier, zellij still uses a content-diff -# strategy because its CLI has no cursor-row/ANSI capture primitive exposed: +# current native agent-state idle-baseline verifier and composer-state +# fallback, zellij still uses a content-diff strategy because its CLI has no +# cursor-row/ANSI capture primitive exposed: # capture the pane right after typing (before any Enter) as the TYPED baseline, # then after each Enter attempt capture again - unchanged means Enter was # swallowed (retry); changed means submitted. This content-diff approach is diff --git a/bin/fm-backend.sh b/bin/fm-backend.sh index 6aa0d439d..5c50dc9b9 100644 --- a/bin/fm-backend.sh +++ b/bin/fm-backend.sh @@ -557,8 +557,8 @@ fm_backend_busy_state() { # } # fm_backend_composer_state: classify the composer/input row of as -# empty|pending|unknown - the SUBMIT-side classifier each adapter already uses -# internally to verify fm_backend_send_text_submit, exposed generically so a +# empty|pending|unknown for callers that need a pre-submit pending-input guard +# or an adapter's conservative submit fallback. It is exposed generically so a # caller other than the send path (the away-mode daemon's supervisor-pane # pending-input guard, bin/fm-supervise-daemon.sh) can ask the same question # without duplicating per-backend composer-reading logic. tmux and herdr both diff --git a/bin/fm-send.sh b/bin/fm-send.sh index 70e21529f..af887ef8e 100755 --- a/bin/fm-send.sh +++ b/bin/fm-send.sh @@ -8,10 +8,9 @@ # Orca currently supports Enter and C-c only, and rejects Escape. # # Text submission is verified: the line is typed ONCE, then Enter is sent and -# retried (Enter only, never retyped) until the target backend reports a -# submitted/cleared composer or an inconclusive send. If a swallowed Enter is -# positively confirmed (the text is still sitting in the composer after all -# retries), fm-send exits NON-ZERO so the caller knows the steer did not land +# retried (Enter only, never retyped) until the target backend confirms a +# submit or reports an inconclusive send. If a swallowed Enter is positively +# confirmed, fm-send exits NON-ZERO so the caller knows the steer did not land # instead of silently leaving an unsubmitted instruction. # Submission dispatches through the target's recorded backend; the tmux adapter # shares its composer/submit core with the away-mode daemon via bin/fm-tmux-lib.sh. @@ -26,8 +25,8 @@ # never reads. A crewmate/scout target, an explicit backend-target escape-hatch # target, and the --key path are never marked - their behavior is unchanged. # After a successful text submit fm-send pauses FM_SEND_SETTLE seconds (default 1, -# 0 disables) before returning: a cleared composer only proves the text was -# submitted, but the harness needs a beat to spin up the turn before its busy +# 0 disables) before returning: submit confirmation only proves the text was +# accepted, but the harness needs a beat to spin up the turn before its busy # footer appears, so an immediate peek would otherwise see the stale idle pane. # The pause is fm-send-only; the shared submit core (used by the away-mode daemon, # which only needs "submitted") does not pay it, and the --key path is unaffected. @@ -102,8 +101,8 @@ else esac retries=${FM_SEND_RETRIES:-3} sleep_s=${FM_SEND_SLEEP:-0.4} - # Type once, submit, verify. Lenient: only a positively-confirmed swallow - # (text still in the composer) is an error; an unreadable pane is assumed sent. + # Type once, submit, verify. Lenient: only a positively-confirmed swallowed + # submit is an error; an unreadable pane is assumed sent. verdict=$(fm_backend_send_text_submit "$TARGET_BACKEND" "$T" "$MARK_PREFIX$*" "$retries" "$sleep_s" "$settle" "$EXPECTED_LABEL") case "$verdict" in pending) @@ -115,8 +114,8 @@ else exit 1 ;; esac - # Submit landed (verdict was not pending/send-failed). The cleared composer only - # proves the text was submitted; the harness still needs a beat to spin up the + # Submit landed (verdict was not pending/send-failed). Confirmation only proves + # the text was accepted; the harness still needs a beat to spin up the # turn before its busy footer shows. Pause so an immediate peek catches the # crewmate actually working instead of the stale idle pane. FM_SEND_SETTLE=0 # disables it. Scoped to this path only, never the shared submit core. diff --git a/bin/fm-supervise-daemon.sh b/bin/fm-supervise-daemon.sh index c3c2da9c3..4fa8d7f1f 100755 --- a/bin/fm-supervise-daemon.sh +++ b/bin/fm-supervise-daemon.sh @@ -704,10 +704,11 @@ window_for_task() { # [state] # - TYPE ONCE, then submit with Enter. Never retype the digest: a swallowed # Enter leaves our text in the composer, and retyping would concatenate two # sentinel-prefixed digests into one corrupted turn. -# - SUBMIT ACK = the dim-ghost-aware and border-aware composer detector reports -# empty after Enter. -# Empty means the text was consumed; pending means Enter was swallowed; unknown -# is treated as undelivered by this strict daemon path. +# - SUBMIT ACK = the backend submit primitive reports `empty` after Enter. +# For tmux that means a cleared composer; for herdr's normal idle-baseline +# path it means native agent-state observed a real turn start. +# Pending means Enter was swallowed; unknown is treated as undelivered by +# this strict daemon path. # - COMPOSER GUARD before typing: if the cursor line already has real content # after dim/faint ghost text and borders are ignored (a human's half-typed # line, or a previous injection's unsent text), defer entirely - injecting @@ -721,7 +722,7 @@ inject_msg() { # [state] afk_active "$state" || { log "inject deferred: afk inactive"; return 1; } # (2) Single-line digest: collapse any embedded newlines so submission via # send-keys + Enter is unambiguous regardless of how the TUI composer treats - # them. Then prepend the sentinel marker — firstmate's afk-exit contract + # them. Then prepend the sentinel marker - firstmate's afk-exit contract # keys off its presence at the start of the message. msg=$(_collapse_newlines "$msg") msg="${FM_INJECT_MARK}${msg}" @@ -747,9 +748,9 @@ inject_msg() { # [state] return 1 fi # (4) Type the digest ONCE, then submit with Enter (retry Enter only, never - # retype) via the shared submit primitive. Success = the composer is confirmed - # EMPTY afterward (the text was consumed). An unconfirmed/unknown pane does NOT - # count as delivered, so the buffer is preserved (strict) rather than cleared. + # retype) via the shared submit primitive. Success = the backend confirms + # submit. An unconfirmed/unknown pane does NOT count as delivered, so the + # buffer is preserved (strict) rather than cleared. # Dispatches through fm_backend_send_text_submit (bin/fm-backend.sh): for # backend=tmux this calls fm_backend_tmux_send_text_submit, a verbatim # re-export of fm_tmux_submit_core - byte-identical to calling it directly. @@ -757,7 +758,7 @@ inject_msg() { # [state] sleep_s=${FM_INJECT_CONFIRM_SLEEP:-$INJECT_CONFIRM_SLEEP_DEFAULT} verdict=$(fm_backend_send_text_submit "$backend" "$target" "$msg" "$retries" "$sleep_s" "$sleep_s") if [ "$verdict" = empty ]; then - return 0 # Composer cleared → submit confirmed. + return 0 # Backend confirmed the submit. fi log "inject failed: submit unconfirmed after $retries retries (verdict=$verdict, text may be in composer)" return 1 diff --git a/docs/architecture.md b/docs/architecture.md index 4e2d7fc8b..068f53ae5 100644 --- a/docs/architecture.md +++ b/docs/architecture.md @@ -39,7 +39,7 @@ The watcher and daemon share `bin/fm-classify-lib.sh` for captain-relevant statu The always-on watcher also uses that library's provably-working predicate on no-verb signals and first-sighting stale panes before status-log terminality is trusted, while the daemon keeps its away-mode stale recheck unchanged. The daemon escalates only captain-relevant events as one batched, single-line digest (prefixed with an in-band sentinel marker so firstmate can tell daemon injections apart from real messages). Its supervisor injection path supports tmux and herdr panes, with `FM_SUPERVISOR_BACKEND` and `FM_SUPERVISOR_TARGET` resolved independently from the task-spawn backend. -Pane existence, busy checks, composer checks, capture, and verified submit route through `bin/fm-backend.sh`: tmux keeps the same submit core used by the tmux send backend, while herdr reuses its native busy state and structural composer classifier. +Pane existence, busy checks, composer checks, capture, and verified submit route through `bin/fm-backend.sh`: tmux keeps the same submit core used by the tmux send backend, while herdr uses native busy state, native agent-state submit confirmation on idle baselines, and its structural composer classifier for pending-input guards and submit fallback. Unsupported supervisor backends refuse at daemon startup. Stalled escalation delivery raises `state/.subsuper-inject-wedged` after `FM_MAX_DEFER_SECS` instead of silently deferring forever. `fm-send.sh` selects a pre-Enter popup-settle for slash commands and for codex `$...` skill invocations using the target's recorded `harness=` meta, then adds its own `FM_SEND_SETTLE` pause after successful text sends so immediate peeks catch the receiving turn starting; the sub-supervisor uses only the shared submit core and does not pay that post-submit pause. diff --git a/docs/configuration.md b/docs/configuration.md index c3d75dcba..e4ba47d26 100644 --- a/docs/configuration.md +++ b/docs/configuration.md @@ -223,9 +223,11 @@ FM_PROJECTS_OVERRIDE= # alternate projects dir, mainly for tests FM_CONFIG_OVERRIDE= # alternate config dir, mainly for tests FM_BACKEND= # optional runtime backend override for new spawns; tmux/herdr/zellij/orca/cmux support ship/scout spawns, codex-app is not accepted HERDR_SESSION=default # herdr-only: named session for normal backend ops; not enough for destructive cleanup (docs/herdr-backend.md) -FM_BACKEND_HERDR_COMPOSER_LINES=20 # herdr-only: tail lines scanned to locate the composer row for submit verification +FM_BACKEND_HERDR_COMPOSER_LINES=20 # herdr-only: tail lines scanned by composer-state guard/fallback paths; idle-baseline submit confirmation uses agent-state FM_BACKEND_HERDR_IDLE_RE='^Type a message\.\.\.$' # herdr-only: empty-composer placeholder regex after border/prompt stripping FM_BACKEND_HERDR_BARE_PROMPT_RE='^[❯›]' # herdr-only: verified agent glyphs recognized as an UNBORDERED (bare) composer row, e.g. claude's ❯ or codex's › (docs/herdr-backend.md "Incident (2026-07-07)") +FM_BACKEND_HERDR_SUBMIT_POLLS=6 # herdr-only: agent-state samples spread across each Enter attempt's budget when confirming a submit (docs/herdr-backend.md "Native agent-state submit confirmation") +FM_BACKEND_HERDR_SUBMIT_MIN_SLEEP=0.6 # herdr-only: minimum per-Enter confirmation budget before polling agent-state after an idle baseline FM_BACKEND_ORCA_COMPOSER_LINES=200 # orca-only: terminal-read lines scanned to locate the composer row for submit verification FM_BACKEND_ORCA_IDLE_RE='^Type a message\.\.\.$' # orca-only: empty-composer placeholder regex after border/prompt stripping FM_ZELLIJ_SESSION=firstmate # zellij-only: named session for normal backend ops and test isolation (docs/zellij-backend.md) diff --git a/docs/herdr-backend.md b/docs/herdr-backend.md index 18af84031..71d7bbc6f 100644 --- a/docs/herdr-backend.md +++ b/docs/herdr-backend.md @@ -183,6 +183,7 @@ Herdr tasks additionally record: | Send literal (unsubmitted) | `herdr pane send-text ` | Does NOT auto-submit, contrary to the original design addendum's guess. Verified directly: a unique marker sent this way sits unexecuted in the composer until a separate Enter. Behaves exactly like tmux's `send-keys -l`. | | Send + submit atomically | `herdr pane run ` | Runs and submits a command in one call; used for the two fixed spawn-time commands (`treehouse get`, the `GOTMPDIR` export) exactly where tmux used one `send-keys ... Enter` call. | | Send key | `herdr pane send-keys ` | Verified names: `enter`, `escape` (alias `esc`), `ctrl+c` (aliases `C-c`, `c-c`). `ctrl+c` verified to interrupt a running foreground process immediately. | +| Submit confirmation (idle baseline) | `herdr agent get ` -> `.result.agent.agent_status` after Enter | `fm_backend_herdr_send_text_submit` records the pre-Enter status and, when it is idle/done, confirms delivery by polling for `working`/`blocked` across the Enter attempt's confirmation budget. Composer-state reads remain the pre-injection guard and the conservative fallback for preexisting submit-active or unreadable baselines; see "Native agent-state submit confirmation". | | Bounded capture | `herdr pane read --source recent --lines N` | See "Verified bug" below - N is never passed through directly. | | Busy state | `herdr agent get ` -> `.result.agent.agent_status` | Verified live against an interactive `claude` session: reports `working` while generating, `done` once idle. Mapped: `working` -> busy; `idle`/`done` -> idle; `blocked` -> idle (surfaced like a stale pane, not suppressed as busy - a blocked agent is stuck waiting on the human, not grinding); anything else -> unknown (the cue for the shared tail-regex fallback). | | Kill | `herdr pane close ` | Closing a tab's only (root) pane also closes the tab - no separate tab-close call needed for this adapter's one-pane-per-tab shape. Best-effort: closing an already-closed pane exits non-zero, matching tmux's `kill-window \|\| true` contract. Teardown itself only ever closes the task's own pane/tab, never the workspace - but closing a workspace's LAST tab (verified real-herdr behavior) deletes the workspace as a side effect, so a home's own workspace persists only while at least one task tab remains; see "Workspace lifecycle" above. | @@ -198,7 +199,7 @@ This was the most significant finding of this verification pass. `herdr pane read --source recent --lines N` returns **completely empty output** when `N` is smaller than the pane's current viewport height, instead of clamping to the last `N` lines. Reproduced deterministically by binary search against a 23-row pane: `--lines 5/6/8/15` all returned zero bytes; `--lines 20` returned a partial read; `--lines 24` and above returned the full expected content, correctly clamping down even at `--lines 1000`. -This silently broke exactly the small bounded reads the adapter needs most - the composer-state verification read inside the send-and-verify path, and would have affected any small `fm-peek.sh` line count too. +This silently broke exactly the small bounded reads the adapter needs most - the composer-state guard/fallback reads around submit and injection, and would have affected any small `fm-peek.sh` line count too. Before the workaround, an early version of the real-herdr smoke test flaked intermittently for exactly this reason. **Workaround:** `fm_backend_herdr_capture` never passes a caller's small requested line count straight through to herdr's own `--lines` flag. @@ -241,21 +242,22 @@ Plain (non-argument) commands like `/new` did submit on the first Enter in the s The tmux backend was NOT affected by this incident: `fm_tmux_composer_state` reads the actual cursor row and classifies it as pending whenever real text remains, so its retry loop correctly issued the second Enter and landed the same live repro; this was confirmed side-by-side against the same real grok pane. -**Fix:** `fm_backend_herdr_composer_state` replaces the delta-based check with a structural read of the composer's OWN row, mirroring what the cursor-row read gives tmux. +**Fix:** `fm_backend_herdr_composer_state` replaced the delta-based check with a structural read of the composer's OWN row, mirroring what the cursor-row read gives tmux. Herdr's CLI exposes no cursor-row primitive, so the composer row is located by shape instead of position. For bordered composers, the row is the only line in a generous tail capture whose trimmed content both starts and ends with the same border glyph (`│`, `┃`, or a plain `|`) - the box's own top/bottom rows use rounded corners and never match, popup item rows and separator rows carry no border glyph at all, and the footer help line uses `│` only as an interior separator (never as the first/last character), so none of those can be mistaken for the composer. For unbordered live composers, added after the 2026-07-07 incident below, the row is a bottom-most trimmed line starting with a verified agent prompt glyph (`❯` for claude or `›` for codex); decorative bordered boxes above it lose to that bottom-most match. -A popup-close-with-placeholder-fill still reads as real content on that row, so it correctly classifies as pending and the retry loop sends the required second Enter, instead of stopping early. +A popup-close-with-placeholder-fill still reads as real content on that row, so composer fallback correctly classifies it as pending; on the normal idle-baseline path, the same first Enter also fails to start a turn, so native agent-state confirmation likewise retries instead of stopping early. Known ghost/placeholder composer text (`Type a message...`, verified grok 0.2.82's empty-composer hint) is recognized and still reads as empty. `FM_BACKEND_HERDR_IDLE_RE` extends that placeholder match, `FM_BACKEND_HERDR_BARE_PROMPT_RE` controls the recognized unbordered prompt glyphs, and `FM_BACKEND_HERDR_COMPOSER_LINES` controls the tail-window scan depth; all three are documented in [`docs/configuration.md`](configuration.md). -See `fm_backend_herdr_composer_state` and `fm_backend_herdr_send_text_submit` in `bin/backends/herdr.sh` for the implementation, and `tests/fm-backend-herdr.test.sh`'s composer-state and send-text-submit sections (including the bordered slash-submit retry and unbordered real claude/codex prompt fixtures) for the fake-harness coverage. +See `fm_backend_herdr_composer_state`, `fm_backend_herdr_wait_for_working`, and `fm_backend_herdr_send_text_submit` in `bin/backends/herdr.sh` for the implementation, and `tests/fm-backend-herdr.test.sh`'s composer-state, wait-for-working, and send-text-submit sections for the fake-harness coverage. -## Composer verification: structural composer-row read, not delta-based +## Composer-state classifier: structural row read, not delta-based -The herdr adapter's submit-verification no longer diffs raw pane content before/after Enter (see the incident above for why that was unsafe). -It instead classifies the composer's own row - located structurally as the bottom-most bordered composer row or verified bare prompt row described above - as empty or pending after each Enter attempt, retried (Enter only, never retyped) until it reads empty or retries are exhausted. -This mirrors tmux's cursor-row classification in spirit, without needing an equivalent cursor-row read primitive from herdr's CLI. -A dedicated composer-state or cursor-row read primitive is still a candidate upstream Herdr feature request; it would let this backend eventually verify with the same precision as tmux's native cursor-row read, rather than a structural approximation over a plain-text capture. +The herdr adapter no longer diffs raw pane content before/after Enter (see the incident above for why that was unsafe). +It keeps `fm_backend_herdr_composer_state` as a structural classifier for the composer's own row - located as the bottom-most bordered composer row or verified bare prompt row described above - and reports `empty`, `pending`, or `unknown`. +That classifier is still the pre-injection pending-input guard for the away-mode daemon and the conservative fallback when `fm_backend_herdr_send_text_submit` cannot use an idle/done native agent-state baseline. +Normal idle-baseline submit confirmation now uses herdr's native agent-state instead; see "Native agent-state submit confirmation" for the current submit path. +A dedicated composer-state or cursor-row/style primitive is still a candidate upstream Herdr feature request; it would let the guard/fallback classifier eventually reach tmux's cursor-row precision instead of relying on a structural approximation over plain-text capture. All implemented backends expose the identical caller-facing verdict vocabulary (`empty`, `pending`, `unknown`, `send-failed`), so `fm-send.sh` needs no backend-specific branching at all. @@ -366,7 +368,9 @@ Fixed by routing through `fm_backend_herdr_cli` (which appends `--session` on to This fix is backend-plumbing, not daemon-specific: it also corrects the same liveness check other callers use (`bin/fm-session-start.sh`'s per-task endpoint-liveness digest read). **Empirical verification (real herdr, isolated session only).** `tests/fm-afk-inject-herdr-e2e.test.sh` mirrors `tests/fm-afk-inject-e2e.test.sh`'s three scenarios (human-partial-input deferral, swallowed-Enter retry, a normal single digest) plus a fourth (a persistently pending composer that never clears must alarm via `state/.subsuper-inject-wedged`, preserve the buffer, and never crash the daemon) against a real, throwaway, NEVER-default `HERDR_SESSION`, torn down with `herdr_safe_stop_and_delete` exactly like `tests/fm-backend-herdr-smoke.test.sh`. -The "supervisor pane" is a tiny deterministic bash loop drawing a bordered composer row (not a real harness), exercising the bordered branch of `fm_backend_herdr_composer_state`; a thin `herdr` PATH shim swallows exactly one `pane send-keys enter` call to simulate the swallowed-Enter scenario, since herdr's real CLI has no built-in way to drop a keystroke. +The "supervisor pane" is a tiny deterministic bash loop, not a real harness binary, that draws a bordered composer row to exercise the bordered branch of `fm_backend_herdr_composer_state`. +Because submit confirmation now uses native agent-state on idle baselines, the fixture also registers itself as a herdr agent via `herdr pane report-agent` and reports an idle->working->idle cycle around each submitted line. +A thin `herdr` PATH shim swallows exactly one `pane send-keys enter` call to simulate the swallowed-Enter scenario, since herdr's real CLI has no built-in way to drop a keystroke. Real claude/codex unbordered prompt coverage lives in `tests/fm-backend-herdr.test.sh`'s captured-fixture regression tests described in the 2026-07-07 incident below. Building that test surfaced one more real finding worth recording for anyone writing a similar herdr-driven composer script: `tput cols`, called from WITHIN a script launched into a herdr pane via `pane run`/`send-text`, reported a stale/default `80` regardless of the pane's actual width, while an interactively-typed one-off `tput cols` in the same pane correctly reported its real width (54, in the environment this was verified in). @@ -385,19 +389,81 @@ Root cause: `fm_backend_herdr_composer_state`'s structural composer-row read (ad Real `claude`'s live input row is a BARE, unbordered `❯ …` - no border glyph anywhere around it - flanked by plain horizontal-rule separator lines, not a box. Claude's own startup welcome banner IS bordered, so immediately after launch the classifier's "last bordered row wins" scan locks onto the banner's own blank interior spacer row and misreads it as the composer (a coincidental, and wrong, "empty"). Once ordinary conversation scrolls that banner out of the 20-line capture window - true of any real supervisor pane with any history at all, which is every production case - NO bordered row exists anywhere in view, so the classifier reports `unknown` for a genuinely empty composer, forever. -`fm_backend_herdr_send_text_submit` treats only `empty` as a confirmed submit; `unknown` counts as failure, so `escalate_flush` never clears the buffer even though the real Enter genuinely submitted the digest to the real pane. +At the time, `fm_backend_herdr_send_text_submit` treated only a composer `empty` verdict as a confirmed submit; `unknown` counted as failure, so `escalate_flush` never cleared the buffer even though the real Enter genuinely submitted the digest to the real pane. Because `pane_input_pending`'s pre-type guard only defers on `pending` (never `unknown`), the next housekeeping tick's flush attempt retypes and resubmits the SAME unmodified buffer content - the redelivery loop. Also discovered while reproducing: real `codex` (0.142.x) has the identical unbordered-live-row shape, using `›` instead of claude's `❯`, confirming this is not claude-specific. Codex additionally shows dynamic tip/hint text in its idle composer (e.g. "Use /skills to list available skills") rather than a fixed placeholder string or a blank row, so no static regex can safely tell a genuinely idle codex composer apart from real pending text. -This narrower gap is NOT fixed here: an idle real-codex supervisor pane under herdr now classifies as `pending` rather than `unknown`, which makes injection defer indefinitely instead of redelivering - a strictly safer failure mode (the buffer is preserved, never silently dropped, and the existing max-defer wedge alarm still fires) but still not a correct read of the composer. -A real fix needs either an upstream herdr cursor-row/style primitive or a codex-specific signal; neither exists today. +This narrower composer-state gap remains whenever `fm_backend_herdr_composer_state` is asked: an idle real-codex supervisor pane under herdr classifies as `pending` rather than `empty`, which makes the pre-injection guard defer instead of injecting - a strictly safer failure mode (the buffer is preserved, never silently dropped, and the existing max-defer wedge alarm still fires) but still not a correct read of the composer. +The native agent-state submit confirmation below removes this gap from the normal idle-baseline submit-confirmation path, but a real fix for the composer classifier itself still needs either an upstream herdr cursor-row/style primitive or a codex-specific signal; neither exists today. **Fix:** `fm_backend_herdr_composer_state` now recognizes TWO composer-row shapes in one scan - the existing bordered shape, and a new bare (unbordered) shape: a trimmed line that STARTS with a verified agent-specific prompt glyph (`❯` claude or `›` codex) with no closing border required at all. The bare-row default is deliberately limited to `❯` and `›`, while generic shell-style glyphs (`>`, `$`, `%`, `#`) stay recognized only after the bordered shape has already identified a composer row, so a no-agent shell fallback cannot be misread as a delivered escalation. Both shapes are checked in the SAME forward scan, keeping whichever match comes LAST (bottom-most on screen), rather than trying bordered-only first and falling back to bare-only when nothing bordered is found: a bordered decorative box (a welcome banner, an update notice) is always rendered ABOVE the live composer, never below it, in every harness observed, so "last match of either shape wins" always resolves to the genuinely live, bottom-most row instead of a stale decorative box still sitting in the capture window. See `fm_backend_herdr_composer_state` in `bin/backends/herdr.sh` for the implementation, and `tests/fm-backend-herdr.test.sh`'s "unbordered (bare) composer rows" section (fixtures captured verbatim from real `claude`/`codex` panes) for the regression coverage - each of those tests read `unknown` before this fix and reads the correct verdict after. +## Native agent-state submit confirmation (fixes the codex idle-tip gap) + +`fm_backend_herdr_send_text_submit` now records a pre-Enter native agent-state baseline before choosing the confirmation signal. +When that baseline is legibly idle or done, it confirms a submit by polling herdr's own semantic agent-state (`agent get`) for a submit-active transition (`working` or `blocked`), via the new `fm_backend_herdr_wait_for_working` helper. +Composer content (`fm_backend_herdr_composer_state`) is unchanged and still used for the pre-injection empty-box guard (`bin/fm-supervise-daemon.sh`'s `pane_input_pending`, dispatched through `fm_backend_composer_state`). +It is also the conservative fallback for submit attempts whose pre-Enter baseline is already submit-active or unreadable, because a preexisting `working`/`blocked` status cannot prove that this Enter landed. +This makes the normal idle-baseline confirmation path cross-agent: it no longer depends on what a harness's idle composer happens to display. + +This directly fixes the practical effect of the codex gap left open by the 2026-07-07 incident above. +A genuinely idle codex composer shows dynamic tip/hint text ("Use /skills to list available skills", or codex's own rotating suggestions) that no static pattern can tell apart from real pending input, so `fm_backend_herdr_composer_state` still misreads it as `pending` - that composer-level gap is NOT fixed, and is not fixable by pattern matching alone (see the "Known gaps" entry below). +What changes is that the normal idle-baseline submit confirmation path no longer asks the composer that question at all, so the gap can no longer block or mis-confirm that path. + +### Design: two failure directions, both guarded + +A message that lands from an idle or done baseline must move the target agent into a submit-active state. +Two ways this signal can be missed, and how the design guards each: + +- **Slow transition.** A single check right after Enter could sample before herdr has updated `agent_status`, wrongly concluding "not submitted" and causing a needless extra Enter (harmless on its own here, since only Enter is retried, never the text - but wasteful and, for a stricter caller, could read as a false negative). + Fix: `fm_backend_herdr_wait_for_working` samples repeatedly (`FM_BACKEND_HERDR_SUBMIT_POLLS`, default 6) across the larger of the caller's per-attempt budget (``) and herdr's own minimum confirmation budget (`FM_BACKEND_HERDR_SUBMIT_MIN_SLEEP`, default 0.6s), instead of checking once at the end. + A transition landing anywhere in that window is caught, and the function returns the instant `busy` is observed, without waiting out the rest of the budget. +- **Instant round-trip.** A turn that starts and returns to idle entirely between two polls would, in the limit, never show as submit-active at all. + This is not eliminated in principle, but it is bounded by how densely `FM_BACKEND_HERDR_SUBMIT_POLLS` samples the budget, and the empirical evidence below shows real turns take far longer than the sampling interval to even START, let alone finish. + On the (unobserved) residual chance this happens, the function reports `pending`, and the caller's own invariant (retry Enter only, never retype) means the worst case is a redundant Enter landing on an already-empty composer - a no-op, not a duplicate delivery of the message text. + +`fm_backend_herdr_wait_for_working` also distinguishes a genuine "not yet submit-active" reading (the target was legibly read at least once, `idle`/`done` was observed, `working`/`blocked` never was) from a hard read failure (every poll in the window failed to read the target at all). +Only the latter reports `unknown` and skips further Enter retries - matching the pre-existing "never retry past an unreadable target" invariant the composer-based design already had. + +### Empirical evidence (2026-07-07, herdr 0.7.1, protocol 14, macOS aarch64) + +Verified against real `claude` (2.1.203) and real `codex` (0.142.1) agents in an isolated, throwaway `HERDR_SESSION` (never the default session), using `herdr_safe_stop_and_delete` for cleanup exactly like every other real-herdr test in this document. + +Method: for each agent, with the pane genuinely idle, `herdr pane send-text ""` followed by `herdr pane send-keys enter`, then `herdr agent get ` polled at roughly 30ms intervals, timestamping the FIRST poll that reports `agent_status: working`. + +Ten repeated trials per agent (a fresh trivial prompt each run, e.g. "reply with just the word pong"): + +| Agent | First-observed-working latency across 10 runs | +|---|---| +| claude 2.1.203 | 0.154s - 0.489s (mean ~0.27s) | +| codex 0.142.1 | 0.087s - 0.435s (mean ~0.25s) | + +Every trial's full turn (working -> idle/done) took at least ~1-3s end to end - orders of magnitude longer than the ~30ms sampling interval used to observe it, which is why an "instant round-trip" miss has not been observed in practice. + +Additional scenarios verified directly against the real binaries: + +- **Never-submitted text stays idle.** Typing real text into either agent's composer WITHOUT pressing Enter leaves `agent_status` unchanged (idle/`done`) indefinitely across repeated polls - confirming that an absence of a `working` observation is a genuine "not submitted" signal, not noise. +- **A popup-selection Enter that does not submit never flips to working.** Sending `/compact` to claude and pressing Enter once submitted immediately in this claude version (no placeholder-fill quirk reproduced here), transitioning to `working` right away - a real submission is what triggers the transition, exactly as designed. + The 2026-07-03 incident's specific failure shape (an Enter that only fills an argument-hint placeholder without submitting) was not literally reproduced against real claude/codex in this pass (grok, the originally affected harness, was not available), but the fix generalizes on logical grounds that do not depend on which harness is used: filling a composer placeholder is not a submission, so by construction no real turn starts and `agent_status` cannot report `working` for that Enter - see `tests/fm-backend-herdr.test.sh`'s `test_send_text_submit_popup_autocomplete_requires_second_enter` for the corresponding fake-CLI regression coverage. +- **A codex idle composer's dynamic tip text does not affect the new idle-baseline confirmation.** With a real, genuinely idle codex pane showing its own rotating suggestion ("Summarize recent commits"), `fm_backend_herdr_composer_state` on that pane still reads `pending` (the known, unfixed gap - see "Known gaps" below), but `fm_backend_herdr_send_text_submit` against the SAME pane correctly reports `empty` (confirmed) based on the observed `working` transition alone, and the message is confirmed to have landed in the pane's own transcript. +- **Confirmation correctly reports `pending` for a genuinely swallowed Enter.** With `fm_backend_herdr_send_key` overridden to a no-op (simulating a dropped keystroke), `fm_backend_herdr_send_text_submit` against a real claude pane reported `pending` after exhausting its retries, and the typed text was confirmed still sitting, unsubmitted, in the real composer afterward - no duplicate, no false confirmation. +- **Confirmation correctly reports `unknown` for a target that cannot be read**, and does not retry past it: with `fm_backend_herdr_agent_status_raw` overridden to always fail, a real send against a real claude pane reported `unknown` after exactly one Enter attempt (no further retries). +- **Submitting to an already submit-active target is not confirmed by preexisting agent-state alone.** A pre-Enter `working` or `blocked` status now falls back to composer-clear confirmation, so a swallowed Enter that leaves the typed message visible reports `pending` instead of falsely accepting the already-active status as proof. + If the composer clears, the adapter still reports `empty`; whether a queued message is reliably processed remains real-harness UI/UX behavior outside this adapter's control. + +### Regression coverage + +`tests/fm-backend-herdr.test.sh`'s "wait_for_working" and "send_text_submit" sections cover both failure directions (a slow transition caught mid-window, an unreadable target that never retries), endpoint-spread timing with no final trailing sleep, the submit-specific `blocked` mapping, the popup-placeholder-fill case using the new mechanism, the already-submit-active baseline fallback, and the core regression this task fixes: `test_send_text_submit_confirms_despite_codex_idle_tip_composer`, which asserts a confirmed `empty` verdict AND that `pane read` is never called on an idle baseline, using a real captured codex idle-tip fixture (`test_composer_state_codex_dynamic_idle_tip_still_reads_pending`) to prove the composer-level gap is untouched while the idle-baseline submit-confirmation path is immune to it. +`test_composer_state_guard_still_refuses_real_pending_text_after_submit_confirmation_change` is a regression guard for the pre-injection empty-box guard itself, confirming it still refuses genuine pending composer text after this change. + +`tests/fm-afk-inject-herdr-e2e.test.sh`'s synthetic supervisor-pane fixture was updated alongside this fix: since confirmation is no longer composer-content-based, a bash script that only DRAWS composer text without being a registered herdr agent would read `agent_not_found` forever and never confirm a submission - discovered when the pre-existing (composer-only) fixture version of that test regressed against the new confirmation code (Scenario B: 0 digests instead of exactly 1, since the daemon treated every injection as unconfirmed and kept retyping it every housekeeping tick, which is exactly the duplicate-send failure mode this design change exists to prevent). +The fix: the fixture now registers itself as a real herdr agent via `herdr pane report-agent --source --agent