Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 2 additions & 1 deletion .agents/skills/bootstrap-diagnostics/SKILL.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@
name: bootstrap-diagnostics
description: >-
Agent-only handling playbook for session-start bootstrap diagnostics.
Use whenever the session-start digest's bootstrap section prints an actionable diagnostic line - MISSING, MISSING_MANUAL, BACKEND_INVALID, NEEDS_GH_AUTH, TANGLE, CREW_DISPATCH invalid, FLEET_SYNC, PR_CHECK_MIGRATION, SECONDMATE_SYNC, SECONDMATE_LIVENESS, NUDGE_SECONDMATES, AXI_SUITE_UPDATED, AXI_SUITE_REVIEW, AXI_SUITE_STUCK, FIRSTMATE_UPDATE_AVAILABLE, FIRSTMATE_UPDATE_STUCK, FORK_SYNC, FORK_SYNC_STUCK, or FMX - or when a standalone bin/fm-bootstrap.sh run prints one of those lines.
Use whenever the session-start digest's bootstrap section prints an actionable diagnostic line - MISSING, MISSING_MANUAL, BACKEND_INVALID, NEEDS_GH_AUTH, TANGLE, SELF_DRIFT, CREW_DISPATCH invalid, FLEET_SYNC, PR_CHECK_MIGRATION, SECONDMATE_SYNC, SECONDMATE_LIVENESS, NUDGE_SECONDMATES, AXI_SUITE_UPDATED, AXI_SUITE_REVIEW, AXI_SUITE_STUCK, FIRSTMATE_UPDATE_AVAILABLE, FIRSTMATE_UPDATE_STUCK, FORK_SYNC, FORK_SYNC_STUCK, or FMX - or when a standalone bin/fm-bootstrap.sh run prints one of those lines.
A silent bootstrap section, or a BOOTSTRAP_INFO fact, means no skill load.
user-invocable: false
metadata:
Expand Down Expand Up @@ -34,6 +34,7 @@ When any diagnostic needs captain attention, report the plain consequence and re
- `TANGLE: <remediation>` - the primary checkout is stranded on a feature branch instead of its default branch; `AGENTS.md` section 8 explains why this guard exists and what it protects.
The work is safe on that branch ref; restore the primary to its default branch with the printed `git -C <root> checkout <default>`, then re-validate that branch in a proper worktree.
This is the only sanctioned firstmate-initiated git write to the primary, and it is a non-destructive branch switch that strands nothing.
- `SELF_DRIFT: <detail>` - the primary checkout's default branch is ahead of, behind, or diverged from its own origin and bootstrap left it untouched; use `/updatefirstmate` only for a clean fast-forward case, otherwise dispatch a manual recover-by-merge crewmate task that preserves the local commits.
- `CREW_DISPATCH: invalid config/crew-dispatch.json - <reason>` - the optional dispatch profile file exists but failed low-cost bootstrap validation; continue with the normal fallback chain, resolve and pass the chosen fallback harness explicitly while the file remains present, fix the malformed schema, unverified harness name, unknown selector, or invalid harness/effort pair when convenient, and do not select a bad profile.
- `FLEET_SYNC: <repo>: skipped: <reason>` - a benign one-off skip (offline, no origin, local-only); bootstrap continued, investigate only if it blocks work.
A skip can also report the bounded fleet-refresh timeout (`FM_FLEET_SYNC_BOOTSTRAP_TIMEOUT`, or a fleet-size-aware default with a 20 second floor); a timeout never blocks startup.
Expand Down
3 changes: 2 additions & 1 deletion AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -454,12 +454,13 @@ When bootstrap prints `FIRSTMATE_UPDATE_AVAILABLE:`, dispatch a crewmate to noti
Fork `main` advances without rewriting history, and every upstream-sync PR must land as a true merge commit rather than a squash.
When the captain invokes `/updatefirstmate` or asks to update firstmate, load the `/updatefirstmate` skill.
It performs guarded fast-forward updates of firstmate and registered secondmate homes, refreshes instructions, and never touches anything under `projects/`.
Bootstrap separately detects when the primary checkout's own default branch has drifted from its own origin and reports `SELF_DRIFT:`; `/updatefirstmate` resolves only the clean fast-forward (behind-only) case, while an ahead or diverged primary needs a manual preserve-and-merge crewmate task instead - `bootstrap-diagnostics` owns the exact remediation text.

## 13. Agent-only reference skills

These skills are not captain-invocable; load them only at their precise triggers.

- `bootstrap-diagnostics` - load whenever the session-start digest's bootstrap section prints an actionable diagnostic line (`MISSING:`, `MISSING_MANUAL:`, `BACKEND_INVALID:`, `NEEDS_GH_AUTH`, `TANGLE:`, `CREW_DISPATCH: invalid`, `FLEET_SYNC:`, `PR_CHECK_MIGRATION:`, `SECONDMATE_SYNC:`, `SECONDMATE_LIVENESS:`, `NUDGE_SECONDMATES:`, `AXI_SUITE_UPDATED:`, `AXI_SUITE_REVIEW:`, `AXI_SUITE_STUCK:`, `FIRSTMATE_UPDATE_AVAILABLE:`, `FIRSTMATE_UPDATE_STUCK:`, `FORK_SYNC:`, `FORK_SYNC_STUCK:`, or `FMX:`); silence and `BOOTSTRAP_INFO:` need no load.
- `bootstrap-diagnostics` - load whenever the session-start digest's bootstrap section prints an actionable diagnostic line (`MISSING:`, `MISSING_MANUAL:`, `BACKEND_INVALID:`, `NEEDS_GH_AUTH`, `TANGLE:`, `SELF_DRIFT:`, `CREW_DISPATCH: invalid`, `FLEET_SYNC:`, `PR_CHECK_MIGRATION:`, `SECONDMATE_SYNC:`, `SECONDMATE_LIVENESS:`, `NUDGE_SECONDMATES:`, `AXI_SUITE_UPDATED:`, `AXI_SUITE_REVIEW:`, `AXI_SUITE_STUCK:`, `FIRSTMATE_UPDATE_AVAILABLE:`, `FIRSTMATE_UPDATE_STUCK:`, `FORK_SYNC:`, `FORK_SYNC_STUCK:`, or `FMX:`); silence and `BOOTSTRAP_INFO:` need no load.
- `diagnostic-reasoning` - load before scoping a reported bug and before acting on a diagnostic report.
- `harness-adapters` - load before spawning or recovering a crewmate or secondmate, handling a trust dialog, sending a harness-specific skill invocation, interrupting or exiting an agent, resuming an exited agent, or verifying a new harness adapter.
- `firstmate-orca` - load before switching to Orca, spawning or supervising Orca-backed work, smoke-testing Orca backend behavior, debugging Orca task state, or reconciling Orca-backed task metadata.
Expand Down
70 changes: 70 additions & 0 deletions bin/fm-bootstrap.sh
Original file line number Diff line number Diff line change
Expand Up @@ -12,6 +12,7 @@
# "FLEET_SYNC: <repo>: skipped|recovered|STUCK: <detail>",
# "PR_CHECK_MIGRATION: <private remediation>",
# "TANGLE: <remediation>",
# "SELF_DRIFT: primary checkout default branch '<branch>' is <N> ahead, <M> behind origin/<branch> (<state>) - needs attention",
# "SECONDMATE_SYNC: secondmate <id>: skipped: <reason>",
# "NUDGE_SECONDMATES: secondmate <id>: send failed: <reason>",
# "BOOTSTRAP_INFO: nudged fm-<id> with '<message>'",
Expand Down Expand Up @@ -46,6 +47,10 @@
# A TANGLE line means the firstmate primary checkout (FM_ROOT) is stranded
# on a feature branch instead of its default branch - a crewmate's work
# landed in the primary instead of its own worktree; restore it per the line.
# A SELF_DRIFT line means that checkout's default branch differs from its
# own origin; the check fetches origin without advancing any local branch,
# is bounded by FM_SELF_DRIFT_BOOTSTRAP_TIMEOUT (default 10s), and silently
# skips missing origins, off-default branches, and fetch failures/timeouts.
# treehouse is also MISSING when its installed version lacks
# "treehouse get --lease" support.
# no-mistakes is also MISSING when its installed version is older than
Expand Down Expand Up @@ -187,6 +192,69 @@ fleet_sync() {
rm -f "$tmp"
}

self_drift_bootstrap_timeout() {
case "${FM_SELF_DRIFT_BOOTSTRAP_TIMEOUT:-}" in
''|*[!0-9]*) echo 10 ;;
*) echo "$FM_SELF_DRIFT_BOOTSTRAP_TIMEOUT" ;;
esac
}

# Detect drift between the primary checkout's checked-out default branch and its
# own origin. Fetch is bounded and best-effort; this never advances a local
# branch, checks out a ref, or reports an off-default TANGLE state twice.
self_drift_check() {
local default current base timeout monitor_was_on pid start elapsed fetch_ok
local ahead behind state

git -C "$FM_ROOT" remote get-url origin >/dev/null 2>&1 || return 0
default=$(fm_default_branch "$FM_ROOT" 2>/dev/null) || return 0
current=$(git -C "$FM_ROOT" symbolic-ref --quiet --short HEAD 2>/dev/null || true)
[ "$current" = "$default" ] || return 0

timeout=$(self_drift_bootstrap_timeout)
monitor_was_on=0
case $- in *m*) monitor_was_on=1 ;; esac
set -m 2>/dev/null || true
git -C "$FM_ROOT" fetch origin --prune --quiet >/dev/null 2>&1 &
pid=$!
start=$SECONDS
fetch_ok=1
while jobs -r -p | grep -qx "$pid"; do
elapsed=$((SECONDS - start))
if [ "$elapsed" -ge "$timeout" ]; then
kill -TERM "-$pid" 2>/dev/null || kill "$pid" 2>/dev/null || true
wait "$pid" 2>/dev/null || true
fetch_ok=0
break
fi
sleep 1
done
if [ "$fetch_ok" -eq 1 ] && ! wait "$pid" 2>/dev/null; then
fetch_ok=0
fi
[ "$monitor_was_on" -eq 1 ] || set +m 2>/dev/null || true
[ "$fetch_ok" -eq 1 ] || return 0

# Recheck after the network wait so a concurrent branch switch cannot turn
# this into a duplicate or stale off-default diagnostic.
current=$(git -C "$FM_ROOT" symbolic-ref --quiet --short HEAD 2>/dev/null || true)
[ "$current" = "$default" ] || return 0
base="origin/$default"
git -C "$FM_ROOT" rev-parse --verify --quiet "$base^{commit}" >/dev/null || return 0
ahead=$(git -C "$FM_ROOT" rev-list --count "$base..$default" 2>/dev/null) || return 0
behind=$(git -C "$FM_ROOT" rev-list --count "$default..$base" 2>/dev/null) || return 0
[ "$ahead" -ne 0 ] || [ "$behind" -ne 0 ] || return 0

if git -C "$FM_ROOT" merge-base --is-ancestor "$default" "$base" 2>/dev/null; then
state=behind
elif git -C "$FM_ROOT" merge-base --is-ancestor "$base" "$default" 2>/dev/null; then
state=ahead
else
state=diverged
fi
echo "SELF_DRIFT: primary checkout default branch '$default' is $ahead ahead, $behind behind $base ($state) - needs attention"
}

secondmate_sync() {
# Local-HEAD secondmate sync: fast-forward every LIVE secondmate home
# to the primary checkout's current default-branch commit. Purely LOCAL - no
Expand Down Expand Up @@ -791,6 +859,8 @@ if [ -n "$tangle_branch" ]; then
else
echo "TANGLE: primary checkout on feature branch '$tangle_branch' (expected '$tangle_default'); the work is safe on that ref - restore the primary with: git -C $FM_ROOT checkout $tangle_default, then re-validate the branch in a proper worktree"
fi
else
self_drift_check
fi
crew=
[ -f "$CONFIG/crew-harness" ] && crew=$(tr -d '[:space:]' < "$CONFIG/crew-harness" || true)
Expand Down
3 changes: 3 additions & 0 deletions docs/architecture.md
Original file line number Diff line number Diff line change
Expand Up @@ -239,6 +239,9 @@ The origin-based updater and the local secondmate sync share the same guarded fa
The mechanics are owned by the `/updatefirstmate` skill and firstmate's operating manual in [`AGENTS.md`](../AGENTS.md) (self-update).
The separately scheduled `bin/fm-firstmate-update-check.sh` reports upstream-only instruction-surface changes without updating the checkout.
The curator-only `bin/fm-fork-sync-check.sh` reports when the curated fork is behind real upstream and points its patch review at `docs/fork-patches.md`.
None of those checks cover drift between the primary checkout's own default branch and its own origin, so bootstrap's `self_drift_check` fills that gap: on the primary's default branch it fetches origin under a bounded `FM_SELF_DRIFT_BOOTSTRAP_TIMEOUT` (default 10s), then compares local and `origin/<default>` with `rev-list`/`merge-base`, and is stateless and detect-only - it never fast-forwards, checks out, merges, or stashes.
A matching branch stays silent, and missing origin, off-default branch, fetch failure, and timeout are benign skips left to the TANGLE and fleet-sync checks above; a real ahead, behind, or diverged mismatch reports `SELF_DRIFT:` with both commit counts, and it runs in both locked and read-only detect-only bootstrap paths.
Remediation is proportional to the mismatch: `/updatefirstmate` only covers the clean fast-forward (behind-only) case, while ahead or diverged history needs a manual preserve-and-merge crewmate task instead.

## Restart-proof

Expand Down
2 changes: 2 additions & 0 deletions docs/configuration.md
Original file line number Diff line number Diff line change
Expand Up @@ -277,6 +277,7 @@ Successful changes emit `AXI_SUITE_UPDATED:`, and bounded registry, permission,
Neither script mutates the checkout or runs from bootstrap, so schedule them externally; their headers and `--help` output own exact overrides and mechanics.
Bootstrap also reports a `TANGLE:` line when `FM_ROOT` is on a named non-default branch; follow the printed checkout remediation rather than treating it as an installable tool problem.
In a read-only session that did not get the fleet lock, the same line is advisory and omits the checkout command.
When `FM_ROOT` sits on its default branch instead, bootstrap reports a `SELF_DRIFT:` line if that branch and its own origin disagree; [architecture.md](architecture.md#self-updates-stay-safe) owns the detection and remediation, and `FM_SELF_DRIFT_BOOTSTRAP_TIMEOUT` below bounds its fetch.
The locked session-start bootstrap step also runs a best-effort project clone refresh through `fm-fleet-sync.sh`.
It emits `FLEET_SYNC:` for skipped refreshes that may matter, recovered self-heals, and `STUCK:` alarms.
Normal completed runs keep local-only and no-origin skips silent.
Expand Down Expand Up @@ -436,6 +437,7 @@ FM_PAUSE_RESURFACE_SECS=3600 # seconds before an idle declared external wa
FM_WEDGE_DEMAND_INSPECT_COUNT=3 # consecutive provably-working stale escalations on the same unchanged pane before demand-deep-inspection is added
FM_WATCH_TRIAGE_LOG_MAX_BYTES=262144 # size cap for the watcher's absorbed-wake debug log
FM_FLEET_SYNC_BOOTSTRAP_TIMEOUT= # optional seconds allowed for bootstrap's best-effort clone refresh; unset/blank defaults to max(20, 5 + 3 * origin-backed-project-count)
FM_SELF_DRIFT_BOOTSTRAP_TIMEOUT=10 # seconds allowed for bootstrap's best-effort origin fetch when checking the primary checkout's default branch for self-drift
FM_FLEET_PRUNE=1 # set to 0 to skip pruning local branches whose upstream is gone
FM_STALE_WORKTREE_LOCK_AGE_SECS=30 # min mtime age before fm-teardown.sh treats a leftover worktree git index.lock as provably stale
FM_TREEHOUSE_RETURN_LOCK_RETRIES=3 # retries after a treehouse return fails on the transient git index.lock signature
Expand Down
177 changes: 177 additions & 0 deletions tests/fm-self-drift.test.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,177 @@
#!/usr/bin/env bash
# Behavior tests for fm-bootstrap.sh's detect-only primary origin-drift check.
#
# The check compares only a primary checkout on its own default branch, reports
# quantified ahead/behind drift after a bounded fetch, and never duplicates the
# existing TANGLE diagnostic for an off-default branch.
set -u

# shellcheck source=tests/lib.sh disable=SC1091
. "$(dirname "${BASH_SOURCE[0]}")/lib.sh"

TMP_ROOT=$(fm_test_tmproot fm-self-drift)
fm_git_identity

make_case() {
local name=$1 seed remote primary writer
seed="$TMP_ROOT/$name-seed"
remote="$TMP_ROOT/$name-origin.git"
primary="$TMP_ROOT/$name-primary"
writer="$TMP_ROOT/$name-writer"
git init -q -b main "$seed"
printf '%s\n' initial > "$seed/tracked"
git -C "$seed" add tracked
git -C "$seed" commit -qm initial
git clone -q --bare "$seed" "$remote"
git clone -q "$remote" "$primary"
git clone -q "$remote" "$writer"
printf '%s|%s\n' "$primary" "$writer"
}

run_bootstrap() {
local repo=$1
FM_ROOT_OVERRIDE="$repo" FM_HOME="$repo" FM_BOOTSTRAP_DETECT_ONLY=1 \
FM_SELF_DRIFT_BOOTSTRAP_TIMEOUT="${FM_SELF_DRIFT_BOOTSTRAP_TIMEOUT:-3}" \
"$ROOT/bin/fm-bootstrap.sh" 2>/dev/null
}

self_drift_line() {
run_bootstrap "$1" | grep '^SELF_DRIFT:' || true
}

commit_file() {
local repo=$1 content=$2
printf '%s\n' "$content" >> "$repo/tracked"
git -C "$repo" add tracked
git -C "$repo" commit -qm "$content"
}

test_clean_matching_is_silent() {
local fixture primary out
fixture=$(make_case clean)
primary=${fixture%%|*}
out=$(self_drift_line "$primary")
[ -z "$out" ] || fail "matching primary emitted drift: $out"
pass "self drift: matching default branch is silent"
}

test_ahead_only_is_quantified() {
local fixture primary out
fixture=$(make_case ahead)
primary=${fixture%%|*}
commit_file "$primary" local
out=$(self_drift_line "$primary")
assert_contains "$out" "is 1 ahead, 0 behind origin/main (ahead)" "ahead-only drift was not quantified"
pass "self drift: ahead-only default branch is quantified"
}

test_behind_only_is_quantified() {
local fixture primary writer out
fixture=$(make_case behind)
primary=${fixture%%|*}
writer=${fixture#*|}
commit_file "$writer" remote
git -C "$writer" push -q origin main
out=$(self_drift_line "$primary")
assert_contains "$out" "is 0 ahead, 1 behind origin/main (behind)" "behind-only drift was not quantified"
pass "self drift: behind-only default branch is quantified"
}

test_diverged_is_quantified() {
local fixture primary writer out
fixture=$(make_case diverged)
primary=${fixture%%|*}
writer=${fixture#*|}
commit_file "$primary" local
commit_file "$writer" remote
git -C "$writer" push -q origin main
out=$(self_drift_line "$primary")
assert_contains "$out" "is 1 ahead, 1 behind origin/main (diverged)" "diverged drift was not quantified"
pass "self drift: diverged default branch is quantified"
}

test_no_origin_is_silent() {
local repo out
repo="$TMP_ROOT/no-origin"
git init -q -b main "$repo"
git -C "$repo" commit -q --allow-empty -m initial
out=$(self_drift_line "$repo")
[ -z "$out" ] || fail "no-origin primary emitted drift: $out"
pass "self drift: primary without origin is silent"
}

test_off_default_reports_only_tangle() {
local fixture primary out
fixture=$(make_case off-default)
primary=${fixture%%|*}
git -C "$primary" checkout -qb fm/feature
commit_file "$primary" feature
out=$(run_bootstrap "$primary")
assert_contains "$out" "TANGLE: primary checkout on feature branch 'fm/feature'" "off-default primary did not report TANGLE"
assert_not_contains "$out" "SELF_DRIFT:" "off-default primary double-reported SELF_DRIFT"
pass "self drift: off-default primary leaves reporting to TANGLE"
}

test_detached_head_is_silent() {
local fixture primary out
fixture=$(make_case detached)
primary=${fixture%%|*}
git -C "$primary" checkout -q --detach
out=$(run_bootstrap "$primary")
assert_not_contains "$out" "SELF_DRIFT:" "detached-HEAD primary emitted drift"
assert_not_contains "$out" "TANGLE:" "detached-HEAD primary emitted tangle"
pass "self drift: detached-HEAD primary is silent"
}

test_fetch_failure_is_silent() {
local fixture primary fakebin real_git out
fixture=$(make_case fetch-fail)
primary=${fixture%%|*}
fakebin=$(fm_fakebin "$TMP_ROOT/fetch-fail-fake")
real_git=$(command -v git)
cat > "$fakebin/git" <<'SH'
#!/usr/bin/env bash
if [ "${1:-}" = -C ] && [ "${2:-}" = "${FM_FAIL_GIT_ROOT:-}" ] && [ "${3:-}" = fetch ]; then
exit 1
fi
exec "$FM_REAL_GIT" "$@"
SH
chmod +x "$fakebin/git"
out=$(PATH="$fakebin:$PATH" FM_REAL_GIT="$real_git" FM_FAIL_GIT_ROOT="$primary" \
self_drift_line "$primary")
[ -z "$out" ] || fail "fetch failure emitted drift instead of skipping: $out"
pass "self drift: plain fetch failure is silently skipped"
}

test_slow_origin_is_bounded_and_silent() {
local fixture primary fakebin real_git out start elapsed
fixture=$(make_case slow)
primary=${fixture%%|*}
fakebin=$(fm_fakebin "$TMP_ROOT/slow-fake")
real_git=$(command -v git)
cat > "$fakebin/git" <<'SH'
#!/usr/bin/env bash
if [ "${1:-}" = -C ] && [ "${2:-}" = "${FM_SLOW_GIT_ROOT:-}" ] && [ "${3:-}" = fetch ]; then
sleep 300
fi
exec "$FM_REAL_GIT" "$@"
SH
chmod +x "$fakebin/git"
start=$SECONDS
out=$(PATH="$fakebin:$PATH" FM_REAL_GIT="$real_git" FM_SLOW_GIT_ROOT="$primary" \
FM_SELF_DRIFT_BOOTSTRAP_TIMEOUT=1 self_drift_line "$primary")
elapsed=$((SECONDS - start))
[ "$elapsed" -lt 5 ] || fail "slow origin exceeded its bootstrap bound (${elapsed}s)"
[ -z "$out" ] || fail "slow origin emitted drift instead of skipping: $out"
pass "self drift: slow origin is bounded and silently skipped"
}

test_clean_matching_is_silent
test_ahead_only_is_quantified
test_behind_only_is_quantified
test_diverged_is_quantified
test_no_origin_is_silent
test_off_default_reports_only_tangle
test_detached_head_is_silent
test_fetch_failure_is_silent
test_slow_origin_is_bounded_and_silent