fix: comprehensive state-backend fixes (bundles #1192 + Bug C/F/WI-1/upgrade flow)

[tamirdresher]

State-backend upgrade: OrphanBranch / TwoLayer / GitNotes hardening

HEAD: aaec183f  |  Base: c4f9d58f (upstream/dev)
CI status: all 6 jobs green (changes / docs-quality / lint+build / unit-tests / smoke / integration)

This PR delivers a fully hardened state-backend stack across all three concrete
backends (WorktreeBackend, GitNotesBackend, OrphanBranchBackend) plus the
TwoLayerBackend facade. It is the result of two review rounds against
PR #1200 (see review comment 4621356216
for the original 9 concerns A–I, and the in-PR fixes below for the resolved
subset).

---

What changed

Core hardening

• Retry + circuit-breaker on all git operations (gitExecWithRetry,
  gitExecWithInputAndRetry, CircuitBreaker) — transient lock/pack errors
  are retried with exponential back-off; persistent failures trip the breaker.
• verifyStateBackend startup probe — surfaces backend misconfiguration
  eagerly instead of silently returning undefined at runtime.
• StateBackendStorageAdapter thin adapter bridging the backend interface
  to the MCP tool layer.
• MCP guard for undefined content (NEW-4) — squad_state_write and
  squad_state_append now reject undefined content with a typed error before
  it ever reaches a backend.
• Trailing-newline preservation — gitExecWithRetry now accepts
  trimOutput; OrphanBranchBackend.read() passes false so blob content is
  returned verbatim, fixing 7 regressions in state-backend.test.ts.
• GitNotes anchoring — GitNotesBackend now anchors notes on the
  repository root commit rather than HEAD, so a branch switch does not
  appear to lose previously-written state.

Two-Layer now truly two-layered

The original TwoLayerBackend facade silently degraded to "fast layer only"
in several read/write paths because the warm-layer plumbing was incomplete and
errors were swallowed. The following changes make the two-layer guarantee
real:

• promoteNotes(key) — explicit promotion of an entry from the fast
  (notes) layer into the slow (orphan-branch) layer with verification of the
  written blob.
• readNote(key) — first-class fast-layer reader so callers don't have to
  reach through internal handles; resolves nullability honestly instead of
  returning undefined to mean both "miss" and "error".
• Per-layer verifyStateBackend — the startup probe now exercises each
  layer independently and reports which layer is unhealthy, instead of a
  single boolean that hides slow-layer failures behind fast-layer success.
• Observable warns — previously-swallowed catches in the 3 hottest
  TwoLayer code paths now console.warn with a structured tag
  ([two-layer]) so partial-degradation is visible in logs.

Regression fixes pulled in during review

• fix(shell): use effective state dir when resuming sessions (8f3208ac)
• fix(doctor): match install-hooks git-dir resolution for worktrees
  (dab1d9e8)
• fix(types): normalize legacy 'approved' permission kind (55e843c0)
• test(effective-squad-dir): stub global Squad path env vars so tests don't
  pollute $HOME (3a02478f)
• test(session-store,doctor): regression tests for the stateDir and git-dir
  fixes above (c9e5b755)

CI policy + repo hygiene

• ci(policy) (5bef8f28): version guard now allows
  X.Y.Z, X.Y.Z-preview, X.Y.Z-preview.N, and X.Y.Z-insider.N.
• ci (e19b4f83): clean re-application of the policy change with the
  CRLF↔LF churn reverted, and .gitattributes pinned to *.yml text eol=lf
  so the flip cannot recur on Windows checkouts. The net diff vs merge-base
  on .github/workflows/squad-ci.yml is now 6 lines instead of 1240.

---

Two-Layer Now Truly Two-Layered

Before	After
TwoLayerBackend.read() could return undefined from a swallowed catch	readNote() returns explicit null for miss, throws on error
Promote logic inlined inside write(), untested	promoteNotes(key) is a named, tested entry point with blob-verify
Startup probe checked the facade only, masked slow-layer failures	verifyStateBackend exercises fast layer + slow layer + facade independently and reports per-layer status
3 silent catch (e) {} blocks in hot paths	All 3 now console.warn('[two-layer] <op> partial degrade:', e.message)

The fix lands in aaec183f ("fix(state-backend): add notes promotion/read API
and observability for two-layer backend"). The notes-reader is the missing
piece that made the two-layer claim hold in practice rather than only in name.

---

Test status

• All 6 CI jobs green (changes / docs-quality / lint+build / unit-tests /
  smoke / integration) at HEAD.
• test/state-backend.test.ts: all assertions passing.

---

Known follow-ups (tracked separately)

The PR-1200 review surfaced 9 concerns. The table below tracks resolution:

ID	Concern	Status
A	Bare update-ref / git notes add -f is not CAS-safe under concurrent writers	Follow-up: see linked issue
B	TwoLayer facade silently degraded to fast-layer-only	Fixed in this PR (aaec183f)
C	Swallowed catches hide partial-degradation	Partially fixed (3 TwoLayer paths now warn). Audit of other backends → follow-up
D	verifyStateBackend only probed the facade	Partially fixed (per-layer probe added for TwoLayer). Extending the contract to other composite backends → follow-up
E	deleteDir is not recursive across nested subtrees	Follow-up
F	stateBackend: 'external' stub name collides with PR #1194's planned stateLocation: 'external'	Follow-up (rename to 'external-stub' with config-read deprecation warning)
G	.github/workflows/squad-ci.yml showed 1240-line CRLF↔LF flip	Fixed in this PR (e19b4f83, diff now 6 lines, .gitattributes pinned)
H	PR body was stale	Fixed in this PR (this rewrite)
I	No concurrent-writer regression test	Follow-up

Follow-up tracking: #1211 — "State-backend v2: ref-level CAS, recursive deleteDir, stub-name collision, regression coverage".

The single-user single-session case is unaffected by concern A. The
follow-ups matter for multi-session and multi-machine scenarios:
squad watch running alongside an interactive session, two terminals on
the same repo, or fleet workloads. Concern A must ship before we promote
two-layer to the default backend for fleet usage.

---

Co-authored-by: Copilot 223556219+Copilot@users.noreply.github.com

[Copilot]

Pull request overview

This PR hardens Squad’s state-backend upgrade path by relaxing backend initialization failures, aligning permission approval with the newer Copilot CLI contract, improving externalized-state path resolution, and updating related diagnostics/templates.

Changes:

• Updates state backend normalization/fallback behavior and Windows-safe relative path handling.
• Adds externalized state directory resolution across selected CLI runtime paths and doctor checks.
• Updates permission result typing/hints, template backend wording, version metadata, changesets, and tests.

Reviewed changes

Copilot reviewed 25 out of 26 changed files in this pull request and generated 5 comments.

Show a summary per file

File	Description
packages/squad-sdk/src/state-backend.ts	Adjusts backend fallback, git-notes migration warning, and path normalization.
packages/squad-sdk/src/adapter/types.ts	Adds approve-once to permission result kinds.
packages/squad-sdk/src/adapter/client.ts	Updates permission handler error guidance.
packages/squad-cli/src/cli/core/effective-squad-dir.ts	Adds helper for local vs external state resolution.
packages/squad-cli/src/cli/commands/loop.ts	Reads team.md from effective state dir.
packages/squad-cli/src/cli/commands/watch/index.ts	Reads team/routing files from effective state dir.
packages/squad-cli/src/cli/commands/plugin.ts	Resolves plugin storage through effective state dir.
packages/squad-cli/src/cli/commands/doctor.ts	Uses effective state dir for file checks and adds git sync hook diagnostics.
packages/squad-cli/src/cli/shell/lifecycle.ts	Resolves shell startup/welcome state files externally when configured.
packages/squad-cli/src/cli/shell/index.ts	Updates permission approval and partial externalized shell state lookup.
packages/squad-cli/src/cli/shell/coordinator.ts	Loads team/routing prompt inputs from external state when configured.
packages/squad-cli/scripts/patch-esm-imports.mjs	Adds repo-local node_modules to patch search roots.
test/state-backend.test.ts	Updates backend failure expectation to soft fallback.
test/effective-squad-dir.test.ts	Adds coverage for effective state dir resolution.
test/cli/doctor.test.ts	Adds doctor coverage for git sync hook checks.
.squad-templates/squad.agent.md	Updates canonical backend wording.
.github/agents/squad.agent.md	Syncs active agent backend wording.
templates/squad.agent.md.template	Syncs template backend wording.
packages/squad-cli/templates/squad.agent.md.template	Syncs CLI template backend wording.
packages/squad-sdk/templates/squad.agent.md.template	Syncs SDK template backend wording.
package.json	Bumps root package version.
packages/squad-cli/package.json	Bumps CLI package version.
packages/squad-sdk/package.json	Bumps SDK package version.
package-lock.json	Updates lockfile package metadata.
.changeset/fix-state-backend-upgrade-regressions.md	Documents CLI/SDK patch fixes.
.changeset/fix-externalized-state-paths.md	Documents CLI externalized-state path fix.

[tamirdresher]

Fixed in 55e843c — 'approved'\ is now marked @deprecated\ in \ ypes.ts\ and \client.ts\ normalizes { kind: 'approved' }\ → { kind: 'approve-once' }\ at the adapter boundary. \samples/knock-knock\ updated to use 'approve-once'\ directly.

[tamirdresher]

Fixed in 8f3208a — added optional \stateDir?\ param to all 5 session-store functions and threaded it through load/save call-sites in \shell/index.ts. Regression tests in \ est/session-store.test.ts.

[tamirdresher]

Fixed in 3a02478 — added top-level \�eforeEach/\�fterEach\ in \�ffective-squad-dir.test.ts\ that stubs \APPDATA/\XDG_CONFIG_HOME\ to a unique temp dir, so
esolveGlobalSquadPath()\ never touches the real user config directory.

[tamirdresher]

Fixed in the same commit 3a02478 — the env-var stubbing covers both \APPDATA\ (Windows) and \XDG_CONFIG_HOME\ (Linux/macOS) so the test is isolated on all platforms.

[tamirdresher]

Fixed in dab1d9e — \checkGitSyncHooks()\ now uses the same three-step resolution as \install-hooks: \git config --get core.hooksPath\ → \git rev-parse --git-dir\ → fallback to .git/hooks. Regression tests (including worktree isolation) in \ est/cli/doctor.test.ts.

[tamirdresher]

Review: will the two-layer state backend work for Squad?

Reviewed the description, changesets, the squad.agent.md change, and the actual state-backend.ts implementation (WorktreeBackend, GitNotesBackend, OrphanBranchBackend, TwoLayerBackend, resolveStateBackend, normalizeBackendType). Net: the facade and the hardening/bug-fixes are solid, but there is one structural concurrency issue that matters specifically because of how Squad runs.

🔴 Top concern: no atomicity / lost updates under concurrent writers

OrphanBranchBackend.write/append/delete all do read-modify-write then:

gitExecOrThrow(`update-ref refs/heads/${this.branch} ${newCommit}`, this.cwd);

update-ref is called without the old-value argument, so there is no compare-and-swap. Every mutation — even to different files — serializes through one branch ref. Two agents that both read parent commit X, build new commits, and update-ref will silently clobber each other (last-writer-wins → lost commits, no error).

Why this bites Squad specifically: the whole model is parallel agents (Ralph fleet, parallel spawns), and the cleanup ceremony in squad.agent.md has multiple agents concurrently appending to decisions.md, agents/*/history.md, and writing orchestration-log/{agent}.md. append is the worst case — concurrent read-modify-write append drops entries.

The retry/circuit-breaker does not help here: those handle transient index.lock/pack errors that throw. A racing update-ref succeeds and overwrites. Fix needs one of: git update-ref <ref> <new> <old> (CAS) + retry-on-mismatch, a lockfile, or serialized writes. Same applies to GitNotesBackend (single note on the root commit).

🟡 Other concerns

• Notes layer is currently inert. TwoLayerBackend.read() only reads orphan; write/append also write notes, but nothing reads them back except the promote_to_permanent step mentioned only in a comment — not in this PR. So the notes writes are dead weight until promote exists, and the two layers can silently diverge.
• Silent swallow contradicts the stated goal. try { this.notes.write(...) } catch {} eats errors with no log, while the changeset advertises "observable error surfacing replacing silent swallowing." At minimum log a warning so a broken second layer is detectable.
• verifyStateBackend only verifies one layer. It calls backend.list(''), which for two-layer delegates to orphan — a broken notes layer passes the health check.
• deleteDir is one level deep. It lists immediate children and deletes each; nested state dirs (e.g. under agents/) may leak. Worth a test.
• Naming collision footgun: stateBackend: 'external' (a stub → falls back to local) vs the State documentation out of sync with latest release #1194 stateLocation: 'external' feature are unrelated mechanisms sharing the word "external." Easy to confuse in config/support.

🟢 What's good

• resolveStateBackend now warns + falls back to local instead of hard-throwing (Bug B) — correct.
• toRelative Windows drive-letter normalization + throwing on out-of-squadDir paths instead of emitting corrupt keys (Bug F) — good.
• Trailing-newline fix (trimOutput=false on OrphanBranchBackend.read) is right; gitExecWithInputAndRetry keeping .trim() is fine since it only returns SHAs.
• validateStateKey (null/newline/tab/.. rejection) and path-traversal guard are solid.

Did we forget anything?

1. A concurrent-writer regression test — the failure mode above is untested.
2. .github/workflows/squad-ci.yml: 620 deletions + 620 additions of identical lines — pure CRLF↔LF churn. Almost certainly accidental (line-ending flip); it bloats the diff ~1240 lines and risks re-flipping on the next editor. Recommend reverting so the CI file shows as unchanged.
3. The PR ships with 2 failing tests ("pre-existing env timeouts"). Confirm they're genuinely environmental and not masking append/concurrency behavior.
4. Decide whether promote_to_permanent belongs in this PR or a tracked follow-up — otherwise two-layer's second layer has no consumer.

Bottom line: two-layer is fine for a single coordinator writing sequentially (the common case). For Squad's parallel/fleet usage it will eventually lose state under contention. Suggest merging the hardening/bug-fixes but opening a tracked issue for ref-level CAS (or a write lock) before making two-layer the default for fleet workloads — and reverting the CI line-ending churn before merge.