Skip to content
This repository was archived by the owner on Jul 15, 2026. It is now read-only.

feat(agent-os): run Firstmate as a portable Kubernetes agent#11

Open
robinbraemer wants to merge 65 commits into
mainfrom
fm/agent-os-firstmate-source-closure-r8
Open

feat(agent-os): run Firstmate as a portable Kubernetes agent#11
robinbraemer wants to merge 65 commits into
mainfrom
fm/agent-os-firstmate-source-closure-r8

Conversation

@robinbraemer

Copy link
Copy Markdown
Member

What Changed

  • Package Firstmate as a portable Kubernetes Agent OS with a persistent container toolchain, isolated recoverable crewmate Pods, an Akua-rendered package, and an OrbStack development profile.
  • Add immutable source and image provenance plus guarded publication and Kubernetes lifecycle operations with installation ownership, freshness checks, conditional mutations, rollback validation, and safe deletion.
  • Synchronize Firstmate runtime improvements across Pi supervision, Zellij recovery, backend dispatch, secondmate routing markers, bootstrap diagnostics, and X-mode follow-up context.

Risk Assessment

✅ Low: Captain, the previously identified publication and Kubernetes ownership races are closed with bounded, fail-closed mechanics, and this pass found no remaining material issue.

Testing

Completed 1 recorded test check.

Pipeline

Updates from git push no-mistakes

⏭️ **intent** - skipped

✅ No issues found.

✅ **Rebase** - passed

✅ No issues found.

🔧 **Review** - 7 issues found → auto-fixed (3) ✅
  • 🚨 bin/fm-bootstrap.sh:351 - Adding akua to COMMON_TOOLS makes it a universal bootstrap dependency, although the documentation defines Akua as optional and only the Kubernetes tooling consumes it. Move this check into the Kubernetes-specific workflow so ordinary Firstmate sessions do not report MISSING and block dispatch.
  • 🚨 bin/agent-os-crewmate.sh:57 - purge --yes deletes the computed Pod and PVC names without verifying that those objects belong to this crewmate. In a shared namespace, a name collision can destroy unrelated persistent data; validate immutable ownership markers before both create/adoption and deletion, and fail closed.
  • ⚠️ tools/agent-os/packages/firstmate/package.k:34 - Installation applies the Agent OS ownership label to an existing namespace, effectively adopting it. The later namespace-deletion guard trusts only that label, so delete-namespace --yes can delete unrelated resources from a pre-existing shared namespace; refuse adoption by default or use a stronger ownership token.
  • 🚨 tools/agent-os/skills-lock.json:5 - The distributed Effect skill bundle records only a repository name and content hash, while the source-provenance and third-party records contain no immutable upstream revision, license, or notice for it. Record the exact source revision and redistribution terms, or remove the vendored bundle before publication.
  • ⚠️ .github/workflows/agent-os-image.yml:151 - The publishing job gives GITHUB_TOKEN package-write authority to actions referenced by mutable major-version tags. Pin login, build/push, checkout, and the other third-party actions to full commit SHAs to prevent tag retargeting from becoming a release supply-chain compromise.
  • ⚠️ bin/fm-x-link.sh:141 - The link command always permits relay lookup while resolving reply context, including in dry-run mode. If local state is incomplete and credentials exist, a supposedly local preview sends request metadata to the relay; pass allow_relay=0 whenever dry-run is enabled.
  • ⚠️ .github/workflows/agent-os-image.yml:101 - Every non-PR push, including arbitrary tag pushes, can publish an image after only the provenance test; the job has no dependency on the repository's behavioral CI. Require the full CI result before publication, or explicitly restrict tags to already-tested main commits.

🔧 Fix: Harden Agent OS ownership, provenance, and publication
4 issues (2 errors, 2 warnings) still open:

  • 🚨 .github/workflows/agent-os-image.yml:101 - Successful CI runs can finish out of order, and this gate does not verify that head_sha is still the current protected-main commit. An older run can therefore publish after a newer one and overwrite latest; reject stale workflow runs immediately before publication or otherwise enforce monotonic releases. GitHub documents that concurrency ordering is not guaranteed.
  • 🚨 bin/agent-os-kubernetes.sh:64 - The ownership checks read each object's UID but discard it before later apply or name-only deletion; the crewmate helper repeats the same pattern. A resource created or replaced between verification and mutation can consequently be adopted or deleted, including a foreign PVC. Use atomic create/resource-version updates and UID-preconditioned deletes, skipping deletion when the verified object was absent. Kubernetes provides UID and resourceVersion deletion preconditions.
  • ⚠️ bin/agent-os-kubernetes.sh:163 - rollback mutates the named StatefulSet without checking its installation UUID, so a wrong namespace can roll back an unrelated workload despite the new ownership boundary. Verify the namespace and StatefulSet ownership before rollout undo.
  • ⚠️ tools/agent-os/packages/firstmate/inputs.example.yaml:2 - The operational helper defaults to this example file, which contains a fixed, publicly reused installation UUID. An install that omits AGENT_OS_INPUTS can treat another default-config namespace as its own; require an explicit input file, generate and persist a fresh UUID, or reject this example UUID for non-local installs.

🔧 Fix: Harden Kubernetes ownership and publication freshness
3 issues (1 error, 2 warnings) still open:

  • 🚨 .github/workflows/agent-os-image.yml:176 - The freshness check runs before a potentially long multi-architecture build, so main can advance before the action pushes latest; no replacement workflow exists to cancel this run until the newer commit's CI finishes. Build without publishing, refresh main again immediately before the registry mutation, then publish the verified digest/tag. GitHub does not guarantee concurrency ordering.
  • ⚠️ bin/agent-os-kubernetes.sh:202 - rollback unconditionally requires Agent OS ownership labels on the namespace, but supported createNamespace: false installations deliberately neither create nor label that shared namespace. Rollback therefore always fails for this package mode; verify namespace ownership conditionally, or persist and compare its UID without adopting it.
  • ⚠️ bin/agent-os-kubernetes-cas.sh:87 - Raw conditional deletes return as soon as the API accepts the DELETE and bypass kubectl delete's waiter. Purge, uninstall, and namespace deletion can report completion while Pods or PVC finalizers remain, causing immediate recreation or reinstall to fail; wait until the captured UID disappears, treating a replacement UID as completion. The kubectl raw-delete path returns directly after the request.

🔧 Fix: Close publication, rollback, and delete races
✅ Re-checked - no issues remain.

✅ **Test** - passed

✅ No issues found.

  • command -v tmux >/dev/null || { echo "tmux is required for e2e tests" >&2; exit 1; }; tmux -V; rc=0; for t in tests/*.test.sh; do echo "== $t =="; bash "$t" || rc=1; done; exit "$rc"
✅ **Document** - passed

✅ No issues found.

🔧 **Lint** - 1 issue found → auto-fixed ✅
  • ⚠️ linter found issues (exit code 1)

🔧 Fix: Captain: suppress literal-only ShellCheck warnings
✅ Re-checked - no issues remain.

✅ **Push** - passed

✅ No issues found.

robinbraemer and others added 30 commits July 12, 2026 02:13
…uid#505)

* fix: guard secondmate own-home turn ends

Remove the .fm-secondmate-home early-exit in fm-turnend-guard.sh so the
'no turn ends blind' backstop fires in a secondmate's own primary session,
matching the cd-guard's scope: the own home is guarded, child crew/scout
worktrees stay exempt via the retained git-dir/git-common-dir test. This
was pure scoping from the guard's primary-only origin and guarded against
no secondmate-specific hazard.

Add secondmate regression tests (blind-turn block, idle-by-default,
stop_hook_active loop guard, deferred-death recovery loop, child-worktree
exemption) and record the autonomous background-notify re-invoke
measurement (Claude Code 2.1.207, 11s) in docs/turnend-guard.md.

* no-mistakes(document): Correct secondmate guard documentation, captain

* fix: force-include marked secondmate homes in turn-end guard

The prior remove-only form (just deleting the .fm-secondmate-home check)
left the DEFAULT secondmate topology unguarded: a treehouse-leased home is
a linked git worktree (git-dir != git-common-dir), which the retained
git-dir exemption still skipped, so its own primary session could still end
a turn blind. Invert the marker: a genuinely-marked home is force-included
as a guarded primary (treehouse-leased linked OR git-cloned plain), and the
git-dir exemption applies only to UNMARKED child worktrees. Marker
validation (regular non-symlink file, non-empty id-token content) blocks a
stray or empty marker from spoofing inclusion.

Add real linked-worktree regression tests: a treehouse-leased LINKED
secondmate home is guarded, a stray/empty marker stays exempt, and the
unmarked child worktree stays exempt - the topology the plain git-init
fixtures masked. Predicates, in-flight gate, and loop guard untouched.

* fix: force ASCII collation in secondmate marker validation

Add a function-scoped local LC_ALL=C in fm_root_is_secondmate_home so the
[A-Za-z0-9._-] id allowlist matches under C collation, not the ambient
locale - a locale-crafted non-ASCII marker id can no longer slip through
the range match and spoof force-inclusion of a linked child worktree.
Add a regression test proving a non-ASCII marker id is rejected and the
linked worktree stays exempt.

* no-mistakes(test): fix backend baseline gate-refusal dependency

* no-mistakes(document): Correct secondmate turn-end guard documentation
* fix: make bootstrap required-tool detection backend-aware

Bootstrap demanded tmux and treehouse for every backend except orca, so a
herdr/zellij/cmux home with tmux absent was wrongly told MISSING: tmux.

Required tools now follow the resolved backend via the single-owner
fm_backend_required_tools helper (bin/fm-backend.sh): each backend's own
session-provider CLI, jq for the JSON-emitting adapters (herdr/zellij/cmux),
and treehouse for session-provider-only backends (orca owns its worktree).
The treehouse lease-support check is gated to backends that use treehouse.

Adds install hints for herdr/zellij/cmux, regression tests for the full
backend dependency matrix (herdr-without-tmux repro plus each boundary),
and updates the authoritative Toolchain docs.

* no-mistakes(review): Captain, prevent executing Herdr install guidance

* no-mistakes(review): Captain, harden backend-aware bootstrap diagnostics

* no-mistakes(review): Captain, separate manual dependency remediation

* no-mistakes(review): Captain, align bootstrap diagnostic consumers

* no-mistakes(document): Align backend adapter dependency comments
robinbraemer and others added 29 commits July 13, 2026 16:34
…id#533)

* fix: preserve secondmate routing markers

* no-mistakes(review): Captain, preserve trailing newlines in marked secondmate sends

* no-mistakes(test): Captain, tolerate bootstrap timeout elapsed drift

* no-mistakes(document): Refresh Herdr marker documentation
* fix: align grok effort docs and spawn with 0.2.99 ceiling

grok 0.2.99 accepts only low|medium|high for --reasoning-effort and
rejects both xhigh and max. Omit unsupported values on spawn, flag them
in crew-dispatch validation, and update harness-adapters.

* no-mistakes(test): Captain: refresh gotmp teardown fixture dependencies

* no-mistakes(document): Clarify Grok effort documentation ownership
@robinbraemer
robinbraemer force-pushed the fm/agent-os-firstmate-source-closure-r8 branch from 62300de to 590418f Compare July 15, 2026 06:37
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants