Deep hardening: crash-safe sessions, safe imports, atomic updates, and visual regression#1
Draft
wyt1234 wants to merge 6 commits into
Draft
Deep hardening: crash-safe sessions, safe imports, atomic updates, and visual regression#1wyt1234 wants to merge 6 commits into
wyt1234 wants to merge 6 commits into
Conversation
Production (src/worker.rs):
- mark_failed_if_owner: reject only an EXPLICIT different owner. A None
worker_pid means the failure happened before the final bind persisted our
pid (e.g. a non-socket sits at the canonical socket path and run_worker
bails pre-bind); at that point we still hold both worker.lock and
socket.lock, so the error is ours to record and must not be swallowed into
a stuck `starting` row. This is the one real production defect behind the
red gate.
Cockpit (tests/cockpit.py) — no production behavior changed:
- keys #9: compare a stable row identity (the age cell ticks 0s->1s between
reads) and wait on the composer repaint instead of fixed sleeps (fixes the
nav flake and the uppercase-W race).
- redesign #13: clear the transient "auto-imported N chat(s)..." footer with a
harmless Left-arrow before asserting the Windows-style key hints.
- distill #15: production runs the distill session FROM (and pre-trusts) the
stable distill root; the corpus is a private per-run child. Assert
cwd == -C == distill_dir trust and corpus-under-runs, not corpus trust.
- never-ready: accept portable-pty 0.8.1's newline+VEOF ("0a04") writer-Drop
cleanup sentinel; still forbid framed/private prompt bytes.
- INJECT: drop the redundant post-DELIVERED detach frame. INJECT is a
one-frame protocol and the worker closes the headless slot the instant it
emits DELIVERED, so the extra frame raced that close and aborted the whole
run with BrokenPipe (which had been hiding the two checks below).
- STOP #58: poll for the guardian's async certification instead of reading the
state file the instant the tree dies (the certified write lags process death).
- distill-prepare #60: the run lock is non-blocking fail-closed (LOCK_EX|
LOCK_NB), so a second overlapping prepare is refused, not queued. Assert the
winners never share a version claim or corpus dir and each loser is exactly
that fail-closed refusal, rather than expecting two concurrent successes.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The updater asked GitHub for main's HEAD sha and (a) offered an update on ANY difference and (b) demanded the downloaded rolling-release binary's embedded marker equal main HEAD. But `releases/latest` is a rolling tag that LAGS main (CI builds after the push), so in practice: - a dev build ahead of the last publish nagged "update available" and, on click, tried to DOWNGRADE; - every install failed "release asset is stale or mismatched" because the release's marker never equalled the current main HEAD. Now resolve the rolling `latest` tag's own commit, ask GitHub's compare API for ahead/behind, and act only when the release is strictly `ahead`: no phantom note, no downgrade, and the marker is validated against the RELEASE's sha so a lagging-but-valid release installs cleanly. The decision is a pure function (classify_release) unit-tested across ahead/behind/diverged/identical/same-sha/ unknown/garbage-sha — including the exact "we are ahead of the last publish" case that used to fail. Verified against the live API: compare df71b6d...latest = behind -> NotNewer. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
build.rs only re-ran on changes to .git/refs/heads/main, so a commit on any feature branch (e.g. agent/deep-rail-hardening) never refreshed the embedded sha and `rail --version` — and the updater's own build_sha — reported an old commit. Watch .git/HEAD, .git/packed-refs, and the exact ref HEAD points at, so the sha stays correct on whatever branch is checked out. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
… output
A too-old codex 400s the configured model and exits at startup, but the session
only showed "codex exited with status N" — the real reason ("requires a newer
version of Codex") was invisible, so a wrong-version codex read as an
unexplained, repeatedly-triggered bug. finish_run now appends codex's last log
line (ANSI/control stripped) to last_error on a quick failure.
To make that reliable, drain the child's final PTY output into the log on
ChildExit before finishing: rx (control) is polled before pty_rx, so ChildExit
could arrive with the last chunk still queued — which both truncated the
transcript tail and hid the reason. Bounded drain (50ms-quiet, 300ms cap).
Tests: hint_from_log_bytes unit test (ANSI/OSC stripping, last readable line,
no-junk); cockpit integration test with a fake codex that prints a version error
and exits 1 -> asserts status=failed and the reason is surfaced ANSI-free.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What changed
/import Ndand exact-session expansion.Why
Rail had several cross-process failure modes that only appear under crashes, concurrent managers, stale workers, or large histories: orphaned Codex trees could retain SQLite locks; sessions could cross-wire rollout identity; prompt injection could race Codex readiness; distillation/update writes were not sufficiently isolated; and pointer movement previously had to be disabled because it mutated durable selection.
This change makes those boundaries explicit and testable instead of relying on timing or process heuristics.
User impact
/procprocess identity.Ctrl+N, and autopilot usesCtrl+A./import 15dor/import <session_id>.latestrelease publishes only the two supported Linux assets and cleans a stale macOS asset if present.Validation
cargo fmt --all -- --checkRUSTFLAGS=-Dwarnings cargo check --locked --all-targetscargo test --locked --all-targets -- --test-threads=1: 66/66cargo test --release --locked --all-targets -- --test-threads=1: 66/66The complete socket/guardian Cockpit suite cannot run inside the current local sandbox because AF_UNIX is denied; this PR's quality workflow runs that full audit on GitHub Actions before any release job can start.