docs(ops): native-inference production rollout plan (#754 unblock)#765
Open
Koh0920 wants to merge 8 commits into
Open
docs(ops): native-inference production rollout plan (#754 unblock)#765Koh0920 wants to merge 8 commits into
Koh0920 wants to merge 8 commits into
Conversation
The full Release-PR CI runs clippy with -D warnings, which surfaced pre-existing lint debt on the 0.7 line (never gated on the nightly feature PRs): - capsule: 2 doc-comment doc_lazy_continuation (line-leading `+` read as a markdown list) — reworded to `and`. - cli/gpu_provision: collapsible_if + match_result_ok (auto-fixed); Configure ProvisionAction variant never constructed (allow dead_code, kept for symmetry). - cli/root: Commands is a wide clap subcommand enum — allow large_enum_variant (boxing fights the derive for no user benefit). - cli/net_client: Client/SyncClient expose the full netd control surface; methods with no caller yet — allow dead_code on the impls. No behavior change. cargo clippy --workspace --all-targets --exclude desktop -- -D warnings now passes.
cargo-dist's plan step rejects v0.7.0 because crates/cli/wix/main.wxs still carried the pre-rename Name='ato-cli' (Product/DiskPrompt/APPLICATIONFOLDER). Regenerated via `dist generate` so `dist generate --check` passes. Windows installer only.
…rename) The desktop bundle build (ato-desktop-xtask) probed the canonical CLI crate at crates/ato-cli, which no longer exists after the ato-cli->cli rename, so all three desktop bundles failed (`cargo build failed for crates/ato-cli/Cargo.toml`). Point the monorepo-canonical path at crates/cli. Same incomplete-rename class as the wix fix.
…etd rename) Second rename-leftover in the desktop xtask: netd_manifest pointed at crates/ato-netd/Cargo.toml (gone after ato-netd->netd; the crate dir is crates/netd, the bin is still ato-netd). This broke the Linux + macOS desktop bundles after the cli path fix (Windows happened to pass). Point netd_manifest at crates/netd.
…758) (#760) v0.7.0 shipped 5 build blockers that only appeared AFTER `git tag`, all crate-rename leftovers (`ato-cli->cli`, `ato-netd->netd`) in release tooling that the normal feature-PR gate never exercised: the wix installer (`Name='ato-cli'`) failed cargo-dist's plan, and the desktop xtask built the gone `crates/ato-cli` / `crates/ato-netd` manifests. rust-ci runs clippy/fmt/test but excludes the desktop/xtask packages and never runs cargo-dist, so these surfaced only when the real release.yml / desktop-release.yml ran on the tag. New `release-dry-run.yml` (PRs to main/dev + dispatch), no publish: - `dist generate --check` — fails on stale generated installer/CI files (the exact wix blocker that forced a v0.7.0 re-tag). - `dist plan` — same manifest/version resolution the release plan job runs. - `cargo test` on the pure-Rust `ato-desktop-xtask` (no GUI deps) — runs `discovered_crate_manifests_all_exist`, an existence check over the cli/netd/desktop/nacelle build manifests that catches any rename leftover. Also fixes the existing `workspace_paths_include_netd_manifest` test, which still asserted the pre-rename `crates/ato-netd` path (latent-red because the xtask was never run in CI), and adds the existence test. Verified locally: dist generate --check exit 0, dist plan resolves, both xtask tests pass.
* fix(runner): dispatch native-inference leases without sandbox (#762) The Connected Runner agent forced `--sandbox` on every dispatched run, so native-inference (host-execution) capsules could not run on the fleet. With the control plane now carrying `runtime` on the lease (ato-api#149), the runner selects dispatch mode from it: - RunnerDispatchMode {Sandboxed, NativeInferenceHost}, chosen ONLY by the exact `runtime = "native-inference"` lease field — no generic host-exec. - run_child_args builds the `ato run` argv by mode BEFORE spawn (never patched after construction): native-inference omits `--sandbox`; everything else keeps it. Source/sandbox dispatch is unchanged. - Advertises a `native-inference` capability in supported_lease_kinds only when the host passes the native-inference doctor (cached probe), so the control plane only dispatches native-inference leases here. - Fail-closed guard: a native-inference lease on a host that cannot run it is rejected with a typed reason (native_inference_unavailable) before any spawn — defence in depth behind the control-plane capability gate. Adds native_inference_doctor::is_ready(). Tests (cli runner_agent, 12 passed): source run still --sandbox; native-inference omits --sandbox; managed-state-root preserved in both modes; only exact native-inference selects host (host-exec/oci/source-native stay sandboxed); fail-closed without capability; resolve_lease_execution sets mode from runtime. cargo check + clippy -D warnings clean. * fix(runner): address review — guard before resolve + testable capability advertise Review follow-ups on the #762 dispatch fix: - Move the native-inference fail-closed guard to the TOP of handle_claimed_lease, BEFORE resolve_lease_execution materializes an inline recipe to disk and before the Preparing report — so a mis-dispatched native-inference lease is rejected with zero side effects (was: checked only inside the spawned task, after those side effects). Release the slot on this reject path AND the adjacent resolve-error reject path (SlotLease has no Drop, so an early return otherwise strands the slot occupied; release() is idempotent). - Split advertised_lease_kinds into a pure advertised_lease_kinds_for(ready: bool) so the conditional native-inference append is unit-tested (ready → advertised, not-ready → not) — closes the capability-advertisement test gap. runner_agent: 21 tests pass; cargo check + clippy -D warnings clean.
…#764) * chore(smoke): cross-runner native-inference smoke harness (#754 gate) A repeatable, manual on-demand smoke verifying a native-inference capsule runs on a Connected Runner via the #149/#763 contract — the final gate before #754 (Store registration). Makes no production changes (dispatches one run, then stops it). - scripts/smoke-connected-native-inference.sh — two evidence tiers: API (dispatch → ready → /health → /v1/chat/completions → stop; dispatch success is itself the #149 capability gate — a non-advertising runner returns 409 runner_capability_required) + runner-SSH (the DIRECT proof: `ato ps` runtime=host, /proc/<pid>/cmdline shows `ato run <ref> -y` with NO --sandbox, engine log shows llama.cpp/Vulkan). Evidence is redacted (session token matched literally via perl \Q\E so +/./- formats are handled; ato_rnr_/cookie/authorization/?token= scrubbed; every sink incl. summary.txt goes through redact()). - docs/ops/connected-runner-native-inference-smoke.md — prerequisites (the 3 #754 gates), run command, observability caveats (lease.command.runtime is NOT operator-API-observable → SSH for the direct proof; the native-inference capability is indirect via dispatch), expected GREEN output, re-run note. - .github/workflows/connected-native-inference-smoke.yml — workflow_dispatch ONLY (never push/PR); secret ATO_SESSION (+ optional ATO_RUNNER_SSH_KEY); uploads redacted evidence as an artifact. Cannot fully RUN until (all deferred per instruction): ato-api #149 in prod (staging-deployed only), a #763-bearing runner advertising native-inference, and local-llm-chat registered (#754). Adversarially reviewed (5-agent endpoint recon + 3-lens harness review); shellcheck clean; redaction verified against +/-/. tokens and tokenized URLs; YAML valid + manual-only. * docs(smoke): fix run path to repo-root scripts/ (match workflow)
Planning/checklist for prod rollout. Surfaces 3 go/no-go blockers found by investigation: (1) a #149 prod deploy bundles 11 unrelated commits (prod@593a0d9, main@e41ed12) — recommend isolating #149 via cherry-pick; (2) no released runner has #763 (not in v0.7.0) — build nightly on the OCI ARM box (CPU sufficient); (3) local-llm-chat registration is a PREREQUISITE for the smoke (chicken-and-egg) so #754's publish must precede the smoke, not follow it. Includes exact deploy/ rollback/build/publish commands, capability OnceLock notes, preflight checklist, and a staging-rehearsal-first recommendation. No execution.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Planning/checklist runbook for taking native-inference on Connected Runners to production and unblocking #754. No execution — prod deploy / fleet build / Store registration all await explicit approval.
Go/No-Go: NO-GO as originally sequenced — 3 decisions first
Investigation (4-agent recon) surfaced 3 live blockers (all anticipated stop-conditions):
593a0d9;mainise41ed12(core: full ExecutionReceiptV2 emission acceptance for #99 (test matrix + CLI surface + smoke) #149).wrangler deploy --env productionships all 12 commits, promoting 11 unrelated ones (Managed-Cloud billing/entitlement feat(ato-cli): migrate validate legacy-lock branch to ExecutionGraphBuilder #140–drive_sync_async: replace panic-on-Pending with a structurally-safe reporter notify path #147, diagnostics desktop: error screen text is not user-selectable #38/cli: --sandbox opt-in for source/native|python should be discoverable from ato run --help #129, …) + migration 0075. → Recommend isolating core: full ExecutionReceiptV2 emission acceptance for #99 (test matrix + CLI surface + smoke) #149 via cherry-pick onto593a0d9.8aa18f1c) on the OCI ARM box (CPU-only, predicted native-inference-ready; CPU sufficient for the first smoke).local-llm-chatregistration is a PREREQUISITE for the smoke (HIGH) — the smoke can't trigger native-inference dispatch until the capsule is registered with aruntime="native-inference"recipe TOML (a github source ref bypasses detection + 404s). → Register local-llm-chat in Store/community recipes after native-inference release #754's publish must precede the smoke (chicken-and-egg), not follow GREEN.Contents
Exact commands (deploy / migration 0075 / rollback
wrangler rollback 8e57dda3…/ OCI nightly build /publish-submission.mjs/ smoke), required secrets (already set on the live Worker; #149 needs none new), chosen runner (OCI, CPU), capabilityOnceLockoperational notes (restart serve after provisioning — no live refresh), rollback per step, preflight checklist, and a staging-rehearsal-first recommendation.Final recommendation
NO-GO until Decisions 1–3; then GO via the corrected sequence — lowest-risk path: isolate #149 → register + smoke on staging → promote to prod.
🤖 Generated with Claude Code
https://claude.ai/code/session_017EB3GkNz44tPCB7WePLth5