UX: wizard-first launch + cross-platform docs + zero-config (adapted from club-3090)#69
Conversation
…boarding GROUP-DOCS wave (additive/corrective; no doc deleted, hero + every link kept). New situation docs (club-3090 per-hardware front-door pattern): - docs/RUN_ON_LINUX.md full-stack front door: warning-first (one heavy model at a time; never interrupt PROD; sndr down not docker stop), workload->preset->ctx->TPS->VRAM table over registry_v2, zero-decision sndr quickstart, expert paths preserved. - docs/RUN_ON_MAC.md honest: the engine needs Linux+CUDA; a Mac drives a rig in client mode (CLI+GUI+memory) via the three-command remote path. - docs/RUN_ON_WINDOWS_WSL.md two lanes: WSL2+NVIDIA passthrough (follow RUN_ON_LINUX; install.sh auto-writes the gptq_marlin_repack override) or client mode; never a native-Windows engine. - docs/REMOTE_ENGINE.md canonical client-mode reference: the SNDR_OPENAI_BASE_URL/SNDR_ENGINE_API_KEY/GENESIS_MEMORY_DSN triplet, where each value comes from, CLI/GUI consumption, memory-DSN persistence, :8000 vs :8102 port story, the missing-Bearer 401 cause, shell-env-wins. Edits (additive/corrective): - README.md: honest "engine needs Linux+CUDA" line under Get running; a task-navigation "I want to... (by machine)" table; reconcile the flagship note so the lone-user recommendation (balanced) matches the auto-pick and multiconc is framed as throughput-only. - docs/GETTING_STARTED.md: "No Linux GPU? Drive a rig instead" subsection (closes the Mac/Windows dead-end) + zero-decision quickstart arc + 4 new rows in Where-to-go. - docs/QUICKSTART.md: reveal the keyed-engine callout on the first curl, clarify balanced-vs-multiconc, strengthen the sndr-down stop-footgun box, add the remote-client three-command alternative. - docs/README.md + mkdocs.yml: register the 4 situation docs. Gate: tests/unit/test_situation_docs_present.py asserts the 4 docs exist, state the Linux+CUDA truth, carry the remote triplet, and are linked from README + docs index. check_doc_sync / audit_links / audit_public_docs / audit_i18n / test_doc_preset_keys all green.
…sistence GROUP-CLI (additive only — `sndr launch <preset>` and every existing seam untouched): - sndr quickstart: zero-decision front door. Detects OS + rig, resolves a fitting preset via the ladder (explicit > pinned default > top-fit), prints the per-card VRAM projection with a hard FAIL gate before any container start, then delegates the bring-up to the existing `sndr up` flow. Remote pivot: no local GPU + SNDR_OPENAI_BASE_URL -> daemon-only client mode; no GPU + no remote -> actionable `sndr remote setup` hint (never a bare "no preset fits"). Every interactive step has a non-interactive escape. - sndr remote setup <url>: client onboarding for Mac/Windows. Validates the http(s)://host:port/v1 form (typed refusal, exit 64), best-effort probes the rig, persists the last remote, and prints the three canonical exports with a "shell env wins" note. - user_prefs: per-user $SNDR_HOME/defaults.toml store (stdlib only). Pinned default preset (registry-validated on set) + last-used remote cache. Shell env SNDR_DEFAULT_PRESET wins over the file; the resolver surfaces the source. - up.py / run.py: consult the pinned default before top-fit; up gains the empty-rig+remote no-engine pivot and a post-boot opt-in "make default?" offer. - chat.py: honors SNDR_OPENAI_BASE_URL + SNDR_ENGINE_API_KEY when no explicit host/port/preset is given (+ --api-key); explicit flags still win. TDD red-first; 25 new tests green, full CLI suite green (964 passed), 0-new-ruff on all touched files. Verified against live rig PROD (:8102, read-only probe) and offline via --fake-gpus.
…t install profile
GROUP-CONFIG (ux-simplification). Additive zero-config layer; every expert
path (sndr launch <preset>, explicit --pin, host.yaml edits) keeps working.
- .env.example (NEW, repo root): copy-and-edit-only-what-you-want scaffold.
Section A = local full stack (all auto-defaulted), Section B = remote-client
triplet (the only manual surface). All lines commented so a zero-edit copy
still boots on code defaults. Precedence/case/CRLF warnings inline. Tracked
via a .gitignore negation (!.env.example) and a security_scan template
exemption (real .env/.env.local stay forbidden).
- host_autoinit.ensure_host_yaml() (NEW) + a one-line seam in host.py's
default-path load miss: first-run host.yaml is auto-written from
detect_paths() so the launch renderer never emits a literal ${models_dir}
(BREAK #2). Idempotent, non-TTY safe, no-ops on a bare host; env overrides
still win at write time (DOWNLOAD-3 ladder); opt out with SNDR_HOST_AUTOINIT=0.
- install.sh: CLIENT PROFILE branch (--client/--no-engine, auto-selected when
no CUDA GPU) installs CLI+GUI only and emits a prefilled Section-B .env,
pointing at docs/RUN_ON_MAC.md. Linux+GPU path auto-copies Section-A .env +
runs host auto-init after a green install. New --dry-run exercises the seams
with no live clone/pip/launch. BREAK #4 repo-reachability preflight (warn).
Fixes a latent `set -u` unbound HF_HOME crash in pick_models_dir.
- scripts/start_sndr_client.sh (NEW): client launcher mirroring the daemon
launcher's env discipline; loads ./.env literally (not sourced) and runs
`sndr up --no-engine`.
TDD: test_env_example, test_host_autoinit (unit) + test_client_profile
(bash-harness running the real install.sh in --dry-run). Server-tested both
dry-run paths on the Mac. 0-new-ruff on all touched .py; make_evidence green
except a pre-existing LAN-IP hit in sndr/cli/commands/remote.py (CLI group).
… examples audit-public-paths gate forbids RFC-1918 addresses in public code; the sndr remote setup docstring, argparse help, and hint examples carried a literal 192.168.1.10. Swap to the <your-host> placeholder. Display-only strings; tests pass the URL programmatically and are gate-excluded.
There was a problem hiding this comment.
Code Review
This pull request introduces a 'client mode' that enables users on Mac or Windows to drive a remote Linux rig as their engine, effectively decoupling the CLI/GUI from the local hardware. Key additions include the sndr quickstart command for automated setup, sndr remote setup for configuring remote connections, and updated documentation for cross-platform deployment. The review feedback highlights necessary improvements for Python 3.10 compatibility, URL validation logic, VRAM projection accuracy, and installer script robustness.
Important
The consumer version of Gemini Code Assist on GitHub is being sunset. Starting June 18, 2026, new organization installations will be blocked, and all code review activity will officially cease on July 17, 2026.
For more details on the timeline and next steps, please review the Help Documentation.
| from pathlib import Path | ||
| from typing import Any | ||
|
|
||
| import tomllib |
There was a problem hiding this comment.
Since Python 3.10 is explicitly supported by the installer, importing tomllib directly will raise a ModuleNotFoundError on Python 3.10 because tomllib was introduced in Python 3.11. To maintain compatibility, use a fallback import to tomli (or set it to None to handle it gracefully with a fallback parser).
| import tomllib | |
| try: | |
| import tomllib | |
| except ImportError: | |
| try: | |
| import tomli as tomllib | |
| except ImportError: | |
| tomllib = None |
| def _read() -> dict[str, Any]: | ||
| path = _prefs_path() | ||
| if not path.is_file(): | ||
| return {} | ||
| try: | ||
| with path.open("rb") as handle: | ||
| return tomllib.load(handle) | ||
| except (OSError, tomllib.TOMLDecodeError): | ||
| # A corrupt/unreadable prefs file must never crash the CLI — treat it as | ||
| # empty (the caller falls through to auto-detected defaults). | ||
| return {} |
There was a problem hiding this comment.
To support Python 3.10 without requiring external dependencies, provide a simple fallback parser when tomllib is unavailable. This also avoids an AttributeError when referencing tomllib.TOMLDecodeError if tomllib is None.
def _read() -> dict[str, Any]:
path = _prefs_path()
if not path.is_file():
return {}
if tomllib is not None:
try:
with path.open("rb") as handle:
return tomllib.load(handle)
except (OSError, Exception):
return {}
# Fallback parser for Python 3.10 when tomli/tomllib are unavailable
try:
data: dict[str, Any] = {}
current: dict[str, Any] = {}
for line in path.read_text(encoding="utf-8").splitlines():
line = line.strip()
if not line or line.startswith("#"):
continue
if line.startswith("[") and line.endswith("]"):
sec = line[1:-1].strip()
current = {}
data[sec] = current
elif "=" in line:
k, v = line.split("=", 1)
current[k.strip()] = v.strip().strip('"').strip("'")
return data
except OSError:
return {}| if parsed.path.rstrip("/") and not parsed.path.rstrip("/").endswith("/v1"): | ||
| return None |
There was a problem hiding this comment.
The check if parsed.path.rstrip("/") evaluates to False when the path is empty or just / (e.g., http://localhost:8102 or http://localhost:8102/). This allows URLs without the required /v1 suffix to pass validation, which will fail later during API requests. Enforce that the path must end with /v1 by removing the falsy check.
| if parsed.path.rstrip("/") and not parsed.path.rstrip("/").endswith("/v1"): | |
| return None | |
| if not parsed.path.rstrip("/").endswith("/v1"): |
| try: | ||
| return kp.project( | ||
| cfg, | ||
| kp.ProjectorRig(vram_gib_per_card=vram_gib, gpu_count=1, name=rig.source), |
There was a problem hiding this comment.
The gpu_count is hardcoded to 1 in ProjectorRig. For multi-GPU rigs, this will calculate the VRAM projection incorrectly as if only a single GPU is present. Use the detected rig.gpu_count instead.
| kp.ProjectorRig(vram_gib_per_card=vram_gib, gpu_count=1, name=rig.source), | |
| kp.ProjectorRig(vram_gib_per_card=vram_gib, gpu_count=getattr(rig, "gpu_count", 1), name=rig.source), |
| # sensible auto-defaults so the common case needs zero manual config. | ||
|
|
||
| # Directory this script lives in (best-effort; curl|bash has no source file). | ||
| SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]:-$0}")" 2>/dev/null && pwd 2>/dev/null || echo "$PWD")" |
There was a problem hiding this comment.
When the installer is run via a piped stream (e.g., curl -sSL ... | bash), ${BASH_SOURCE[0]} is empty/unset, and $0 defaults to the shell executable path (like /bin/bash or /usr/bin/bash). This causes dirname "$0" to resolve to /bin or /usr/bin, which adds them to PYTHONPATH and searches them for .env.example. Avoid falling back to $0 when ${BASH_SOURCE[0]} is not set.
| SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]:-$0}")" 2>/dev/null && pwd 2>/dev/null || echo "$PWD")" | |
| SCRIPT_DIR="$PWD" | |
| if [ -n "${BASH_SOURCE[0]:-}" ]; then | |
| SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" 2>/dev/null && pwd 2>/dev/null || echo "$PWD")" | |
| fi |
| key="${line%%=*}"; val="${line#*=}" | ||
| key="${key#"${key%%[![:space:]]*}"}" # ltrim key | ||
| key="${key%"${key##*[![:space:]]}"}" # rtrim key | ||
| case "$key" in [A-Za-z_]*) : ;; *) continue ;; esac |
There was a problem hiding this comment.
The pattern [A-Za-z_]* only validates the first character of the key. If the key contains invalid characters (like hyphens or spaces) later in the string, it will pass this check but cause a syntax or runtime error during indirect expansion ${!key} or export. Ensure the entire key contains only valid alphanumeric characters and underscores.
| case "$key" in [A-Za-z_]*) : ;; *) continue ;; esac | |
| case "$key" in [A-Za-z_]*) : ;; *) continue ;; esac | |
| case "$key" in *[!A-Za-z0-9_]*) continue ;; esac |
| try: | ||
| user_prefs.set_default_preset(preset_id) | ||
| em.ok(f"default set — future `sndr quickstart` will boot {preset_id}") | ||
| except ValueError: |
| try: | ||
| user_prefs.set_default_preset(preset_id) | ||
| em.ok(f"default set — future `sndr up` will boot {preset_id}") | ||
| except ValueError: |
| # and no-ops on a bare host, so nothing is written unless a real path | ||
| # is detectable. Opt out with SNDR_HOST_AUTOINIT=0. | ||
| if path is None and os.environ.get("SNDR_HOST_AUTOINIT", "1") != "0": | ||
| from sndr.model_configs.host_autoinit import ensure_host_yaml |
|
|
||
| from typing import TYPE_CHECKING | ||
|
|
||
| from sndr.model_configs import host as _host |
|
|
||
| pytest.importorskip("pydantic") | ||
|
|
||
| import sndr.cli.commands.quickstart as qs # noqa: E402 |
|
|
||
| pytest.importorskip("pydantic") | ||
|
|
||
| import sndr.cli.commands.remote as remote_mod # noqa: E402 |
|
|
||
| pytest.importorskip("pydantic") | ||
|
|
||
| import sndr.cli.commands.up as up_mod # noqa: E402 |
| _clear_host_env(monkeypatch) | ||
| host_yaml = tmp_path / ".sndr" / "host.yaml" | ||
|
|
||
| import sndr.model_configs.host_autoinit as mod |
tomllib is stdlib only on 3.11+, but the project supports >=3.10 and CI runs the test matrix on 3.10 — the unconditional `import tomllib` broke collection of every tests/unit/cli module (ModuleNotFoundError). Prefer tomllib, fall back to the tomli backport if installed, else a built-in reader for the exact flat [section] key = "value" subset _dumps emits, so reads round-trip writes with zero new dependency. Adds a regression test that forces the no-tomllib path.
…d sibling over -multiconc Fast-follow to the UX wizard (workflow-flagged): the fit ranking sorts by measured metric desc, floating the -multiconc throughput variant (max_num_seqs=8, ~100% per-card VRAM) above balanced. A lone newcomer got the peak-VRAM/OOM-risk preset while QUICKSTART.md documents prod-qwen3.6-35b-balanced as the auto-pick. _lone_user_first is surgical: when the top auto-pick is -multiconc it promotes that SAME model's single-stream sibling if it also fits — model choice + cross-model order untouched, nothing dropped (multiconc stays if it's the only fit). TDD: 4 tests (sibling swap, cross-model preserved, no-op when top is safe); existing test_dry_run restored; 969 cli tests green.
What & why
Newcomer onboarding was expert-only: to reach a working
curlyou had to know a preset key, a pin, a models dir, and that the engine only runs on Linux+CUDA. This branch adds a zero-decision front door on top of the existing expert surface — adapting club-3090's per-hardware front-door pattern to our stack. Nothing was removed:sndr launch <preset>, explicit--pin, all existing verbs and flags keep working unchanged.What got simpler (before -> after)
sndr quickstartauto-detects GPUs, projects per-card VRAM, picks a fitting preset (explicit > pinned-default > top-fit)sndr launch <preset>+ flagssndr quickstartdelegates tosndr up; can save the pick as defaultsndr remote setup <url>one-liner -> client mode against a remote rig; per-OS docs (RUN_ON_MAC / RUN_ON_WINDOWS_WSL).env.examplescaffold;cp .env.example .envboots on built-in defaults; host.yaml auto-init on first runinstall.shauto-selects client vs full-stack by CUDA presenceFiles
Added (nothing removed):
sndr/cli/user_prefs.py,sndr/cli/commands/quickstart.py,sndr/cli/commands/remote.py.env.example,sndr/model_configs/host_autoinit.py,scripts/start_sndr_client.shdocs/RUN_ON_LINUX.md,docs/RUN_ON_MAC.md,docs/RUN_ON_WINDOWS_WSL.md,docs/REMOTE_ENGINE.mdEdited (additive/corrective only):
sndr/cli/commands/{__init__,up,run,chat}.py,sndr/model_configs/host.py,install.sh,scripts/security_scan.py,.gitignore,README.md,docs/{GETTING_STARTED,QUICKSTART,README}.md,mkdocs.yml,tests/unit/scripts/test_security_scan.py.Verification
make_evidence65/65 green;check_doc_sync --strictclean (329 patches);test_doc_preset_keys2/2.audit-public-pathsclean (fixed a literal LAN IP inremote.pyexamples -><your-host>)..py(ruff 0.15.11).vllm-35b-dev748untouched:sndr quickstart --dry-run --no-inputauto-detected OS + 2x A5000 via nvidia-smi and reached the boot decision with no manual config;sndr remote setupprobed live PROD :8102 from the Mac (reachable);install.sh --dry-run(full-stack + client) scaffolded.envand host.yaml auto-init.Known follow-up (not blocking)
prod-qwen3.6-35b-multiconc(throughput-tuned, peak ~100% per-card VRAM) while README documentsprod-qwen3.6-35b-balancedas the lone-user default. Reconcile the ranking bias or the doc claim in a fast-follow.