Box config: make /etc/lager writable by the box user, and fail loudly when a render fails

[danielrmerskine]

Problem

lager box config apply reported success while applying nothing. Adding a pip package and applying it printed Applied box config on <ip>, exited 0, stamped the applied-hash — and the package was not installed. Re-running then printed Config unchanged since last apply; skipping restart, so the config could never be applied at all.

This affected every box. It was invisible because the two provisioning paths disagreed about who owns /etc/lager:

last provisioning step	/etc/lager	box config renders
lager install	33:33, mode 755 (owner-only)	fail silently — nothing is applied
lager update	chmod 777 (world-writable)	work, but the directory is writable by every local account

A box whose last step was an update looked healthy. A freshly installed box silently applied nothing, and any box that had rendered files from an earlier era kept running that stale output — so the config on disk and the config in the container could drift indefinitely with no error anywhere.

Root cause

apply runs start_box.sh on the box as the login user. Its four box-config renderers create files in /etc/lager (box_config.docker.sh, user_requirements.txt, cargo_packages.txt, npm_packages.txt). Creating a file requires write permission on the directory, so an owner-only /etc/lager fails every render with EACCES.

Renders are soft-failed by design — the container must always come up — which is what turned a hard permission error into a silent no-op:

render fails (EACCES)  ->  rendered file never written
                       ->  `[ -s "$PIP_REQS_FILE" ]` is false
                       ->  the in-container install step is skipped
                       ->  start_box.sh exits 0
                       ->  apply reports success and stamps the applied-hash

Stamping the applied-hash is what sealed it shut: the retry after any fix short-circuits on "Config unchanged since last apply".

Fix

Ownership. /etc/lager is now 33:<box-user-group>, mode 2775 (setgid), from both lager install and lager update. Owner stays www-data (uid 33) so the container is unaffected; the group is the box's login user, so start_box.sh can write. This also removes the chmod 777: world-write was never needed, and that directory holds box_config.json, saved_nets.json and the org secrets. lager update repairs existing boxes in place — no reinstall.

The narrow two-argument chown/chmod forms are deliberate: the sudoers spec is chown * /etc/lager, which matches exactly one argument before the path, so chown -R ... would not match it. Only the directory's own group and mode matter — the renderers and store_build_hash replace files via tmp+rename, which needs permission on the directory, not on the file being replaced.

Fail loudly. A render failure used to print a one-line warning and a raw Python traceback. The four duplicated _write() helpers are hoisted into box_config/config.py as write_atomic(), which reports which file could not be written, why, and how to repair it. start_box.sh then exits 3 — a new code meaning the container is up, but the config on disk was not applied to it.

apply treats rc 3 as: print the box-side error, do not stamp the applied-hash, do not roll back. Rolling back is wrong here because the container is healthy and the fault is environmental — restoring the previous config and re-bouncing would fail identically. A genuine bounce failure (container possibly down) still rolls back exactly as before. The in-container pip/cargo/npm install failures now exit 3 for the same reason: they run after docker run and already reported "container is up but ... may be incomplete", while exiting 1 and triggering a rollback of a healthy container.

Unrelated but adjacent: lager_excepthook read LAGER_HOST_MODULE_FOLDER with os.environ[...]. That variable is set by lager python; anything else that execs a script into the container leaves it unset, so the exception handler raised KeyError. Python then printed Error in sys.excepthook: plus a traceback from inside lager, and only then the user's real exception — so a one-line ModuleNotFoundError arrived buried under a stack trace pointing at lager internals. With no host paths to rewrite, the traceback is now printed unchanged.

Validation

Unit: 1638 pass. (Three pre-existing failures on this checkout — test_box_authorize, test_lock_state, test_monitor_state — fail identically on main and touch none of these files.)

Hardware, full A/B on PRD-1:

	before	after, perms still broken	after, perms repaired
start_box.sh	exit 0, silent	exit 3, names each file + the repair	exit 0
apply	"Applied box config", exit 0	exit 1, "NOT applied"	"Applied box config"
applied-hash	stamped (seals the bug)	not stamped	stamped
rollback	—	none (container is healthy)	—
requested package	not installed	not installed, correctly reported	installed

Also verified after the change that lager update leaves /etc/lager as drwxrwsr-x www-data:<box user> (not 777), and that renders, the in-container installs, and store_build_hash all still work.

Risk

The permission change is the only thing that touches existing boxes, and it runs through the sudoers grants that already exist. A box that has not yet been updated keeps working exactly as it does today, except that a failed apply is now loud instead of silent.

[gemini-code-assist]

Code Review

This pull request addresses permission issues with /etc/lager by changing its ownership and mode to allow both the container and host users to write to it safely, preventing silent configuration render failures. It also improves error handling during configuration rendering (exiting with code 3 instead of rolling back healthy containers) and fixes a bug in the exception hook that buried actual errors under a KeyError. However, there are unresolved merge conflicts in CHANGELOG.md and cli/__init__.py that must be resolved before merging.

Important

The consumer version of Gemini Code Assist on GitHub is being sunset. Starting June 18, 2026, new organization installations will be blocked, and all code review activity will officially cease on July 17, 2026.
For more details on the timeline and next steps, please review the Help Documentation.

[gemini-code-assist]

Merge conflict markers found in CHANGELOG.md. Please resolve the conflict and remove the markers.

[gemini-code-assist]

Merge conflict markers found in CHANGELOG.md. Please resolve the conflict and remove the markers.

[gemini-code-assist]

Merge conflict markers found in cli/init.py. Please resolve the conflict and remove the markers.