Skip to content

Fix first-run blockers on Linux/macOS + clarify image upgrade procedure#1

Merged
Z-Blocks merged 5 commits into
lightchain-protocol:mainfrom
DimitarTAtanasov:fix/env-sh-silent-failure-on-first-run
May 21, 2026
Merged

Fix first-run blockers on Linux/macOS + clarify image upgrade procedure#1
Z-Blocks merged 5 commits into
lightchain-protocol:mainfrom
DimitarTAtanasov:fix/env-sh-silent-failure-on-first-run

Conversation

@DimitarTAtanasov

Copy link
Copy Markdown
Contributor

Summary

Four fixes that surfaced while onboarding a fresh testnet worker on macOS Apple Silicon. Three are first-run blockers (every clean Linux/macOS install hits them); the fourth is a docs improvement around the day-2 image-upgrade flow.

What's in this PR

Commit Change Affects
fix(bash): env.sh fails silently on first run... env.sh set -e propagation bug All Linux + macOS + WSL/Git Bash users
fix(bash): mark phase + ops scripts as executable Execute bit on 14 bash scripts All Linux + macOS + WSL/Git Bash users
Update ollama default host (by @KostadinBuglov) PowerShell OLLAMA_URL default Windows users
docs(operations): expand 'Upgrading the worker image'... Pre-check + NETWORK re-export + 01-resolve step All users following the upgrade docs

Per-commit detail

1. env.sh fails silently on first run

scripts/bash/env.sh ended with:

```bash
[[ -f "$_SCRIPT_DIR/resolved.env" ]] && source "$_SCRIPT_DIR/resolved.env"
[[ -f "$_SCRIPT_DIR/secrets.env" ]] && source "$_SCRIPT_DIR/secrets.env"
```

On a clean first-run install, neither file exists yet (Phase 00 creates `secrets.env`, Phase 01 creates `resolved.env`). The failed `[[ -f ... ]]` test returns exit 1, the `&&` chain inherits it, and because every numbered script uses `set -euo pipefail`, that exit 1 propagates back through `source common.sh` → `source env.sh` and silently kills the parent script before any output is produced.

So the script that's supposed to create `secrets.env` was gated on `secrets.env` existing. Pure chicken-and-egg.

Why it wasn't caught in development: maintainers' working trees always had `secrets.env` left over from earlier runs, so the test never failed.

Fix: replace the short-circuit with explicit `if [[ ]]; then ... fi` blocks — these are specifically exempt from `set -e` propagation, so a missing optional file no longer kills the parent.

Verified the bug reproduces identically on bash 3.2.57 (Apple stock) and bash 5.3.9 (Homebrew); both die with exit 1 on the minimal repro below. Native Windows users using the PowerShell scripts are unaffected.

Minimal repro (no project dependencies):

```bash

inner.sh

[[ -f /tmp/nonexistent ]] && source /tmp/nonexistent

outer.sh

set -euo pipefail; echo before; source inner.sh; echo after
```

Expected: prints `before` and `after`. Actual: prints `before` only, exit 1.

2. Scripts shipped without execute bit

`./scripts/bash/00-generate-key.sh` (and every other script the README invokes directly) returned `zsh: permission denied` on a fresh clone because the files were committed at mode `100644`. The README's documented usage assumes `./scripts/bash/...` invocation, but that requires the execute bit. Workaround would have been `chmod +x scripts/bash/*.sh` or `bash ./scripts/bash/...`, neither of which is documented.

Fix: marked the 14 scripts the README invokes directly as `100755`. Left the three files that are only sourced or copied at `100644`:

  • `common.sh` — sourced from every phase script
  • `env.sh` — sourced from `common.sh`
  • `secrets.example.sh` — copied (not executed) by `00-generate-key.sh`

3. PowerShell `OLLAMA_URL` default

Authored by @KostadinBuglov. Switches the default from the explicit IPv4 (`http://192.168.65.254:11434\`) to `http://host.docker.internal:11434\`. Including in this PR so all related first-run fixes ship together.

4. Upgrade procedure docs

The original three-line `stop` → `pull` → `run` snippet was correct in spirit but had an important omission: it assumed the operator was still in the shell session from their original onboarding. In practice operators come back days/weeks later in a fresh terminal where `NETWORK` is no longer exported, the scripts silently default to mainnet, and they end up pulling the wrong image and connecting to the wrong gateway.

Changes:

  • Added a pre-upgrade digest check so operators can confirm whether `:latest` has actually moved before stopping the container.
  • Added explicit `export NETWORK=…` / `$env:NETWORK = …` as step 1, with a comment explaining why.
  • Added `01-resolve-addresses` as step 2, defensive (idempotent in the happy path, self-healing if `resolved.env` got deleted, protective against the rare case of proxy redeployment).
  • Footnote making explicit that no other onboarding phase (00, 02, 04-07) needs to be re-run.

Test plan

End-to-end on macOS Apple Silicon (M2 Pro, Docker Desktop 27.4.0, bash 5.3.9):

  • Verified `env.sh` first-run bug reproduces on a clean clone (no `secrets.env`)
  • Confirmed the `if` block fix unblocks the same scenario
  • Completed full testnet onboarding (Phases 00 → 08) on the fixed branch
  • Confirmed worker registered, authenticated, and received the `websocket connected to gateway` signal
  • Tested the documented upgrade procedure end-to-end (`NETWORK` re-export + `01-resolve` + stop + pull + run) — confirmed correct image swap, native arm64 selection from multi-arch index, gateway reconnect
  • Confirmed the digest-check commands return matching digests when the running image is already `:latest`

DimitarTAtanasov and others added 5 commits May 20, 2026 11:08
…lved.env are missing

env.sh used the `[[ -f file ]] && source file` short-circuit pattern to
conditionally load two optional config files. When either file does not exist,
the failed `[[ ]]` test returns exit 1 and the `&&` chain inherits that status.
Because env.sh is the last file sourced through common.sh from every numbered
script (which all use `set -euo pipefail`), the non-zero return propagates back
up the source chain and triggers `errexit` in the parent script — silently,
before any command can print output.

The bug fires on every truly clean first-run install, which is exactly the
flow the README promises will work. It was not caught in development because
the maintainers' working trees always had `secrets.env` left over from previous
runs, so the test never failed.

Replace the short-circuit with explicit `if [[ ]]; then ... fi` blocks. The
`if` construct is specifically exempt from `set -e` propagation, so a missing
optional file no longer kills the parent.

Reproduces on bash 3.2.57 (Apple stock) and bash 5.3.9 (Homebrew); affects
all Linux + macOS + WSL/Git Bash users. Native Windows users using the
PowerShell scripts are unaffected.
The bash scripts shipped as mode 100644, so on a fresh clone every direct
invocation (./scripts/bash/00-generate-key.sh, status.sh, tail-jobs.sh, etc.)
fails with `zsh: permission denied`. Users had to either `chmod +x scripts/bash/*.sh`
manually or fall back to `bash ./scripts/bash/...` — neither is documented in
the README, both contradict the documented one-liner usage.

Mark the 14 scripts the README invokes directly as 100755. Leave the three
files that are only sourced or copied at 100644:

  - common.sh: sourced from every phase script
  - env.sh:    sourced from common.sh
  - secrets.example.sh: copied (not executed) by 00-generate-key.sh
…port + 01-resolve

The previous three-line stop/pull/run snippet was correct but assumed the
operator was in the same shell session as their original onboarding. In
practice, operators frequently come back to upgrade days or weeks later
in a fresh terminal where NETWORK is no longer exported — the scripts
then silently default to mainnet and pull the wrong image / connect to
the wrong gateway.

This change:

  * Adds an explicit NETWORK re-export as step 1 so the procedure works
    from any shell.
  * Adds 01-resolve-addresses as a defensive step 2 — idempotent in the
    happy path (just re-writes resolved.env with the same on-chain
    addresses), self-healing if resolved.env got nuked, and protective
    against the rare scenario where Lightchain replaces a proxy.
  * Documents what every other step would do and why no earlier phase
    (00, 02, 04, 05, 06, 07) needs to be re-run — pre-empts the natural
    question "if 01 then why not all of them".
  * Adds a 'Verifying the new image' subsection listing the three
    health-signal log lines to look for after restart.
  * Adds a 'Checking whether there's actually a new image to pull'
    subsection with the manifest-inspect + RepoDigest commands so
    operators can compare digests without doing a full pull.

No code change.
macOS's default /bin/bash is 3.2 due to GPLv3 licensing. Two scripts
(04-import-key.sh, 06-fund-worker.sh) use the ${var,,} lowercase
expansion introduced in bash 4.0 and fail with `bad substitution` on
a stock Mac, leaving the operator in a half-done state.

- README: blockquote callout in the Linux/macOS quick-start
- docs/installation-macos.md: required-software row + install step +
  verification line + gotcha entry with the actual error message
- docs/installation-linux.md: bash 4.0+ row with a note that any
  mainstream distro from the last decade satisfies it
@DimitarTAtanasov DimitarTAtanasov force-pushed the fix/env-sh-silent-failure-on-first-run branch from 8f17b97 to 4c4987a Compare May 20, 2026 12:20
@HristiyanG HristiyanG requested a review from lightchainai May 21, 2026 10:05
@Z-Blocks Z-Blocks merged commit 38307f4 into lightchain-protocol:main May 21, 2026
1 of 3 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants