Skip to content

fix(toolchain): invalidate target/ + sccache after a nightly bump#384

Merged
githubrobbi merged 2 commits into
mainfrom
fix/toolchain-sync-cache-invalidation
Jun 10, 2026
Merged

fix(toolchain): invalidate target/ + sccache after a nightly bump#384
githubrobbi merged 2 commits into
mainfrom
fix/toolchain-sync-cache-invalidation

Conversation

@githubrobbi

Copy link
Copy Markdown
Collaborator

Problem

After just toolchain-sync bumps the rust-toolchain.toml nightly pin, the next local build dies with:

error[E0514]: found crate `assert_cmd` compiled by an incompatible version of rustc
  = note: ... compiled by rustc 1.98.0-nightly (6bdf43094 2026-06-01)
  = help: please recompile that crate (consider running `cargo clean` first)

(also hit winnow, tempfile, toml_parser). This blocked pre-push gates this session.

Root cause — two stale caches, not one

toolchain-sync probes candidate nightlies in a per-candidate scratch CARGO_TARGET_DIR (it already knows sharing the main target/ across toolchains throws E0514), then commits the bump and exits without touching the main caches:

  1. target/ — rlibs/rmeta carry the old rustc commit-hash; cargo fingerprints don't reliably force a full rebuild across a nightly bump.
  2. the sccache daemon — the non-obvious one. The long-lived sccache server caches its compiler-version detection keyed by the rustc binary's mtime/size. Under rustup, ~/.cargo/bin/rustc is a toolchain-agnostic proxy (symlink → rustup) whose bytes never change across a bump, so the daemon keeps serving the OLD nightly's objects and re-poisons target/ right after a cargo clean. This is why cargo clean alone didn't fix it.

Fix

After toolchain-sync commits a bump (only in the "found newer nightly" branch, not "no bump available"), run:

  • cargo clean — clear the old-nightly target/.
  • sccache --stop-server — force the daemon to re-probe rustc -vV on next invocation, restoring correct cache keying. A full 30 GiB cache wipe is unnecessary; the daemon restart is the targeted fix.

Verification

  • Reproduced E0514 locally; fixed with cargo clean + sccache --stop-server; full cargo build --workspace --all-targets clean afterward.
  • This PR's own pre-push gate passed all Rust compilation gates (cargo-check, lint-tests, tests, lint-ci-windows) — the exact gates that were failing with E0514 before the cache fix.

toolchain-sync probes candidate nightlies in a per-candidate scratch

CARGO_TARGET_DIR, then commits the bump and exits without touching the

main caches. Both retain the OLD nightly's artifacts, so the next

build dies with E0514 'found crate X compiled by an incompatible

version of rustc' (observed: assert_cmd/winnow/tempfile/toml_parser).

Two caches must be invalidated, not one:

  1. cargo clean        — main target/ rlibs/rmeta carry the old rustc

                          commit-hash; cargo fingerprints do not force a

                          full rebuild across a nightly bump.

  2. sccache --stop-server — the long-lived sccache daemon caches its

                          compiler-version detection keyed by the rustc

                          binary mtime/size. Under rustup, rustc is a

                          toolchain-agnostic proxy (~/.cargo/bin/rustc ->

                          rustup) whose bytes never change across a bump,

                          so the daemon keeps serving the OLD nightly's

                          objects and re-poisons target/ right after a

                          cargo clean. Restarting forces a fresh rustc -vV

                          probe. A full 30 GiB cache wipe is unnecessary.

Both run only in the 'found newer nightly' success branch, not the

'no bump available' branch.
@githubrobbi githubrobbi enabled auto-merge (squash) June 10, 2026 01:08
@githubrobbi githubrobbi merged commit dc8f326 into main Jun 10, 2026
25 checks passed
@githubrobbi githubrobbi deleted the fix/toolchain-sync-cache-invalidation branch June 10, 2026 01:25
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant