arm64: guest PAC keys not preserved across checkpoint/restore (SIGILL on restore)

[mstrathman]

Description

We've been exercising runsc checkpoint/restore on arm64 and hit a reproducible failure. Any
guest built with pointer authentication (-mbranch-protection, the default in many distro
toolchains and in glibc) is killed by SIGILL on restore.

A trivial binary restores fine, so it took a PAC-free control to isolate it. The guest's PAC
keys aren't surviving the checkpoint: on restore the guest threads come up with fresh keys, so
the first autiasp against a return address signed by the old key (paciasp, pre-checkpoint)
faults.

• Expected: the restored process keeps running with PAC still functional, as on x86_64 (no
  PAC) and as PAC-free arm64 binaries already do.
• Observed: SIGILL immediately on restore; the container comes back stopped with
  boot.go: application exiting with killed by signal 4 in the boot log.

Possibly related: #12917 (PAC state on the KVM platform, sentry VDSO __kernel_getrandom).
That's a runtime crash on KVM and is reported not to occur on systrap; this one is guest-key
preservation across checkpoint/restore on systrap, so it looks like a distinct gap.

Steps to reproduce

// pac_counter.c — a counter so you can see whether it kept running after restore
#include <stdio.h>
#include <unistd.h>
int main(void) { for (long i = 0; ; i++) { printf("counter=%ld\n", i); fflush(stdout); sleep(1); } }

gcc -O2 -static -mbranch-protection=standard -o pac_counter pac_counter.c
# pac_counter as the entrypoint of a minimal OCI bundle ($B):
runsc --platform=systrap run        --bundle "$B" cid &
sleep 3
runsc --platform=systrap checkpoint --leave-running --image-path=/tmp/img cid
runsc --platform=systrap create     --bundle "$B" rid
runsc --platform=systrap restore    --image-path=/tmp/img rid   # SIGILL here

Rebuilt with -mbranch-protection=none, the same program checkpoints, restores, and keeps
counting.

runsc version

runsc version release-20260608.0
spec: 1.2.1

docker version (if using docker)

n/a — reproduced with raw runsc, no Docker.

uname

Linux 7.0.0-22-generic #22-Ubuntu SMP PREEMPT_DYNAMIC Mon May 25 15:37:49 UTC 2026 aarch64 GNU/Linux

kubectl (if using Kubernetes)

n/a

repo state (if built from source)

n/a — reproduced against the released runsc binary, not a source build.

runsc debug logs (if available)

Restore-side boot log (PAC binary, --platform=systrap), trimmed to the fault:

task_signals.go:464] [   1:   1] Notified of signal 4
task_signals.go:200] [   1:   1] Signal 4, PID: 1, TID: 1, fault addr: 0x404470: terminating thread group
task_exit.go:388]    [   1:   1] Init process terminating, killing namespace
boot.go:675] application exiting with killed by signal 4
cli.go:310] Exiting with status: 4

---

What we found (observations, not a proposed design)

We chased this far enough to get a working prototype, so a few notes in case they're useful.
You know the internals better than we do, so treat these as observations:

• The saved arch state (arch.State, arch_aarch64.go) carries GP/FP registers but no PAC
  keys, so they're absent from the serialized image.
• On systrap, the kernel exposes the keys via PTRACE_GET/SETREGSET with NT_ARM_PACA_KEYS
  (0x407) and NT_ARM_PACG_KEYS (0x408): grab them at checkpoint, restore them after.
• On KVM they're a handful of sysregs (APIA/APIB/APDA/APDB/APGA), presumably the same get/set
  path you already use for the other EL1 sysregs.
• The subtlety that cost us time: systrap pools and reuses stub subprocesses, and a guest
  thread's PAC keys also back gVisor's own PAC-signed sysmsg handler on that thread, so a thread
  can't be safely re-keyed after it's running. We kept one key consistent at a coarser scope
  than per-thread; where that scope belongs is your call.

We have a working systrap version that survives C/R with PAC still functional and no isolation
regression, and we're glad to share it or open a PR if that helps. We wanted to file the bug and
what we know first.

Impact: arm64 guests built with branch protection (most real binaries now) can't be
checkpointed and restored, which rules out snapshotting, live migration, and restoring multiple
copies from one image on arm64 until the keys are preserved.

[konstantin-s-bogom]

Yeah that makes sense, PAC keys should be restored for Systrap/ptrace. I think the simplest solution is to just generate a random key at startup that we use for all created stub threads, but we'll still need to save it in case the app changed it at some point. You're right about the subprocess pool, that it will need to check if the keys match -- but is it actually an issue on restore, because I would think that on restore the subprocess pool is empty and we're creating everything from scratch? Anyway feel free to open a PR for this.

KVM has PAC disabled across the board after #12917, because unlike with Systrap where you can read the stub process' keys, there's no way for the sentry to read its own keys. Since we don't enable PAC for the sentry in gr0, we don't enable PAC for workloads running in the guest either.

[mstrathman]

Thanks, that matches what we found.

On the subprocess pool: you're right, it isn't a restore problem. At restore the pool starts empty and every stub is rebuilt from scratch, so with the key persisted and seeded the same way there's nothing to match against. The pool only mattered for the running case. Systrap reuses stub subprocesses, and the keys also back gVisor's own signed sysmsg handler, so we couldn't safely re-key a live thread. One key per sandbox makes that correct by construction, which is the route we went.

We have it working: generate a key set once at kernel Init, serialize it on the Kernel so it survives C/R, and SETREGSET it (NT_ARM_PACA_KEYS / NT_ARM_PACG_KEYS) on each guest thread before it runs any guest code. A PAC binary that used to SIGILL on restore now restores running, 3/3, with no regression on non-PAC binaries or on amd64 (which falls back to the plain address space).

One thing to settle before the PR. A key generated at Init is simple but won't follow an app that re-keys itself with PR_PAC_RESET_KEYS mid-run. The faithful alternative is the CRIU route: GETREGSET the real keys at checkpoint, SETREGSET them back at restore. Any preference? Either is fine by us. The Init version is less machinery; the checkpoint-capture version is correct for the re-key case.

Agreed KVM is out of scope here since PAC is disabled there after #12917, so this is systrap-only.

Will open the PR shortly.