feat: runtime hot-mount (limactl mount add|remove|list) for QEMU and VZ by riccardosabatini · Pull Request #5154 · lima-vm/lima

riccardosabatini · 2026-06-24T21:26:34Z

First off, thank you for Lima. It's become the backbone of how a lot of us run Linux on macOS, and digging through the codebase to build this, I came away genuinely impressed: the driver abstraction, the host-agent/guest-agent split, and the overall clarity made a fairly deep feature far more approachable than it had any right to be. The VsockEventEmitter optional-interface pattern in particular is lovely, and it's what this PR builds on.

Huge respect to the maintainers and contributors. 🙏

What this adds

A new command to mount/unmount host folders into a running instance, with no restart:

  limactl mount add    <inst> <host-path> <guest-path> [--type virtiofs|9p|reverse-sshfs] [--writable]
  limactl mount remove <inst> <guest-path>
  limactl mount list   <inst>

Mounts are ephemeral (not written to lima.yaml). The motivation: people currently reach for NFS/sshfs to get a folder into a live VM, and the throughput/memory behavior is rough. virtiofs is dramatically better: in testing here a hot-mounted folder reads at ~5 GB/s on VZ, identical to a statically-configured virtiofs mount.

How it works

It's exposed as a new optional driver.FSHotPlugger capability interface (modeled on VsockEventEmitter), so the CLI, the ha.sock /v1/mounts API, and the host-agent mount registry are all driver-agnostic, a driver opts in by implementing the interface, and everything else "just works."

QEMU (Linux host): virtiofs (default) and 9p are hot-plugged as PCIe devices over QMP (chardev-add/device_add, fsdev_add); reverse-sshfs works over SSH. To make this possible, each Linux QEMU instance now boots with shareable
memory (memory-backend-memfd,share=on) and 8 spare pcie-root-port controllers as hot-plug slots. The guest-side mount runs over the existing SSH connection.
VZ (macOS host, Linux guest): virtiofs. VZ has no device hot-plug API, so instead the driver reserves spare empty virtio-fs devices at boot and sets their share at runtime via VZVirtioFileSystemDevice.share. This needs a small
addition to the Code-Hex/vz binding (see notes below).

The host-agent/guest-agent gRPC proto is untouched, guest mounts use the existing ssh.ExecuteScript helper.

Testing

Unit tests: mount-option helper, host-agent mount registry (add/remove/list, rollback, validation), /v1/mounts handlers, QMP command builders + PCIe slot allocator (QEMU), spare-device slot allocator (VZ), and CLI arg validation.
Integration: hack/test-hot-mount.sh exercises virtiofs/9p/reverse-sshfs end-to-end (read/write both directions).
Real hardware:
- QEMU/Linux, validated on a nested-KVM rig: hot-mount + unmount of virtiofs, 9p, and reverse-sshfs, with PCIe hot-plug/unplug confirmed in guest dmesg.
- VZ/macOS (Apple silicon), native: hot-mount/unmount, two concurrent mounts, slot reuse, writable vs read-only, and ~5 GB/s sequential read.
Build + go vet clean across Linux and the darwin/vz CGo path; rebased cleanly on current master.

Notes & deliberate tradeoffs

A few things are intentional choices that are really the maintainers' call, flagging them transparently so nothing's hidden:

The Code-Hex/vz dependency. VZ hot-mount needs one new method on the binding (*VirtualMachine).SetVirtioFileSystemDeviceShareAtIndex (runtime share mutation). For now it's carried as an in-tree vendored fork under third_party/Code-Hex-vz (via a go.mod replace) so the branch builds standalone, with the isolated change also saved as hack/patches/code-hex-vz-runtime-share.patch. The clean end state is to upstream that method to Code-Hex/vz and bump the dep here; I'm glad to open that PR and drop the fork.
Scope. This is effectively two features (QEMU/Linux and VZ/macOS) and can be split — the QEMU half has no external dependency and could land independently.
Boot-time change. Every Linux QEMU instance gets memfd shareable memory + 8 spare PCIe root ports. It's an always-on change for QEMU users; it can equally be gated behind an opt-in field (e.g. hotMount: true) if that's preferred.
Process. I went straight to a working, hardware-validated prototype to de-risk the uncertain parts (notably whether VZ's runtime share change is actually visible to a Linux guest — it is). I realize a change this size would normally start with a design discussion.

For full transparency

This was developed with AI assistance (hence the Co-Authored-By trailers); all commits are DCO signed-off by me, and I've reviewed and tested everything on real hardware.

I'm offering the feature and the implementation here as a complete, working starting point. If it's of interest to the project, I'll happily defer to your senior maintainers on how you'd like to take it forward — splitting it,upstreaming the vz change first, gating the boot behavior, or reworking the packaging.

Thanks again for the project, and for taking the time to look. 🙇

Adds the approved design for runtime hot-mount/unmount of host folders into a running QEMU/Linux VM. Primary transport is virtiofs (high-IO), with 9p and reverse-sshfs alternatives. Hot-mounts are ephemeral. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Signed-off-by: Riccardo Sabatini <riccardo.sab@gmail.com>

Adds the task-by-task implementation plan and records three planning-time refinements in the spec: optional FSHotPlugger capability interface (no macOS/Windows driver edits), Linux-gated memfd memory backing, and deferral of the external-driver gRPC proxy. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Signed-off-by: Riccardo Sabatini <riccardo.sab@gmail.com>

Extract the 9p/virtiofs/sshfs mount-option and fstype computation from cidata into pkg/mountutil so the runtime hot-mount path and the boot-time cloud-init path share one source of truth. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Signed-off-by: Riccardo Sabatini <riccardo.sab@gmail.com>

On Linux, back guest memory with memory-backend-memfd (share=on) so virtiofs devices can be hot-plugged at runtime, and reserve 8 spare pcie-root-port controllers on q35/virt as hot-plug slots. Non-Linux qemu hosts keep the existing /dev/shm backing for static virtiofs. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Signed-off-by: Riccardo Sabatini <riccardo.sab@gmail.com>

Add FSHotPlugger plus HotPlugFS/HotUnplugFS request/response types and ErrFSHotPlugUnsupported. ConfiguredDriver delegates to the wrapped driver when it implements the capability, so non-supporting drivers (VZ, WSL2, krunkit) need no changes. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Signed-off-by: Riccardo Sabatini <riccardo.sab@gmail.com>

Add runtime attach/detach of virtio-9p-pci and vhost-user-fs-pci devices via QMP: PCIe slot allocator over the spare lima-hp-* root ports, virtiofsd lifecycle per virtiofs share, chardev-add/device_add (virtiofs) and fsdev_add/device_add (9p), and device_del waiting on DEVICE_DELETED before freeing the slot. virtiofsd leaks are reaped on Stop. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Signed-off-by: Riccardo Sabatini <riccardo.sab@gmail.com>

Add a runtime hot-mount registry to the host agent. reverse-sshfs reuses the existing setupMount; 9p/virtiofs attach a device via the driver's FSHotPlugger capability and mount it in the guest over SSH (with rollback on failure). Mounts are torn down on shutdown. guestExec is injected for testability. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Signed-off-by: Riccardo Sabatini <riccardo.sab@gmail.com>

Add POST/GET/DELETE /v1/mounts to the host agent HTTP API plus matching client methods and a httpclientutil.Delete helper. Backend now depends on a narrow Agent interface (satisfied by *HostAgent), which decouples the server from the hostagent package and enables handler tests. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Signed-off-by: Riccardo Sabatini <riccardo.sab@gmail.com>

Add 'limactl mount add|remove|list' to mount and unmount host directories into a running instance at runtime over the host agent /v1/mounts API. --type defaults to virtiofs; --writable controls write access. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Signed-off-by: Riccardo Sabatini <riccardo.sab@gmail.com>

Document 'limactl mount add|remove|list' in the mount docs (QEMU/Linux only, ephemeral, virtiofs default, 8-mount limit) and add a hack integration test exercising virtiofs, 9p, and reverse-sshfs hot-mounts host<->guest. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Signed-off-by: Riccardo Sabatini <riccardo.sab@gmail.com>

- shell-escape guest mount/umount paths (avoid injection/breakage on unusual mount points) - retry the guest mount to tolerate PCI hot-plug enumeration latency - reap virtiofsd in closeHotPlug (avoid zombies) via shared killProcess - reject 9p host paths containing spaces/commas that would corrupt the HMP fsdev_add command - do not fail/block host agent shutdown on hot-mount teardown errors - log chardev-remove rollback failures - set SSHFS defaults only for reverse-sshfs; reuse qemuConfig() in Start() Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Signed-off-by: Riccardo Sabatini <riccardo.sab@gmail.com>

VZ's Virtualization.framework has no device hot-plug API, so instead of adding a device per mount (as QEMU does), the VZ driver reserves 8 spare empty virtio-fs devices at boot and populates their share at runtime. - pkg/driver/vz/vm_darwin.go: reserve spare virtio-fs devices (tags lima-hotmount-0..7) on Linux guests, at directory-share indices 0..7. - pkg/driver/vz/hotplug_darwin.go: implement driver.FSHotPlugger by setting/clearing a spare device's share via a new binding method. - pkg/driver/driver.go: HotPlugFSResponse.Tag lets a driver with fixed pre-reserved device tags (VZ) tell the host agent which tag to mount. - third_party/Code-Hex-vz: vendored fork of Code-Hex/vz adding (*VirtualMachine).SetVirtioFileSystemDeviceShareAtIndex (runtime share mutation); isolated patch in hack/patches/, to be upstreamed. The host agent, /v1/mounts API, and limactl mount command are unchanged: VZ slots in purely by implementing the FSHotPlugger capability interface. Verified on native VZ (macOS 26, Apple silicon): hot-mount/unmount, two concurrent mounts, slot reuse, writable/read-only, ~5 GB/s read. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Signed-off-by: Riccardo Sabatini <riccardo.sab@gmail.com>

…t-mount Document the macOS/VZ build requirements for runtime hot-mount: Xcode CLT for the CGo VZ driver, the automatic virtualization-entitlement codesign, and the in-tree vendored Code-Hex/vz fork (with go.mod replace) that carries the runtime share-mutation method until it is upstreamed. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Signed-off-by: Riccardo Sabatini <riccardo.sab@gmail.com>

Cover the bin/share layout limactl expects relative to its binary, which guest agents and external deps (qemu, virtiofsd) to bundle per driver, macOS codesigning/entitlements and notarization for the embedded limactl, and runtime configuration (LIMA_HOME, PATH). Notes that the vendored Code-Hex/vz fork is compile-time only and that hot-mount works embedded. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Signed-off-by: Riccardo Sabatini <riccardo.sab@gmail.com>

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Signed-off-by: Riccardo Sabatini <riccardo.sab@gmail.com>

These brainstorm spec/plan files are local workflow notes, not part of the Lima codebase. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Signed-off-by: Riccardo Sabatini <riccardo.sab@gmail.com>

- exclude vendored third_party/Code-Hex-vz from codespell, ls-lint, and editorconfig-checker (upstream code, not subject to our conventions) - fix golangci-lint findings in the hot-mount code: intrange, noctx, empty-block (revive), slices.Contains, unnecessary-format, unconvert, unnamedResult, forbidigo (t.Fatal*) and usetesting (t.Context) Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Signed-off-by: Riccardo Sabatini <riccardo.sab@gmail.com>

…ndows - ltag: add third_party to -excludes (vendored fork carries upstream headers, not the Lima SPDX boilerplate) - validateHotMount: validate the guest mount point with path.IsAbs (Unix semantics) instead of filepath.IsAbs, which rejects /mnt/... on Windows hosts and broke the unit tests Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Signed-off-by: Riccardo Sabatini <riccardo.sab@gmail.com>

AkihiroSuda · 2026-06-25T03:11:51Z

Please send this patch to the upstream

Hi @AkihiroSuda agree, this must go upstream, for now I copied the VZ code and added the batch in this branch not to rely on the release from VZ (so you guys can test it). I'll submit a PR to their repo, hopefully this will be fast. Once done I'll remove the patch and the VZ code from this PR (and other pieces managing the compilation, etc)

AkihiroSuda · 2026-06-25T03:12:20Z

New tests should be written in BATS
https://github.com/lima-vm/lima/tree/master/hack/bats

AkihiroSuda · 2026-06-25T03:15:57Z

Why fork the entire vz into third_party/Code-Hex-vz?

AkihiroSuda · 2026-06-25T03:16:18Z

+> **Warning**
+> Runtime hot-mount is experimental
+
+| ⚡ Requirement | Lima >= 2.1.0; QEMU driver on a Linux host, or the VZ driver on macOS |


Not in 2.1.0.

AkihiroSuda · 2026-06-25T03:16:57Z

+> `com.apple.security.virtualization` entitlement (from `vz.entitlements`). This signature is
+> required for the VZ driver, including runtime [hot-mount](../../config/mount#runtime-hot-mounts).
+> If you re-sign or strip the binary, re-apply the entitlement with
+> `codesign -f --entitlements vz.entitlements -s - <binary>`.


Why does this need to be documented?

Same thing, to remove once VZ accept the PR

AkihiroSuda · 2026-06-25T03:19:43Z

 - `git`
 - `go`
 - `make`
+- On **macOS**: the Xcode Command Line Tools (`xcode-select --install`), required to build


Not relevant to host-mounts

AkihiroSuda · 2026-06-25T03:21:22Z

marked this pull request as ready

Not ready until unforking vz

AkihiroSuda · 2026-06-25T03:30:14Z

+
+// FSType returns the guest filesystem type string for a mount type and guest OS.
+// It returns an empty string for unknown mount types.
+func FSType(mountType limatype.MountType, os limatype.OS) string {


Why is this function needed?

AkihiroSuda · 2026-06-25T03:33:01Z

+sudo mkdir -p %[1]s
+i=0
+while [ "$i" -lt 30 ]; do
+	if sudo mount -t %[2]s -o %[3]s %[4]s %[1]s; then exit 0; fi


sudo is being discouraged
#5109 (comment)

The guestagent should have a proper API for this?

riccardosabatini · 2026-06-25T03:51:12Z

@AkihiroSuda let me know if this is something you guys would like to integrate, once polished - happy to work on the upstream VZ PR in case

AkihiroSuda · 2026-06-25T04:00:48Z

@AkihiroSuda let me know if this is something you guys would like to integrate, once polished

Yes

happy to work on the upstream VZ PR in case

Does this PR work?

feat: expose runtime directory-sharing devices with live share swap Code-Hex/vz#216

riccardosabatini · 2026-06-25T04:11:06Z

@AkihiroSuda let me know if this is something you guys would like to integrate, once polished

Yes

happy to work on the upstream VZ PR in case

Does this PR work?

feat: expose runtime directory-sharing devices with live share swap Code-Hex/vz#216

Interesting, might be, I'll give a check tomorrow (getting late on my side)

Riccardo Sabatini and others added 16 commits June 24, 2026 17:14

docs(readme): point to source-install/embedding guide for hot-mount

3da558f

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Signed-off-by: Riccardo Sabatini <riccardo.sab@gmail.com>

chore: remove superpowers design/plan workflow artifacts from branch

e7a24a2

These brainstorm spec/plan files are local workflow notes, not part of the Lima codebase. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Signed-off-by: Riccardo Sabatini <riccardo.sab@gmail.com>

riccardosabatini marked this pull request as ready for review June 24, 2026 21:31

riccardosabatini marked this pull request as draft June 24, 2026 21:34

Riccardo Sabatini and others added 2 commits June 24, 2026 18:23

riccardosabatini marked this pull request as ready for review June 24, 2026 22:50

AkihiroSuda added impact/changelog component/qemu/9p labels Jun 25, 2026

AkihiroSuda reviewed Jun 25, 2026

View reviewed changes

Comment thread third_party/Code-Hex-vz/cmd/addtags/main.go

AkihiroSuda Jun 25, 2026

Copy link
Copy Markdown

Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why fork the entire vz into third_party/Code-Hex-vz?

AkihiroSuda reviewed Jun 25, 2026

View reviewed changes

AkihiroSuda marked this pull request as draft June 25, 2026 03:21

AkihiroSuda reviewed Jun 25, 2026

View reviewed changes

Uh oh!

Conversation

riccardosabatini commented Jun 24, 2026

Uh oh!

Choose a reason for hiding this comment

Uh oh!

riccardosabatini Jun 25, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

AkihiroSuda commented Jun 25, 2026

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

riccardosabatini commented Jun 25, 2026

Uh oh!

AkihiroSuda commented Jun 25, 2026

Uh oh!

riccardosabatini commented Jun 25, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

riccardosabatini Jun 25, 2026 •

edited

Loading