VM platform: file sharing (DAX), USB passthrough, Rosetta x86, security + GitHub issue fixes#5
Merged
Merged
Conversation
Own VMM on Hypervisor.framework: guest RAM is process-owned mmap (the foundation for real memory reclaim), in-kernel GICv3 via hv_gic with MMIO register forwarding, PL011 console, PSCI over SMC, arm64 Image loader, and an FDT builder. Kernel reaches the expected VFS root panic; block devices land next. Architecture doc included.
…n 0.4s Split virtqueue engine with bounds-checked descriptor walks, the virtio-mmio v2 register surface, a zero-copy block device, and a host CSPRNG entropy device. The dind rootfs boots to a fully initialized Docker Engine (API listen on /var/run/docker.sock) 405ms after kernel entry, verified with an in-guest docker version round trip.
…eclaim virtio-balloon negotiates VIRTIO_BALLOON_F_REPORTING; reported ranges are hv_vm_unmap'd (stage-2 pins pages, madvise alone is a no-op) then marked MADV_FREE_REUSABLE, and remapped lazily on guest fault with REUSE for honest accounting. PL031 RTC gives the guest real wall time. Measured: 2GiB VM fills to 900MB, guest frees it, host footprint falls to 107MB in seconds; 800MB integrity checksum survives the cycle; idle CPU 0.2%. Guest-side cache trimming via cgroup2 memory.reclaim + compaction closes the page-cache ratchet (680MB -> ~330MB and falling).
…_ENGINE virtio-net speaks the vfkit datagram protocol to a gvproxy sidecar: guest DHCPs 192.168.127.2, full outbound NAT/DNS/TLS verified with a real docker pull. gvproxy also serves ~/.dory/engine.sock as a unix socket straight into guest dockerd. dory-hv engine mode owns the whole lifecycle (one-time dind unpack to a PERSISTENT disk: images survive restarts, fixing the VZ helper's re-clone regression), PL031 RTC for correct guest wall time, SIGTERM reaps the sidecar. SharedVMProvisioner prefers dory-hv when DORY_HV_ENGINE=1, falling back to the VZ helper then the container CLI. Verified end to end: the Dory app provisioned dory-hv and ran postgres:16 through its own hypervisor.
…adroom Wires the in-process helper as the preferred engine with env-tunable memory policy, counts dory-vmboot in readiness memory probes, and covers the new host-support ladder in RuntimeSupportTests. Full suite green.
Secondary vCPUs are created eagerly (so the kernel's redistributor walk sees every GIC frame), park on a condition until PSCI CPU_ON releases them at their entry point, and run one dedicated thread each per the framework's create/run/destroy-on-one-thread rule. Per-vCPU MPIDR and redistributor-frame mapping by the GIC-assigned base. stopAll cancels running vCPUs exactly once via hv_vcpus_exit; run() joins every secondary before returning so hv_vm_destroy never races a live thread. Guest boots 4 CPUs (nproc=4).
Reclaim under SMP hit a real race: releaseRange unmapped a page before marking it released, so a concurrent vCPU faulting in that window saw an unmapped-but-unmarked page and crashed it as a genuine fault. The bitmap flip and the stage-2 map/unmap now happen atomically under one lock in both releaseRange and restorePage, and a stage-2 RAM fault (which can only be a page we unmapped) always remaps instead of crashing. Adds released/restored byte counters for the reclaim gauge. Also serialize virtio-net RX drain against MMIO queue reconfig via the transport register lock, and guard Virtqueue.push with size>0, closing the reported divide-by-zero when a guest resets the RX ring mid-drain. Verified: 5 rounds of 200MB memory churn under 4 vCPUs, no crash.
Split the single persistent disk into a throwaway APFS-cloned rootfs (recreated each boot, so system state can't rot) plus a separate journaled ext4 mounted at /var/lib/docker for docker state. An unclean quit can no longer corrupt the engine: the rootfs is disposable and the data disk journals (verified: EXT4-fs vdb recovery completes on restart, images survive). SIGTERM/SIGINT now ask the guest to sync+unmount+power off through a shutdown socket, with a 10s hard-kill watchdog and gvproxy reaped. Guest trim profile v2: proportional memory.reclaim toward a 128MB cache floor every 15s, vfs_cache_pressure 200, so the page cache stops ratcheting the footprint up.
- Data-disk mount failure now powers the guest off instead of letting dockerd write to the throwaway rootfs (was silent total data loss on the next boot's re-clone). - Rootfs unpack and data-disk format are built at a .partial path, fsynced, then atomically renamed, so an interrupted first run can't leave a truncated file the fileExists guard treats as complete forever. - PID 1 is now tini (docker-init, via exec so it keeps PID 1 while dockerd + loops run as its children): container shims orphan children onto PID 1; without a reaper they pile up as zombies until PID exhaustion. Verified 0 zombies after 50 --rm containers. - SIGTERM shutdown touch retries until the gvproxy forward and guest listener exist, instead of firing one discarded connect that a boot-time signal would lose. Verified end to end: images + volumes + volume data survive restart, ext4 journal recovery clean, graceful shutdown ~2s, 4 vCPUs.
Live measurement of the signed app exposed two defects in the trim loop, both confirmed against the running guest: - compact_memory migrates pages and re-faults the ones free page reporting already handed back to the host. Measured 129 MiB restored per 45 s of pure idle: pure churn (wasted CPU + inflated footprint, transient 1.7 GiB after a heavy image pull). Removed. - memory.reclaim on the ROOT cgroup is write-rejected, so the cache-drop half of the loop did nothing (silent, behind 2>/dev/null). Now: free page reporting handles free pages automatically; the guest gets vm.min_free_kbytes=256M so the kernel keeps cold cache moving back to the free list, and a gentle pagecache drop fires only when cache bloats past ~384 MiB, leaving active working sets intact.
…im guards 16 tests, no VM required (pure logic + GuestMemory mmap): FDT header + string dedup, arm64 Image header parse + bad-magic reject, data-abort syndrome decode, MMIO bus routing, PL011 transmit/flags, a full virtqueue pop->write->push round trip, push-after-reset divide-by-zero guard, out-of-bounds descriptor reject, and releaseRange/restorePage bitmap invariants.
bundle-engine.sh now builds and signs dory-hv with com.apple.security.hypervisor and injects a gvproxy binary (from DORY_GVPROXY / podman / PATH) alongside the legacy dory-vm helper. The HV-engine provisioner path falls back to the bundled compressed kernel when no container toolchain is installed, so a self-contained install boots the new engine with no external dependencies.
Slightly tighter pagecache cap trims the idle footprint without disturbing active working sets. Documents that the residual floor is guest cache + sub-16KiB fragmentation (the future lever is a custom 4KiB-granule reporting device), and that restore churn is now ~4-5% of released bytes vs ~58% before the compaction fix.
The dory-hv engine runs on Hypervisor.framework's in-kernel GICv3 (macOS 15+) and ships its own kernel + userspace networking, so it needs neither macOS 26 nor Apple's container toolchain. hostSupport now returns supported on macOS 15+ Apple silicon whenever the dory-hv engine is available (DORY_HV_ENGINE opt-in + bundled helper/gvproxy/kernel), falling back to the legacy container/VZ evaluation otherwise. New DoryHVSupport type + tests cover the broadened matrix; existing container-path tests pin the fallback. All DoryTests green.
Dory now controls the whole stack for uniform performance: our own Hypervisor.framework VMM (dory-hv) is the one and only shared-VM engine. Removed the Apple `container` CLI provisioning path, the older Virtualization.framework dory-vm helper path, the AppleContainerRuntime backend, AppleContainerSupport (macOS 26 + toolchain gate), and the guided container-toolchain install UX (EngineSetupView, needsContainer- Toolchain, toolchainInstall*). - SharedVMProvisioner is dory-hv only: hostSupport = DoryHVSupport (macOS 15+ Apple silicon), no toolchain required. Engine is default-on (opt out with DORY_HV_ENGINE=0). - AppStore backend selection drops the Apple per-container fallback; Intel / older macOS still fall back to a Docker-compatible socket. - Copy updated (macOS 15, 'Dory's engine', no 'Apple container engine'). - Tests updated; DoryTests + DoryHVTests green; app builds. Known follow-up: published container ports (docker -p) need gvproxy per-port expose wiring for the dory-hv path.
Closes the last dory-hv gap: docker run -p is now reachable from the host. The engine polls the docker socket and keeps gvproxy's forwards in sync with the live set of published ports (expose on publish, unexpose on stop), using gvproxy's bare host:port TCP remote form. The app no longer binds ports itself for the shared-VM path (that would race gvproxy); it points *.dory.local domains at the gvproxy-exposed localhost ports. Verified: nginx -p 18080:80 -> localhost:18080 HTTP 200 in ~9s, and released within 3s of docker rm.
Two defects caught by adversarial review of the sole-engine refactor: - hostSupport() had stopped folding in hvEngineAvailable(), so a DORY_HV_ENGINE=0 opt-out and a missing/partial engine install both slipped through as 'supported' and then surfaced as a misleading 'engine could not start'. hostSupport now reports capable-but- unavailable honestly (via the engineAvailable param, default hvEngineAvailable()), so the app falls back to a Docker-compatible engine instead. Covered by a new test. - PortForwarder.unexpose dropped a port from its tracked set even when the gvproxy call failed, leaking a stale host forward with no retry. Removal is now gated on success, mirroring the expose path, so a transient failure is retried on the next tick.
A fresh Dory download now includes everything: the k8s features shell out to a bundled kubectl and the optional docker context uses a bundled docker CLI, so users install nothing beyond the app. New HostTools resolves the bundled Contents/Helpers copy first, falling back to a system install for dev builds. bundle-engine.sh fetches the darwin/arm64 kubectl (stable) and the static docker CLI and signs them into the bundle. Verified both download URLs resolve; DoryTests green. With this, dory-hv (engine) + gvproxy (networking) + kernel + kubectl + docker are all in the app; only the docker:dind image is fetched once on first launch (the OrbStack model).
… pull) Adds a --rootfs flag to the engine and a bundledRootfs config field. When an offline build ships a decompressed engine rootfs (via the app's prepareCompressedResource on dory-engine-rootfs.ext4.zst), the engine installs it as the pristine rootfs on first launch instead of fetching docker:dind over the network. Online builds omit the resource and fetch once as before.
…P, binfmt; guest kernel+agent Track 0 (vsock + guest agent), Track 1 (virtio-fs, FUSE server, DAX window + coherence probe), Track 3 (usbip + IOUSBHost shim), Track 5 (DirectIP bridge, machine port watcher). DAX flag-gated via --share :dax; dory-hv daxprobe proves hv_vm_map file-backed coherence. Rosetta opt-in on shared vz engine.
…r bridges, DAX/Rosetta UX Track 2 (recipe schema + RecipeStore + SSHConfigWriter + catalog), Track 3 UI (UsbDevicesView + UsbAttachmentStore), Track 5 (TunRouter), Track 6 (CredentialBridge, ExposeTunnel). Recipe tilde-mount expansion fix; USB replay idempotency; credential fail-fast logging; DockerShim shared stdcopy frame; amd64 machines point users to dory vm --rosetta.
…, usb matrix, gpu research
…binary Audit findings: SSHConfigWriter was built+tested but never called, so ~/.dory/ssh/config (for 'ssh <machine>' + VS Code Remote-SSH) was dead code — now regenerated on every machine refresh. Also untrack + gitignore the stray 4.4MB guest/agent/agent Mach-O build artifact.
…fig to build+boot Discovered by actually building the guest kernel and booting a real guest: - dory.config lacked a serial console (PL011) + devtmpfs, so a from-source kernel booted blind; and defconfig pulled in GPU/DRM (nouveau) that fails to build. Disable GPU/sound/media/wlan/fb; add PL011 console + devtmpfs. - Recorded fio 4k randread / dd write / git status: Dory virtio-fs works end to end; write+metadata competitive with OrbStack, random-read ~5.6x behind (motivates DAX + off-vCPU-thread FUSE I/O). DAX guest-mount not yet active (host hv_vm_map proven via daxprobe; guest dax_device handshake deferred).
…g off vCPU thread Root cause of the ~6x file-sharing gap vs OrbStack: the guest page cache was never retained, so every read (even a re-read) took a full FUSE round-trip. Two attr-reply bugs combined under FUSE_AUTO_INVAL_DATA: LOOKUP returned attr_valid=0 (revalidate every access), and every attr carried mtime_nsec=0, so an unchanged host file looked modified on each revalidation and lost its cache. Fix: 1s attr_valid/entry_valid window + real host atime/mtime/ctime nsecs (HostFSAttributes gains *Nsec, encodeAttr emits them). Keep AUTO_INVAL_DATA — with correct nsecs it invalidates only on genuine host changes. Result (fio 4k randread, cache-resident): 18k -> 1,075k IOPS, vs OrbStack 1,011k (1.06x ahead). Raw cache-bypassed plane is 18.8k vs 223k = per-request latency, the DAX/passthrough target (not the plain-path gate). Also: move FUSE request handling off the vCPU thread into a bounded drainer pool (a blocking pread in the MMIO exit froze the guest and collapsed deep queues to depth 1); out-of-order completion; @unchecked Sendable on the manually-synchronized virtqueue/transport primitives. And FUSE_MAX_PAGES so the advertised 256-page request size is honored (guest was clamping to 128KiB).
FUSE READ used to alloc a read buffer, build a response array, then copy it back into guest memory. Now preadv scatter-reads the payload straight into the guest's device-writable descriptor segments and writes the fuse_out_header in place, with an array-path fallback when the first segment can't hold the header. Raw single- stream 4k randread 16.7k -> 20.1k IOPS (~20%); removes ~10us Swift per-request overhead and the allocator pressure across concurrent workers. Test proves the zero-copy output is byte-identical to the array path.
…beats OrbStack) DAX collapses the per-4k-read guest<->host round-trip: file pages are mapped straight into the guest via hv_vm_map, so reads are direct memory access with zero FUSE traffic. Three bugs blocked it, found by booting a real guest: 1. guest/kernel/dory.config: FUSE_DAX silently dropped because the FS_DAX -> DAX -> ZONE_DEVICE chain wasn't enabled (defconfig leaves ZONE_DEVICE off, so olddefconfig disabled FUSE_DAX even though set =y). Pull in ZONE_DEVICE + FS_DAX + deps. The guest now compiles DAX and queries the SHM window. 2. FuseSetupMappingIn was 48 bytes; the real fuse_setupmapping_in is 40 (5x u64). Every guest SETUPMAPPING was rejected as a short frame -> EINVAL. 3. virtio-fs sends fh=-1 for inode read-only mappings (resolve by node id), and Apple's hv_vm_map rejects a PROT_READ-only host region (HV_ERROR) -> map the host read-write while the guest stage-2 stays read-only. Read-only files degrade to plain FUSE reads. fio 4k randread, cache-bypassed: plain 20.1k -> DAX 1,074k IOPS (OrbStack 223k). Cached 1,186k; concurrent-16 4,147k. 110 engine tests pass.
…r leave stale data) The setattr handler accepted mode no-ops but silently dropped FATTR_SIZE, so truncate() and O_TRUNC opens (atomic_o_trunc is not negotiated, so the guest relies on a separate SETATTR size=0) returned success without touching the host file, leaving a stale tail after an overwrite. Add HostFS.truncate (by node id and by open handle) and wire FATTR_SIZE through, returning the post-truncate attrs. Affects both plain virtio-fs and DAX writes. Verified end-to-end on a real dory-hv guest with -o dax=always: O_TRUNC (41->3 bytes), ftruncate shrink (16->5), ftruncate grow (3->100, NUL-padded), extending write (0->4MiB), create+write, in-place overwrite, and write/read coherency all persist correctly to the host. +2 unit tests.
…th no flags File sharing was never wired into the app engine: SharedVMProvisioner passed no --share, so `docker run -v ~/project:/app` had nothing to bind. Share the user's home at its identical guest path (home=$HOME:rw:at=$HOME) so macOS paths resolve transparently in containers, the OrbStack 'just works' default. Add a guestMountPoint to the share config (:at=/guest/path) so a host dir can mount at its real path instead of /mnt/dory/<tag>; also fold dax=always into the engine's guest mount options. Plain virtio-fs (no DAX) is the default: it matches OrbStack on realistic cache-resident workloads and has none of DAX's window-thrashing or read-only-file caveats, so it is safe across the whole home tree. Verified in a real dory-hv guest: a host dir mounted at /Users/dorytest read a host file and a guest write-back persisted to the host. +4 unit tests.
…se blocker) USB passthrough is not wired end-to-end: components exist + are unit-tested (discovery, IOUSBHost I/O, codec, dispatcher, guest vhci writer, UI, CLI) but nothing instantiates the usbip server, there is no vsock data channel, and the guest agent can't produce the connected fd vhci needs. Add a code-grounded, adversarially-verified completion plan (3.6a-n): guest-dials/host-listens data channel, VsockConnection EOF signal for cleanup, host UsbipBridge serve loop, engine control plane, entitlement scope, readiness reconcile, loopback test, and the COMPATIBILITY.md over-claim fix. Mark 3.1/3.2/3.4 done.
The server handles FUSE_READDIRPLUS but not plain FUSE_READDIR, and never advertised DO_READDIRPLUS — so the guest sent READDIR, got ENOSYS, and every `ls` of a shared directory returned empty (lookup/cat still worked). Force readdirplus (without READDIRPLUS_AUTO, which would let the kernel fall back to the unhandled plain readdir). Verified in a guest: a shared dir now lists.
The auto-share exposed all of $HOME read-write to every container, including ~/.ssh, ~/.aws, ~/.gnupg, ~/.kube, shell rc files, and ~/Library — a real key exfiltration / rc-poisoning risk (exactly what launch feedback warned against). HostFS gains hiddenNames: a hidden name fails lookup as not-found and is omitted from listings, so no node id is ever minted for it (no read/open/write path). The home share now passes :safe (VirtioFSShareConfiguration.sensitiveNames). Interim defense-in-depth; the stronger guarantee is per-bind-mount on-demand sharing (shim create-interception + engine authorization channel) — a tracked follow-up. Verified in a guest: ~/.ssh/~/.aws/~/.zshrc invisible, ~/project reads+writes+lists. +HostFS/parse/engine-args tests.
…e passthrough) dory vm --devices attaches a VZXHCIController with no host device, so the ✅ 'USB passthrough delivered' row was inaccurate — 'USB controllers attached: 1' confirms a controller, not a passthrough. Split audio (real, VZVirtioSoundDevice) from USB device passthrough (🚧, real path is usbip Track 3.6). Fix the section header's 'all four delivered' framing accordingly.
…ort from OrbStack') Import auto-selected a single source via DockerEngineRuntime.detect(), which returns the highest-priority responding docker socket. With Docker Desktop also installed, /var/run/docker.sock outranked OrbStack and there was no way to choose it — so 'import from OrbStack' silently used the wrong engine or showed nothing. - Enumerate ALL present source engines, labeled by vendor (DockerSourceEngine + availableSources/engineLabel), and add a source picker to the Migrate panel. - Preflight + import use the selected source (probed to confirm it's reachable). - Surface MigrationSummary.failures in the UI instead of a bare success count. - Defensively exclude ~/.dory/engine.sock too (not just dory.sock) so a migration can never pick Dory as its own source. Note: the earlier hypothesis that the 'dory' context self-selected engine.sock was wrong — the context points to dory.sock, already excluded (verified). The real gap was the missing source choice. +6 discovery tests.
…,c,f,m) The missing host-side glue: UsbipBridge pumps one guest usbip vsock connection against one claimed device via UsbipServer — OP_REQ_IMPORT handshake, then the CMD_SUBMIT/UNLINK loop with correct 48-byte-header + OUT-payload framing, on its own queue so a blocking device submit never stalls the vsock queue. Ends and releases the device on peer close. Adds VsockPorts.usbip (1025) and a VsockConnection.isPeerClosed EOF signal so the long-lived bridge can tell 'idle' from 'guest gone' (read returns 0 for both) — without it a claimed device leaks on guest reboot. Makes InProcessConnection thread-safe (bridge reads off the vsock queue). Loopback integration test proves import + submit round-trip + unknown-busID rejection + EOF cleanup, no hardware. 21 tests green.
…ke (Track 3.6d,e) attachUSB previously required a caller-supplied socket_fd, which can't work — the fd vhci needs must live in the guest's own fd table. Now when no fd is given it dials VMADDR_CID_HOST on vsock_port, performs OP_REQ_IMPORT (drains the 312-byte descriptor), and hands that guest-owned fd to vhci; the socket_fd path stays a test seam. Closes the fd after the vhci write (kernel dups it) to avoid a leak. connectVsock/usbipImport/closeFD live in the linux file with darwin stubs so the dev-host build stays green; the 40-byte OP_REQ_IMPORT encoder is extracted and unit-tested. Builds linux/arm64 + darwin; existing socket_fd seam tests green.
…Track 3.6f,g) UsbipManager holds the set of claimed host devices by busID and registers the usbip listener on VsockPorts.usbip in EngineMode: an accepted guest dial is served by a UsbipBridge backed by whatever devices are registered, and the guest's OP_REQ_IMPORT busID picks which one. The listener is a no-op until the control plane claims a device, so it is always safe to register. Bridge refactored to a multi-device UsbipServer (busID read from the wire) with a single-device convenience init. 124 engine tests green.
…ansfer smoke test Claims a real host device via HostUsbDeviceFactory.open and drives one GET_DESCRIPTOR through the exact usbip submit path the guest uses — a host-side hardware smoke with no guest/VM. Validated on real hardware (SanDisk mass storage, busID 1-2): the claim code path executes correctly and hits macOS's security gate exactly as documented — .capture needs the restricted com.apple.vm.device-access (ad-hoc binaries carrying it are SIGKILLed at launch, exit 137), and .userAuthorized IOServiceAuthorize returns notPermitted (0xe00002c6) for a driver-owned device from a headless CLI. Confirms v1 scope: driverless devices via a properly-authorized app; capture-class needs the notarized entitlement.
…er+notify (Track 3.6h,i,j) Completes the USB last mile. UsbControlHandler (engine-side): on attach it claims the host device (HostUsbDeviceFactory.open), registers it with the UsbipManager, allocates a vhci port, and tells the guest agent to dial+vhci-attach — rolling the claim back if the guest notify fails so the device returns to macOS. detach notifies the guest, unregisters, frees the port. UsbControlServer serves this over a unix socket (~/.dory/hv/usb-control.sock, newline-JSON), wired in EngineMode with real deps (device open + AgentChannel usb.attach/detach). dory-hv usb attach/detach are the client; scripts/dory routes through them (mode defaults userAuthorized; capture needs the notarized entitlement). Handler logic + codec fully unit-tested (claim/register/notify, distinct-port allocation, duplicate reject, rollback-on-guest-failure, detach, round-trip codec). Client verified against a down engine. 130 engine tests green. Only the real device-claim + guest dial remain hardware-gated.
… via proxy, native Intel later Make the platform split explicit and correct the stale 'macOS 26 (Tahoe)' floor (the shared VM needs macOS 15). Apple silicon = full self-contained experience; Intel = native front-end for a separately-installed Docker engine, with a native Intel engine (Virtualization.framework) planned for a later update.
The DNS/HTTP/HTTPS proxy ports were hardcoded constants (15353/8080/8443) and the *.dory.local domain feature started unconditionally — 8080 collides with common dev servers, and MDM-managed DNS can't be pointed at Dory, making domains unusable rather than optional. Promote the ports to UserDefaults-backed settings, add a domainsEnabled toggle (gates startLocalNetworking), and applyNetworkingSettings() does a clean teardown+restart so listeners rebind. New Network settings panel: domains on/off + three port fields. Also corrected the settings copy that overclaimed certs are 'trusted system-wide' (the system-trust install is not wired). Published ports stay on localhost regardless. Builds.
…imit (#3) #4: k3s runs its own embedded containerd, separate from the shared dockerd store, so 'built images immediately usable in Pods, no registry push' was false (→ ImagePullBackOff). Correct both claim sites (KubernetesProvisioner doc + the Kubernetes empty-state) to say built images must be pushed to a registry or imported into k3s. Real auto image-sync is a tracked follow-up. #3: document that heavy amd64 images (SQL Server, Oracle, AVX-heavy) can segfault under qemu-user on the default engine — a qemu-user limit, not a Dory bug — and point to the Rosetta fast path (dory vm --arch amd64 --rosetta). Rosetta can't run on dory-hv (raw Hypervisor.framework), so this is a documented limitation.
The single-container VZ path hardcoded a 1 GB ceiling, which silently blocks heavy amd64 images (SQL Server refuses to start under 2 GB) regardless of Rosetta. PROVEN on real hardware: with DORY_VM_MEM_MB=4096, SQL Server 2022 (linux/amd64) starts under Rosetta and reaches 'ready for client connections' on port 1433 in ~4s — where qemu-user segfaults (signal 11). Rosetta emulates AVX2 + the atomics SQL Server needs; qemu-user does not.
…ver) Adds a manual Settings toggle that switches the shared engine from dory-hv to the Virtualization.framework engine with Rosetta, so heavy amd64 images run reliably — SQL Server 2022 (linux/amd64) was PROVEN to reach 'ready for client connections' in ~4s under Rosetta on real hardware, where qemu-user segfaults. - Config.rosettaX86 (persisted, UserDefaults) routes provision() to a new provisionWithRosettaEngine: launches dory-vm --shared-engine + DORY_ENGINE_ROSETTA=1 + 3 GB (SQL Server needs >2 GB), publishing to the same engine.sock so the shim and docker context are unchanged. Bundled dory-vm helper + dory-vm-kernel/initfs. - AppStore.setRosettaX86 restarts the engine on toggle; Engine settings gets the toggle (honest memory-tradeoff copy). Off = dory-hv + the memory advantage. Mechanism proven end-to-end manually; the same VZ engine also unlocks the future Intel self-contained track. Docker Desktop's model. App builds, 28 tests green.
…rename Extend the :safe hiddenNames gate beyond lookup/readdir: requireVisible() now guards every entry-creating or entry-removing op, so a container can't create, delete, or rename a hidden name (e.g. drop a new ~/.ssh/authorized_keys or clobber a hidden path) before the host is touched. Plus small USB test refinements. Engine 131 tests pass.
…nner host death) The gate's retry only fired when too few tests ran, so a PARTIAL host death (one xctest worker crashes, its tests report 'failed' at 0.000s while 300+ others pass) tripped the unexpected-failures check and exited 1 without retrying, turning an infra flake into a red check. Now retry on any non-clean attempt and fail only if failures persist across both. Real failures still fail (they reproduce); host deaths get their intended second chance. Full DoryTests suite passes locally (525 tests, 0 failures).
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What this does
Builds out Dory's VM platform (dory-hv guest stack) and fixes the issues reported after launch. It spans file sharing, USB passthrough, x86/Rosetta, a security fix, and the four open GitHub issues. 30 commits, all building green (engine 130 tests, app RuntimeSupport tests, Go agent tests).
Highlights
File sharing (virtio-fs)
attr_valid+mtime_nsec=0underFUSE_AUTO_INVAL_DATA). After the fix, plain virtio-fs matches OrbStack on realistic workloads.hv_vm_map: 1,074k IOPS on cache-bypassed 4k random read, ~4.8x past OrbStack. Opt-in via:dax.docker run -v ~/project:/appresolves with no config ($HOMEshared at its identical guest path).SETATTRnow honorsFATTR_SIZE, sotruncate/O_TRUNCno longer leave stale data. Verified in a real guest.FUSE_DO_READDIRPLUSadvertised so shared directories actually enumerate (lswas returning empty).Security
:safedenylist on the$HOMEauto-share hides credential stores and shell rc files (~/.ssh,~/.aws,~/.kube,Library, ...) from every container. This closes the risk that the whole home tree was exposed read-write. On-demand per-mount sharing is the tracked follow-up.USB passthrough (usbip-over-vsock, Track 3.6)
UsbipBridgeserves URBs against a claimedHostUsbDevice, and an engine control plane (dory usb attach/detach) claims the device, registers it, and triggers the guest dial. Loopback integration test proves the chain without hardware.com.apple.vm.device-accessentitlement + a notarized build; driverless devices (microcontrollers, serial) work withIOServiceAuthorize.x86 / Rosetta (GitHub #3)
GitHub issues fixed
*.dory.localcan be turned off (for MDM-managed DNS).Platform note
This release's own engine (dory-hv) is Apple silicon only; on Intel, Dory runs as a native front-end for an installed Docker engine. README + website updated.