perf(vk_native): chain renderer atlas-blit → pre-DP submit via semaphore

[leaiss]

Summary

Tier 3 #10 (partial): eliminates the first of two per-frame CPU stalls in the in-process vk_native compositor by chaining the renderer's atlas-blit submit and the compositor's pre-DP submit via a binary semaphore instead of a vkQueueWaitIdle between them. The second stall (pre-DP → DP self-submit) requires a DP vtable change and is deferred.

Before

renderer.draw():
    vkQueueSubmit(atlas blit)
    vkQueueWaitIdle             ← CPU stall A (eliminated by this PR)
    free cmd buffer

compositor frame loop:
    vkQueueSubmit(pre-DP cmds: WS-layer composite + atlas crop)
    vkQueueWaitIdle             ← CPU stall B (deferred — needs DP vtable change)
    display_processor.process_atlas(NULL cmd)  // self-submits internally

After

renderer.draw():
    vkQueueSubmit(atlas blit, signal=frame_done_sem, fence=frame_done_fence)
    return  // no waitIdle, no cmd buffer free

compositor frame loop:
    vkQueueSubmit(pre-DP cmds, wait=frame_done_sem @ TRANSFER|FRAGMENT_SHADER|COLOR_ATT)
    vkQueueWaitIdle             ← stall B still present
    display_processor.process_atlas(NULL cmd)

The CPU now skips stall A entirely. Stall B remains until the DP gets a wait-semaphore parameter on its self-submit path — that's a separate plug-in-coordinated PR.

Implementation

Renderer (comp_vk_native_renderer.{h,c})

• frame_done_sem + frame_done_fence allocated at create time. Fence starts signaled so the first draw() doesn't stall waiting on a non-existent prior submit.
• draw() waits on the fence at the start (back-pressure from previous frame), frees pending_cmd from previous frame, resets fence, records & submits, defers the cmd-buffer free.
• A defensive drain at the top of draw() consumes a stuck previous-frame signal in the rare case (downstream submit failed, or resize() ran between frames) — avoids the binary-semaphore double-signal UB.
• New public accessor comp_vk_native_renderer_take_frame_done_semaphore(): returns the handle once per draw, then returns 0. Lets the compositor consume the signal at whichever per-frame submit site fires first.

Compositor (comp_vk_native_compositor.c)

• Static helper vk_native_wire_renderer_wait(c, &submit_info, &sem_storage, &stage_storage) called at each per-frame vkQueueSubmit site. Wait-stage mask is TRANSFER | FRAGMENT_SHADER | COLOR_ATTACHMENT_OUTPUT — the union of stages where downstream paths read the atlas. The take-once-per-frame semantics in the accessor mean only one submit per frame actually waits; the rest get a 0 handle and are no-ops.
• Wired into all five per-frame submit sites: shared-image path (pre-DP, post-DP, fallback) and target/window-present path (pre-DP, final).
• The one-shot debug vk_native_capture_atlas_to_png helper keeps its vkQueueSubmit + vkQueueWaitIdle — not per-frame, not worth the state.

Correctness

The change is functionally equivalent to the prior vkQueueWaitIdle pattern:

	Pre-change (waitIdle)	Post-change (semaphore)
Execution ordering	Submission order on a queue is guaranteed; waitIdle is over-sync	Submission order + semaphore wait at correct stages
Memory visibility	Implicit via full GPU drain	Explicit via wait at TRANSFER/FRAGMENT_SHADER/COLOR_ATT stages
Cmd buffer lifetime	Freed immediately after waitIdle	Deferred to next draw() start, gated on fence
First-frame behavior	Same	Fence created signaled → first wait is a no-op
Zero-copy path	renderer.draw() skipped, no waitIdle needed	renderer.draw() skipped, accessor returns 0, no wait wired
Resize race	renderer.draw() doesn't run during resize	Defensive drain at next draw() handles unclaimed signal

Validation

• Windows runtime build: clean (scripts\build_windows.bat build).
• macOS + Android builds: running via CI.
• Hardware perf measurement: deferred. Emulator can't reach the frame loop (Vulkan device-extension wall at xrCreateVulkanDeviceKHR). Windows test apps default to service mode which routes around in-process vk_native compositor. Real perf delta will land when Lume Pad arrives or someone runs the macOS test apps with sim_display.

Test plan

• CI: Windows + macOS + Android builds all green.
• On Lume Pad (post-arrival): cube_handle_vk_android renders cleanly with no validation errors and no visible regression. Perfetto/RenderDoc trace shows the renderer submit + pre-DP submit chained on a single queue with no CPU gap.
• On macOS: cube_handle_vk_macos renders the cube as before.

Follow-up

Tier 3 #10 part 2: extend xrt_display_processor vtable with get_self_submit_wait_semaphores (or accept a wait sem on process_atlas). Plug-in side: CNSDK plug-in honors the wait. Eliminates stall B. Coordinated runtime + plug-in PR; better measured on hardware with real frame timings.

🤖 Generated with Claude Code

[leaiss]

Self-audit pass surfaced two bugs in the prior commit, fixed in 8ad6de5f2:

1. Fence-reset deadlock. vkResetFences ran at the top of draw() before vkAllocateCommandBuffers. If alloc (or any early-return) fired, the fence was left unsignaled with no submit to signal it → next draw() blocks on vkWaitForFences forever. Fix: defer fence reset to just before the main vkQueueSubmit so all early-return paths leave the fence in its prior signaled state.

2. Drain-failure double-signal risk. The semaphore-drain block cleared signal_pending unconditionally, even on vkQueueSubmit failure. With signal_pending=false but the binary semaphore still in the signaled-pending state, the next main submit's signal operation is UB per Vulkan spec. Fix: only clear signal_pending when the drain submit succeeded; on failure log + return XRT_ERROR_VULKAN.

Builds clean on Windows; CI re-running.