Epic: shared helper module (displayxr-common) + unified capture API — kill the 6× Kooima/common drift

[dfattal]

Why this epic

The same native helper code — above all the Kooima off-axis projection math (display3d_view.{c,h}, camera3d_view.{c,h}, leia_math.h) — is vendored by copy in at least six places across the org and is already drifting (the clip-plane fix in #393 had to be hand-applied to one copy with no way to sync the others). The #389 work (VK native compositor now reliably composites many window-space layers; new windowspace_handle_*_win test apps) turns the window-space-layer + HUD + input scaffolding into a genuinely reusable building block too. This epic makes the shared code a versioned dependency so editing-a-local-copy becomes structurally impossible.

#393 originally scoped this to the two C++ demos and excluded the engines. That was too narrow: the engine plugins compile the same math C files. This epic supersedes that framing; #393 is re-scoped to be layer 2 (the C++ scaffolding) of the design below.

Drift inventory — every current copy

Copy location	Repo	Form	Notes
test_apps/common/	displayxr-runtime	C/C++	this repo's native test apps
common/	displayxr-demo-modelviewer	C/C++	Vulkan [0,1] projection fix landed here
common/	displayxr-demo-gaussiansplat	C/C++	+ unmerged feat/zdp-clip-soft-fade-pick WIP (near/far, selftest, center_view)
include/	kooima-projection	C	the intended math home — seeded then abandoned (last push 2026-05-04)
Source/DisplayXRCore/Private/Native/	displayxr-unreal	C	compiled into the UE module (see ADR-003 UE-native off-axis projection)
native~/ (display3d_view.h, displayxr_kooima.{h,cpp})	displayxr-unity	C++ → P/Invoke	wraps the same Kooima math (see ADR-006 window-relative Kooima)

~20 of the ~28 common/ files are byte-identical between the two demos; nearly all divergence is accidental.

Architecture — one repo, two CMake targets (no new repo)

Repurpose the dormant kooima-projection repo → rename to displayxr-common, exposing two targets so the math core is shared by everything while engines never link the C++/Win32 scaffolding:

displayxr-common   (renamed from kooima-projection — net new repos: 0)
  ├─ displayxr::math     pure C: display3d_view, camera3d_view, leia_math — ZERO deps
  │     └─ linked by: displayxr-unreal (UE module), displayxr-unity (native~)   ← engines, math only
  └─ displayxr::common   C++ scaffolding (HUD/D2D, input/Win32, window mgr,
        xr_session_common, window-space-layer UI, stb, view_params,
        manifest cmake) — depends on displayxr::math
        (NOTE: atlas/screenshot capture is NOT shared here — it is removed in
         favour of a runtime API; see W6.)
        └─ linked by: runtime test apps, modelviewer, gaussiansplat            ← C++ apps

Consumed via CMake FetchContent pinned to a tag (same pattern already used for tinygltf/glm/OpenXR-loader). Local co-dev via FETCHCONTENT_SOURCE_DIR_DISPLAYXRCOMMON=../displayxr-common.

Why one repo, not two: FetchContent clones the whole repo regardless of target, so a separate math repo wouldn't make engines pull less source — the real link-level isolation comes from the displayxr::math target carrying no transitive Win32/D2D deps. A second repo would only earn its keep if the math needed an independent release cadence or had an external (non-DisplayXR) consumer; neither holds today. One repo = one CI, one tag, one branch-protection/CODEOWNERS/versions.json entry.

Divergence policy: mechanism in the lib, policy at the call site. Parameterize (e.g. display3d_compute_view(..., near_offset, far_offset, ...) outputs both the projection matrix and resolved near_z/far_z — modelviewer uses hardware clip, gauss feeds software cull); inject renderer-specific bits via callback; keep genuinely app-local files app-local (e.g. stb_image_impl_macos.cpp). Never #ifdef MODELVIEWER / #ifdef GAUSS inside the lib.

Workstreams & ordering

W1 — Reconcile the math core (blocking prerequisite; highest drift risk).
Land gauss's feat/zdp-clip-soft-fade-pick, then unify display3d_view.{c,h} + camera3d_view.{c,h} into one superset: Vulkan [0,1] projection + near_offset/far_offset API + near_z/far_z outputs + Unity's window-relative Kooima (ADR-006) + any Unreal delta. Validate against each consumer before landing.

W2 — Stand up displayxr-common. Rename kooima-projection; add the displayxr::math target seeded from the reconciled core; add Win+Mac CI on a tiny consumer harness; tag v0.1.0 (math only).

W3 — Adopt the math core everywhere (engines included). Replace each vendored math copy with the pin:

• displayxr-runtime test apps (*_handle_*_win, windowspace_handle_*_win)
• displayxr-demo-modelviewer
• displayxr-demo-gaussiansplat
• displayxr-unreal (UE Build.cs references the fetched C core instead of Private/Native/)
• displayxr-unity (native~ build compiles the fetched C core)

W4 — Extract the C++ scaffolding layer (this is re-scoped #393). ✅ DONE 2026-06-06. Reconciled hud/input/xr_session deltas into the lib superset; added the single platform-gated displayxr::common target (displayxr-common#5); migrated the three C++ consumers (runtime #462, gauss #34, modelviewer #24 — each common/ deleted down to a thin shim + app-local leia_math.h, ~45k duplicated lines removed). Tagged v0.3.0 (the planned v0.2.0 was taken by W3's Layer-1 work). All user-verified on the Leia display. Details: W4-complete comment.

W5 — Release discipline. ✅ DONE 2026-06-05/06. displayxr-common tags disciplined since v0.1.0 (now v0.3.0); consumers bump the pin on their own cadence. The drift-guard went from optional to shipped: no-vendored-math.yml in all 5 consumers (math filenames), extended 2026-06-06 in the 3 C++ consumers to also catch re-vendored scaffolding filenames.

W6 — Replace app-side capture with a runtime API (supersedes sharing atlas_capture via the lib).
The I-key / Ctrl+Shift+C screenshot is reimplemented 6× today — 5 per-API readbacks in test_apps/common/atlas_capture_{d3d11,d3d12,gl,vk,metal} (forked byte-for-byte into both demos' common/), plus Unity Runtime/DisplayXRScreenshot.cs and Unreal DisplayXRAtlasCapture.cpp. Rather than fold atlas_capture into displayxr::common, the runtime gains one official xrCaptureAtlasEXT (new XR_EXT_atlas_capture, PROJECTION_ONLY / POST_COMPOSE flag) and every app deletes its readback. Only the platform flash-overlay + filename-numbering UX stays app-local. Full design + per-repo deletion list: docs/roadmap/unified-atlas-capture.md.

• Runtime: XR_EXT_atlas_capture + in-process dispatch (mcp_capture / u_capture_intent) — Phase 1, landed in PR feat(oxr): XR_EXT_atlas_capture / xrCaptureAtlasEXT (in-process) [#396 W6] #397 (non-blocking latch; verified e2e on D3D11)
• Runtime: IPC dispatch (per-session capture) — Phase 2, PR feat(capture): per-session xrCaptureAtlasEXT over IPC + numbering fix [#396 W6 2] #398. PROJECTION_ONLY captures the calling client's own crop atlas (service-side); internal IPC_CAPTURE_FLAG_PROJECTION_ONLY (no workspace spec bump). POST_COMPOSE over IPC = combined/workspace atlas only.
• displayxr-runtime test apps (Windows): all cube_handle_*_win migrated to xrCaptureAtlasEXT behind a shared dxr_capture::RequestRuntimeAtlasCapture helper — feat(oxr): XR_EXT_atlas_capture / xrCaptureAtlasEXT (in-process) [#396 W6] #397 (d3d11) + refactor(test-apps): migrate Windows cube_handle capture to xrCaptureAtlasEXT [#396 W6] #400 (d3d12/gl/vk; per-API fallbacks dropped). Deleted dead atlas_capture_d3d11.cpp + atlas_capture_d3d12.cpp.
• test apps (macOS): cube_handle_{metal,gl,vk}_macos migrated to xrCaptureAtlasEXT — refactor(test-apps): macOS cube_handle capture → xrCaptureAtlasEXT + mip parity [#396 W6] #405 (rebased on refactor(test-apps): migrate Windows cube_handle capture to xrCaptureAtlasEXT [#396 W6] #400, merged; macOS CI green). As the second-merger it deleted the last readback TUs, so all 5 per-API readbacks are now gone (atlas_capture_{d3d11,d3d12,gl,vk}.cpp + _metal.mm); also added metal/vk wood-texture mip-chain parity (matching fix(#396 W6): GL/VK in-process capture parity + consistent cube test-app rendering #403).
• displayxr-demo-gaussiansplat: migrate; delete forked common/atlas_capture* (PR Native D3D12 compositor #27, merged; Win+macOS CI green, Win capture verified e2e)
• displayxr-demo-modelviewer: migrate; delete forked common/atlas_capture* (PR Upstream Monado contribution: feature extraction tracker #18, merged; Win+macOS CI green, Win capture verified e2e)
• displayxr-unity: gut DisplayXRScreenshot.cs readback + on-demand re-render; extension opt-in; keep flash (editor preview-session path out of scope — no runtime session). Not native~/displayxr_readback.* (that is macOS offscreen-preview readback, unrelated)
• displayxr-unreal: gut ProcessRequest_RenderThread readback; extension opt-in; keep flash; repackage DisplayXR_5.7
• (optional) shell: adopt PROJECTION_ONLY for a pre-HUD variant
• extensions/docs: header auto-sync to displayxr-extensions; spec doc; promote the roadmap doc → ADR on acceptance

Scope note: W6 expands this epic beyond the original Kooima/common math dedup — it adds a runtime OpenXR extension and engine capture reimplementation (the engines' capture code was never in the math drift inventory). Folded here per the shared common/ surface and migration coordination; the runtime extension itself is independently releasable and need not gate on W1–W5.

W7 — RAW inputs / rig generators / render-ready output (spike → design → implement).
A design-first workstream that reshapes how view geometry is served, and changes what displayxr::math is for. Design doc WRITTEN 2026-06-06: docs/roadmap/raw-vs-render-ready-views.md (e99f0c39b + terminology/per-view pass 7710d5bd6); model summary:

• Output is a fixed point: render-ready = standard XrView { pose, skewed XrFovf }, already converged on the plane — rig-agnostic. Legacy/non-aware apps and the runtime's weaver consume exactly this.
• Rig is an input-side generator, not an output field. Display-centric and camera-centric are two paradigms that route raw inputs → the same output shape. They live in displayxr::math (re-roling W1: display3d_view = display-centric rig, camera3d_view = camera-centric rig — two intentional generators, not two drifting copies). Future rigs = new generators, no API/output change.
• Raw = the generator inputs: eye positions, display-plane pose, effective canvas rect (handle = window, texture = subrect — no app-class branching), timestamp_ns, is_tracking. The escape hatch for any future modifier (IPD/parallax/convergence/ortho/clip) the runtime can't anticipate — promise complete inputs + a stable output, predict nothing.
• Runtime links displayxr::math to compute render-ready, so "runtime's render-ready" ≡ "aware app's own computation from raw under the default rig" by construction — kills the "runtime view ≠ my view" bug class.

Session findings (code as of 2026-06-02 — verified, with file:line):

• The runtime IS the suspected 7th copy. src/xrt/auxiliary/math/m_camera3d_view.{c,h} + m_display3d_view.{c,h} + m_multiview.{c,h} are FOV-only xrt-typed ports of app-side test_apps/common/camera3d_view.c / display3d_view.c (header: "Runtime-side port (xrt types, FOV-only — no matrices)"); consumed by oxr_session.c (#include "math/m_camera3d_view.h" :42). Hand-synced → real drift. ⇒ displayxr::math must expose two layers: FOV-only (runtime consumer) and FOV+matrix (app consumers).
• Render-ready already ships — as a full TWO-rig system with live modifiers, default camera-centric @ 0.5 D (CORRECTED — the earlier "always display-centric" note was wrong). oxr_session.c:1384-1426 branches on view_state.camera_mode: camera3d_compute_views() (camera-centric) vs display3d_compute_views() (display-centric), mirrored server-side at ipc_server_handler.c:~504. Default is camera-centric, cam_convergence = 0.5 D / 2 m (qwerty_device.c:642,647) — i.e. the intended legacy default already holds. External-window apps (handle/texture, real HWND) are forced display-centric via the !sess->has_external_window gate (window = portal). WebXR-over-IPC gets render-ready (bridge runs on d3d11_service, so the normal path — not the narrow xc==NULL headless raw fast-path at ipc_server_handler.c:415-435).
• The rig + tunables are driven ONLY by the qwerty debug device, not an app API — that is the actual W7 gap. qwerty_view_state (camera_mode + cam/disp spread/parallax/convergence, qwerty_device.h:51-60) is read by the compositors (comp_gl:872, comp_metal:1238, comp_multi_system:1925) and fed into the view computation. Rig toggle = P key (qwerty_win32.c:546-549 / qwerty_macos.m:431-434 → qwerty_toggle_camera_mode, qwerty_device.c:1066); convergence/spread on other keys. Apps cannot select rig or set convergence/IPD through OpenXR today.
• Measured-vs-predicted is already plumbed at the DP. Single accessor xrt_display_processor::get_predicted_eye_positions() (xrt_display_processor.h:145) → xrt_eye_positions already carries timestamp_ns + is_tracking + valid (xrt_display_metrics.h:51-60); MANAGED/MANUAL (docs/specs/vendor/eye-tracking-modes.md) is the predict-vs-passthrough knob. ⇒ no new DP accessor; just surface timestamp_ns/is_tracking into the raw channel (stops at the compositor today).

Reframed W7 goal: render-ready + two rigs + modifiers + camera-centric-0.5D default already exist and are correct — but controllable only via the qwerty debug keyboard. W7 = promote that control to an app-facing OpenXR extension (app selects rig + sets convergence/spread/parallax via XrViewLocateRigEXT), add a raw channel for aware apps that bring their own generator, and keep qwerty as the dev/default driver. Not "add render-ready."

Open decisions for the design doc:

• First impl-session check — ANSWERED: cube_handle_* ignore XrView.fov in 3D mode and recompute from .pose + their own displayxr::math calls (cube_handle_d3d11_win/main.cpp:773-775); the runtime fov is consumed only in mono/legacy mode. So the rig request makes the existing render-ready output finally consumable, and the raw channel formalizes what 3D apps already treat .pose as.
• Default is settled (camera-centric @ 0.5 D, already in code). Decided: chaining a rig descriptor lifts the external-window forcing — an explicit XrCameraRigEXT is the knowledge the guard was substituting for; sessions that chain nothing keep today's behavior exactly.
• API shape decided in the doc — one extension XR_EXT_view_rig (types 1000999140-142): request = chain exactly one of XrDisplayRigEXT{pose, virtualDisplayHeight, ipdFactor, parallaxFactor, perspectiveFactor} / XrCameraRigEXT{pose, ipdFactor, parallaxFactor, convergenceDiopters, verticalFov} on XrViewLocateInfo::next; result = XrViewDisplayRawEXT (raw display-space eyes + display pose + canvas rect + sampleTimeNs + isTracking) on XrViewState::next. Descriptors carry no clip and no placement params. Rig state is per-session; qwerty = debug fallback for non-extension sessions. Open sub-questions tracked in the doc (raw chain point, clamp-vs-reject, workspace hook, tracked-vs-synthesized raw eyes).
• Make the runtime a displayxr::math consumer — decided + designed (doc section "Equivalence by construction"): type-neutral core layer in displayxr-common (own dxr_* types, zero OpenXR dep) + byte-compatible OpenXR-typed wrapper (the 5 existing pins untouched) + new xrt-typed wrapper replacing m_display3d_view/m_camera3d_view/m_multiview. Lands with the W7 implementation so the rig path is born on the shared core.

Status: design doc LANDED 2026-06-06 — implementation is a follow-up session (extension wiring + per-session state + next-chain walk in oxr_xrLocateViews + the math fold-in).

Per-repo touch list (incl. native-build changes)

Repo	Change	Native build impact
kooima-projection → displayxr-common	rename, two targets, CI, tags	—
displayxr-runtime	test apps consume math + common pin	CMake only
displayxr-demo-modelviewer	delete common/, pin	CMake only
displayxr-demo-gaussiansplat	land WIP, delete common/, pin	CMake only
displayxr-unreal	drop Private/Native/ math, pin + W6: drop DisplayXRAtlasCapture readback	Build.cs
displayxr-unity	drop native~ math copy, pin + W6: gut DisplayXRScreenshot.cs readback	native~ CMake / build-win.bat
displayxr-runtime	W6: XR_EXT_atlas_capture + delete atlas_capture_*	runtime + CMake
displayxr-extensions	W6: header auto-sync	—
displayxr-runtime	W7: fold m_multiview/m_display3d_view into displayxr::math (runtime becomes a math consumer); add raw-channel + rig-select to xrLocateViews	runtime + CMake

Decisions taken

• One repo, two CMake targets (not two repos) — avoids proliferation; link isolation via targets.
• Repurpose existing kooima-projection (net zero new repos); it already holds the math files.
• FetchContent-by-tag; independent per-consumer cadence preserved.
• Capture is removed, not shared: the atlas_capture family is deleted in favour of a runtime xrCaptureAtlasEXT (W6), so it is intentionally excluded from displayxr::common.
• The runtime is the 7th math consumer (W7): m_multiview/m_display3d_view are FOV-only ports of the app-side generators; fold into displayxr::math so render-ready ≡ app-from-raw by construction. display3d_view/camera3d_view are the two canonical rig generators, not drifting copies.

Open questions — all resolved

• Exact lib surface → settled in W4: everything goes in including the stb impl TUs and the macOS platform glue (the lib owns the only STB_IMAGE[_WRITE]_IMPLEMENTATION per platform); only leia_math.h stays app-local. Extension headers via consumer-provided DISPLAYXR_EXTENSIONS_INCLUDE_DIR.
• Versioning → settled in practice: tag per meaningful change (v0.1.0→v0.3.0), consumers pin and bump on their own cadence; no minimum-version gating needed so far.
• Engine math consumption mechanics → settled in W3: Unreal = git submodule at Source/ThirdParty/displayxr-common (UBT has no per-file source exclusion; #include shims compile the C core), Unity native~ = standard FetchContent.

Related: #393 (re-scoped to W4), #389 (window-space-layer building block), docs/roadmap/unified-atlas-capture.md (W6 capture-API design + per-repo deletion list), docs/roadmap/raw-vs-render-ready-views.md (W7 view-model design — landed 2026-06-06), displayxr-unity ADR-006, displayxr-unreal ADR-003.

[dfattal]

Session hand-off 2026-06-02 (epic scoping)

Landed:

• Title broadened to "+ unified capture API".
• W6 added (unify atlas/screenshot capture → runtime xrCaptureAtlasEXT); design doc docs/roadmap/unified-atlas-capture.md pushed to main (incl. Unity/Unreal corrections).
• W7 added as a design-first SPIKE; arch block + per-repo list + decisions updated; atlas_capture explicitly carved OUT of displayxr::common (removed, not shared).

Decisions:

• Capture is removed via runtime API, not shared via the lib.
• W7 view model: render-ready = fixed point XrView{pose, skewed fov}; rigs = input-side generators; raw = complete inputs (escape hatch).
• Progress tracking = this issue only (body checkboxes = live state; comments = session hand-off log; auto-memory = pointer). No throwaway tracker doc.

Corrected finding (important): render-ready is NOT missing — it already ships as a two-rig system (oxr_session.c:1384-1426), default camera-centric @ 0.5 D, selected by view_state.camera_mode; external-window apps forced display-centric; WebXR-over-IPC gets render-ready. Rig/tunables driven ONLY by the qwerty debug device (toggle = P key, not V). So W7's real gap = no app-facing API → expose via XrViewLocateRigEXT (request) + XrViewDisplayRawEXT (raw result), extension and qwerty writing the same view_state. Runtime is the hidden 7th displayxr::math consumer (m_camera3d_view/m_display3d_view/m_multiview, FOV-only ports).

State: scoping complete. No code written yet. docs/roadmap/raw-vs-render-ready-views.md (W7 design) still TODO.

Next: start W1 (reconcile the math core — blocking prereq) or W6 runtime (XR_EXT_atlas_capture + in-process/IPC dispatch). W7 stays design-only until its doc lands. New branch off main, no worktree; this clone has concurrent sessions — verify branch before commit, don't clobber other WIP. First W7 check: do cube_handle_* consume XrView.fov or recompute from .pose?

[dfattal]

Session hand-off 2026-06-02 (W6 runtime — in-process MVP)

Landed: PR #397 (feat/396-w6-atlas-capture-ext, open for review) — XR_EXT_atlas_capture / xrCaptureAtlasEXT wired to the in-process native compositors, plus cube_handle_d3d11_win migrated as the proof consumer.

• New header XR_EXT_atlas_capture.h (types 1000999120/121, stage enum == mcp_capture_mode); registration mirrors EXT_spatial_workspace; new oxr_capture.c; non-blocking oxr_mcp_tools_submit_capture().

Key decision / correction to the plan: the in-process path is a non-blocking latch, not the blocking mcp_capture_blocking_handler. The D3D11 native compositor runs layer_commit (and the capture poll) synchronously on the app's xrEndFrame thread — blocking there deadlocks (the poll can't run while that thread is blocked). The MCP tool only gets away with blocking because it's on a separate server thread. So xrCaptureAtlasEXT latches the intent via mcp_capture_get_installed() and returns; the readback runs at the next layer_commit. XR_SUCCESS = accepted, not "file on disk".

Verified e2e (D3D11 native, real Leia display):

• Advertised (v1) → AVAILABLE → xrCaptureAtlasEXT: resolved → PNG written, no deadlock.
• Stage flag correct: POST_COMPOSE includes the window-space HUD layer (panel on both tiles); PROJECTION_ONLY excludes it.
• Size correct: window 1280×720 × scale 0.5 = 640×360/tile → 2×1 = 1280×360 (content region, not the worst-case 3840×2160 allocation).

In-process result metadata: atlasWidth/Height, displayWidthM/HeightM, eyeWidth/Height, timestampNs from xrt_system_compositor_info; eyeLeftM/RightM and tileColumns/Rows are 0 (no in-process accessor / eye-pose plumbing — the design doc open question).

Known minor: the runtime forces a _atlas.png suffix, so the app's NextCaptureNum (scans <stem>-<N>_<cols>x<rows>.png) doesn't see captures → index stays -1, re-captures overwrite. Ties to the "Result path ownership" open question; harmless for a debug capture.

State: W6 Phase 1 (in-process) in review (#397). W1–W5, W7 untouched. No clang-format locally (CI gate unchecked).

Next: W6 Phase 2 — IPC-session routing (reuse comp_ipc_client_compositor_workspace_capture_frame + stage→flag map), XR_EXT_spatial_workspace PROJECTION_ONLY flag bump + service-side honoring, then the broader consumer migration (other test apps, demos, Unity/Unreal) + atlas_capture_* deletions. (Or pick up W1 — reconcile the math core — independently.)

[dfattal]

Session hand-off 2026-06-03 (W6 Phase 2 — IPC per-session capture)

Landed: PR #398 (feat/396-w6-phase2-ipc-capture, open for review) — xrCaptureAtlasEXT now works for IPC / service-mode sessions, capturing the calling session's own atlas, plus a capture-numbering fix and a CLAUDE.md IPC-testing note.

Key correction (live IPC test drove this): the original "2a routing reuses the workspace bridge" was the wrong primitive. comp_d3d11_service_capture_frame read mc->combined_atlas (the multi-compositor's whole-workspace composite), which is null for a single non-workspace IPC client and is the wrong buffer (everyone, not the caller) under the shell. So per-session capture needed real service-side work — folded in (the planned 2a/2b split collapsed).

How it works now:

• oxr capture_ipc() routes IPC sessions over the existing capture bridge; stage→flag (PROJECTION_ONLY→new IPC_CAPTURE_FLAG_PROJECTION_ONLY, POST_COMPOSE→ATLAS); views_written check → UNSUPPORTED for an unwritten stage.
• service PROJECTION_ONLY captures the active client's content-sized render.crop_texture (the real DP input; projection-only by construction since window-space only lands in combined_atlas; exists for single clients). POST_COMPOSE stays combined_atlas (workspace-only). Source chosen from flag + active_compositor (known-valid) — no blind cast of the caller's xc, no signature change.
• No XR_EXT_spatial_workspace spec bump — stage selection lives in the internal IPC flag.

Verified e2e (D3D11): in-process numbering writes -1/-2/-3; over IPC (XRT_FORCE_MODE=ipc + service) PROJECTION_ONLY writes a tight 1280×360 two-view atlas captured by the service from the client's crop atlas (flags=0x2 written=0x2), matching the in-process capture. POST_COMPOSE over a single IPC client correctly reports unsupported.

How to drive an IPC session (now in CLAUDE.md): start _package\bin\displayxr-service.exe, then launch any app with XRT_FORCE_MODE=ipc (a running service alone does NOT make hosted/handle apps IPC — they're in-process). The app must be focused to stay in 3D (it drops to 2D when unfocused).

State: W6 runtime done — in-process (#397, merged) + IPC per-session (#398, in review).

Next: broader W6 consumer migration (gauss/modelviewer demos, remaining cube_* test apps, Unity/Unreal — all on the in-process API) + atlas_capture_* deletions. Smaller follow-ups: per-arbitrary-caller selection under multi-client workspace (currently active/focused client); POST_COMPOSE-over-IPC per-session if ever needed. Or pick up W1 (math core) independently.

[dfattal]

Future capture requirement — workspace capture scopes (multi-compositor)

Noted by @dfattal: under shell / workspace mode (multi-compositor), we want two distinct, app-selectable capture scopes, currently conflated:

1. Whole workspace output — all apps composed to the final render (= the multi-compositor's combined_atlas; what xrCaptureWorkspaceFrameEXT and xrCaptureAtlasEXT POST_COMPOSE capture today).
2. An individual app's content — what a specific client submitted to the multi-compositor, before the cross-app combine (= that client's per-session render.crop_texture / atlas_texture).

Where #398 leaves it: PROJECTION_ONLY over IPC captures a per-session atlas, but the service sources active_compositor (the focused client), so under multi-client workspace it can't yet target an arbitrary caller; and the proj-vs-post stage flag doubles as the de-facto scope selector rather than an explicit one.

Likely shape: keep XrAtlasCaptureStageEXT (projection vs post-compose) and add an explicit scope (this-session vs whole-workspace), with per-client addressing for a privileged workspace capturer (caller→multi-comp slot mapping, comp_d3d11_service_workspace_find_slot_by_xc). Belongs with the broader workspace/multi-client capture work, after the in-process consumer migration.

[dfattal]

Session hand-off 2026-06-03 (W6 consumer migration — Windows test apps)

Landed: PR #400 (open for review) — all four Windows cube_handle_*_win apps migrated to xrCaptureAtlasEXT, with the shared common/ capture code refactored.

• Refactor: new dxr_capture::RequestRuntimeAtlasCapture(xr, appName, cols, rows, hwnd) in sr_common_base (atlas_capture.cpp) — the single API-agnostic capture path; each app's ~30-line I-key block → one call.
• Migrated d3d11/d3d12/gl/vk: main.cpp calls the helper (per-API CaptureAtlasRegion* fallback dropped); xr_session.cpp (d3d12/gl/vk) detect+enable+resolve the ext.
• Deleted the now-dead Windows-only atlas_capture_d3d11.cpp + atlas_capture_d3d12.cpp (+ decls + CMake refs). Net −350 lines.
• Kept atlas_capture_{gl,vk}.cpp + atlas_capture_metal.mm — the macOS apps still use them.

Verified e2e (in-process): d3d11 + d3d12 each I → tight 1280×360 two-view PNG via xrCaptureAtlasEXT; gl/vk build+link with the identical pattern.

Next (W6 consumer migration remaining):

1. macOS cube_handle_{metal,gl,vk}_macos (needs a Mac) → then delete atlas_capture_{gl,vk}.cpp + _metal.mm.
2. Demos: displayxr-demo-gaussiansplat, displayxr-demo-modelviewer (delete their forked common/atlas_capture*, call xrCaptureAtlasEXT).
3. Engines: Unity (DisplayXRScreenshot.cs) + Unreal (DisplayXRAtlasCapture.cpp) per the design doc.

[dfattal]

Runtime capture follow-ups (observed during W6 test-app verification)

Two cross-API inconsistencies in the in-process *_capture_atlas_to_png paths (runtime-side, independent of the consumer migration):

1. VK capture tone — the cube_handle_vk_win capture reads slightly warmer/lighter than d3d11/d3d12/gl. Likely an SRGB/linear-encoding nuance in vk_native_capture_atlas_to_png (the atlas format / channel-swap / gamma handling on readback). Cosmetic; tracked for a later fix.
2. Capture dimensions differ by API — d3d11/d3d12 capture a tight content region (1280×360 for the cube), while gl/vk capture the full display tile (3840×1080). Being fixed now (gl/vk → content-region crop to match d3d11/d3d12).

[dfattal]

Correction + handoff: gl/vk capture-crop fix (deferred to a fresh session)

Scoped the gl/vk capture-dimension issue; it's not a simple crop. Root cause:

• gl/vk compositors set c->view_width = mode->view_width_pixels (the display mode's native per-view size, ~1920×1080) and the capture uses content = tile_columns × c->view_width → 3840×1080. (comp_gl_compositor.cpp ~L1214-1216, L972/1055; comp_vk_native_compositor.c analog.)
• d3d11/d3d12 use the app's actual per-view render size (~640×360) via get_view_dimensions → 1280×360.
• The gl/vk apps render at renderW = windowW × scale (~640), smaller than the mode tile (~1920) — so there's an app-render-size vs compositor-mode-size mismatch entangled with the weave, not just a capture crop.

Fix target (next session, fresh context): in gl_compositor_capture_atlas_to_png / vk_native_capture_atlas_to_png, crop to the actually-rendered content region (match d3d11/d3d12's get_view_dimensions source) rather than the full mode tile — verifying against the gl/vk apps' real render dims and without disturbing the weave/DP-input path. Pairs with the VK capture-tone fix noted above.

(Deferred from the W6 test-app migration session — that work shipped in #400; this is a separate runtime-compositor change.)