2D surround on hooked path — high-res 2D over woven 3D (#131)#133
Merged
Conversation
) Foundation for handle-mode 2D surround on the hooked path (built apps). Runtime source (openxr-3d-display) confirms the surround strip-blit runs on the HWND path too (comp_d3d12_compositor.cpp:1670), so the built app gets surround in its current handle mode — no shared-texture-mode rework. - displayxr_extensions.h: add PFN_xrSetSharedTextureSurround2DEXT (v6) + PFN_xrSetSharedTextureSurround2DFenceEXT (v7) typedefs; bump XR_EXT_WIN32_WINDOW_BINDING_SPEC_VERSION 5 -> 7 (CreateInfo struct unchanged). - displayxr_hooks: the existing displayxr_set_canvas_rect now caches the rect and hooked_xrEndFrame re-applies it each frame (matches the cube_texture reference), so the 3D canvas sub-rect survives runtime state and can be set before session-ready. Default invalid -> full-window canvas (no behavior change). C# P/Invoke added. Surround texture manager + DisplayXRSurround component + tiger bubble demo follow. Built clean (MSVC x64). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The standalone session loader used plain LoadLibraryA(full_path) to load the DisplayXR runtime DLL. Windows resolves a target DLL's imports against the host process search path (Unity.exe dir + System32 + PATH), never the loaded DLL's own directory. When the runtime gained a folder-local dependency (pthreadVCE3.dll, shipped beside DisplayXRClient.dll), the load failed with ERROR_MOD_NOT_FOUND (126) and the preview/play-mode standalone session could not start. Switch to LoadLibraryExA(..., LOAD_WITH_ALTERED_SEARCH_PATH) so Windows searches the runtime DLL's own folder for its dependencies. Requires an absolute path, which resolve_library_path already provides. The hooked built-app path is unaffected (Khronos loader already uses altered search). Binary intentionally omitted: another build was in flight on this branch. Rebuild Runtime/Plugins/Windows/x64/displayxr_unity.dll (build-win.bat) or let CI produce it. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
) Implements 2D surround (xrSetSharedTextureSurround2DFenceEXT, spec v7) on the hooked path so built apps get a full-window, post-weave, full-native-resolution 2D layer filling the non-canvas region — no shared-texture-mode rework (the runtime applies the surround strip-blit on the HWND path). Native (D3D12 backend): - Allocate a SHARED RGBA8 surround texture + SHARED ID3D12Fence on Unity's device; export NT handles. surround_update() copies the registered Unity RT into the surround texture on the runtime queue (explicit barriers, mirrors the wsui copy), signals the fence (monotonic), and returns handles + value. Shared resources left in COMMON for cross-queue handoff; no CPU wait (the runtime waits on the fence before its strip blit). Native (hooks): - Resolve xrSetSharedTextureSurround2DFenceEXT; new GraphicsBackend:: surround_update/surround_release virtuals (D3D12 only, others no-op). - hooked_xrEndFrame copies + signals + registers each frame on the render thread (same context as wsui), BEFORE s_real_end_frame. - C ABI: displayxr_surround_set_texture / displayxr_surround_clear. C#: - DisplayXRNative P/Invokes; DisplayXRSurround MonoBehaviour: renders a Canvas into a panel-sized RGBA8 RT via a dedicated ortho camera (mirrors wsui), registers it, and sets the 3D canvas sub-rect so the surround region exists. Transparent apps: the surround copy preserves alpha verbatim and the runtime blits post-weave/pre-present, so an alpha-0 surround with an opaque bubble should present transparent-except-bubble via DComp (verify on device). Built clean (MSVC x64). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The surround-manager commit's binary predated dc7de52 (LoadLibraryExA / LOAD_WITH_ALTERED_SEARCH_PATH so the standalone loader finds the runtime's folder-local DLLs). Rebuild so the shipped DLL carries both that fix and the 2D surround code. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
OnEnable may run before the runtime reports display pixel dims; the surround texture must match the window client area or the runtime rejects it. Defer RT/ camera creation into TrySetup() and retry from LateUpdate until dims resolve. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
So the surround bubble shows in the Editor Preview Window and Play Mode, not just built apps. The standalone session runs on its own D3D12 device, so this mirrors the hooked manager but routes through a cross-device bridge (same pattern as the wsui/atlas bridges). Native (SA D3D12 backend): - surround_create_bridge: SHARED RGBA8 surround texture on the SA device opened on Unity's device (C# renders into it) + a SHARED ID3D12Fence. surround_signal bumps + signals the fence on the SA queue and returns the NT handles + value. Native (standalone session): - Resolve xrSetSharedTextureSurround2DFenceEXT; sa_push_canvas_rect_to_runtime pushes a registered sub-rect (so a surround region exists) instead of the full window; submit_frame_atlas signals + registers each frame before xrEndFrame. - C ABI: displayxr_standalone_get_surround_bridge_texture / _surround_set_active / _surround_clear. C#: - DisplayXRSurround now branches on Application.isEditor: editor (standalone) copies the RT into the bridge + registers via the standalone API; built app (hooked) registers the RT directly. DisplayXRNative P/Invokes added. D3D12 editor only (Vulkan-editor standalone surround is a follow-on). Built clean (MSVC x64). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…#131) Diagnostics on hardware showed the bug: the runtime weaves into the bound HWND client area (3153x2248 on Leia SR — the SR weaver oversizes/crops the window), NOT the display panel dims (3840x2160). The surround texture was sized to the panel, so the runtime silently skipped the strip blit on the dim mismatch (comp_d3d12_compositor.cpp:2484) even though registration succeeded. - native: displayxr_get_render_target_size() returns the bound HWND client area (GetClientRect, physical px) — the size the surround texture + canvas sub-rect must match. Added one-shot DIAG logs (surround DIAG / surround COPY DIAG). - C#: DisplayXRSurround.TryGetTargetSize() resolves the weave-target size (render-target size for built apps, preview-window size in editor) and ResolveResolution uses it instead of display info. TigerSpeechBubble uses the same size for its canvas-rect fractions. Built clean (MSVC x64). Click-through sub-rect alignment is the remaining item. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
- Map the transparent-overlay click-through silhouette into the active canvas sub-rect (new displayxr_get_canvas_rect_px) so it tracks the shrunk weave; round trailing edges outward to avoid truncation clipping the tiger edge. - New displayxr_set_overlay_surround_rect: union an opaque 2D rect (e.g. a high-res text bubble drawn in the surround) into the SetWindowRgn region so it catches clicks while the empty surround keeps routing past to the desktop. - DisplayXRTransparentOverlay: render ALL submeshes into the silhouette mask (was submeshIndex 0 only) — fixes interior click-through holes over parts on other material slots (e.g. tiger face/arm). - Remove the one-shot surround DIAG / COPY DIAG diagnostics. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…es (#131) The hit-mask projection uses the cached Kooima matrices (near=0.05, far=100), which are tighter than the camera's actual render clip. Foreground geometry nearer than 0.05 (the tiger's cheek/arm leaning toward the camera) was near-plane-clipped only in the mask, leaving wedge-shaped click-through holes the woven render doesn't have. Pin o.pos.z = 0.5w (mid-frustum, valid in both D3D and GL clip ranges) so every in-front vertex survives near AND far clipping. The mask only needs x/y coverage — depth is irrelevant (ZTest Always, no depth buffer) — so overriding z is safe. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Implements 2D surround (
xrSetSharedTextureSurround2DFenceEXT, spec v7) on the hooked path (built apps), so a built DisplayXR app can show full-native-resolution flat 2D content (text bubbles, HUD chrome) in the non-canvas region, composited post-weave over the woven 3D. Closes the implementation half of #131; canvas sub-rect plumbing also addresses #34.Why no shared-texture-mode rework
Reading the runtime source (
openxr-3d-display) settled the open question: the surround strip-blit runs on the HWND path too (comp_d3d12_compositor.cpp:1670), blitting the surround into the back buffer the runtime presents. So the built app gets surround in its current handle mode — no present-ownership rework (the standalone shared-texture-mode work in #132 is parked, not needed for this).How
displayxr_set_canvas_rectnow caches + re-applies eachxrEndFrame(matches the cube_texture reference), defining where the 3D weaves; the rest is the surround region.ID3D12Fenceon Unity's device;surround_update()copies the registered Unity RT into it on the runtime queue (explicit barriers, mirrors the wsui copy), signals the fence (monotonic), returns handles + value. No CPU wait — the runtime waits on the fence before its strip blit.hooked_xrEndFramecopies + signals + registers each frame on the render thread (same context as wsui), befores_real_end_frame. C ABI:displayxr_surround_set_texture/displayxr_surround_clear.DisplayXRSurroundMonoBehaviour renders a Canvas into a panel-sized RGBA8 RT via a dedicated ortho camera, registers it, and sets the 3D canvas sub-rect.Transparency
The surround copy preserves the surround texture's alpha verbatim and the runtime blits it post-weave / pre-present, so an alpha-0 surround with an opaque bubble should present transparent-except-bubble through DComp in a transparent app. Verify on device.
Status
TigerSpeechBubble) staged indisplayxr-unity-test-transparentworking tree (pinned to this local plugin).🤖 Generated with Claude Code