Off topic but ...

[Noemata]

Since you are among the best uwp rendering gurus at Microsoft, I was wondering what thoughts you might have about a somewhat similar rendering problem but in a very different context.

I've been toying with how best to render a XAML surface and have it displayed within a Unity texture. For now I'm mostly interested in the rendering side of things and not worrying about input just yet. What would be the best approach from a performance perspective?

I would like to leverage XAML for the many complex 2D panels I'll need in my Unity app. XAML is so much more efficient for 2D work and has such a rich API, it seems wasteful not to leverage what's there when running on a Mixed Reality device.

I'm also curious why XAML surfaces aren't a thing in any of the WinRT 3D API's currently available from Microsoft?

Thoughts, ideas, suggestions?

[mediabuff]

You may try Xaml islands

[Noemata]

Interesting. Not sure if this is a clue just yet. Are the XAML island underpinnings available on the HoloLens 1 and 2? If so, I can see that working nicely. Given the HoloLens 2 is running on Core, it seems less likely it would have those bits? I presume this would most certainly work with desktop MR?

[mediabuff]

It's desktop only - for hosting XAML content in Win32 Windows. You need a HWND.

Conversely, you may want to invert the problem. Make UWP/Xaml as your primary host and draw your game content onto SwapChainPanel.

[Noemata]

Sorry mediabuff, I thought you might have had some specific insight into under the hood plumbing. This is a tricky problem. I've already tried 3 different approaches that work, just that I'm not happy with the perf level as they all involve shuffling data across threads or processes and are not directly tied to the Unity texture. So I'm probably copying the same data around at least three times and likely more for two of the approaches I've tried.

[Noemata]

FYI, the best perf so far has been derived by leveraging this approach:

https://github.com/shweaver-MSFT/MediaPlayback

[mediabuff]

Thanks for sharing the links. Seems to me a glorified WRL wrapper for MediaPlayer winrt api.

Much of that has been superceded and can be simplified using cppwinrt.

Could you please share what other approaches you tried ? Any pointers.

[Noemata]

(1) UWP Screen Sharing APIs + DirectX. (2) MediaFoundation + Screen Sharing APIs, and (3) DirectX approache I can't mention publicly. There's no "modern" example out there of how to do this. There is a Unity app that uses the older DWM API, from what I recall. Anyone that has done it isn't sharing.