fix(wayland): support DMA-BUF modifiers for wlroots capture

[joeknock90]

Explicitly query supported DMA-BUF modifiers from the compositor and use them when creating GBM buffers. This fixes capture failures on Sway when using the Vulkan renderer, which rejects the implicit INVALID modifier.

Explicitly query supported DMA-BUF modifiers from the compositor and use them when creating GBM buffers. This fixes capture failures on Sway when using the Vulkan renderer, which rejects the implicit INVALID modifier.

DISCLAIMER: I am not a developer, programmer, or really any sort of smart person at all. I punched this issue into Gemini-CLI. I manually approved all of he changes it made with very little understanding of them, but did learn a little bit. Never thought I'd be a vibe coder 🤮

For full transparency, here is the prompt I used:

In this project something is broken when trying to stream a swaywm desktop using the WLR_RENDERER=vulkan option. According to some reports, this issues happens in Linux 7.0 but works as intended in Linux 6.18.25. According to the swaywm developers: "The wlroots capture method in Sunshine is using the implicit INVALID modifier, which is rejected in Vulkan."

I am currently testing my branch and it seems to be working well for me so far.

Screenshot

N/A

Issues Fixed or Closed

Fixes #4050

Roadmap Issues

N/A

Type of Change

• feat: New feature (non-breaking change which adds functionality)
• fix: Bug fix (non-breaking change which fixes an issue)
• docs: Documentation only changes
• style: Changes that do not affect the meaning of the code (white-space, formatting, missing semicolons, etc.)
• refactor: Code change that neither fixes a bug nor adds a feature
• perf: Code change that improves performance
• test: Adding missing tests or correcting existing tests
• build: Changes that affect the build system or external dependencies
• ci: Changes to CI configuration files and scripts
• chore: Other changes that don't modify src or test files
• revert: Reverts a previous commit
• BREAKING CHANGE: Introduces a breaking change (can be combined with any type above)

Checklist

• Code follows the style guidelines of this project
• Code has been self-reviewed
• Code has been commented, particularly in hard-to-understand areas
• Code docstring/documentation-blocks for new or existing methods/components have been added or updated
• Unit tests have been added or updated for any new or modified functionality

AI Usage

• None: No AI tools were used in creating this PR
• Light: AI provided minor assistance (formatting, simple suggestions)
• Moderate: AI helped with code generation or debugging specific parts
• Heavy: AI generated most or all of the code changes

[sonarqubecloud]

Quality Gate passed

Issues
0 New issues
0 Accepted issues

Measures
0 Security Hotspots
0.0% Coverage on New Code
0.0% Duplication on New Code

See analysis details on SonarQube Cloud