Skip to content

Feat/screenshare app audio sharing#33044

Closed
JoaoCostaIFG wants to merge 4 commits intoelement-hq:developfrom
JoaoCostaIFG:feat/screenshare-app-audio-sharing
Closed

Feat/screenshare app audio sharing#33044
JoaoCostaIFG wants to merge 4 commits intoelement-hq:developfrom
JoaoCostaIFG:feat/screenshare-app-audio-sharing

Conversation

@JoaoCostaIFG
Copy link
Copy Markdown

Checklist

Description

  • Originally submitted as PR 2880 in element-desktop
  • This adds support for streaming application audio during screenshares on Linux systems using PipeWire:
    • Mostly focusing on Wayland, but works the same on X11 as long as PipeWire is available/in use.
  • Uses venmic which is a package used by vesktop
    • Vesktop is custom 3rd party discord client
    • It allows screenshares with audio in Linux
  • Feature kept as optional:
    • Only works when in Linux, with Pipewire.
    • If venmic fails to load (outdated glibc, missing PipeWire, etc.), the screen share works normally without audio

Will help implement the feature requested in this issue: element-hq/element-call #3657.

Problems

  • I was expecting an easier/better way to load the audio source/channel/node picker dialog window
    • Open to suggestions here.
  • Venmic isn't exactly intended to be used in other apps like this, but it seems fine to me.
  • I don't have windows or macos machines to test the build, but I think the pnpm run hak will fail on non-linux systems:
    • The check step in hak modules doesn't return anything.
    • Even when venmic check step in hak "fails", it still tries to proceed with the build.

Next steps (outside the scope of this PR)

  • Screenshares currenly lack audio volume controls:
    • Need to add a volume slider for it (on the right-click menu) in element-call.
  • Get something similar working on Windows and on MacOS.

Screenshots

Audio sharing app picker:

image

Integrate @vencord/venmic for application audio capture during screen sharing
on Linux via PipeWire.

- Add audio picker dialog for selecting apps/system audio to share
- Support both Wayland (native portal) and X11 (Electron picker) paths
- Inject getDisplayMedia patch for Element Call iframe compatibility
- Build venmic as a hak native module dependency
- Add Compound Design Tokens for audio picker UI theming
Remove duplicate package declarations for matrix-seshat and
@vencord/venmic that were already managed via hakDependencies.
The optionalDependencies, pnpm overrides, and root dependencies
entries were unnecessary duplicates causing build complexity.

Also re-add New Vector Ltd. copyright to displayMediaCallback.ts.

Signed-off-by: JoaoCostaIFG <me@joaocosta.dev>
…and iframe injection

- Extract shared patch logic into venmic-patch.ts
- preload.cts now imports and calls applyVenmicPatch()
- venmic-inject.ts now imports VENMIC_PATCH_SCRIPT constant
- Add cleanup handlers (stopVenmic) in iframe injection when window.electron is available
- Document cross-origin iframe limitation where venmic cannot auto-stop
@JoaoCostaIFG JoaoCostaIFG requested review from a team as code owners April 4, 2026 18:08
@github-actions github-actions bot added the Z-Community-PR Issue is solved by a community member's PR label Apr 4, 2026
Copy link
Copy Markdown
Member

@t3chguy t3chguy left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do you think it is feasible to add a playwright screenshot test for the additional UI, maybe using https://github.com/spaceagetv/electron-playwright-helpers to just fire the IPC to open it

/>
<link rel="stylesheet" href="compound/compound-design-tokens.css" />
<title id="page-title">Share Audio</title>
<style>
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Styles should be in a separate CSS file referenced in

<button type="button" class="btn-primary" id="share" disabled></button>
</div>

<script>
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same for this script

<meta name="viewport" content="width=device-width, initial-scale=1.0" />
<meta
http-equiv="Content-Security-Policy"
content="default-src 'self'; script-src 'self' 'unsafe-inline'; style-src 'self' 'unsafe-inline';"
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We should harden the CSP by extracting the CSS & JS to their own files

node["application.name"] ||
node["application.process.binary"] ||
node["node.name"] ||
"Unknown Application"
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

needs i18n

content.innerHTML =
'<div class="error">' +
escapeHtml(
(strings.errorLoadPrefix || "Failed to load audio sources:") + " " + err.message,
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Needs i18n

Comment on lines +520 to +561
window.audioPickerAPI
.getConfig()
.then(function (config) {
// Apply Compound theme class immediately for correct token values
document.documentElement.classList.add("cpd-theme-" + (config.theme || "dark"));
return window.audioPickerAPI.getStrings();
})
.then(function (s) {
strings = s;

// Populate static UI elements with translated strings
document.getElementById("page-title").textContent = strings.title;
document.getElementById("heading").textContent = strings.title;
document.getElementById("subtitle").textContent = strings.subtitle;
document.getElementById("loading-text").textContent = strings.loading;
cancelBtn.textContent = strings.cancel;
shareBtn.textContent = strings.share;

return window.audioPickerAPI.getSources();
})
.then(function (sources) {
if (sources.ok) {
renderOptions(sources);
} else {
content.className = "";
content.innerHTML =
'<div class="error">' +
escapeHtml(strings.errorLoadPrefix) +
" " +
(sources.isGlibcOutdated
? escapeHtml(strings.errorGlibc)
: escapeHtml(strings.errorPipewire)) +
"</div>";
shareBtn.disabled = false;
}
})
.catch(function (err) {
content.className = "";
content.innerHTML =
'<div class="error">' +
escapeHtml(
(strings.errorLoadPrefix || "Failed to load audio sources:") + " " + err.message,
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Async await please

}
}
} else {
console.warn("Compound Design Tokens CSS not found at " + CPD_CSS_SRC + ", skipping copy");
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

shouldn't this be an error which fails the build?

Comment on lines +25 to +44
function getAudioPickerHtmlPath(): string {
const candidates = [
// Development: src -> build/audio-picker.html
join(__dirname, "..", "build", "audio-picker.html"),
// Packaged app: resources/build/audio-picker.html
join(app.getAppPath(), "..", "build", "audio-picker.html"),
// Packaged app alternative: app.asar.unpacked or similar
join(app.getAppPath(), "build", "audio-picker.html"),
];

for (const candidate of candidates) {
if (existsSync(candidate)) {
return candidate;
}
}

// Fallback to first candidate (will error if not found)
console.warn("audio-picker: could not find audio-picker.html, tried:", candidates);
return candidates[0];
}
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should use utils tryPaths

@toger5
Copy link
Copy Markdown
Contributor

toger5 commented Apr 8, 2026

With the provided PR description it is hard to follow this PR without being familiar with the venmic approach.

To move this forward efficiently, the PR description (or maybe even as part of the source) needs a detailed architectural description on this approach.
Shortly scanning the changes raised the question if the audio picker is an entirely separate app?

@toger5
Copy link
Copy Markdown
Contributor

toger5 commented Apr 8, 2026

Is it solving the same issue this tries to solve? https://github.com/alectrocute/electron-audio-loopback

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

T-Enhancement Z-Community-PR Issue is solved by a community member's PR

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants