Add media player notification widget and allow background playback

[bxdxnn]

Content

Add a MediaStyle notification for the media player and allow background playback

Motivation and context

Media playback stops when the app goes to background because MediaVideoView and MediaAudioView explicitly pause/release ExoPlayer on lifecycle events. This adds a MediaSessionService that owns the ExoPlayer so playback survives backgrounding.

Screenshots / GIFs

Tests

• Test that all notification buttons are working
• Test background playback in-app
• Test background playback when not in-app
• Test that you can go back to the currently played file: tap on the notification or on the file in the timeline
• Test that the in-app player and the notification are synced and are correctly reflecting the current state

Tested devices

• Physical
• Emulator
• OS version(s):

Checklist

• Changes have been tested on an Android device or Android emulator with API 24
• UI change has been tested on both light and dark themes
• Accessibility has been taken into account. See https://github.com/element-hq/element-x-android/blob/develop/CONTRIBUTING.md#accessibility
• Pull request is based on the develop branch
• Pull request title will be used in the release note, it clearly define what will change for the user
• Pull request includes screenshots or videos if containing UI changes
• You've made a self review of your PR

[github-actions]

Thank you for your contribution! Here are a few things to check in the PR to ensure it's reviewed as quickly as possible:

• Your branch should be based on origin/develop, at least when it was created.
• The title of the PR will be used for release notes, so it needs to describe the change visible to the user.
• The test pass locally running ./gradlew test.
• The code quality check suite pass locally running ./gradlew runQualityChecks.
• If you modified anything related to the UI, including previews, you'll have to run the Record screenshots GH action in your forked repo: that will generate compatible new screenshots. However, given Github Actions limitations, it will prevent the CI from running temporarily, until you upload a new commit after that one. To do so, just pull the latest changes and push an empty commit.

[bxdxnn]

@bmarty FYI, this was done almost fully by LLM

[bmarty]

Thanks for the PR. I am not sure if we want to have such feature in the application. Can you post a video demoing it so I can ask the product team? Thanks

[bxdxnn]

Thanks for the PR. I am not sure if we want to have such feature in the application. Can you post a video demoing it so I can ask the product team? Thanks

Hello, I can't make a demo video as there are some technical bugs and minor design questions regarding the metadata in the notification since the last rebase, but I can continue if the main behavior will be accepted by the product team.
The main behavior changes will be:

1. Show an Android media notification widget that allows to control the player and the output device, similar to VLC and other audio players (see the example screenshot)
2. The player will automatically continue to the next media in the timeline when the app is in background or the player is closed, creating a playlist experience
3. The playback continues if you close the player or the app goes background

If this feature won't be accepted, can we allow background playback at least just when the app goes background? It'll be also great to be able to listen to media while you're in the app and the player is hidden, similar to Telegram, but this leads to the need of this feature because there would be no player controls without this PR.

Example screenshot of the widget (no cover art, in a DM):

[bmarty]

Thanks for your reply. We are building a messaging application and not a media player. The application is able to play media (video, voice message) when in the foreground as a facility, but I believe that users expect all media playing to just stop when the app is put into background. But I let @mxandreas give the final words here.

[mxandreas]

I have checked the behavior on WhatsApp and Signal, and they do continue playing whatever was playing when you lock the screen or switch to another app, but they stop once the current file is finished. Thus, this as a behavior is fine as long as it can be done with what the OS/system provides and we do not need to design for any of the audio players, etc.

I am curious though to understand in which situations are you @bxdxnn seeing this as a friction personally - what are you listening to, why is it challenging to keep the app in the foreground, etc. And in particular why would you even like to continue with the next/previous file in the timeline?

Also, for the next PRs which are about new features (visual or not), please open a a Enhancement Request issue in the https://github.com/element-hq/element-meta/issues repo - describing the the use case and the why behind the feature. This way you can get feedback earlier if/what can we have in the product before we start reviewing the code which is always more effort.

[bxdxnn]

@mxandreas

I have checked the behavior on WhatsApp and Signal, and they do continue playing whatever was playing when you lock the screen or switch to another app, but they stop once the current file is finished. Thus, this as a behavior is fine as long as it can be done with what the OS/system provides and we do not need to design for any of the audio players, etc.

Do you want me to only implement the background widget (allowing background playback), without any other behavioral/UI changes to the app (cover arts, metadata, automatic forward/previous while in background, etc.)?

I am curious though to understand in which situations are you @bxdxnn see this as a friction personally - what are you listening to, why is it challenging to keep the app in the foreground, etc. And in particular why would you even like to continue with the next/previous file in the timeline?

I want to create a read-only channel in a space where I publish my (~1 hour duration) audio books. Continuing it makes more convenient to listen to all medias in the room.
It's also challenging to listen to any media which's length exceeds screen timeout.

Also, for the next PRs which are about new features (visual or not), please open a a Enhancement Request issue in the https://github.com/element-hq/element-meta/issues repo - describing the the use case and the why behind the feature. This way you can get feedback earlier if/what can we have in the product before we start reviewing the code which is always more effort.

• Many issues from outside contributors don't get feedback until there's a PR for it.
  • For example, one of the most liked features (max_selections polls Support for the max_selections poll parameter element-meta#2694) didn't get any feedback for almost 4 years until I've made a PR for it.
• Making a PR makes it faster to proceed on an issue, any discussions can be easily resolved later.
• It makes it easier to discuss when you can already test the implementation and how it'll work.

[bmarty]

It's also challenging to listen to any media which's length exceeds screen timeout.

We should have a wake lock when playing media to mitigate this. I though we had one, but it's apparently not the case.

[bxdxnn]

The widget has an ability to show the cover art, but what should be shown when there's no cover art for an audio?

• Nothing (current behavior)
• Room's avatar
• Sender's avatar

[mxandreas]

We should have a wake lock when playing media to mitigate this. I though we had one, but it's apparently not the case.

Good point, @bmarty. Do you have any strong arguments for "wake lock" vs "continue in the background" for this scenario?

The widget has an ability to show the cover art, but what should be shown when there's no cover art for an audio?

I'd say the first iteration would be fine probably without anything, but if it easy to make, then probably sender's avatar. @americanrefugee do you agree?

[americanrefugee]

@mxandreas : I'd say the first iteration would be fine probably without anything, but if it easy to make, then probably sender's avatar. @americanrefugee do you agree?

Yes, that's fine with me.

[mxandreas]

I have checked the behavior on WhatsApp and Signal, and they do continue playing whatever was playing when you lock the screen or switch to another app, but they stop once the current file is finished.

I have to correct myself regarding this, in fact they continue playing, but there are some differences in behaviour, see details in #6454 (comment)

[bxdxnn]

I have removed the playlist (play next timeline media after ending) feature and the notification now uses sender's avatar for the cover art, addressing the product's decision review. Waiting for code review now.

About WA/Signal behavior: I'm not sure what it means here, auto-playing voice messages is a different case from auto playing media or playing it in background.