Switch to earpiece if device is held to ear when playing voice messages

[bxdxnn]

Content

Be able to listen to voice message using the earpiece when a device is held to ear

Motivation and context

This is a highly desired feature and the standard behavior in all messaging apps. The earpiece sensor feature remains available even when external devices are connected, due to privacy considerations. Here's why:

1. Android does not provide information on whether a connected Bluetooth device is a speaker or a headset.
2. Even if we were certain (we are with wired headsets) the connected device is a private headset:
   • It could be used as a speaker at maximum volume.
   • It might be in use by another person.
   • The user might forget he have an unused Bluetooth headset connected, which could lead to confusion or inconvenience when audio is unexpectedly absent.

Screenshots / GIFs

Tests

• Test that switching to earpiece mode occurs when the device is held to the ear.
• Ensure the output device priority order:
  1. Wired headset
  2. Bluetooth device
  3. Speakers
  4. Ensure that the earpiece is used if the device is held to the ear, overriding the above options.
• Test that audio playback works over external devices.
• Test that audio doesn't unexpectedly switches to external device if it's currently on the earpiece and a device gets connected (e.g. over Bluetooth).
• Test that audio switches to an external device as expected when it gets connected and the device is not held to the ear.
• Test that audio is not played through the earpiece unless the device is explicitly held to the ear.

Tested devices

• Physical
• Emulator
• OS version(s):

Checklist

• Changes have been tested on an Android device or Android emulator with API 24
• UI change has been tested on both light and dark themes
• Accessibility has been taken into account. See https://github.com/element-hq/element-x-android/blob/develop/CONTRIBUTING.md#accessibility
• Pull request is based on the develop branch
• Pull request title will be used in the release note, it clearly define what will change for the user
• Pull request includes screenshots or videos if containing UI changes
• You've made a self review of your PR

[github-actions]

Thank you for your contribution! Here are a few things to check in the PR to ensure it's reviewed as quickly as possible:

• Your branch should be based on origin/develop, at least when it was created.
• The title of the PR will be used for release notes, so it needs to describe the change visible to the user.
• The test pass locally running ./gradlew test.
• The code quality check suite pass locally running ./gradlew runQualityChecks.
• If you modified anything related to the UI, including previews, you'll have to run the Record screenshots GH action in your forked repo: that will generate compatible new screenshots. However, given Github Actions limitations, it will prevent the CI from running temporarily, until you upload a new commit after that one. To do so, just pull the latest changes and push an empty commit.

[bxdxnn]

@ganfra any updates?

[bxdxnn]

@bmarty can this be reviewed?

[mxandreas]

This is a highly desired feature and the standard behavior in all messaging apps.

So, again, I did some testing on both iOS and Android using Signal and WhatsApp. The results are:

1. If you do not have a BT device connected, it does indeed switch to earpiece when you move your phone to your ear.
2. However, if the BT device was connected, then it kept playing on the BT device.

The results were consistent across both platforms and apps. I am not sure if this can be really used as a "safety feature", I see it more as a convenience feature and thus, I'd rather keep playing on the BT if that is connected.

@fkwp what are we currently doing for Element Call calls regarding this?

[fkwp]

This is a highly desired feature and the standard behavior in all messaging apps.

So, again, I did some testing on both iOS and Android using Signal and WhatsApp. The results are:

1. If you do not have a BT device connected, it does indeed switch to earpiece when you move your phone to your ear.
2. However, if the BT device was connected, then it kept playing on the BT device.

The results were consistent across both platforms and apps. I am not sure if this can be really used as a "safety feature", I see it more as a convenience feature and thus, I'd rather keep playing on the BT if that is connected.

@fkwp what are we currently doing for Element Call calls regarding this?

I do agree with @mxandreas a headset (independent of bt or wired) is overruling earpiece)

[bxdxnn]

@mxandreas

It will play on BT/external devices, but if held to an ear, it will switch to the earpiece, i.e. it won't disable the sensor feature like in WhatsApp in case you still need it.

WhatsApp and other apps disable the sensor feature when external devices are connected, which is personally frustrating in cases like when the phone is connected to a car. I could be out of a car and on parking, and with the sensor disabled I will have to disable Bluetooth first, which is also very inconvenient if I have Android Auto connected and the passengers need it at the same time. With this PR, I can just hold the device to ear if I need for it to play on the earpiece instead of speakers without disconnecting BT.
Another case, I might also have music playing on the external device, and I'd like to be able to listen to the voice message on the earpiece while the music is still playing and without having to disconnect anything first.

If you don't want it to play to the ear (keep playing on BT), don't hold the device to the ear ̄\_ (ツ)_/ ̄

This is a privacy feature, so IMO we should prioritize privacy here instead of copying other apps. If you want to disable this feature if an external device is connected, please tell.

[mxandreas]

Thanks for providing the example scenarios.

My previous point was primarily about the initial claim that this is "standard in messaging apps" while it actually does not seem to be so. At least not in WhatsApp and Signal which Element is compared to and has overlapping use cases with.

When we consider changing the behavior, we have to gain some certainty that this is a step in the right direction, and something that is appreciated by most users. Without the ability to conduct a ad-hoc UX research, using other apps as a reference is the first low-hanging fruit. If this is a privacy concern, then it should equally to apply to messages sent via WhatsApp or Signal which are also secure messengers.

What we do not know well is if the WA and Signal behavior is intentional or accidental. And if it is (or was) accidental, then why haven't they done anything to change it.

Having said that - I kind of agree that it may be safe to try overriding BT, because indeed one can simply not move the phone to his ear when they do not want to use earpiece. @fkwp do you have any specific arguments against, like situation where this could be non-ideal?