[Question] Does the Android deployment support audio understanding (audio + text prompt)?

[starrydreamawa]

I'm trying to deploy MiniCPM-o on the Android platform. I would like to know whether MiniCPM-o on Android supports audio understanding, specifically:

Input: audio file (e.g., WAV/MP3) + natural language prompt
Output: textual understanding of the audio content (e.g., "What is the sound in this recording?")

I've deployed MiniCPM-o with llama.cpp by Android NDK and adb tool (direct building with Termux failed), checked the llama-minicpmv-cli help menu and found no --audio or similar parameter for audio input.

[tc-mb]

Hi, thanks for trying MiniCPM-o on Android! The llama-minicpmv-cli currently supports vision (image) input only and does not have audio input support. For full omni-modal capabilities including audio, please check out our llama.cpp-omni project: https://github.com/tc-mb/llama.cpp-omni — it supports audio input/output along with vision. You may try building it for Android as well, though we haven't officially tested Android deployment for llama.cpp-omni yet.