Running on QCS8550

[Xu-honghao]

Hello，
I observed that the project supports SM8550, so I wanted to try running it on QCS8550, but the following log and error appeared：

Argmax, Inc.

Audio Codec: aac
Input #0, mov,mp4,m4a,3gp,3g2,mj2, from '(null)':
Metadata:
major_brand : M4A
minor_version : 0
compatible_brands: M4A mp42isom
creation_time : 2024-08-07T16:38:45.000000Z
iTunSMPB : 00000000 00000840 000000D4 00000000000766EC 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
Duration: 00:00:11.05, start: 0.047891, bitrate: 73 kb/s
Stream #0:00x1: Audio: aac (mp4a / 0x6134706D), 44100 Hz, mono, fltp, 31 kb/s (default)
Metadata:
creation_time : 2024-08-07T16:38:45.000000Z
vendor_id : [0][0][0][0]
tflite_init input: {"cache":"/data/local/tmp/cache","ch":1,"debug":false,"dur":11052,"fmt":1,"freq":44100,"lib":"/data/local/tmp/lib","root_path":"/storage/emulated/0/argmax/tflite","size":"base"}
SoC: QCS8550 -> QNN HTP
root dir: /storage/emulated/0/argmax/tflite
lib dir: /data/local/tmp/lib
cache dir: /data/local/tmp/cache
INFO: Initialized TensorFlow Lite runtime.
INFO: [QNN Delegate] Caching: cache_filename of /data/local/tmp/cache/qnn_binary_8316569388647923909.bin isn't available, caching in SAVE MODE.
ERROR: [QNN Delegate] Failed to create device_handle for Backend ID 6, error=14001
ERROR: Restored original execution plan after delegate application failure.
INFO: [QNN Delegate] Caching: cache_filename of /data/local/tmp/cache/qnn_binary_12912538988619984752.bin isn't available, caching in SAVE MODE.
ERROR: [QNN Delegate] Failed to create device_handle for Backend ID 6, error=14001
ERROR: Restored original execution plan after delegate application failure.
INFO: [QNN Delegate] Caching: cache_filename of /data/local/tmp/cache/qnn_binary_792083847501657808.bin isn't available, caching in SAVE MODE.
ERROR: [QNN Delegate] Failed to create device_handle for Backend ID 6, error=14001
ERROR: Restored original execution plan after delegate application failure.
INFO: [QNN Delegate] Caching: cache_filename of /data/local/tmp/cache/qnn_binary_18177980569190481050.bin isn't available, caching in SAVE MODE.
ERROR: [QNN Delegate] Failed to create device_handle for Backend ID 6, error=14001
ERROR: Restored original execution plan after delegate application failure.
Stream: freq - 44100, channels - 1, format - 32784, target_buf size - 1440000
postproc vocab size: 51864
INFO: [QNN Delegate] Caching: cache_filename of /data/local/tmp/cache/qnn_binary_16138577325637000973.bin isn't available, caching in SAVE MODE.
ERROR: [QNN Delegate] Failed to create device_handle for Backend ID 6, error=14001
ERROR: Restored original execution plan after delegate application failure.
tflite_init done..
[aac @ 0xb400007289646790] Could not update timestamps for skipped samples.

Final Text: And so my fellow America asked not what your country can do for you, ask what you can do for your country.
Deleted interpreter & delegate for post_proc
Deleted interpreter & delegate for whisper_decoder
Deleted interpreter & delegate for whisper_encoder
Deleted interpreter & delegate for mel_spectrogram
Deleted interpreter & delegate for audio_input
tflite_close done..

Model latencies:
Audio Input: 1 inferences, median:0.81 ms
Melspectro: 1 inferences, median:62.56 ms
Encoder: 1 inferences, median:931.88 ms
Decoder: 29 inferences, median:15.70 ms
Postproc: 29 inferences, median:0.59 ms

Total Duration: 1968.000 ms

From the results, it seems that the NPU is not fully utilized. Is it due to the difference between QCS8550 and SM8550?

[bpkeene]

Hi Xu-honghao,

Thank you for surfacing the issue! From the chip specification, that does look like a similar (same?) chip as what we have for testing and development.

ERROR: [QNN Delegate] Failed to create device_handle for Backend ID 6, error=14001

This looks like the relevant issue, and is often a 'soft' failure -> the QNN Delegate will use available backends

tflite_init input: {"cache":"/data/local/tmp/cache","ch":1,"debug":false,"dur":11052,"fmt":1,"freq":44100,"lib":"/data/local/tmp/lib","root_path":"/storage/emulated/0/argmax/tflite","size":"base"}
SoC: QCS8550 -> QNN HTP
root dir: /storage/emulated/0/argmax/tflite
lib dir: /data/local/tmp/lib
cache dir: /data/local/tmp/cache

Are all of these directories read-write accessible on the device? Perhaps the QNN delegate doesn't have access to the required resources or libraries for setup at runtime.

[Xu-honghao]

Hi @bpkeene:
Thanks for your response.
Yes, the chips of QCS8550 and SM8550 are the same.
Regarding the read and write permission issue, there shouldn't be any, because I'm running the demo with root privileges, so what other reasons could there be?

[keith4ever]

hi @Xu-honghao ,

This is puzzling, and I see that it is trying to delegate to QNN (HTP) compute core (per your log):

SoC: QCS8550 -> QNN HTP

I am guessing you have tried more than twice.. if it is not the privilege issue, then I don't have much idea why.
Is this a smartphone or tablet form factor? I'm curious why it has QCS8550 instead of SM8550 SoC..
What is the Android version?

[Xu-honghao]

Hi @keith4ever ,

Thanks for your response.
QCS8550 is Qualcomm's IoT platform device. Since the original code does not have QCS8550 support options, I modified the code to use QCS8550. The code is as follows

SM8550 is Qualcomm's hardware for smartphones, and QCS is hardware for smart IoT, but the core chip should be the same. In addition, I am using Android 13 version.

[keith4ever]

I see, yes I agree you did the right modification.
If it still doesn't work, I think it's somehow causing an issue inside qnn libraries. We have tested all SM8x50 SoCs and none of them had an issue.
To give you an extra tip: please use 'debug' option at the command line to print out detailed delegation info:

whisperax_cli input_audio.aac tiny debug

And send us the log. This will help us analyze the issue.

[Xu-honghao]

Hi @keith4ever ,
OK,I just ran the program using debug, but it seems that the output file is not a log. I saved it, and I also saved the running log in log.txt, but due to screen limitations, it was not saved completely. I hope it helps.
output.json
log.txt