Skip to content

Adreno GPU support broken after commit 896718e #3935

Description

@egelja

Found using git bisect on a Rubik Pi 3 with a QCS6490 chipset.

Command run at each bisect step:

rm -rf build \
    && cmake -S . -B build -G Ninja -DCMAKE_BUILD_TYPE=Debug -DBUILD_SHARED_LIBS=OFF -DGGML_OPENCL=ON \
    && ninja -C build \
    && ./build/bin/whisper-cli -m models/ggml-base.en.bin -f samples/jfk.wav -nfa

Flash attention was disabled; enabling it would lead to garbage output on GPU. Runtime would also be ~10x slower.

Failing output

Commit 896718e

whisper_init_from_file_with_params_no_state: loading model from 'models/ggml-base.en.bin'
whisper_init_with_params_no_state: use gpu    = 1
whisper_init_with_params_no_state: flash attn = 0
whisper_init_with_params_no_state: gpu_device = 0
whisper_init_with_params_no_state: dtw        = 0
ggml_opencl: selected platform: 'QUALCOMM Snapdragon(TM)'

ggml_opencl: device: 'QUALCOMM Adreno(TM) 643 (OpenCL 3.0 Adreno(TM) 643)'
ggml_opencl: default device: 'QUALCOMM Adreno(TM) 643 (OpenCL 3.0 Adreno(TM) 643)'
register_backend: registered backend OpenCL (1 devices)
register_device: registered device GPUOpenCL (QUALCOMM Adreno(TM) 643)
register_backend: registered backend CPU (1 devices)
register_device: registered device CPU (CPU)
whisper_init_with_params_no_state: devices    = 2
whisper_init_with_params_no_state: backends   = 2
whisper_model_load: loading model
whisper_model_load: n_vocab       = 51864
whisper_model_load: n_audio_ctx   = 1500
whisper_model_load: n_audio_state = 512
whisper_model_load: n_audio_head  = 8
whisper_model_load: n_audio_layer = 6
whisper_model_load: n_text_ctx    = 448
whisper_model_load: n_text_state  = 512
whisper_model_load: n_text_head   = 8
whisper_model_load: n_text_layer  = 6
whisper_model_load: n_mels        = 80
whisper_model_load: ftype         = 1
whisper_model_load: qntvr         = 0
whisper_model_load: type          = 2 (base)
whisper_model_load: adding 1607 extra tokens
whisper_model_load: n_langs       = 99
Segmentation fault (core dumped)
GDB Backtrace
(gdb) bt full
#0  ggml_backend_opencl_buffer_type_get_alignment (buffer_type=0xaaaaab11d6d0) at /home/nmaruszewski3/whisper.cpp/ggml/src/ggml-opencl/ggml-opencl.cpp:7617
        dev_ctx = 0xaaaaab11d650
#1  0x0000aaaaaae08b94 in ggml_backend_buft_get_alignment (buft=0xaaaaab11d6d0) at /home/nmaruszewski3/whisper.cpp/ggml/src/ggml-backend.cpp:49
No locals.
#2  0x0000aaaaaae07ea0 in ggml_backend_alloc_ctx_tensors_from_buft_impl (ctx=0xaaaaab9d9240, buft=0xaaaaab11d6d0, nbytes_total=0xffffffffe120, no_alloc=false) at /home/nmaruszewski3/whisper.cpp/ggml/src/ggml-alloc.c:1171
        alignment = 187649985929176
        max_size = 0
        buffers = 0xfffff7b666f0 <global_max_fast>
        n_buffers = 208
        cur_buf_size = 34
        first = 0xfffff7ffdb58 <_rtld_global_ro>
        __func__ = "ggml_backend_alloc_ctx_tensors_from_buft_impl"
        buffer = 0xffffffffe100
#3  0x0000aaaaaae0828c in ggml_backend_alloc_ctx_tensors_from_buft (ctx=0xaaaaab9d9240, buft=0xaaaaab11d6d0) at /home/nmaruszewski3/whisper.cpp/ggml/src/ggml-alloc.c:1243
        nbytes_total = 0
#4  0x0000aaaaaabd04e0 in whisper_model_load (loader=0xffffffffe4b8, wctx=...) at /home/nmaruszewski3/whisper.cpp/src/whisper.cpp:1852
        buft = 0xaaaaab11d6d0
        ctx = 0xaaaaab9d9240
        buf = 0x17450
        p = {first = 0xaaaaab11d6d0, second = 0xaaaaab9d9240}
        __for_range = std::map with 1 element = {[0xaaaaab11d6d0] = 0xaaaaab9d9240}
        __for_begin = {first = 0xaaaaab11d6d0, second = 0xaaaaab9d9240}
        __for_end = {first = 0x1, second = 0xaaaaab11a2a0}
        __func__ = "whisper_model_load"
        t_start_us = 481658338674
        model = @0xaaaaab11d828: {type = MODEL_BASE, hparams = {n_vocab = 51864, n_audio_ctx = 1500, n_audio_state = 512, n_audio_head = 8, n_audio_layer = 6, n_text_ctx = 448, n_text_state = 512, n_text_head = 8, n_text_layer = 6, n_mels = 80, ftype = 1, eps = 9.99999975e-06}, filters = {n_mel = 80, n_fft = 201,
            data = std::vector of length 16080, capacity 16080 = {0, 0.0248625949, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
              0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
              0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0...}}, e_pe = 0xaaaaab9f0fe0, e_conv_1_w = 0xaaaaab9f1150, e_conv_1_b = 0xaaaaab9f12c0, e_conv_2_w = 0xaaaaab9f1430, e_conv_2_b = 0xaaaaab9f15a0, e_ln_w = 0xaaaaab9f1710, e_ln_b = 0xaaaaab9f1880, d_pe = 0xaaaaab9f9b50,
          d_te = 0xaaaaab9f9cc0, d_ln_w = 0xaaaaab9f9e30, d_ln_b = 0xaaaaab9f9fa0, layers_encoder = std::vector of length 6, capacity 6 = {{attn_ln_0_w = 0xaaaaab9f2290, attn_ln_0_b = 0xaaaaab9f2400, attn_ln_1_w = 0xaaaaab9f2ca0, attn_ln_1_b = 0xaaaaab9f2e10, attn_q_w = 0xaaaaab9f2570, attn_q_b = 0xaaaaab9f26e0,
              attn_k_w = 0xaaaaab9f2850, attn_v_w = 0xaaaaab9f29c0, attn_v_b = 0xaaaaab9f2b30, mlp_ln_w = 0xaaaaab9f19f0, mlp_ln_b = 0xaaaaab9f1b60, mlp_0_w = 0xaaaaab9f1cd0, mlp_0_b = 0xaaaaab9f1e40, mlp_1_w = 0xaaaaab9f1fb0, mlp_1_b = 0xaaaaab9f2120}, {attn_ln_0_w = 0xaaaaab9f3820, attn_ln_0_b = 0xaaaaab9f3990,
              attn_ln_1_w = 0xaaaaab9f4230, attn_ln_1_b = 0xaaaaab9f43a0, attn_q_w = 0xaaaaab9f3b00, attn_q_b = 0xaaaaab9f3c70, attn_k_w = 0xaaaaab9f3de0, attn_v_w = 0xaaaaab9f3f50, attn_v_b = 0xaaaaab9f40c0, mlp_ln_w = 0xaaaaab9f2f80, mlp_ln_b = 0xaaaaab9f30f0, mlp_0_w = 0xaaaaab9f3260, mlp_0_b = 0xaaaaab9f33d0,
              mlp_1_w = 0xaaaaab9f3540, mlp_1_b = 0xaaaaab9f36b0}, {attn_ln_0_w = 0xaaaaab9f4db0, attn_ln_0_b = 0xaaaaab9f4f20, attn_ln_1_w = 0xaaaaab9f57c0, attn_ln_1_b = 0xaaaaab9f5930, attn_q_w = 0xaaaaab9f5090, attn_q_b = 0xaaaaab9f5200, attn_k_w = 0xaaaaab9f5370, attn_v_w = 0xaaaaab9f54e0,
              attn_v_b = 0xaaaaab9f5650, mlp_ln_w = 0xaaaaab9f4510, mlp_ln_b = 0xaaaaab9f4680, mlp_0_w = 0xaaaaab9f47f0, mlp_0_b = 0xaaaaab9f4960, mlp_1_w = 0xaaaaab9f4ad0, mlp_1_b = 0xaaaaab9f4c40}, {attn_ln_0_w = 0xaaaaab9f6340, attn_ln_0_b = 0xaaaaab9f64b0, attn_ln_1_w = 0xaaaaab9f6d50,
              attn_ln_1_b = 0xaaaaab9f6ec0, attn_q_w = 0xaaaaab9f6620, attn_q_b = 0xaaaaab9f6790, attn_k_w = 0xaaaaab9f6900, attn_v_w = 0xaaaaab9f6a70, attn_v_b = 0xaaaaab9f6be0, mlp_ln_w = 0xaaaaab9f5aa0, mlp_ln_b = 0xaaaaab9f5c10, mlp_0_w = 0xaaaaab9f5d80, mlp_0_b = 0xaaaaab9f5ef0, mlp_1_w = 0xaaaaab9f6060,
              mlp_1_b = 0xaaaaab9f61d0}, {attn_ln_0_w = 0xaaaaab9f78d0, attn_ln_0_b = 0xaaaaab9f7a40, attn_ln_1_w = 0xaaaaab9f82e0, attn_ln_1_b = 0xaaaaab9f8450, attn_q_w = 0xaaaaab9f7bb0, attn_q_b = 0xaaaaab9f7d20, attn_k_w = 0xaaaaab9f7e90, attn_v_w = 0xaaaaab9f8000, attn_v_b = 0xaaaaab9f8170,
              mlp_ln_w = 0xaaaaab9f7030, mlp_ln_b = 0xaaaaab9f71a0, mlp_0_w = 0xaaaaab9f7310, mlp_0_b = 0xaaaaab9f7480, mlp_1_w = 0xaaaaab9f75f0, mlp_1_b = 0xaaaaab9f7760}, {attn_ln_0_w = 0xaaaaab9f8e60, attn_ln_0_b = 0xaaaaab9f8fd0, attn_ln_1_w = 0xaaaaab9f9870, attn_ln_1_b = 0xaaaaab9f99e0,
              attn_q_w = 0xaaaaab9f9140, attn_q_b = 0xaaaaab9f92b0, attn_k_w = 0xaaaaab9f9420, attn_v_w = 0xaaaaab9f9590, attn_v_b = 0xaaaaab9f9700, mlp_ln_w = 0xaaaaab9f85c0, mlp_ln_b = 0xaaaaab9f8730, mlp_0_w = 0xaaaaab9f88a0, mlp_0_b = 0xaaaaab9f8a10, mlp_1_w = 0xaaaaab9f8b80, mlp_1_b = 0xaaaaab9f8cf0}},
          layers_decoder = std::vector of length 6, capacity 6 = {{attn_ln_0_w = 0xaaaaab9fa9b0, attn_ln_0_b = 0xaaaaab9fab20, attn_ln_1_w = 0xaaaaab9fb3c0, attn_ln_1_b = 0xaaaaab9fb530, attn_q_w = 0xaaaaab9fac90, attn_q_b = 0xaaaaab9fae00, attn_k_w = 0xaaaaab9faf70, attn_v_w = 0xaaaaab9fb0e0,
              attn_v_b = 0xaaaaab9fb250, cross_attn_ln_0_w = 0xaaaaab9fb6a0, cross_attn_ln_0_b = 0xaaaaab9fb810, cross_attn_ln_1_w = 0xaaaaab9fc0b0, cross_attn_ln_1_b = 0xaaaaab9fc220, cross_attn_q_w = 0xaaaaab9fb980, cross_attn_q_b = 0xaaaaab9fbaf0, cross_attn_k_w = 0xaaaaab9fbc60,
              cross_attn_v_w = 0xaaaaab9fbdd0, cross_attn_v_b = 0xaaaaab9fbf40, mlp_ln_w = 0xaaaaab9fa110, mlp_ln_b = 0xaaaaab9fa280, mlp_0_w = 0xaaaaab9fa3f0, mlp_0_b = 0xaaaaab9fa560, mlp_1_w = 0xaaaaab9fa6d0, mlp_1_b = 0xaaaaab9fa840}, {attn_ln_0_w = 0xaaaaab9fcc30, attn_ln_0_b = 0xaaaaab9fcda0,
              attn_ln_1_w = 0xaaaaab9fd640, attn_ln_1_b = 0xaaaaab9fd7b0, attn_q_w = 0xaaaaab9fcf10, attn_q_b = 0xaaaaab9fd080, attn_k_w = 0xaaaaab9fd1f0, attn_v_w = 0xaaaaab9fd360, attn_v_b = 0xaaaaab9fd4d0, cross_attn_ln_0_w = 0xaaaaab9fd920, cross_attn_ln_0_b = 0xaaaaab9fda90,
              cross_attn_ln_1_w = 0xaaaaab9fe330, cross_attn_ln_1_b = 0xaaaaab9fe4a0, cross_attn_q_w = 0xaaaaab9fdc00, cross_attn_q_b = 0xaaaaab9fdd70, cross_attn_k_w = 0xaaaaab9fdee0, cross_attn_v_w = 0xaaaaab9fe050, cross_attn_v_b = 0xaaaaab9fe1c0, mlp_ln_w = 0xaaaaab9fc390, mlp_ln_b = 0xaaaaab9fc500,
              mlp_0_w = 0xaaaaab9fc670, mlp_0_b = 0xaaaaab9fc7e0, mlp_1_w = 0xaaaaab9fc950, mlp_1_b = 0xaaaaab9fcac0}, {attn_ln_0_w = 0xaaaaab9feeb0, attn_ln_0_b = 0xaaaaab9ff020, attn_ln_1_w = 0xaaaaab9ff8c0, attn_ln_1_b = 0xaaaaab9ffa30, attn_q_w = 0xaaaaab9ff190, attn_q_b = 0xaaaaab9ff300,
              attn_k_w = 0xaaaaab9ff470, attn_v_w = 0xaaaaab9ff5e0, attn_v_b = 0xaaaaab9ff750, cross_attn_ln_0_w = 0xaaaaab9ffba0, cross_attn_ln_0_b = 0xaaaaab9ffd10, cross_attn_ln_1_w = 0xaaaaaba005b0, cross_attn_ln_1_b = 0xaaaaaba00720, cross_attn_q_w = 0xaaaaab9ffe80, cross_attn_q_b = 0xaaaaab9ffff0,
              cross_attn_k_w = 0xaaaaaba00160, cross_attn_v_w = 0xaaaaaba002d0, cross_attn_v_b = 0xaaaaaba00440, mlp_ln_w = 0xaaaaab9fe610, mlp_ln_b = 0xaaaaab9fe780, mlp_0_w = 0xaaaaab9fe8f0, mlp_0_b = 0xaaaaab9fea60, mlp_1_w = 0xaaaaab9febd0, mlp_1_b = 0xaaaaab9fed40}, {attn_ln_0_w = 0xaaaaaba01130,
              attn_ln_0_b = 0xaaaaaba012a0, attn_ln_1_w = 0xaaaaaba01b40, attn_ln_1_b = 0xaaaaaba01cb0, attn_q_w = 0xaaaaaba01410, attn_q_b = 0xaaaaaba01580, attn_k_w = 0xaaaaaba016f0, attn_v_w = 0xaaaaaba01860, attn_v_b = 0xaaaaaba019d0, cross_attn_ln_0_w = 0xaaaaaba01e20, cross_attn_ln_0_b = 0xaaaaaba01f90,
              cross_attn_ln_1_w = 0xaaaaaba02830, cross_attn_ln_1_b = 0xaaaaaba029a0, cross_attn_q_w = 0xaaaaaba02100, cross_attn_q_b = 0xaaaaaba02270, cross_attn_k_w = 0xaaaaaba023e0, cross_attn_v_w = 0xaaaaaba02550, cross_attn_v_b = 0xaaaaaba026c0, mlp_ln_w = 0xaaaaaba00890, mlp_ln_b = 0xaaaaaba00a00,
              mlp_0_w = 0xaaaaaba00b70, mlp_0_b = 0xaaaaaba00ce0, mlp_1_w = 0xaaaaaba00e50, mlp_1_b = 0xaaaaaba00fc0}, {attn_ln_0_w = 0xaaaaaba033b0, attn_ln_0_b = 0xaaaaaba03520, attn_ln_1_w = 0xaaaaaba03dc0, attn_ln_1_b = 0xaaaaaba03f30, attn_q_w = 0xaaaaaba03690, attn_q_b = 0xaaaaaba03800,
              attn_k_w = 0xaaaaaba03970, attn_v_w = 0xaaaaaba03ae0, attn_v_b = 0xaaaaaba03c50, cross_attn_ln_0_w = 0xaaaaaba040a0, cross_attn_ln_0_b = 0xaaaaaba04210, cross_attn_ln_1_w = 0xaaaaaba04ab0, cross_attn_ln_1_b = 0xaaaaaba04c20, cross_attn_q_w = 0xaaaaaba04380, cross_attn_q_b = 0xaaaaaba044f0,
              cross_attn_k_w = 0xaaaaaba04660, cross_attn_v_w = 0xaaaaaba047d0, cross_attn_v_b = 0xaaaaaba04940, mlp_ln_w = 0xaaaaaba02b10, mlp_ln_b = 0xaaaaaba02c80, mlp_0_w = 0xaaaaaba02df0, mlp_0_b = 0xaaaaaba02f60, mlp_1_w = 0xaaaaaba030d0, mlp_1_b = 0xaaaaaba03240}, {attn_ln_0_w = 0xaaaaaba05630,
              attn_ln_0_b = 0xaaaaaba057a0, attn_ln_1_w = 0xaaaaaba06040, attn_ln_1_b = 0xaaaaaba061b0, attn_q_w = 0xaaaaaba05910, attn_q_b = 0xaaaaaba05a80, attn_k_w = 0xaaaaaba05bf0, attn_v_w = 0xaaaaaba05d60, attn_v_b = 0xaaaaaba05ed0, cross_attn_ln_0_w = 0xaaaaaba06320, cross_attn_ln_0_b = 0xaaaaaba06490,
              cross_attn_ln_1_w = 0xaaaaaba06d30, cross_attn_ln_1_b = 0xaaaaaba06ea0, cross_attn_q_w = 0xaaaaaba06600, cross_attn_q_b = 0xaaaaaba06770, cross_attn_k_w = 0xaaaaaba068e0, cross_attn_v_w = 0xaaaaaba06a50, cross_attn_v_b = 0xaaaaaba06bc0, mlp_ln_w = 0xaaaaaba04d90, mlp_ln_b = 0xaaaaaba04f00,
              mlp_0_w = 0xaaaaaba05070, mlp_0_b = 0xaaaaaba051e0, mlp_1_w = 0xaaaaaba05350, mlp_1_b = 0xaaaaaba054c0}}, ctxs = std::vector of length 1, capacity 1 = {0xaaaaab9d9240}, buffers = std::vector of length 0, capacity 0, n_loaded = 0, tensors = std::map with 245 elements = {
            ["decoder.blocks.0.attn.key.weight"] = 0xaaaaab9faf70, ["decoder.blocks.0.attn.out.bias"] = 0xaaaaab9fb530, ["decoder.blocks.0.attn.out.weight"] = 0xaaaaab9fb3c0, ["decoder.blocks.0.attn.query.bias"] = 0xaaaaab9fae00, ["decoder.blocks.0.attn.query.weight"] = 0xaaaaab9fac90,
            ["decoder.blocks.0.attn.value.bias"] = 0xaaaaab9fb250, ["decoder.blocks.0.attn.value.weight"] = 0xaaaaab9fb0e0, ["decoder.blocks.0.attn_ln.bias"] = 0xaaaaab9fab20, ["decoder.blocks.0.attn_ln.weight"] = 0xaaaaab9fa9b0, ["decoder.blocks.0.cross_attn.key.weight"] = 0xaaaaab9fbc60,
            ["decoder.blocks.0.cross_attn.out.bias"] = 0xaaaaab9fc220, ["decoder.blocks.0.cross_attn.out.weight"] = 0xaaaaab9fc0b0, ["decoder.blocks.0.cross_attn.query.bias"] = 0xaaaaab9fbaf0, ["decoder.blocks.0.cross_attn.query.weight"] = 0xaaaaab9fb980,
            ["decoder.blocks.0.cross_attn.value.bias"] = 0xaaaaab9fbf40, ["decoder.blocks.0.cross_attn.value.weight"] = 0xaaaaab9fbdd0, ["decoder.blocks.0.cross_attn_ln.bias"] = 0xaaaaab9fb810, ["decoder.blocks.0.cross_attn_ln.weight"] = 0xaaaaab9fb6a0, ["decoder.blocks.0.mlp.0.bias"] = 0xaaaaab9fa560,
            ["decoder.blocks.0.mlp.0.weight"] = 0xaaaaab9fa3f0, ["decoder.blocks.0.mlp.2.bias"] = 0xaaaaab9fa840, ["decoder.blocks.0.mlp.2.weight"] = 0xaaaaab9fa6d0, ["decoder.blocks.0.mlp_ln.bias"] = 0xaaaaab9fa280, ["decoder.blocks.0.mlp_ln.weight"] = 0xaaaaab9fa110,
            ["decoder.blocks.1.attn.key.weight"] = 0xaaaaab9fd1f0, ["decoder.blocks.1.attn.out.bias"] = 0xaaaaab9fd7b0, ["decoder.blocks.1.attn.out.weight"] = 0xaaaaab9fd640, ["decoder.blocks.1.attn.query.bias"] = 0xaaaaab9fd080, ["decoder.blocks.1.attn.query.weight"] = 0xaaaaab9fcf10,
            ["decoder.blocks.1.attn.value.bias"] = 0xaaaaab9fd4d0, ["decoder.blocks.1.attn.value.weight"] = 0xaaaaab9fd360, ["decoder.blocks.1.attn_ln.bias"] = 0xaaaaab9fcda0, ["decoder.blocks.1.attn_ln.weight"] = 0xaaaaab9fcc30, ["decoder.blocks.1.cross_attn.key.weight"] = 0xaaaaab9fdee0,
            ["decoder.blocks.1.cross_attn.out.bias"] = 0xaaaaab9fe4a0, ["decoder.blocks.1.cross_attn.out.weight"] = 0xaaaaab9fe330, ["decoder.blocks.1.cross_attn.query.bias"] = 0xaaaaab9fdd70, ["decoder.blocks.1.cross_attn.query.weight"] = 0xaaaaab9fdc00,
            ["decoder.blocks.1.cross_attn.value.bias"] = 0xaaaaab9fe1c0, ["decoder.blocks.1.cross_attn.value.weight"] = 0xaaaaab9fe050, ["decoder.blocks.1.cross_attn_ln.bias"] = 0xaaaaab9fda90, ["decoder.blocks.1.cross_attn_ln.weight"] = 0xaaaaab9fd920, ["decoder.blocks.1.mlp.0.bias"] = 0xaaaaab9fc7e0,
            ["decoder.blocks.1.mlp.0.weight"] = 0xaaaaab9fc670, ["decoder.blocks.1.mlp.2.bias"] = 0xaaaaab9fcac0, ["decoder.blocks.1.mlp.2.weight"] = 0xaaaaab9fc950, ["decoder.blocks.1.mlp_ln.bias"] = 0xaaaaab9fc500, ["decoder.blocks.1.mlp_ln.weight"] = 0xaaaaab9fc390,
            ["decoder.blocks.2.attn.key.weight"] = 0xaaaaab9ff470, ["decoder.blocks.2.attn.out.bias"] = 0xaaaaab9ffa30, ["decoder.blocks.2.attn.out.weight"] = 0xaaaaab9ff8c0, ["decoder.blocks.2.attn.query.bias"] = 0xaaaaab9ff300, ["decoder.blocks.2.attn.query.weight"] = 0xaaaaab9ff190,
            ["decoder.blocks.2.attn.value.bias"] = 0xaaaaab9ff750, ["decoder.blocks.2.attn.value.weight"] = 0xaaaaab9ff5e0, ["decoder.blocks.2.attn_ln.bias"] = 0xaaaaab9ff020, ["decoder.blocks.2.attn_ln.weight"] = 0xaaaaab9feeb0, ["decoder.blocks.2.cross_attn.key.weight"] = 0xaaaaaba00160,
--Type <RET> for more, q to quit, c to continue without paging--
            ["decoder.blocks.2.cross_attn.out.bias"] = 0xaaaaaba00720, ["decoder.blocks.2.cross_attn.out.weight"] = 0xaaaaaba005b0, ["decoder.blocks.2.cross_attn.query.bias"] = 0xaaaaab9ffff0, ["decoder.blocks.2.cross_attn.query.weight"] = 0xaaaaab9ffe80,
            ["decoder.blocks.2.cross_attn.value.bias"] = 0xaaaaaba00440, ["decoder.blocks.2.cross_attn.value.weight"] = 0xaaaaaba002d0, ["decoder.blocks.2.cross_attn_ln.bias"] = 0xaaaaab9ffd10, ["decoder.blocks.2.cross_attn_ln.weight"] = 0xaaaaab9ffba0, ["decoder.blocks.2.mlp.0.bias"] = 0xaaaaab9fea60,
            ["decoder.blocks.2.mlp.0.weight"] = 0xaaaaab9fe8f0, ["decoder.blocks.2.mlp.2.bias"] = 0xaaaaab9fed40, ["decoder.blocks.2.mlp.2.weight"] = 0xaaaaab9febd0, ["decoder.blocks.2.mlp_ln.bias"] = 0xaaaaab9fe780, ["decoder.blocks.2.mlp_ln.weight"] = 0xaaaaab9fe610,
            ["decoder.blocks.3.attn.key.weight"] = 0xaaaaaba016f0, ["decoder.blocks.3.attn.out.bias"] = 0xaaaaaba01cb0, ["decoder.blocks.3.attn.out.weight"] = 0xaaaaaba01b40, ["decoder.blocks.3.attn.query.bias"] = 0xaaaaaba01580, ["decoder.blocks.3.attn.query.weight"] = 0xaaaaaba01410,
            ["decoder.blocks.3.attn.value.bias"] = 0xaaaaaba019d0, ["decoder.blocks.3.attn.value.weight"] = 0xaaaaaba01860, ["decoder.blocks.3.attn_ln.bias"] = 0xaaaaaba012a0, ["decoder.blocks.3.attn_ln.weight"] = 0xaaaaaba01130, ["decoder.blocks.3.cross_attn.key.weight"] = 0xaaaaaba023e0,
            ["decoder.blocks.3.cross_attn.out.bias"] = 0xaaaaaba029a0, ["decoder.blocks.3.cross_attn.out.weight"] = 0xaaaaaba02830, ["decoder.blocks.3.cross_attn.query.bias"] = 0xaaaaaba02270, ["decoder.blocks.3.cross_attn.query.weight"] = 0xaaaaaba02100,
            ["decoder.blocks.3.cross_attn.value.bias"] = 0xaaaaaba026c0, ["decoder.blocks.3.cross_attn.value.weight"] = 0xaaaaaba02550, ["decoder.blocks.3.cross_attn_ln.bias"] = 0xaaaaaba01f90, ["decoder.blocks.3.cross_attn_ln.weight"] = 0xaaaaaba01e20, ["decoder.blocks.3.mlp.0.bias"] = 0xaaaaaba00ce0,
            ["decoder.blocks.3.mlp.0.weight"] = 0xaaaaaba00b70, ["decoder.blocks.3.mlp.2.bias"] = 0xaaaaaba00fc0, ["decoder.blocks.3.mlp.2.weight"] = 0xaaaaaba00e50, ["decoder.blocks.3.mlp_ln.bias"] = 0xaaaaaba00a00, ["decoder.blocks.3.mlp_ln.weight"] = 0xaaaaaba00890,
            ["decoder.blocks.4.attn.key.weight"] = 0xaaaaaba03970, ["decoder.blocks.4.attn.out.bias"] = 0xaaaaaba03f30, ["decoder.blocks.4.attn.out.weight"] = 0xaaaaaba03dc0, ["decoder.blocks.4.attn.query.bias"] = 0xaaaaaba03800...}}
        vocab = @0xaaaaab11d970: {n_vocab = 51864, token_to_id = std::map with 51864 elements = {[""] = 188, ["\001"] = 189, ["\002"] = 190, ["\003"] = 191, ["\004"] = 192, ["\005"] = 193, ["\006"] = 194, ["\a"] = 195, ["\b"] = 196, ["\t"] = 197, ["\n"] = 198, ["\n\n"] = 628, ["\n "] = 44320, ["\v"] = 199,
            ["\f"] = 200, ["\r"] = 201, ["\016"] = 202, ["\017"] = 203, ["\020"] = 204, ["\021"] = 205, ["\022"] = 206, ["\023"] = 207, ["\024"] = 208, ["\025"] = 209, ["\026"] = 210, ["\027"] = 211, ["\030"] = 212, ["\031"] = 213, ["\032"] = 214, ["\033"] = 215, ["\034"] = 216, ["\035"] = 217, ["\036"] = 218,
            ["\037"] = 219, [" "] = 220, [" !"] = 5145, [" !!"] = 37867, [" !="] = 14512, [" \""] = 366, [" \"\""] = 13538, [" \"\"\""] = 37227, [" \"#"] = 25113, [" \"$"] = 17971, [" \"$:/"] = 32047, [" \"%"] = 36521, [" \"'"] = 24018, [" \"("] = 30629, [" \"+"] = 43825, [" \","] = 33172, [" \"-"] = 27444,
            [" \"."] = 27071, [" \"..."] = 27896, [" \"/"] = 12813, [" \"<"] = 33490, [" \"@"] = 44212, [" \"["] = 12878, [" \"\\"] = 37082, [" \"_"] = 45434, [" \"{"] = 45144, [" \"…"] = 29368, [" #"] = 1303, [" ##"] = 22492, [" ###"] = 44386, [" #####"] = 46424, [" $"] = 720, [" $$"] = 32382, [" $("] = 29568,
            [" $\\"] = 39280, [" $_"] = 40111, [" ${"] = 25597, [" %"] = 4064, [" %%"] = 43313, [" &"] = 1222, [" &&"] = 11405, [" '"] = 705, [" ''"] = 10148, [" '("] = 29513, [" ',"] = 46083, [" '."] = 45302, [" '/"] = 31051, [" '["] = 44438, [" ("] = 357, [" (!"] = 22759, [" (\""] = 5855, [" (#"] = 17426,
            [" ($"] = 7198, [" ($)"] = 45491, [" (%"] = 37633, [" (%)"] = 11509, [" (&"] = 35494, [" ('"] = 19203, [" (("] = 14808, [" ()"] = 7499, [" ())"] = 32865, [" ());"] = 38377, [" (),"] = 29994, [" ()."] = 27972, [" ();"] = 13979, [" (*"] = 20789, [" (+"] = 11502...},
          id_to_token = std::map with 51864 elements = {[0] = "!", [1] = "\"", [2] = "#", [3] = "$", [4] = "%", [5] = "&", [6] = "'", [7] = "(", [8] = ")", [9] = "*", [10] = "+", [11] = ",", [12] = "-", [13] = ".", [14] = "/", [15] = "0", [16] = "1", [17] = "2", [18] = "3", [19] = "4", [20] = "5", [21] = "6",
            [22] = "7", [23] = "8", [24] = "9", [25] = ":", [26] = ";", [27] = "<", [28] = "=", [29] = ">", [30] = "?", [31] = "@", [32] = "A", [33] = "B", [34] = "C", [35] = "D", [36] = "E", [37] = "F", [38] = "G", [39] = "H", [40] = "I", [41] = "J", [42] = "K", [43] = "L", [44] = "M", [45] = "N", [46] = "O",
            [47] = "P", [48] = "Q", [49] = "R", [50] = "S", [51] = "T", [52] = "U", [53] = "V", [54] = "W", [55] = "X", [56] = "Y", [57] = "Z", [58] = "[", [59] = "\\", [60] = "]", [61] = "^", [62] = "_", [63] = "`", [64] = "a", [65] = "b", [66] = "c", [67] = "d", [68] = "e", [69] = "f", [70] = "g", [71] = "h",
            [72] = "i", [73] = "j", [74] = "k", [75] = "l", [76] = "m", [77] = "n", [78] = "o", [79] = "p", [80] = "q", [81] = "r", [82] = "s", [83] = "t", [84] = "u", [85] = "v", [86] = "w", [87] = "x", [88] = "y", [89] = "z", [90] = "{", [91] = "|", [92] = "}", [93] = "~", [94] = "\241", [95] = "\242",
            [96] = "\243", [97] = "\244", [98] = "\245", [99] = "\246"...}, token_eot = 50256, token_sot = 50257, token_translate = 50357, token_transcribe = 50358, token_solm = 50359, token_prev = 50360, token_nosp = 50361, token_not = 50362, token_beg = 50363}
        __PRETTY_FUNCTION__ = "bool whisper_model_load(whisper_model_loader*, whisper_context&)"
        wtype = GGML_TYPE_F16
        vtype = GGML_TYPE_F16
        hparams = @0xaaaaab11d82c: {n_vocab = 51864, n_audio_ctx = 1500, n_audio_state = 512, n_audio_head = 8, n_audio_layer = 6, n_text_ctx = 448, n_text_state = 512, n_text_head = 8, n_text_layer = 6, n_mels = 80, ftype = 1, eps = 9.99999975e-06}
        n_audio_layer = 6
        n_text_layer = 6
        n_tensors = 259
        ctx_map = std::map with 1 element = {[0xaaaaab11d6d0] = 0xaaaaab9d9240}
        get_ctx = {__ctx_map = std::map with 1 element = {[0xaaaaab11d6d0] = 0xaaaaab9d9240}, __n_tensors = @0xffffffffe1e0, __model = @0xaaaaab11d828}
        buft_list = std::vector of length 3, capacity 4 = {{first = 0xaaaaab1d73a0, second = 0xaaaaab11d6d0}, {first = 0xaaaaab071cf8 <ggml_backend_cpu_reg_get_device(ggml_backend_reg*, unsigned long)::ggml_backend_cpu_device>,
            second = 0xaaaaab071e30 <ggml_backend_cpu_repack_buffer_type()::ggml_backend_cpu_buffer_type_repack>}, {first = 0xaaaaab071cf8 <ggml_backend_cpu_reg_get_device(ggml_backend_reg*, unsigned long)::ggml_backend_cpu_device>,
            second = 0xaaaaab072730 <ggml_backend_cpu_buffer_type::ggml_backend_cpu_buffer_type>}}
        create_tensor = {__hparams = @0xaaaaab11d82c, __buft_list = std::vector of length 3, capacity 4 = {{first = 0xaaaaab1d73a0, second = 0xaaaaab11d6d0}, {first = 0xaaaaab071cf8 <ggml_backend_cpu_reg_get_device(ggml_backend_reg*, unsigned long)::ggml_backend_cpu_device>,
              second = 0xaaaaab071e30 <ggml_backend_cpu_repack_buffer_type()::ggml_backend_cpu_buffer_type_repack>}, {first = 0xaaaaab071cf8 <ggml_backend_cpu_reg_get_device(ggml_backend_reg*, unsigned long)::ggml_backend_cpu_device>,
              second = 0xaaaaab072730 <ggml_backend_cpu_buffer_type::ggml_backend_cpu_buffer_type>}}, __get_ctx = @0xffffffffe2b8, __model = @0xaaaaab11d828}
#5  0x0000aaaaaabd6b88 in whisper_init_with_params_no_state (loader=0xffffffffe4b8, params=...) at /home/nmaruszewski3/whisper.cpp/src/whisper.cpp:3723
        __func__ = "whisper_init_with_params_no_state"
        ctx = 0xaaaaab11d7e0
#6  0x0000aaaaaabd66f0 in whisper_init_from_file_with_params_no_state (path_model=0xaaaaab119e80 "models/ggml-base.en.bin", params=...) at /home/nmaruszewski3/whisper.cpp/src/whisper.cpp:3659
        __func__ = "whisper_init_from_file_with_params_no_state"
        fin = <incomplete type>
        loader = {context = 0xffffffffe4d8, read = 0xaaaaaabd64d4 <_FUN(void*, void*, size_t)>, eof = 0xaaaaaabd6560 <_FUN(void*)>, close = 0xaaaaaabd65c8 <_FUN(void*)>}
        ctx = 0x0
#7  0x0000aaaaaabd6c54 in whisper_init_from_file_with_params (path_model=0xaaaaab119e80 "models/ggml-base.en.bin", params=...) at /home/nmaruszewski3/whisper.cpp/src/whisper.cpp:3736
        ctx = 0xffffffffee90
#8  0x0000aaaaaaab9e2c in main (argc=6, argv=0xfffffffff208) at /home/nmaruszewski3/whisper.cpp/examples/cli/cli.cpp:1039
        params = {n_threads = 4, n_processors = 1, offset_t_ms = 0, offset_n = 0, duration_ms = 0, progress_step = 5, max_context = -1, max_len = 0, best_of = 5, beam_size = 5, audio_ctx = 0, word_thold = 0.00999999978, entropy_thold = 2.4000001, logprob_thold = -1, no_speech_thold = 0.600000024,
          grammar_penalty = 100, temperature = 0, temperature_inc = 0.200000003, debug_mode = false, translate = false, detect_language = false, diarize = false, tinydiarize = false, split_on_word = false, no_fallback = false, output_txt = false, output_vtt = false, output_srt = false, output_wts = false,
          output_csv = false, output_jsn = false, output_jsn_full = false, output_lrc = false, no_prints = false, print_special = false, print_colors = false, print_confidence = false, print_progress = false, no_timestamps = false, log_score = false, use_gpu = true, flash_attn = false, gpu_device = 0,
          suppress_nst = false, carry_initial_prompt = false, language = "en", prompt = "", font_path = "/System/Library/Fonts/Supplemental/Courier New Bold.ttf", model = "models/ggml-base.en.bin", grammar = "", grammar_rule = "", tdrz_speaker_turn = " [SPEAKER_TURN]", suppress_regex = "",
          openvino_encode_device = "CPU", dtw = "", fname_inp = std::vector of length 1, capacity 1 = {"samples/jfk.wav"}, fname_out = std::vector of length 0, capacity 0, grammar_parsed = {symbol_ids = std::map with 0 elements, rules = std::vector of length 0, capacity 0}, vad = false, vad_model = "",
          vad_threshold = 0.5, vad_min_speech_duration_ms = 250, vad_min_silence_duration_ms = 100, vad_max_speech_duration_s = 3.40282347e+38, vad_speech_pad_ms = 30, vad_samples_overlap = 0.100000001}
        vec_args = std::vector of length 0, capacity 0
        cparams = {use_gpu = true, flash_attn = false, gpu_device = 0, dtw_token_timestamps = false, dtw_aheads_preset = WHISPER_AHEADS_NONE, dtw_n_top = -1, dtw_aheads = {n_heads = 0, heads = 0x0}, dtw_mem_size = 134217728}
        ctx = 0x2
        __func__ = "main"

Expected output

Commit ad717a6 (parent of broken commit)

whisper_init_from_file_with_params_no_state: loading model from 'models/ggml-base.en.bin'
whisper_init_with_params_no_state: use gpu    = 1
whisper_init_with_params_no_state: flash attn = 0
whisper_init_with_params_no_state: gpu_device = 0
whisper_init_with_params_no_state: dtw        = 0
ggml_opencl: selected platform: 'QUALCOMM Snapdragon(TM)'

ggml_opencl: device: 'QUALCOMM Adreno(TM) 643 (OpenCL 3.0 Adreno(TM) 643)'
ggml_opencl: OpenCL driver: OpenCL 3.0 QUALCOMM build: 0838.2 Compiler E031.50.15.00
ggml_opencl: vector subgroup broadcast support: true
ggml_opencl: device FP16 support: true
ggml_opencl: mem base addr align: 128
ggml_opencl: global mem size: 3642 MB
ggml_opencl: max mem alloc size: 910 MB
ggml_opencl: device max image buffer size (pixels): 134217728
ggml_opencl: device max image2d size: 16384 x 16384
ggml_opencl: device max workgroup size: 1024
ggml_opencl: SVM coarse grain buffer support: true
ggml_opencl: SVM fine grain buffer support: true
ggml_opencl: SVM fine grain system support: false
ggml_opencl: SVM atomics support: true
ggml_opencl: flattening quantized weights representation as struct of arrays (GGML_OPENCL_SOA_Q)
ggml_opencl: using kernels optimized for Adreno (GGML_OPENCL_USE_ADRENO_KERNELS)
ggml_opencl: loading OpenCL kernels....................................................................................................................................
ggml_opencl: default device: 'QUALCOMM Adreno(TM) 643 (OpenCL 3.0 Adreno(TM) 643)'
register_backend: registered backend OpenCL (1 devices)
register_device: registered device GPUOpenCL (QUALCOMM Adreno(TM) 643)
register_backend: registered backend CPU (1 devices)
register_device: registered device CPU (CPU)
whisper_init_with_params_no_state: devices    = 2
whisper_init_with_params_no_state: backends   = 2
whisper_model_load: loading model
whisper_model_load: n_vocab       = 51864
whisper_model_load: n_audio_ctx   = 1500
whisper_model_load: n_audio_state = 512
whisper_model_load: n_audio_head  = 8
whisper_model_load: n_audio_layer = 6
whisper_model_load: n_text_ctx    = 448
whisper_model_load: n_text_state  = 512
whisper_model_load: n_text_head   = 8
whisper_model_load: n_text_layer  = 6
whisper_model_load: n_mels        = 80
whisper_model_load: ftype         = 1
whisper_model_load: qntvr         = 0
whisper_model_load: type          = 2 (base)
whisper_model_load: adding 1607 extra tokens
whisper_model_load: n_langs       = 99
whisper_model_load:       OpenCL total size =   147.37 MB
whisper_model_load: model size    =  147.37 MB
whisper_backend_init_gpu: device 0: GPUOpenCL (type: 1)
whisper_backend_init_gpu: found GPU device 0: GPUOpenCL (type: 1, cnt: 0)
whisper_backend_init_gpu: using GPUOpenCL backend
whisper_init_state: kv self size  =    6.29 MB
whisper_init_state: kv cross size =   18.87 MB
whisper_init_state: kv pad  size  =    3.15 MB
whisper_init_state: compute buffer (conv)   =   17.24 MB
whisper_init_state: compute buffer (encode) =   85.88 MB
whisper_init_state: compute buffer (cross)  =    4.66 MB
whisper_init_state: compute buffer (decode) =   97.29 MB

system_info: n_threads = 4 / 8 | WHISPER : COREML = 0 | OPENVINO = 0 | CPU : NEON = 1 | ARM_FMA = 1 | OPENMP = 1 | REPACK = 1 |

main: processing 'samples/jfk.wav' (176000 samples, 11.0 sec), 4 threads, 1 processors, 5 beams + best of 5, lang = en, task = transcribe, timestamps = 1 ...


[00:00:00.000 --> 00:00:10.620]   So my fellow Americans, ask not what your country can do for you, ask what you can do for your country.

whisper_print_timings:     load time =   293.40 ms
whisper_print_timings:     fallbacks =   0 p /   0 h
whisper_print_timings:      mel time =    77.43 ms
whisper_print_timings:   sample time =  1200.01 ms /   133 runs (     9.02 ms per run)
whisper_print_timings:   encode time =  2401.16 ms /     1 runs (  2401.16 ms per run)
whisper_print_timings:   decode time =    66.17 ms /     2 runs (    33.09 ms per run)
whisper_print_timings:   batchd time =  2108.31 ms /   127 runs (    16.60 ms per run)
whisper_print_timings:   prompt time =     0.00 ms /     1 runs (     0.00 ms per run)
whisper_print_timings:    total time =  6274.81 ms

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Fields

    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions