constrain auto-detect to a provided set of candidate languages

[ilhamsyahids]

In my case, the audio is known to contain only a subset of languages. However, when I use auto-detect, it sometimes selects a similar or unrelated language. For example, Indonesian audio may be detected as Malay.

This PR constrains auto-detect to a user-provided set of candidate languages.

• Related issue: Feature Request: Improved Multi-Language Audio Transcription #3334

Test:

% cmake --build build --target test-lang-auto-detect && \
    ctest -R ^test-lang-auto-detect$ --test-dir build --output-on-failure -VV

whisper-cli output:

% ./build/bin/whisper-cli -m ./models/ggml-base.bin -f samples/jfk.wav \
    --language auto --language-candidates en,es

whisper_init_from_file_with_params_no_state: loading model from './models/ggml-base.bin'
whisper_init_with_params_no_state: use gpu    = 1
whisper_init_with_params_no_state: flash attn = 1
whisper_init_with_params_no_state: gpu_device = 0
whisper_init_with_params_no_state: dtw        = 0
ggml_metal_device_init: tensor API disabled for pre-M5 and pre-A19 devices
ggml_metal_library_init: using embedded metal library
ggml_metal_library_init: loaded in 0.011 sec
ggml_metal_rsets_init: creating a residency set collection (keep_alive = 180 s)
ggml_metal_device_init: GPU name:   MTL0 (Apple M3 Max)
ggml_metal_device_init: GPU family: MTLGPUFamilyApple9  (1009)
ggml_metal_device_init: GPU family: MTLGPUFamilyCommon3 (3003)
ggml_metal_device_init: GPU family: MTLGPUFamilyMetal4  (5002)
ggml_metal_device_init: simdgroup reduction   = true
ggml_metal_device_init: simdgroup matrix mul. = true
ggml_metal_device_init: has unified memory    = true
ggml_metal_device_init: has bfloat            = true
ggml_metal_device_init: has tensor            = false
ggml_metal_device_init: use residency sets    = true
ggml_metal_device_init: use shared buffers    = true
ggml_metal_device_init: recommendedMaxWorkingSetSize  = 30150.67 MB
whisper_init_with_params_no_state: devices    = 3
whisper_init_with_params_no_state: backends   = 3
whisper_model_load: loading model
whisper_model_load: n_vocab       = 51865
whisper_model_load: n_audio_ctx   = 1500
whisper_model_load: n_audio_state = 512
whisper_model_load: n_audio_head  = 8
whisper_model_load: n_audio_layer = 6
whisper_model_load: n_text_ctx    = 448
whisper_model_load: n_text_state  = 512
whisper_model_load: n_text_head   = 8
whisper_model_load: n_text_layer  = 6
whisper_model_load: n_mels        = 80
whisper_model_load: ftype         = 1
whisper_model_load: qntvr         = 0
whisper_model_load: type          = 2 (base)
whisper_model_load: adding 1608 extra tokens
whisper_model_load: n_langs       = 99
whisper_model_load:         MTL0 total size =   147.37 MB
whisper_model_load: model size    =  147.37 MB
whisper_backend_init_gpu: device 0: MTL0 (type: 1)
whisper_backend_init_gpu: found GPU device 0: MTL0 (type: 1, cnt: 0)
whisper_backend_init_gpu: using MTL0 backend
ggml_metal_init: allocating
ggml_metal_init: found device: Apple M3 Max
ggml_metal_init: picking default device: Apple M3 Max
ggml_metal_init: use fusion         = true
ggml_metal_init: use concurrency    = true
ggml_metal_init: use graph optimize = true
whisper_backend_init: using BLAS backend
whisper_init_state: kv self size  =    6.29 MB
whisper_init_state: kv cross size =   18.87 MB
whisper_init_state: kv pad  size  =    3.15 MB
whisper_init_state: compute buffer (conv)   =   17.24 MB
whisper_init_state: compute buffer (encode) =   26.31 MB
whisper_init_state: compute buffer (cross)  =   38.45 MB
whisper_init_state: compute buffer (decode) =   97.29 MB

system_info: n_threads = 4 / 14 | WHISPER : COREML = 0 | OPENVINO = 0 | MTL : EMBED_LIBRARY = 1 | CPU : NEON = 1 | ARM_FMA = 1 | FP16_VA = 1 | MATMUL_INT8 = 1 | DOTPROD = 1 | ACCELERATE = 1 | OPENMP = 1 | REPACK = 1 | 

main: processing 'samples/jfk.wav' (176000 samples, 11.0 sec), 4 threads, 1 processors, 5 beams + best of 5, lang = auto, task = transcribe, timestamps = 1 ...
main: constrained auto-detect candidates = en,es

whisper_full_with_state: auto-detected language: en (p = 0.995665)

[00:00:00.000 --> 00:00:10.500]   And so my fellow Americans ask not what your country can do for you, ask what you can do for your country.

whisper_print_timings:     load time =    66.74 ms
whisper_print_timings:     fallbacks =   0 p /   0 h
whisper_print_timings:      mel time =     3.52 ms
whisper_print_timings:   sample time =    22.08 ms /   137 runs (     0.16 ms per run)
whisper_print_timings:   encode time =    56.77 ms /     2 runs (    28.39 ms per run)
whisper_print_timings:   decode time =     4.58 ms /     3 runs (     1.53 ms per run)
whisper_print_timings:   batchd time =    46.38 ms /   133 runs (     0.35 ms per run)
whisper_print_timings:   prompt time =     0.00 ms /     1 runs (     0.00 ms per run)
whisper_print_timings:    total time =   210.29 ms
ggml_metal_free: deallocating

[ilhamsyahids]

@ggerganov Would you mind to take a look on this? Thanks