Question: Score of all species per detection

[batthiloreng]

Maybe redundant question (couldn't extract a certain answer from the paper), but from the FAQ i read it like the algorithm gives back a confidence score for all species present in the classes of the model for each call similar to BirdNet (list of all species with associated score per call)

Calls that are clearly belonging to the same call sequence are being predicted as coming from different species?
Currently we do not do any sophisticated post processing on the results output by the model. We return a probability associated with each species for each call. You can use these predictions to clean up the noisy predictions for sequences of calls.

This would be very interesting to us for our research aproach in evaluating and improving the model preditcitons as the algorithm is missing a species present in our dataset. After analysing the data the output only included one species classification and a single score.

Is it possible to modify the output to get all species and associated scores for each detected call or does the model not work in a comparable way to BirdNet?

Happy to hear your response and sorry if the question is already answered in the paper.
All the best
Thilo

[mbsantiago]

Hi @batthiloreng,

Thanks for the question!

For each detection the model produces, it outputs several pieces of information:

• Detection score
  This is a confidence score that the detected event is an echolocation call similar to those in the training data.
  Technically, it's the sum of the confidence scores across all species.
  We've used it to find calls from species outside the UK (and not seen during training), though performance depends on how acoustically similar those calls are. For instance, we've noticed weaker results for species with lower frequency ranges than typical UK bats.

• Per-species confidence scores
  The model also assigns a score to every species in the training set.
  In that sense it's somewhat like BirdNET, however BatDetect2 assumes one valid species per detection (single-label classification), whereas BirdNET was designed for multiple species per clip (multi-label classification). So while the structure is similar, the training objectives and prediction resolution differ.

• Feature vector
  Each detection includes a 32-dimensional feature vector describing only the sound within the predicted bounding box, not the whole clip.
  These features are localised (and appear to be somewhat robust to background noise) and can be treated a bit like measured call parameters.
  They can also be used for tasks like clustering or unsupervised analysis, though our tests so far haven't shown strong results for transfer learning.

• Bounding box
  Each detection also has a bounding box in time–frequency space, defining the region of the spectrogram the classification and features refer to.

Historically, the packaged output only includes the top predicted species and its score ("class_prob") along with the overall detection score ("det_prob").
However, the full set of per-species scores and features are generated internally, you'd just need to go a bit deeper into the code to extract them.

Happy to point you to where that happens if you'd like to access those directly.

Cheers,
Santiago

[batthiloreng]

Hi @mbsantiago

Thanks for the detailed explanation.

Historically, the packaged output only includes the top predicted species and its score ("class_prob") along with the overall detection score ("det_prob").
However, the full set of per-species scores and features are generated internally, you'd just need to go a bit deeper into the code to extract them.
Happy to point you to where that happens if you'd like to access those directly.

That would be amazing, if it's not too much effort.

We've been using the algorithm this past few months for our long-term bat monitoring in Germany. Hopefully we can feedback something for the algorithm or the overall goal of the GloBat Network soonish.

All the best
Thilo

[mbsantiago]

Hey @batthiloreng,

I've had a look and I believe you can use this piece of code:

from batdetect2 import api
from batdetect2.utils.detector_utils import _process_spectrogram

# Load audio (any audio loader will work)
wav = api.load_audio("<path_to_audio_file>")

# Generate a spectrogram using BatDetect2's preprocessing pipeline.
# It's best not to create spectrograms manually, as the process is quite
# specific to the model.
spec = api.generate_spectrogram(wav)

# Run the model to get detections and feature embeddings
# You get a PredictionsResult object (see its definition in
# `batdetect2/types.py`) and a numpy array (`feats`) containing the
# 32-dimensional features for each detection.
predictions, feats = _process_spectrogram(
    spec,
    api.TARGET_SAMPLERATE_HZ,
    api.model,
    api.config,
)

# Example: convert detections to a DataFrame
# This includes detection data and all species scores
import pandas as pd

df = pd.concat(
    [
        pd.DataFrame(
            {
                "det_prob": predictions["det_probs"],
                "start_time": predictions["start_times"],
                "end_time": predictions["end_times"],
                "low_freq": predictions["low_freqs"],
                "high_freq": predictions["high_freqs"],
            }
        ),
        pd.DataFrame(
            predictions["class_probs"].T, columns=api.config["class_names"]
        ),
        pd.DataFrame(
            feats.T,
            columns=["feat_" + str(ii) for ii in range(feats.shape[1])],
        ),
    ],
    axis=1,
)

print(df)

This should give you a DataFrame with detection details and species probabilities.

Hope this helps. Let me know if you have issues.

Cheers,
Santiago