Pico 2 Audio Stream PoC

What this is

• ADC0 (GP26) microphone capture at 16 kHz
• DMA ping-pong capture (2 x 256-sample buffers)
• Fixed-point frontend filter (DC blocker + optional LP + gain)
• USB-CDC packet stream:
  • type=1 audio blocks (int16, little-endian)
  • type=2 runtime stats (queue, drop, clip, USB write counters)
  • type=3 event messages

Build

cd pico2_audio_stream
export PICO_SDK_PATH=${PICO_SDK_PATH:-$HOME/pico/pico-sdk}
cmake -S . -B build -G Ninja -DPICO_BOARD=pico2
cmake --build build -j"$(nproc)"

Expected artifacts:

• build/pico2_audio_stream.elf
• build/pico2_audio_stream.bin

Flash

• Development flash (recommended, no UF2 step required):

picotool load -x -f pico2_audio_stream/build/pico2_audio_stream.bin -t bin -o 0x10000000

Alternative UF2 path:

cd pico2_audio_stream/build
picotool uf2 convert pico2_audio_stream.elf pico2_audio_stream.uf2

Then hold BOOTSEL while plugging Pico 2 over USB and copy UF2 to the mounted RPI-RP2 drive.

No-button reflash flow (after this firmware is already running once):

# 1) Tell firmware to jump to BOOTSEL over USB-CDC
python3 tools/pico_control.py --port /dev/ttyACM0 bootsel

# 2) Flash next UF2
picotool load -x pico2_audio_stream/build/pico2_audio_stream.bin -t bin -o 0x10000000

Host debug tool

python3 tools/pico_stream_spectrogram.py --port /dev/ttyACM0 --sample-rate 16000 --plot-seconds 3

Lower host CPU load (recommended while tuning aggressive LP values):

python3 tools/pico_stream_spectrogram.py --port /dev/ttyACM0 --refresh-ms 150 --spec-refresh-ms 500

Stopping cleanly:

• Close the spectrogram window and wait for [info] Plot window closed, exiting.
• Or press Ctrl-C while the window is still open.

Optional one-shot WAV capture:

python3 tools/pico_stream_spectrogram.py --port /dev/ttyACM0 --save-wav capture.wav

Labeled capture UI (for command dataset collection):

python3 tools/pico_stream_spectrogram.py \
  --port /dev/ttyACM0 \
  --dataset-dir data/commands \
  --allowed-commands "up,down,left,right,start,stop,yes,no" \
  --default-label up \
  --min-capture-seconds 0.25

Record controls:

• Use on-plot buttons: Start Rec, Stop+Save, Discard
• Or keyboard shortcuts: r = start, s = stop+save, d = discard
• Captures are saved to dataset_dir/<label>/<label>_timestamp.wav
• Metadata is appended to dataset_dir/metadata.csv with filter settings (hp_q15/lp_q15/gain_q8)

Small-vocabulary recognizer PoC (host-side, non-ML template matching):

# Train MFCC+DTW templates from labeled WAV captures
python3 tools/pico_command_poc.py train \
  --dataset-dir data/commands \
  --model-out data/models/cmd10_dtw.npz \
  --sample-rate 16000 \
  --commands "up,down,left,right,start,stop,yes,no" \
  --negative-labels "noise,other_speech" \
  --min-files-per-label 5 \
  --templates-per-label 3

# Classify one WAV file
python3 tools/pico_command_poc.py predict \
  --model data/models/cmd10_dtw.npz \
  --wav data/commands/up/example.wav

# Live serial command recognition with VAD segmentation
python3 tools/pico_command_poc.py live \
  --model data/models/cmd10_dtw.npz \
  --port /dev/ttyACM0 \
  --print-stats

If --reject-best/--reject-ratio are omitted, predict and live use thresholds embedded in the trained model.

Spectrogram with live detection bounding boxes:

python3 tools/pico_stream_spectrogram.py \
  --port /dev/ttyACM0 \
  --detect-model data/models/cmd10_dtw.npz \
  --detect-vad-abs-floor 20

This draws a highlighted box on the spectrogram for each detected segment, with the predicted label and score. By default, detect reject thresholds are loaded from the trained model calibration metadata.

On-device recognizer PoC (MFCC+DTW on Pico 2):

# 1) Export trained .npz model into a C header used by firmware
python3 tools/export_pico_model_header.py \
  --model data/models/hello_dtw_auto_latest.npz \
  --output pico2_audio_stream/src/pico_asr_model_generated.h \
  --symbol-prefix pico_asr

# 2) Build + flash firmware
cd pico2_audio_stream
export PICO_SDK_PATH=${PICO_SDK_PATH:-$HOME/pico/pico-sdk}
cmake --build build -j"$(nproc)"
python3 tools/pico_control.py --port /dev/ttyACM0 bootsel
picotool load -x -f pico2_audio_stream/build/pico2_audio_stream.bin -t bin -o 0x10000000

# 3) Verify on-device model path
python3 tools/pico_control.py --port /dev/ttyACM0 asr-selftest
python3 tools/pico_control.py --port /dev/ttyACM0 asr-stats

ASR firmware control commands:

python3 tools/pico_control.py --port /dev/ttyACM0 asr-on
python3 tools/pico_control.py --port /dev/ttyACM0 asr-off
python3 tools/pico_control.py --port /dev/ttyACM0 asr-stats
python3 tools/pico_control.py --port /dev/ttyACM0 asr-reset
python3 tools/pico_control.py --port /dev/ttyACM0 asr-reset-stats
python3 tools/pico_control.py --port /dev/ttyACM0 asr-set-vad 1.02 1.005 1 1 2

ASR streaming policy:

• When ASR is ON, firmware suppresses type=1 audio packets to reduce USB overhead and only emits stats/events.
• The stats field supp_asr (audio_packets_suppressed_asr) tracks how many audio blocks were intentionally not streamed.

Control commands:

python3 tools/pico_control.py --port /dev/ttyACM0 ping
python3 tools/pico_control.py --port /dev/ttyACM0 stats
python3 tools/pico_control.py --port /dev/ttyACM0 bootsel
python3 tools/pico_control.py --port /dev/ttyACM0 getcfg
python3 tools/pico_control.py --port /dev/ttyACM0 set-hp 0.969
python3 tools/pico_control.py --port /dev/ttyACM0 set-lp 1.0
python3 tools/pico_control.py --port /dev/ttyACM0 set-gain 8.0
python3 tools/pico_control.py --port /dev/ttyACM0 set-hp-hz 80
python3 tools/pico_control.py --port /dev/ttyACM0 set-lp-hz 3000
python3 tools/pico_control.py --port /dev/ttyACM0 reset-filter-state

Notes

• The firmware uses a short USB write timeout (PICO_STDIO_USB_STDOUT_TIMEOUT_US=2000) to avoid long stalls.
• If host-side consumption is too slow, drops are counted in stats packets instead of blocking indefinitely.
• If you want SDK-managed picotool post-processing enabled in CMake, configure with: -DENABLE_PICOTOOL_POSTPROCESS=ON