Room→SIP egress mutes continuous low-level audio in a periodic rebuffer cycle (fine to WebRTC subscribers, 55-64% loss on PSTN leg)

[gmunhoz0810]

Summary

Continuous low-level background audio (office-ambience bed mixed under an AI agent's voice) is delivered intact to WebRTC subscribers but arrives on the PSTN leg with 55–64% of its frames replaced by digital silence, in a metronomic pattern (~100–150 ms of audio passing every ~600–750 ms). Agent speech is never affected — only the quiet continuous bed between utterances. Callers hear the background "pumping" in and out whenever the agent stops talking.

Setup

• LiveKit Cloud + SIP (inbound), Twilio Elastic SIP Trunking
• Python agent (pipecat) publishing 16 kHz mono Opus; TrackPublishOptions with dtx: false defaults; track carries TTS speech + a constant ambience bed (~-38 dBFS effective, never below -45 dBFS in any 50 ms window — measured)
• Inbound trunk: krisp_enabled: false, no media encryption
• Dispatch rule: dispatch_rule_individual

Evidence (same-call dual tap)

For a single PSTN call we recorded simultaneously:

• Tap A — a hidden room participant subscribing to the agent track (what the SFU forwards)
• Tap B — Twilio dual-channel call recording (what LiveKit SIP egress sends to the trunk)

50 ms RMS windows, "bed" = -60..-30 dBFS, "dropout" = below -60 dBFS:

	non-speech windows	bed present	dropout
Tap A (room subscriber)	688	96%	4%
Tap B (PSTN, same call)	634	28%	61%

Tap A contains unbroken 5+ second stretches of continuous bed. Tap B never has more than ~150 ms of bed at a time outside agent speech; the rest is digital silence (-90 dBFS), alternating with the bed in a regular ~750 ms cycle. Levels confirm Tap A is the real bed (median -41.8 dBFS tracking the asset's dynamics), not decoder comfort noise.

Three separate probe calls reproduce the same pattern at the same magnitude.

Codec-independent

Forcing the SIP leg to PCMU via the dispatch rule's media config (only_listed_codecs: true, codecs: [PCMU/8000]) changed nothing (61% vs 64% dropout), so this is not Opus DTX on the SIP leg.

Where it seems to come from

The behavior matches the mixer input state machine in media-sdk (mixer/mixer.go):

func (i *Input) readSample(bufMin int, out msdk.PCM16Sample) (int, error) {
    if i.buffering {
        if i.buf.Len() < bufMin {
            return 0, nil // keep buffering -> mixer emits silence for this input
        }
        i.buffering = false
    }
    n, err := i.buf.Read(out)
    if n == 0 {
        i.buffering = true // starving; pause the input and start buffering again
    }
    ...
}

A momentary starvation mutes the input until bufMin re-accumulates, then it plays briefly and starves again — i.e. a short producer hiccup is amplified into a repeating mute/burst cycle on the phone leg. During TTS speech the upstream buffers are full (TTS delivers faster than real time), which would explain why speech never chops while the just-in-time-paced bed does.

Questions

1. Is this a known issue? (Related reports: Audio stream stutter/lag when talking over SIP #348, Audio quality issue with sip outbound call agents#4026 — "fine on WebRTC, choppy only over SIP".)
2. Is there a Cloud-side flag to relax/disable the rebuffer-mute (or pre-buffer more) on the room→SIP direction?
3. If useful we can share both WAV captures and the analysis script.

[gmunhoz0810]

Workaround data point that may help others (and narrows down the mechanism):

The gate behaves like a speech-activity detector, not an energy threshold:

bed content	level	non-speech windows muted
steady office room tone	0.30 gain (~-38 dBFS)	61–64%
steady office room tone	0.50 gain (~-33 dBFS)	20%
continuous murmur/walla (same recording, 4 time-offset layers, same RMS)	0.35 gain	0–4%

Same probe methodology as the report (Twilio dual-channel recording of the PSTN leg, 50 ms RMS windows). Continuous speech-like content holds the gate open at a level where stationary noise is muted ~60%.

We'd still much prefer to ship plain room tone — happy to test a fix or flag on the Cloud side.

[gmunhoz0810]

Additional data point: the gating is intermittent per call/route. After the walla workaround we measured 0% / 4% / 0% dropout across three probe calls, then a real call minutes later was audibly gated again, then the next probe measured 0% again. Same trunk, same region, same published audio. Whatever state the egress gate keys on appears to vary between calls — worth checking whether different media pods/routes apply different VAD/buffering behavior.

We now run an automated probe every 6 hours (recorded silent call → RMS-window analysis of the PSTN leg) and can share a longitudinal dataset if useful.

[gmunhoz0810]

Root cause found — and it spans repos. The gating we reported is the interaction of THREE things:

1. Publisher-side Opus DTX defaults ON. The Rust SDK's TrackPublishOptions default is dtx: true (options.rs); the FFI fills unset proto fields with that default (opts.dtx.unwrap_or(default_publish_options.dtx)). Clients that construct bare options (pipecat's LiveKit transport does) publish with DTX on without knowing it — and the Python proto accessor reads the unset field as False, which makes this very easy to mis-verify. With DTX on, libopus's internal speech-VAD stops emitting packets on quiet non-speech audio — exactly an ambience bed between agent turns. That's why our reversed-speech bed mostly passed (speech-like input keeps the VAD active), louder beds partially passed (energy), and level/codec changes on the SIP leg changed nothing.

2. The SIP bridge has no concealment on room→SIP. The opus decoder drops empty packets (commit c7231b6) and never synthesizes PLC/CNG — media-sdk PR #14 which adds exactly this has been parked since June 2025. WebRTC subscribers conceal DTX gaps with comfort noise, which is why the room side always sounded fine; the PSTN leg gets hard silence.

3. The mixer's rebuffer threshold can't re-arm on DTX refresh frames. DTX emits ~one 20ms frame per ~400ms; readSample's buffering state needs ≥3 frames (60ms) to unmute — so a single refresh frame can never reopen the input, producing the metronomic ~150ms-pass/~600ms-mute signature we measured.

Fix on our side, verified in progress: publish with explicit dtx=False (presence-marked). Suggest (a) reviving media-sdk #14 so DTX publishers degrade gracefully on SIP, and (b) documenting the DTX default prominently for SIP use cases — anyone shipping ambience/music through the bridge will hit this.

One question for the team: is enable_jitter_buffer_prob non-zero on the Cloud SIP fleet? Per-call randomization there would explain residual call-to-call variance we observed even with content/level held constant.