Skip to content

Implement context-aware interruption handling layer#503

Open
vnayakde wants to merge 1 commit into
Dark-Sys-Jenkins:mainfrom
vnayakde:feature/interrupt-handler-vnayakde
Open

Implement context-aware interruption handling layer#503
vnayakde wants to merge 1 commit into
Dark-Sys-Jenkins:mainfrom
vnayakde:feature/interrupt-handler-vnayakde

Conversation

@vnayakde
Copy link
Copy Markdown

Overview

This PR implements a context-aware interruption handling layer for the LiveKit Voice Agent framework.

Previously, the Voice Activity Detection (VAD) would immediately pause the agent's playback upon detecting any user audio, creating awkward stuttering when users simply provided "backchanneling" feedback (e.g., "yeah", "ok", "hmm"). This PR ensures the agent elegantly ignores passive feedback while actively speaking, while still correctly interpreting commands and responding when silent.

Implementation Details

Rather than hacking low-level VAD behavior, this solution was built completely within the AgentActivity event loop, ensuring safe, modular execution.

  • Configurable Ignore List: Added backchannel_ignore_words to AgentSessionOptions, making the array easily configurable and highly manageable for developers.
  • _is_backchanneling Helper: Implemented a transcript verification function that strips whitespace/punctuation and evaluates if the incoming transcript consists purely of backchannel items.
  • Interruption Filtering: Injected logic into _interrupt_by_audio_activity and on_end_of_turn. The interruption is gracefully bypassed (returning early) only if the agent is actively speaking (self._current_speech is not None).

Evaluation Matrix Passed

  • Agent Speaking + User says "Yeah/Ok": Bypasses interruption. Audio continues seamlessly.
  • Agent Speaking + User says "Stop": Detects semantic command. Instantly cuts audio playback.
  • Agent Silent + User says "Yeah/Ok": Detects the agent isn't talking, skips the filter, and responds normally.

Proof of Execution

The demonstration covering all evaluation criteria can be found here:
View Video Proof of Execution

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants