Add voice-only Textual TUI for live agent cascade sessions#250
Merged
Conversation
`assembly live` (the agent cascade) now runs in a Textual TUI by default for an interactive mic session: a scrolling transcript above an animated voice bar that tracks listening / thinking / speaking. There is no text input — it's a hands-free spoken experience. The new `agent_cascade/tui.py` (`LiveAgentApp`) reuses the `assembly code` TUI's chrome: the ASSEMBLY wordmark splash (`code_agent.banner`), the transcript message widgets (`code_agent.messages`), and the voice-bar rendering — extracted into `code_agent.tui_status.voicebar_markup`/`VOICE_FRAMES` so both front-ends share one source. `UserMessage` grows an interim transcript in place via a new `set_text`. The blocking `engine.run_cascade` runs on a worker thread and reaches the UI through a `_TuiRenderer` (the `engine.Renderer` protocol) that hops each call onto the UI thread; a quit closes the duplex audio, ending the mic iterator and unblocking the worker. `_exec._should_use_tui` gates the front-end: file/sample input, `--json`/`-o text`, and non-TTY runs keep the plain `AgentRenderer` line output. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01WfPLNLa2h4B5khteUmYnFc
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Introduces
LiveAgentApp, a hands-free voice-only Textual UI forassembly live(the agent cascade), complementing the existing code-agent TUI. The new TUI drops the text prompt entirely since input is purely voice-based, while reusing the shared chrome (ASSEMBLY wordmark, animated voice bar, transcript message widgets).Key changes
New
aai_cli/agent_cascade/tui.py: ImplementsLiveAgentApp(the Textual app) and_TuiRenderer(marshals cascade callbacks onto the UI thread). The app displays a scrolling transcript above an animated voice bar that tracks the session phase (listening/thinking/speaking). A worker thread drives the blocking cascade viarun_conversation, withon_stopclosing the audio to unblock that worker on quit.TUI selection logic in
aai_cli/commands/agent_cascade/_exec.py:_should_use_tui(): Determines when to use the TUI (interactive mic session in human mode on a TTY, not file/sample input, not--json/-o text, not piped stdin/stdout)._run_live_tui(): Wires the duplex audio, cascade dependencies, and hands the app arun_conversationclosure pluson_stopcallback._web_search_note()to return the notice string (orNone) so it can be passed to the TUI as a notification and still emitted to stderr in line-renderer mode.Shared voice-bar infrastructure in
aai_cli/code_agent/tui_status.py:VOICE_FRAMES(animated meter) and_VOICE_PHASES(phase labels + colors) as module-level constants so both the code and live TUIs use identical animations and styling.voicebar_markup()helper to generate the voice bar's content (meter, label, accent color, optional hint).Message widget enhancements in
aai_cli/code_agent/messages.py:UserMessage.set_text()to update the user prompt in place (used by the live TUI to grow interim voice transcripts without mounting new widgets)._user_markup()helper so the styling is consistent between constructor andset_text().Comprehensive test suite in
tests/test_live_tui.py(342 lines):_TuiRendererthrough a scripted cascade to cover the off-thread hop, error path, and teardown without a mic/speaker/socket.run_agent_cascade.Test updates in
tests/test_code_messages.pyandtests/test_code_tui_status.py: Added tests forUserMessage.set_text()andvoicebar_markup().Implementation details
call_from_thread(). Once the app tears down (quit mid-turn), that call raisesRuntimeError— the event is moot, so it's dropped rather than surfaced as an unhandled exception.on_stopto close the audio, which ends the mic iterator and unblocks the cascade worker, then exits the app.--json,-o text), and non-TTY environments — all fall back to the line renderer.https://claude.ai/code/session_01WfPLNLa2h4B5khteUmYnFc