feat: Call Mode — brutalist stream widget, hooks narration, participant system#10
Open
sameeeeeeep wants to merge 7 commits intomainfrom
Open
feat: Call Mode — brutalist stream widget, hooks narration, participant system#10sameeeeeeep wants to merge 7 commits intomainfrom
sameeeeeeep wants to merge 7 commits intomainfrom
Conversation
feat: Vision-powered screen context, face recognition & speaker attribution Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
…leave detection - CallRoom.swift: Participant model (llama/claudeCode/connection kinds), state machine (idle/thinking/streaming/paused), gesture-based addressing (finger count → slot), pause/remove lifecycle, claudeCodeIsActive gate for MCP transcript - MCPServer.swift: initialize handshake fires onJoined callback; 60s inactivity timer fires onLeft; autoclawd_get_audio_transcript returns standby message when paused - AppState: callRoom singleton added - AppDelegate: wires isPausedProvider, onJoined, onLeft into MCPServer.start() - MainPanelView: CallModeZoomView (3-panel zoom-call layout), hides PixelWorld in callMode - CameraPreviewView: screen preview constrained to parent frame (fixes chevron disappearing) - PillMode, PipelineModels, ChunkManager, WidgetCanvasViews, SkillStore: call mode wiring Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- CallModeRoomView: replaces CallModeZoomView as the panel's call mode view
- Horizontal scrollable participant tiles (tap or left-hand finger count to address)
- ParticipantTileView: slot badge, pause/remove buttons, state label, mascot
- ParticipantMascotView: SF Symbol icon with state-driven animations
(idle: breathing pulse, thinking: spinner, streaming: shimmer ring, paused: static)
- ThinkingDotsView: animated three-dot thinking indicator
- Shared call feed: voice transcript + callModeSession messages unified
- Bottom bar: camera thumb, screen thumb, play/pause/stop, active participant indicator
- AppState: leftFingerCount gesture → callRoom.selectByGesture() when in callMode
- MainPanelView: swapped CallModeZoomView → CallModeRoomView
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
4 new MCP tools let Claude Code orchestrate a live cast of participants: - autoclawd_invite_participant(id, name, system_image) → tile joins call - autoclawd_set_participant_state(id, state) → animate tile - autoclawd_send_participant_message(id, name, text) → attributed feed bubble - autoclawd_remove_participant(id) → tile leaves CallRoom: connection kind now carries systemImage + consistent hash-derived tile color so each plugin always gets the same hue. CallMessage gains participantID/participantName so feed rows show the correct plugin name, icon, and color rather than a generic "Plugin" label. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
These tools (invite/set_state/send_message/remove) would have appeared in Claude Code's tools/list and been called during normal work, burning credits on pure UI bookkeeping. Callbacks and CallRoom plumbing remain for future UI-driven or stream-inferred participant management. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- HookNarrationService: parses PostToolUse/Stop hook events; calls Llama to produce a 1-sentence natural-language narration (e.g. "Claw'd is reading package.json to understand the project"); falls back to template descriptions if Ollama is unavailable - MCPServer: adds POST /hook endpoint — Claude Code hooks curl here on every tool event; fires onHookEvent callback on @mainactor - MCPConfigManager: writeHooksConfig() merges PostToolUse + Stop hook entries into ~/.claude/settings.json automatically on launch - AppDelegate: wires onHookEvent → narrate → appendParticipantMessage so every Claude Code tool call appears as a Claw'd feed bubble in the call room; also calls writeHooksConfig() at startup The call room now acts as a live video-call window into a Claude Code session — each tool use becomes a narrated story beat in the feed. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- Add CallStreamWidget.swift — always-on-top floating NSPanel (420×560)
that overlays any session during Call Mode; animates in/out with slide+fade;
draggable from anywhere; snaps to bottom-right by default
- Add CallStreamWidgetView.swift — full brutalist redesign:
· No circles, no soft rounding — sharp rectangular geometry throughout
· Header: REC dot + CALL STREAM label + session timer + close button
· Mission bar: orange left-accent shows user's spoken goal (first message)
· Participant strip: sharp tiles with thick colored top-border accent (no circles)
Square mascot icon with NSImage(named: "mascot-{id}") fallback to SF Symbol
· Task queue: top 3 pending StructuredTodos; active task highlighted orange
· Stream feed: brutalist group-chat format — NAME ─── time / message / image
Generated/reaction messages render at 70% opacity with ~ marker
· Spotlight panel: auto-surfaces latest image (full-width) or filename
detected via regex from stream messages
· Bottom bar: animated waveform + event count + END CALL red square button
- CallModeSession.swift: add createdAt: Date to CallMessage for timestamps;
add imageData and isGenerated fields; extend appendParticipantMessage
- HookNarrationService.swift: complete implementation — parse hook events,
extract ToolParticipant from mcp__server__tool names, extract image data
from 3 tool_response payload patterns, Llama narration with template
fallback, optional AutoClawd reaction for MCP/image events
- CallModeRoomView.swift: CallFeedMessageRow with 2px left-border style;
solid gradient for real messages, dashed segments for generated ones;
inline image support via imageData
- AppDelegate.swift: wire $pillMode observer to show/hide CallStreamWidget;
lazy-init on first show; animate out on mode change; onHookEvent handler
auto-joins tool participants and posts structured feed messages
- SettingsManager.swift: add callStreamWidgetEnabled (default true)
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
This PR ships the full Call Mode feature for AutoClawd — a real-time overlay that turns Claude Code sessions into a live, story-driven group call experience.
Call Stream Widget (new floating panel)
CallStreamWidget.swift— always-on-topNSPanel(420×560) that overlays any app during Call Mode; slide+fade animate in/out; draggable; bottom-right default positionCallStreamWidgetView.swift— brutalist design language throughout:NSImage(named: "mascot-{id}")PNG support + SF Symbol fallbackStructuredTodos; active task highlighted orangeNAME ─── time/ message / inline image; generated/reaction messages render at 70% opacity with~marker■ END CALLred square buttonHooks Integration (real-time narration)
HookNarrationService.swift— translates raw Claude Code hook events into human-readableNarrationBundles:mcp__server__toolnames →ToolParticipantwith SF Symbol mapping (Pencil, Figma, GitHub, Linear, Notion, Google Sheets)Read,Write,Edit,Bash,Glob,Grep,Task,WebFetch,WebSearch,TodoWrite, etc.)Participant System
CallRoom.swift—CallParticipantmodel with state machine (idle / thinking / streaming / paused), tile color, mascot SF SymbolAppDelegate.swift—$pillModeobserver auto-shows/hides widget;onHookEventhandler auto-joins tool participants and posts structured feed messagesCallModeRoomView.swift—CallFeedMessageRowwith 2px colored left-border; solid gradient for real events, dashed for generated; inline image renderingSupporting Changes
CallModeSession.swift—CallMessagegainscreatedAt,imageData,isGenerated;appendParticipantMessageextended accordinglySettingsManager.swift—callStreamWidgetEnabledtoggle (defaulttrue)Test plan
CallStreamWidgetanimates in at bottom-rightYOUmessage with timestamp in stream feed.swiftfile → filename card appears in Spotlightmascot-pencil.pngintoResources/→ appears in participant tile (no rebuild needed after copy)🤖 Generated with Claude Code