Skip to content

feat(office): add Agent Office 3D real-time visualization#60

Open
therichardngai-code wants to merge 1 commit intonextlevelbuilder:mainfrom
therichardngai-code:feat/agent-office-3d
Open

feat(office): add Agent Office 3D real-time visualization#60
therichardngai-code wants to merge 1 commit intonextlevelbuilder:mainfrom
therichardngai-code:feat/agent-office-3d

Conversation

@therichardngai-code
Copy link
Contributor

Summary

Adds GET /office — a browser-based 3D isometric view of the GoClaw agent office. Every agent appears as an animated character on its own platform zone. State streams via SSE with a 3-second server-side heartbeat as safety net.

What's New

Agent Office UI (/office)

  • Isometric orthographic camera — fixed angle, orbit-aware pan/zoom (buttons + mouse drag + wheel)
  • Platform zones — 🏠 Home (idle) · 📱 channel platforms (Telegram/Discord/WhatsApp/Feishu/Zalo) · 👥 team platforms (managed mode)
  • Animated characters — 14 distinct Kenney models (7 male, 6 female, 1 gamer), deterministic per agent ID; idle + walking animations
  • Agent labels — neon-green name + inline status dot (colour reflects state)
  • Speech bubblesProcessing... always · ••• tool during tool calls · live LLM text when channel streaming is enabled
  • Delegation arcs — marching-ants dashed line during handshake; agents walk toward each other
  • Platform decoration — L-shaped walls (arcade or market theme, deterministic by key hash), corner crate, 2 machines, back-left column
  • Space nebula background

Real-time State Pipeline

Bus event (EventAgent) → Bridge.handleAgentEvent()
                       → OfficeState mutation
                       → SSEHub.BroadcastState()   (non-blocking, cap 512)
                       → SSE clients → updateScene()

office.Start() heartbeat (3 s) → BroadcastState()   ← safety net for drops

New Go Package internal/office/

File Description
state.go Mutex-protected OfficeState — agents, teams, delegations, notifications
bridge.go Bus subscriber — maps all EventAgent subtypes to state mutations
sse.go SSE hub — fan-out via buffered channel (cap 512)
handler.go HTTP: /office, /office/state, /office/events, /office/assets/
office.go Top-level controller + 3-second heartbeat goroutine

Wiring Changes to Existing Files

  • internal/agent/loop.go — emit channel name in run.started payload
  • internal/agent/router.goAgentInfo gains DisplayName + AgentType
  • internal/gateway/methods/teams.gopublishTeamState() on team create/update
  • internal/gateway/server.goSetOfficeHandler()
  • cmd/gateway.go — create Office, Bridge, seed agents + teams at startup
  • pkg/protocol/events.goEventTeamUpserted constant

Assets (CC0 Kenney.nl, embedded via go:embed)

  • 14 character GLBs + animated variants (idle, walk, talk)
  • Arcade walls, market walls, machines, crate, column + colormaps

Tests

File Lines
state_test.go 509
bridge_test.go 649
sse_test.go 377

How to Access

http://localhost:PORT/office?token=YOUR_GATEWAY_TOKEN

Same token as gateway.token in config.json. Auth bypassed when no token configured.

Security

  • GET /office — no auth (HTML shell only, no data)
  • GET /office/state and GET /office/events — require gateway token, constant-time compare

Known Limitations (follow-up issues)

  1. Team platform removalteams.delete does not remove the team platform; persists until page refresh
  2. Machine scale tuning — all machines normalised to TARGET_H = 1.3; some models may look slightly off
  3. Camera far-clipcamera.far = 300; may clip on very large scenes (>20 platforms)
  4. Live LLM text in bubbles — requires dm_stream: true / group_stream: true in channel config; Processing... fallback shown otherwise

Adds GET /office — a browser-based 3D isometric view of the agent office.
Every agent appears as an animated character on its own platform zone.
State streams via SSE with a 3-second server-side heartbeat as safety net.

## UI
- Isometric orthographic camera with orbit pan/zoom (buttons + mouse)
- Platform zones: Home (idle), channel (Telegram/Discord/etc), team (managed mode)
- 14 animated Kenney characters, deterministic per agent ID
- Neon-green agent labels with live status dot
- Speech bubbles: "Processing..." (always), "••• tool" (tool calls),
  live LLM text (when channel streaming enabled)
- Delegation arcs: marching-ants during handshake, agents walk toward each other
- Platform decoration: L-shaped walls (arcade/market theme), corner crate,
  2 machines per platform, back-left column
- Space nebula background

## Backend (new package internal/office/)
- state.go    — mutex-protected OfficeState (agents, teams, delegations)
- bridge.go   — bus subscriber, maps EventAgent subtypes to state mutations
- sse.go      — SSE hub, fan-out via buffered channel (cap 512)
- handler.go  — HTTP: /office, /office/state, /office/events, /office/assets/
- office.go   — top-level controller + 3-second heartbeat goroutine

## Wiring changes
- internal/agent/loop.go       — emit channel name in run.started payload
- internal/agent/router.go     — AgentInfo gains DisplayName + AgentType
- internal/gateway/methods/teams.go — publishTeamState() on create/update
- internal/gateway/server.go   — SetOfficeHandler()
- cmd/gateway.go               — create Office, Bridge, seed agents+teams
- pkg/protocol/events.go       — EventTeamUpserted constant

## Assets (CC0 Kenney.nl, embedded via go:embed)
- 14 character GLBs + animated variants (idle, walk, talk)
- Arcade walls, market walls, machines, crate, column + colormaps

## Tests
- state_test.go   509 lines
- bridge_test.go  649 lines
- sse_test.go     377 lines
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant