feat: live dashboard monitor + serve loop improvements

[msitarzewski]

Summary

• ncurses dashboard — htop-style terminal monitor (dashboard.c) that reads /tmp/flash-moe-stats.json and shows real-time inference status, progress bars, TTFT, tok/s, and rolling averages
• SSE streaming — per-token Server-Sent Events for /v1/chat/completions (OpenAI-compatible streaming)
• Dashboard stats reporting — server writes live state (prefill progress, generation metrics, uptime) to JSON for the dashboard
• Tool call parsing — detects <tool_call> blocks in model output and returns structured tool_calls in the response
• Session state — save/restore KV cache and linear attention state for multi-turn conversations
• GPU KV buffer — increased pre-allocation from 8K to 32K tokens
• CPU 2-bit expert path — fallback compute path for 2-bit quantized experts

Testing

Tested end-to-end on Apple M5 Max (128GB RAM):

• make && make dashboard builds cleanly
• ./infer --serve 6601 --2bit + ./dashboard — live monitoring works across idle/prefilling/generating states
• ./infer --serve 6601 (4-bit) — 10.5 tok/s served with correct SSE streaming
• Terminal resize, disconnect/reconnect, and q exit all work correctly
• Verified dashboard borders render correctly at various terminal widths

Test plan

• make clean && make && make chat && make dashboard — all targets build
• ./infer --serve 6601 --2bit + ./chat --port 6601 — interactive chat works
• ./dashboard shows live stats during generation
• Dashboard shows DISCONNECTED when server is stopped
• Dashboard adapts to terminal resize

🤖 Generated with Claude Code