Add AI Assistant for guided NWB conversion

[bendichter]

Summary

Adds an AI Assistant mode to NWB GUIDE — a chat-based interface that leads users through converting their neurophysiology data to NWB format and publishing on DANDI Archive.

The assistant is powered by Claude (via the Claude Agent SDK) and guides users through 7 phases:

1. Experiment Discovery — understand what data was collected and how
2. Data Inspection — scan files, identify formats, map to NeuroConv interfaces
3. Metadata Collection — gather subject, session, and experiment metadata
4. Synchronization Analysis — align clocks across recording modalities
5. Code Generation — produce a pip-installable conversion repo
6. Testing & Validation — run conversions, validate with NWB Inspector
7. DANDI Upload — create a Dandiset and upload NWB files

What's included

Frontend (Lit web components):

• AIAssistantPage — two-view layout: session history list + active chat with a real-time todo/phase panel
• ChatMessage — renders markdown, collapsible tool-use cards, and multiple-choice answer buttons (<choices> convention)
• ChatInput — message input with send and interrupt controls
• SettingsPanel — API key and model configuration

Backend (Flask-RESTX):

• /ai namespace with session CRUD, message send, interrupt, and SSE streaming endpoints
• ConversionAgent class wrapping ClaudeSDKClient for multi-turn agent sessions
• Session persistence as JSON files in ~/NWB_GUIDE/ai-sessions/
• Skill loader that resolves $file: includes from bundled phase/knowledge files
• Monitoring hooks for transcript logging

Bundled skill files:

• 7 phase instruction files with detailed conversion workflow guidance
• Knowledge base: 68 NeuroConv interfaces catalog, NWB best practices, repo structure patterns, conversion patterns from real CatalystNeuro repos
• fetch_paper.py tool for extracting methods sections from DOI/PMID references

Architecture

NWB GUIDE (Electron)
├── Existing pages: Home | Validate | Explore | Upload | Settings
├── NEW: "AI Assistant" page
│   ├── Chat UI (SSE streaming, markdown, tool cards, choices)
│   ├── Phase/Todo tracking panel
│   └── Settings (API key, model)
│
├── Flask Backend
│   ├── /neuroconv  (existing)
│   ├── /dandi      (existing)
│   └── /ai         (NEW)
│       ├── POST /sessions              → create agent session
│       ├── GET  /sessions              → list saved sessions
│       ├── POST /sessions/<id>/message → send user message
│       ├── POST /sessions/<id>/interrupt → interrupt agent
│       └── GET  /sessions/<id>/events  → SSE stream
│
└── Agent Runtime (in Flask process)
    ├── agent.py         → ClaudeSDKClient wrapper
    ├── skill_loader.py  → SKILL.md $file: resolver
    ├── session_store.py → JSON-based chat history
    └── skill/           → bundled conversion instructions + knowledge

Test plan

• npm start → dev mode → click "AI Assistant" in sidebar → page loads
• Pick a data folder → start conversation → agent responds via SSE
• Verify session appears in session list after navigating away and back
• Verify multiple-choice buttons render and send selection as user message
• Verify todo panel tracks phases and action items
• Test interrupt button stops agent mid-turn
• Test with BYO Anthropic API key in settings panel
• Verify ophys_testing_data/ and secrets are NOT committed

🤖 Generated with Claude Code