Skip to content

Commit 3ee1aea

Browse files
committed
feat: add {{Research:}} tag — autonomous AI experiment loop
- New {{Research:}} tag inspired by Karpathy's autoresearch - Propose→Execute→Evaluate→Keep/Discard loop via Pyodide (Python in WASM) - Configurable @Metric, @direction, @max_iterations, @model, @goal - Multiline @code: | (mutable) and @test: | (fixed harness) fields - Metric extraction from stdout (METRIC:xxx) with numeric fallback - AI prompt with experiment history and strategy shift hints - Live results table with status badges and delta indicators - Glass card UI with purple accents and pulse animation - js/research-loop.js (~795 lines) + css/research-loop.css (~300 lines) - Integrated into ai-docgen.js, renderer.js, main.js
1 parent 3eed508 commit 3ee1aea

File tree

7 files changed

+1276
-6
lines changed

7 files changed

+1276
-6
lines changed

README.md

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -37,7 +37,7 @@
3737
| **Desktop** | Native app via Neutralino.js with system tray and offline support |
3838
| **Code Execution** | 7 languages in-browser: Bash ([just-bash](https://justbash.dev/)), Math (Nerdamer), LaTeX (MathJax + Nerdamer evaluation), Python ([Pyodide](https://pyodide.org/)), HTML (sandboxed iframe, `html-autorun` for widgets/quizzes), JavaScript (sandboxed iframe), SQL ([sql.js](https://sql.js.org/) SQLite) · 25+ compiled languages via [Judge0 CE](https://ce.judge0.com): C, C++, Rust, Go, Java, TypeScript, Kotlin, Scala, Ruby, Swift, Haskell, Dart, C#, and more · **▶ Run All** notebook engine — one-click sequential execution with preflight dialog (block table with model/status), pre-execution model loading (AI + TTS auto-loaded before blocks run), progress bar, abort, per-block status badges, detailed console logging, and SQLite shared context store |
3939
| **Security** | Content Security Policy (CSP), SRI integrity hashes, XSS sanitization (DOMPurify), ReDoS protection, Firestore write-token ownership, API keys via HTTP headers, postMessage origin validation, 8-char password minimum, sandboxed code execution, Cloudflare Turnstile CAPTCHA on email endpoint, automated security scanner (`scripts/security-check.sh`) with pre-commit integration |
40-
| **AI Document Tags** | `{{@AI:}}` text generation (`@think: Yes` for deep reasoning), `{{@Image:}}` image generation (Gemini Imagen), `{{@OCR:}}` image-to-text extraction (Text/Math/Table modes via Granite Docling 258M, Florence-2 230M, or GLM-OCR 1.5B, 📷 live camera capture + 📎 image/PDF upload, PDF page rendering via pdf.js), `{{@TTS:}}` text-to-speech playback (Kokoro TTS per card, language selector, ▶ Play / ⬇ Save WAV), `{{@STT:}}` speech-to-text dictation (engine selector: Whisper/Voxtral/Web Speech API, 11 languages, Record/Stop/Insert/Clear), `{{@Translate:}}` translation (target language selector, integrated TTS pronunciation, cloud model routing), `{{@Game:}}` game builder (AI-generated or pre-built, Canvas 2D/Three.js/P5.js, import/export HTML), `{{@Draw:}}` whiteboard (Excalidraw + Mermaid, AI diagram generation with per-card model selector + 🚀 Generate, robust JSON repair for local models, Insert/PNG/SVG export, 📚 Library Browser with 29 bundled packs in 6 categories), `{{@Tools:}}` web tools (Jina Reader scrape + Jina Search, multi-URL scraping, API key modal, Accept/Reject/Copy results) — `@` prefix syntax on all tag types + metadata fields (`@name`, `@use`, `@think`, `@search`, `@prompt`, `@step`, `@upload`, `@model`, `@engine`, `@lang`, `@prebuilt`); `@model:` field persists selected model per card with intelligent defaults (OCR→`granite-docling`, TTS→`kokoro-tts`, STT→`voxtral-stt`, Image→`imagen-ultra`); editable `@prompt:` textarea and `@step:` inputs in preview cards; description/prompt separation (bare text = label, `@prompt:` = AI instruction); 📎 image/PDF upload for multimodal vision analysis; per-card model selector with document-portable model persistence, concurrent block operations |
40+
| **AI Document Tags** | `{{@AI:}}` text generation (`@think: Yes` for deep reasoning), `{{@Image:}}` image generation (Gemini Imagen), `{{@OCR:}}` image-to-text extraction (Text/Math/Table modes via Granite Docling 258M, Florence-2 230M, or GLM-OCR 1.5B, 📷 live camera capture + 📎 image/PDF upload, PDF page rendering via pdf.js), `{{@TTS:}}` text-to-speech playback (Kokoro TTS per card, language selector, ▶ Play / ⬇ Save WAV), `{{@STT:}}` speech-to-text dictation (engine selector: Whisper/Voxtral/Web Speech API, 11 languages, Record/Stop/Insert/Clear), `{{@Translate:}}` translation (target language selector, integrated TTS pronunciation, cloud model routing), `{{@Game:}}` game builder (AI-generated or pre-built, Canvas 2D/Three.js/P5.js, import/export HTML), `{{@Draw:}}` whiteboard (Excalidraw + Mermaid, AI diagram generation with per-card model selector + 🚀 Generate, robust JSON repair for local models, Insert/PNG/SVG export, 📚 Library Browser with 29 bundled packs in 6 categories), `{{@Tools:}}` web tools (Jina Reader scrape + Jina Search, multi-URL scraping, API key modal, Accept/Reject/Copy results), `{{@Research:}}` autonomous experiment loop (Pyodide-powered, AI-driven code optimization with live results table, keep/discard metric tracking, configurable @metric/@direction/@max_iterations) — `@` prefix syntax on all tag types + metadata fields (`@name`, `@use`, `@think`, `@search`, `@prompt`, `@step`, `@upload`, `@model`, `@engine`, `@lang`, `@prebuilt`); `@model:` field persists selected model per card with intelligent defaults (OCR→`granite-docling`, TTS→`kokoro-tts`, STT→`voxtral-stt`, Image→`imagen-ultra`); editable `@prompt:` textarea and `@step:` inputs in preview cards; description/prompt separation (bare text = label, `@prompt:` = AI instruction); 📎 image/PDF upload for multimodal vision analysis; per-card model selector with document-portable model persistence, concurrent block operations |
4141
| **🔌 API Calls** | `{{API:}}` REST API integration — GET/POST/PUT/DELETE methods, custom headers, JSON body, response stored in `$(api_varName)` variables; inline review panel; toolbar GET/POST buttons |
4242
| **🔗 Agent Flow** | `{{Agent:}}` multi-step pipeline — define Step 1/2/3, chain outputs, per-card model + search provider selector, live step status indicators (⏳/✅/❌), review combined output; `@cloud: yes/no` with ☁️ Cloud / 🖥️ Local badge; `@agenttype:` dropdown selector (openclaw/openfang) for external agents; Docker-based local execution via `agent-runner/server.js`; Agent Execution Settings UI (Codespaces/Local Docker/Custom endpoint); GitHub Codespaces cloud execution via ☁️ toggle; **📦 Agent Containers panel** — floating toolbar panel showing running Docker containers with live status, uptime, instant stop (`docker rm -f`), badge count, daemon readiness check, startup container recovery, and floating toggle accessible when header is fully hidden |
4343
| **🔍 Web Search** | Toggle web search for AI — 7 providers: DuckDuckGo (free), Brave Search, Serper.dev, Tavily (AI-optimized), Google CSE, Wikipedia, Wikidata; search results injected into LLM context; source citations in responses; per-agent-card search provider selector |
@@ -544,6 +544,7 @@ TextAgent has undergone significant evolution since its inception. What started
544544

545545
| Date | Commits | Feature / Update |
546546
|------|---------|-----------------:|
547+
| **2026-04-01** | | 🔬 **Research Loop Tag** — new `{{@Research:}}` tag for autonomous AI-driven experiment optimization (inspired by Karpathy's autoresearch); Propose→Execute→Evaluate→Keep/Discard loop runs entirely in-browser via Pyodide (Python in WASM); configurable `@metric`, `@direction` (lower/higher), `@max_iterations`, `@model`, `@goal`; multiline `@code: \|` (mutable) and `@test: \|` (fixed harness) fields; metric extraction from stdout (`METRIC:xxx`); AI prompt includes experiment history with strategy shift hints after 3 consecutive failures; live results table with status badges (baseline/keep/discard/crash), delta indicators, progress bar; per-card model selector; Start/Stop controls; glass card UI with purple accents and pulse animation; `js/research-loop.js` (~795 lines) + `css/research-loop.css` (~300 lines) |
547548
| **2026-04-01** | | 🎨 **AI Panel UI Redesign** — centered initial chat state (Claude-like welcome with input + model selector clustered in middle, transitions to bottom-pinned on first message via CSS `:has(.ai-welcome-message)`); merged separate Attach File (paperclip) and Screenshot (camera) buttons into single `+` button with unified dropdown menu (Attach File / Capture Page / Capture Screen / Upload Image, `+` rotates to `×` when open); merged header bar and status bar into single compact header (status text inline below title, download progress bar still standalone); fixed download progress bar stuck at 96% after model load |
548549
| **2026-03-31** | | 📷 **Screenshot to AI** — new 📷 camera button in the AI chat input bar with three capture modes: Capture Page (`html2canvas` full-page snapshot, AI panel hidden during capture), Capture Screen (`getDisplayMedia` screen-share with frame extraction from a hidden DOM-attached video element), and Upload Image (file picker); captured image auto-injected into `pendingAttachments` and sent to AI for analysis; fixed black-screen capture bug (video must be in DOM for GPU decoder, wait for `timeupdate` event not just `requestAnimationFrame`); self-healing button injection via `injectButtonIfMissing()` with 2s fallback poll; CSS-independent dropdown via inline `style.display` toggling; vision model warning toast; `js/ai-screenshot.js` new module (~300 lines) + `css/ai-panel.css` styles + `js/modal-templates.js` template update + `src/main.js` registration |
549550
| **2026-03-31** | | 🦀 **OpenClaw Integration Blog Post** — published detailed technical post (`CHANGELOG-openclaw-textagent-integration.md`) documenting how OpenClaw runs natively inside TextAgent's Docker-based Agent Flow; covers `AGENT_CLI_MAP`, native CLI invocation (`openclaw agent --message ... --json`), API key forwarding, structured JSON response parsing, multi-step context chaining, cloud mode via GitHub Codespaces, and security boundaries |
Lines changed: 55 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,55 @@
1+
# Research Loop Tag — Autonomous AI Experiment Loop
2+
3+
- Added `{{Research:}}` tag — an autonomous experiment loop inspired by Karpathy's autoresearch
4+
- Created `js/research-loop.js` (~795 lines): tag parser, card renderer, loop engine, metric extraction, AI prompt builder
5+
- Created `css/research-loop.css` (~300 lines): glass card with purple accents, pulse animation, results table, status badges
6+
- Added `Research` to tag regex in `ai-docgen.js` (lines 157, 380) with pass-through skip for independent transform
7+
- Added `transformResearchMarkdown` to renderer.js transform chain and `data-research-index` to DOMPurify allowed attrs
8+
- Added `bindResearchPreviewActions` to renderer.js post-render bindings
9+
- Added `research-loop.js` and `research-loop.css` imports to `src/main.js` (phase 3g-ext)
10+
- Loop runs entirely in-browser via Pyodide (Python in WASM) — zero external API calls
11+
- Supports `@runtime`, `@metric`, `@direction`, `@max_iterations`, `@model`, `@goal`, `@code: |`, `@test: |` fields
12+
- Implements Propose→Execute→Evaluate→Keep/Discard cycle with live results table
13+
- AI prompt includes experiment history and strategy shift hints after 3 consecutive failures
14+
- Gracefully handles AI errors and code crashes — logs as "crash" and continues loop
15+
- Registered `research` runtime adapter for exec-controller integration
16+
17+
---
18+
19+
## Summary
20+
Implemented the `{{Research:}}` tag, bringing Karpathy's autoresearch paradigm into TextAgent as a fully client-side, in-browser autonomous experiment loop. Users define a goal, mutable code, and a fixed test harness, then the system iteratively proposes AI modifications, executes them via Pyodide, and keeps only improvements — all visualized in a live results table.
21+
22+
---
23+
24+
## 1. Research Loop Controller
25+
**Files:** `js/research-loop.js`
26+
**What:** New module implementing the full Research tag lifecycle — parsing multi-line `@code: |` and `@test: |` fields, rendering experiment cards with model dropdown/config badges/code preview, driving the autonomous keep/discard loop, extracting metrics from stdout, and building AI prompts with experiment history context.
27+
**Impact:** Users can now define optimization experiments that run autonomously, iterating on code until max iterations or manual stop. The loop handles all failure modes gracefully (AI errors, runtime crashes, missing metrics).
28+
29+
## 2. Research Card Styling
30+
**Files:** `css/research-loop.css`
31+
**What:** Complete card CSS with dark glass aesthetic, purple accent borders, pulse animation during active loops, sticky-header results table with colored status badges (baseline/keep/discard/crash), green/red delta indicators, progress bar with shimmer effect, and light theme overrides.
32+
**Impact:** Premium visual experience matching the existing TextAgent card design language.
33+
34+
## 3. Tag System Integration
35+
**Files:** `js/ai-docgen.js`, `js/renderer.js`
36+
**What:** Added `Research` to the tag regex so the parser recognizes it, with a pass-through skip in `transformDocgenMarkdown()` so the tag flows to `research-loop.js`'s own transform. Added `transformResearchMarkdown` to the renderer transform chain and `bindResearchPreviewActions` to post-render bindings. Added `data-research-index` to DOMPurify allowed attributes.
37+
**Impact:** Research tags integrate cleanly into the existing tag system without affecting other tag types.
38+
39+
## 4. Module Loading
40+
**Files:** `src/main.js`
41+
**What:** Added CSS import for `research-loop.css` and dynamic JS import for `research-loop.js` after linux-docgen (phase 3g-ext).
42+
**Impact:** Module loads at the correct point in the dependency chain — after AI models and Pyodide runtime are available.
43+
44+
---
45+
46+
## Files Changed (6 total)
47+
48+
| File | Lines Changed | Type |
49+
|------|:---:|------|
50+
| `js/research-loop.js` | +795 | NEW — Core loop controller |
51+
| `css/research-loop.css` | +300 | NEW — Card styling |
52+
| `js/ai-docgen.js` | +11 −2 | Tag regex + pass-through skip |
53+
| `js/renderer.js` | +7 −3 | Transform chain + bindings |
54+
| `src/main.js` | +4 | CSS + JS imports |
55+
| `index.html` | 0 | No changes needed (Vite handles CSS) |

0 commit comments

Comments
 (0)