feat: browser automation, workspace memory, semantic search, doc editing#28
Open
GlamgarOnDiscord wants to merge 13 commits into
Open
feat: browser automation, workspace memory, semantic search, doc editing#28GlamgarOnDiscord wants to merge 13 commits into
GlamgarOnDiscord wants to merge 13 commits into
Conversation
New sinew-browser crate exposing 18 agent tools: browser_open, browser_screenshot, browser_dom, browser_click, browser_type, browser_eval, browser_console, browser_network, browser_wait, browser_scroll, browser_select, browser_hover, browser_close, browser_record_start, browser_record_stop, browser_resize, browser_back, browser_keys, browser_find, browser_pdf, browser_upload, browser_cookies, browser_iframe. - BrowserSession launches Chrome/Edge via CDP (chromiumoxide) - Isolated profile in %TEMP%/sinew-browser-profile to avoid conflicts with running browser instances - Selector hint system: lists visible interactive elements on failure - GIF session recording (recording.rs) - Stealth overrides to hide CDP automation fingerprint
New workspace_memory tool backed by .sinew/memory.md: - Actions: read, append, update, clear - File created automatically on first use - Auto-injected into the system prompt at session start so the agent retains workspace knowledge across conversations
New sinew-search crate + semantic_search tool: - tree-sitter AST chunking per top-level declaration (Rust, TypeScript, JavaScript, Python, Go; line fallback) - fastembed v4 local ONNX embeddings (AllMiniLML6V2, 384 dims) - SQLite index: FTS5 BM25 + cosine similarity, RRF fusion (k=60) - Incremental indexing via SHA256 per file - Respects .gitignore, skips target/ and node_modules/ - Two tools: index_workspace + semantic_search
New doc_read + doc_edit tools backed by a stateless Python sidecar (sinew-sidecar/sinew_docs.py) communicating over stdin/stdout JSON: Supported formats: - DOCX: python-docx — find/replace preserving run[0] formatting, paragraph insert/delete via lxml - PDF: PyMuPDF redaction API — surgical text replacement without recreating the file - XLSX: openpyxl — cell-level string replacement - PPTX: python-pptx — run-level text replacement sidecar discovery: SINEW_DOCS_SCRIPT env var -> next to exe (prod) -> sinew-sidecar/sinew_docs.py relative to CWD (dev)
- tool_names: constants for all new tools (BROWSER_*, WORKSPACE_MEMORY, INDEX_WORKSPACE, SEMANTIC_SEARCH, DOC_READ, DOC_EDIT) - TurnContext: new Arc fields (browser, workspace_memory, semantic_search, doc_tool) - tool_dispatch: dispatch branches for all new tools - turn: descriptor registration + both run_tool call sites updated - sinew-app/lib.rs: export new types - store.rs: browser_enabled setting - tool_run: err_with_images helper for browser failure screenshots - Workspace crates: sinew-browser + sinew-search added to workspace - Tauri schemas regenerated
ToolCard: - BrowserGlyph icon for all browser_* tools - Rich title with action label + contextual meta (URL, selector, text) - Thumbnail preview of screenshot in collapsed card ChatPane: - ArrowUp/Down navigates through message history in composer (draft preserved, restored on ArrowDown past end) TodoStrip: - isStreaming prop gates the spinner on in_progress tasks (paused square when agent is not actively streaming) App: - Skip updater check in DEV mode (avoids boot delay during dev) - tauri.conf.json: updater active=false for local builds
GNU tar on Windows interprets drive letters (C:\) as network hostnames, causing zip extraction to fail with 'Cannot connect to C: resolve failed'. Use PowerShell Expand-Archive for .zip files on win32.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Four new AI agent capabilities, each independently useful:
1. CDP Browser Automation (
feat(browser))New
sinew-browsercrate + 18 agent tools backed by chromiumoxide (Chrome DevTools Protocol):%TEMP%/sinew-browser-profile) — no conflicts with running Chrome/Edge2. Persistent Workspace Memory (
feat(memory))workspace_memorytool backed by.sinew/memory.mdper workspaceread,append,update,clear3. Semantic Code Search (
feat(search))New
sinew-searchcrate +index_workspace/semantic_searchtools:.gitignore4. Surgical Document Editing (
feat(docs))doc_read+doc_edittools via a stateless Python sidecar (sinew-sidecar/sinew_docs.py)5. UI improvements (
feat(ui))6. Build fix (
fix(build))prepare-sidecars.mjs: useExpand-Archive(PowerShell) instead oftarfor.zipon Windows — GNU tar interpretsC:\as a network hostnameCommits
feat(browser): CDP browser automation via chromiumoxidefeat(memory): persistent workspace memory toolfeat(search): semantic code search with local embeddingsfeat(docs): surgical document editing via Python sidecarfeat(agent): wire all new tools into the agent pipelinefeat(ui): browser tool cards, composer history, streaming task indicatorfix(build): use PowerShell Expand-Archive for zip extraction on WindowsTest plan
workspace_memory appendthen start new conversation; verify content appears in system promptindex_workspacethensemantic_search "authentication"on any code projectdoc_readon a.docx;doc_edit find_replaceverifies surgical edit (requirespip install python-docx PyMuPDF openpyxl python-pptx)npm run prepare-sidecarscompletes without tar errorsNotes
index_workspace(~25 MB, cached in HuggingFace hub)