Skip to content

Latest commit

 

History

History
604 lines (411 loc) · 40.1 KB

File metadata and controls

604 lines (411 loc) · 40.1 KB

Multi-Document Support — Development Plan

Terminology: This doc previously used workspace as a placeholder for “the set of files the agent can see and read.” The final product term was decided as document_research. Code and config keys use the document_research domain.

Living document: Update this file as phases ship, decisions get made, or scope changes. Link PRs and topic docs from Related docs and the Changelog.


Problem

Today WriterAgent is effectively single-document: chat resolves one LO model from the sidebar frame, injects [DOCUMENT CONTENT] for that doc only, and tools operate on ToolContext.doc. Users want cross-document workflows without typing full paths:

  • “Grab the Q4 numbers from our budget spreadsheet and add them to this table.” (Writer or Calc as the active doc)
  • “Pull slide 3 talking points from the deck into this Writer doc.”

Unlike a code repo with thousands of files, a typical office folder has tens of documents; fuzzy names (“budget 2026”) should resolve without exact filenames. The @ mention UI is nice-to-have; backend listing + read matters more.


MVP (Phase 0) — shipped

Status: Phase 0 is implemented in-tree (2026-05-17). Automated tests: test_nearby.py, test_nearby_specialized.py, test_nearby_uno.py. Manual Calc acceptance (#1) still recommended before calling the feature “done” in production.

Phase 0 provides same-directory discovery and cross-file read via two-tier delegation. Main adds only document_research to the existing delegate enum — no new core tools on main.

User flow: one delegate_to_specialized_*_toolset(domain="document_research", task="…") call → outer sub-agent lists/opens → inner sub-agent(s) read with production read tools → compact result to main → main writes active doc only.

  • Same folder as the active saved document’s parent directory (no config dirs, no recursion).
  • Untitled active doc: list from LibreOffice profile Work folder (get_work_directory via com.sun.star.util.thePathSettings); if that path is missing, fall back to other open LO file URLs only (see Edge cases).
  • Later phases add config directories, prompt injection, UI, headless, and polish — see Phased implementation.

Design principles

Principle Detail
Same read tools, not main’s schema The inner sub-agent calls the same production read tools as today (get_document_content, get_document_tree, search_in_document, get_sheet_summary, read_cell_range, Draw read tools, …) via ToolRegistry.execute — not a parallel extractor. Main chat does not register those tools; it only has core tools plus delegate_to_specialized_*_toolset. Discovery (list_nearby_files on the outer agent) and targeting (open hidden/read-only, point ToolContext.doc at that model) live in the document_research domain.
Read-only on other files Mutation tools apply to the active document only. Sibling files are opened read-only; write APIs are not exposed on those models in Phase 0 (see Read-only enforcement and Open questions #4).
Writer + Calc + Draw Calc is first-class (budget → table). Draw/Impress: list_pages + shape/text tools where applicable.
Active doc unchanged Cross-doc reads must not steal focus or mutate the user’s window; hidden open in the same LO process for Phase 0.
Fresh contexts when tool sets change Main chat keeps one OpenAI-style history; don’t swap its wire tool schema mid-thread. Sub-agents are ephemeral task runs: focused tool context, compact result, smol runtime torn down afterward.
Two-tier delegation Outer picks the file; inner gets read tools for that doc_type. See Two-tier sub-agent model.
Headless later Target: headless / separate process for large files by default; defer until in-process hidden path is solid (Phase 4).

Data contracts

FileEntry

Listing tools return a consistent shape (TypedDict or dataclass in plugin/doc/nearby.py):

Field Type Notes
path str Absolute local path
name str Basename
url str file:///… for LO + MCP parity
modified float mtime; used for newest-first sort
size_bytes int Optional; used in Phase 3+ large-file heuristic
doc_type_guess "writer" | "calc" | "draw" | "image" | "unknown" From extension
is_open bool Desktop already has this URL

Exclude the active document’s path from list_nearby_files results (do not offer “read self as sibling”).

Extension allowlist

Constants in plugin/doc/document_research.py, referenced by tests. list_nearby_files takes optional file_kind: "documents" (default) or "images" (mutually exclusive listing modes).

file_kind=documentsNEARBY_FILE_EXTENSIONS: .odt, .ott, .ods, .ots, .odp, .otp, .odg, .fodt, .fods, .fodp

file_kind=imagesNEARBY_IMAGE_EXTENSIONS: .png, .jpg, .jpeg, .gif, .webp, .bmp, .svg (discovery only; not readable via delegate_read_document)

Exclude by convention: LibreOffice lock/temp files (~$*), *.tmp, *.bak, and extensions outside the selected allowlist.

open_document_for_read(path | url) -> (model, doc_type)

flowchart LR
    pathIn[path_or_name] --> abs[abspath]
    abs --> url[file URL]
    url --> resolve[resolve_document_by_url]
    resolve -->|hit| model[use open component]
    resolve -->|miss| load["loadComponentFromURL Hidden + ReadOnly"]
Loading
  • resolve_document_by_url scans already open components only — it does not open closed files.
  • New load props: Hidden=True and ReadOnly=True (today plugin/writer/format.py uses Hidden only for temp docs).
  • If a sibling is already open editable in another window, reuse that component via URL match; inner agent still gets a read-only allowlist (schema enforcement, not LO mode alone).

Fuzzy name matching (Phase 0)

MVP: basename substring filter on list_nearby_files(filter=…) plus newest-first bias when multiple matches (e.g. Budget_2026.ods vs Budget_2025.ods). No Levenshtein required for Phase 0.


Architecture

Phase 0 path: main agent → outer document_research sub-agent → inner read-only sub-agent(s) → back to main for writes on the active doc only.

flowchart TD
    UserMsg["User: grab Q4 from budget"]
    MainAgent["Main agent — active doc only"]
    Delegate["delegate_to_specialized_*_toolset domain=document_research"]
    Outer["Outer sub-agent — orchestration"]
    ListOpen["list_nearby_files, open hidden ReadOnly"]
    Inner["Inner sub-agent — one opened file"]
    ReadTools["Existing read tools: get_document_tree, read_cell_range, get_document_content, ..."]
    MoreFiles{"More files needed?"}
    OuterReturn["Return extracted facts to main"]
    WriteActive["apply_document_content / write_formula_range on active doc"]

    UserMsg --> MainAgent
    MainAgent --> Delegate --> Outer
    Outer --> ListOpen --> Inner
    Inner --> ReadTools --> Outer
    ReadTools --> MoreFiles
    MoreFiles -->|yes| ListOpen
    MoreFiles -->|no| OuterReturn --> MainAgent
    MainAgent --> WriteActive
Loading

Document resolution today: sidebar uses frame.getController().getModel() (SendButtonListener._get_document_model); MCP uses X-Document-URL + resolve_document_by_url. Nearby reads add: filesystem path → file URL → match open component or loadComponentFromURL.

See Threading / nested delegation for UNO thread rules.


Tool API contract (in-process parity, read-only elsewhere)

Main chat today: sidebar exposes core-tier tools for the active doc plus one delegate_to_specialized_{writer|calc|draw}_toolset gateway per app. That delegate’s domain parameter is an enum of specialized domains (shapes, python, web_research, …) — not the specialized tools themselves. ToolRegistry.get_schemas excludes specialized / specialized_control from the main wire list; specialized work runs in a sub-agent when USE_SUB_AGENT is on (DelegateToSpecializedBase).

Multi-document extension:

Scope APIs
Active document Unchanged for main — same core read/write tools on ToolContext.doc, plus delegate gateway with document_research in the domain enum.
Other files (document_research) Same read tools as in-process chat, invoked on a temporarily opened model (Hidden + ReadOnly). Outer exposes list_nearby_files and delegate_read_document. No write tools on sibling models unless we add a deliberate later feature.

Implementation uses two nested smol runs (outer, then inner per file). Both call ToolRegistry.execute — no duplicate tool bodies.

Headless / out-of-process readers (Phase 4), when added, expose the same logical read API to the inner sub-agent; only the transport to LO changes.

Phase 0 prompt: document_research to research (my / our) personal documents; web_research to research public topics. embedded is active-doc OLE only. Main chat: one delegation with described file(s) and task (vague labels OK); outer agent matches via list_nearby_files.


Two-tier sub-agent model

Cross-file reads use two ephemeral sub-agent runs (outer, then one or more inners). Sub-agents are not part of the user’s main chat history. The main agent keeps the same wire tool list it has todayno new tools on main beyond adding workspace to the delegate domain enum. Rationale: Why two tiers.

Roles

Layer Responsibility Tool surface
Main agent User intent; edits active doc only. Core tools on active ToolContext.doc + delegate_to_specialized_{writer|calc|draw}_toolset (domain enum includes document_research). No specialized-tier tools on main’s wire schema.
Outer sub-agent Natural-language task from main. Lists nearby files, resolves names, calls inner per file, aggregates. list_nearby_files, delegate_read_document(path, task), specialized_workflow_finished / final answer. No apply_* / write_*.
Inner sub-agent One run per opened file; doc_type known. Read tools for that type only — Writer inner never sees read_cell_range; Calc inner never sees get_document_tree. Same production schemas; no writes.

Reference scenario

User (on active report or sheet): “Get Q4 numbers and put them in a table.”

Step Who What happens
1 Main Parses intent: read Q4 from a budget file; write table on active doc. Does not open siblings.
2 Main delegate_*(domain="document_research", task="Find Q4 numbers in the budget spreadsheet and return structured data").
3 Outer list_nearby_files(filter="budget") → match candidates; newest-first bias.
4 Outer delegate_read_document(path, task) — hidden read-only open → inner (e.g. “extract Q4 revenue”).
5 Inner On Calc model: get_sheet_summaryread_cell_range. On Writer budget: search_in_document / get_document_content. Returns to outer; inner discarded.
6 Outer Repeat step 4 if needed; aggregate; final_answer to main.
7 Main write_formula_range, apply_document_content, etc. on active doc only.

Important: Main issues one workspace delegation, then edits the active document with the returned payload.

Shorter control-flow checklist

  1. User: “Get Q4 numbers and put in a table.”
  2. Main → delegate document_research with read task.
  3. Outer → list → delegate_read_documentinner per file → aggregate → return to main.
  4. Mainwrite_* / apply_* on active doc only.

Why two tiers

1. Within-session tool stability on main — Sub-agents avoid mid-thread schema churn on main; results fold back as a single tool result.

2. Right tools only after we know the file — Inner allowlist depends on opened doc_type (.ods → Calc tools; .odt → Writer tools).

3. Outer vs inner split — Outer: discovery, open, multi-file loop, aggregate. Inner: one file, one modality.

4. Multi-file orchestration — Outer calls delegate_read_document separately per file before one return to main.

5. Main stays lean — Users who never use workspace pay zero extra tool-schema cost on main.

6. Anti-pattern: read tools on main — Pollutes the schema for everyone.

Implementation note: Same gateway pattern as delegate_*(domain="shapes"), but document_research adds a nested inner smol run via delegate_read_document (not a single-level domain like python’s tools). Infrastructure: DelegateToSpecializedBase, build_toolcalling_agent — see writer-specialized-toolsets.md and smol-main-chat-tool-architecture.md.

Handoff mechanism (implemented)

  • DelegateReadDocument: dedicated tool — not recursive DelegateToSpecializedBase.execute. Resolves name via resolve_path_or_name, opens via open_document_for_read on the main thread (execute_on_main_thread).
  • run_inner_read_agent: builds ToolContext(doc=opened_model, read_only_target=True, …); runs smol with READ_TOOLS_BY_DOC_TYPE[doc_type] + specialized_workflow_finished.
  • Outer document_research sub-agent is still launched by DelegateToSpecializedBase with active_domain="document_research" (tools from nearby_tools.py + DelegateReadDocument).
  • Inner ends with specialized_workflow_finished / final_answer; outer accumulates and returns one payload to main.

Registration (doc module)

Document research is cross-app: active Writer may read a Calc .ods budget. Register on plugin/doc/ with specialized_cross_cutting = True (same pattern as RunVenvPythonScript) so Writer, Calc, and Draw delegate gateways all expose domain="document_research".

Component tier specialized_domain Notes
list_nearby_files specialized document_research Outer agent only
delegate_read_document specialized document_research Spawns inner smol; outer only
Inner read tools Not a separate domain on main registry; allowlist inside delegate_read_document

Gateway enum discovery: marker subclasses with specialized_domain = "document_research" on each app base — ToolWriterDocumentResearchBase, ToolCalcDocumentResearchBase, ToolDrawDocumentResearchBase in writer/specialized_base.py, calc/base.py, draw/base.py. Tools register once in plugin/doc/__init__.py via auto_discover(nearby_tools, nearby_specialized).


Read-only enforcement

Schema omission alone is insufficient; Phase 0 enforces at execution time:

  1. ToolContext.read_only_target — set on the inner agent’s context in run_inner_read_agent.
  2. Inner allowlistREAD_TOOLS_BY_DOC_TYPE in nearby_specialized.py; inner uses registry.get_tools(..., names=allowlist) — not active_domain="document_research" on the opened model.
  3. Defense in depthToolRegistry.execute returns READ_ONLY_TARGET when ctx.read_only_target and tool.detects_mutation().

Threading / nested delegation

Layer Thread UNO
Main tool loop Main Sync tools via execute_safe
Outer smol Worker (DelegateToSpecializedBase.is_async) List/open via execute_on_main_thread or async open tool
Inner smol Same worker as outer Read tools via SmolToolAdapter(..., main_thread_sync=True)execute_on_main_thread

Phase 0 tests (implemented):

  • test_nearby_specialized.py: mock inner agent (no live LLM); document_research on all three delegates; USE_SUB_AGENT=False error; READ_ONLY_TARGET guard; outer document_research tool surface; delegate_read_document is not a delegate gateway recurse.
  • test_nearby_uno.py: hidden+read-only open; list excludes active file; inner path reads sibling via read_cell_range on opened model.
  • stop_checker is copied from parent to inner ToolContext in run_inner_read_agent.

See streaming-and-threading.md.


Reusing existing read tools (not new extractors)

The inner sub-agent calls production read tools via ToolRegistry.execute:

  1. Outer resolves path / name from catalog.
  2. Outer obtains UNO model via open_document_for_read (see Data contracts).
  3. Inner receives ToolContext(doc=model, doc_type=…, read_only_target=True, ctx=…).
  4. Inner invokes read tools only:
Source doc type Typical inner allowlist
Writer get_document_content (content.py; optional include_images, default false), get_document_tree (outline.py), search_in_document
Calc get_sheet_summary (sheets.py), read_cell_range (cells.py)
Draw / Impress list_pages (shapes.py), get_draw_tree (tree.py) — see READ_TOOLS_BY_DOC_TYPE in nearby_specialized.py

Calc example: User has Q4_Report.odt open → main delegates document_research → outer opens Budget_2026.ods → inner get_sheet_summary + read_cell_range → main apply_document_content on active doc.

Large Writer docs: Inner should prefer get_document_tree / search_in_document before full get_document_content (existing truncation limits apply).


Edge cases and failure modes

Case Behavior
Untitled / unsaved active doc Implemented: resolve_listing_directory uses LO profile Work path (thePathSettingsWork); if no directory, list other open LO documents only; if none, tool error with a clear message.
Active file in directory listing Exclude self from list_nearby_files.
Permission denied / I/O error Tool error with details.path.
File modified on disk after open Phase 0: no cache; re-open on next delegate if needed. Phase 6 may add mtime-keyed metadata.
Same path already open Reuse via resolve_document_by_url.
Non-file URL (google-docs:, etc.) get_document_pathNone; treat like untitled.
Huge directory Cap list (e.g. 100 entries); set truncated: true in tool result.
Sensitive neighbors Phase 1+ optional exclude globs (.*, *.pem, …) in config.

Scope boundaries

Surface Note
Sidebar chat Primary target for workspace delegation.
Menu “Chat with Document” No tool-calling today (AGENTS.md) — workspace not supported until menu gains tools.
USE_SUB_AGENT = False DelegateToSpecializedBase returns DOCUMENT_RESEARCH_REQUIRES_SUB_AGENT for domain="document_research" (no in-place domain switch).
Librarian mode No document_research reads until document mode (switch_mode / switch_to_document_mode).
Extend / Edit selection Single-doc selection path; out of scope unless explicitly wired.
Hermes ACP Defer; same registry tools if host exposes delegate gateway later.

Open questions

Record decisions here as we learn.

# Question Notes
1 Catalog in main system prompt? Option A: inject [NEARBY FILES] every turn. Option B: outer calls list_* when needed (default for Phase 0). Option C: hybrid when ≤N files or user mentions another doc. Revisit in Phase 2.
2 Main vs sub-agent for multi-file tasks Decision: two-tier delegation in Phase 0 — main only delegate_*(domain="document_research", …); outer orchestrates; inner reads per file. Same gateway as domain="shapes"; unique nested inner smol via delegate_read_document.
3 Untitled / unsaved active doc Decision (Phase 0): saved parent dir → else Work path → else open-docs-only list. Unit-tested (test_get_work_directory, test_resolve_listing_directory_work_fallback).
4 Write-back to nearby files Out of scope: siblings read-only; writes on active doc only.
5 Final naming Decision: named document_research to best describe document-centric cross-file reading.
6 Inner allowlist Decision (Phase 0): static READ_TOOLS_BY_DOC_TYPE in nearby_specialized.py.
7 Max files per list Cap (e.g. 100) — document in list_nearby_files result.

Phased implementation

Phases are incremental; each shippable with make test. Phase dependency:

flowchart LR
    P0[Phase0_two_tier_same_dir]
    P1[Phase1_config_dirs]
    P2[Phase2_prompt_optional]
    P3[Phase3_open_hardening]
    P4[Phase4_headless]
    P5[Phase5_UI]
    P0 --> P1
    P0 --> P2
    P0 --> P3
    P3 --> P4
    P0 --> P5
Loading

Phase 0 — Same-directory MVP (two-tier delegation) — shipped

Goal: End-to-end cross-file read from the active document’s parent directory (or Work / open-docs fallbacks).

Main (unchanged wire shape):

  • delegate_to_specialized_{writer|calc|draw}_toolset(domain="document_research", task="…") only.
  • No list_nearby_files, no read tools, no new core tools on main.
  • Delegate gateway descriptions mention domain=document_research (Writer/Calc/Draw).
  • USE_SUB_AGENT must be on for document_research (DOCUMENT_RESEARCH_REQUIRES_SUB_AGENT otherwise).

Outer sub-agent (specialized_domain="document_research"):

  • ListNearbyFileslist_nearby_files: NEARBY_FILE_EXTENSIONS, newest first, exclude active path, optional filter, truncated when capped (default 100).
  • DelegateReadDocument: path_or_name + task; hidden+read-only open (or reuse open); spawns inner; returns { path, doc_type, result }.
  • specialized_workflow_finished / final answer to main (same as other specialized domains).

Inner sub-agent (per opened file):

  • READ_TOOLS_BY_DOC_TYPE allowlist; run_inner_read_agent.
  • ToolContext.doc = opened model; read_only_target=True — see read-only enforcement.
  • One inner run per delegate_read_document call; discarded after return.

Library (plugin/doc/nearby.py):

  • FileEntry, NEARBY_FILE_EXTENSIONS, get_document_directory, get_work_directory, resolve_listing_directory, list_nearby_files, resolve_path_or_name, open_document_for_read, guess_doc_type_from_path.

Module wiring: plugin/doc/__init__.py auto_discover for nearby_tools and nearby_specialized. No new keys in module.yaml.

Out of scope Phase 0: config directories, [NEARBY FILES] prompt injection, @ UI, headless, metadata cache, write-back to siblings.

Done when:

  • Calc acceptance scenario #1 (Q4 budget) passes manually (recommended before release notes).
  • make test / pytest green: test_nearby.py, test_nearby_specialized.py.
  • UNO: hidden open, list excludes self, sibling read via read_cell_rangetest_nearby_uno.py.
  • Active window focus unchanged after cross-file read (manual check).

Phase 1 — Config-expanded directories — deferred (low priority)

Status (2026-05-17): Deferred. Phase 0 (same-folder / two-tier workspace) is sufficient for now. Revisit when users need folders outside the active document’s parent (e.g. shared project drive). A draft implementation plan exists in .cursor/plans/ but is not scheduled.

Goal: User-configured extra roots beyond same-folder listing.

  • Config key: workspace_extra_directories (JSON array of absolute paths) — name TBD per open question #5.
  • list_nearby_files merges: active doc’s parent dir + configured dirs, deduped, newest-first.
  • Settings UI: text field in dialog_views.py / manifest registry (see config.py).
  • Tests: merge, dedupe, invalid path handling.

Done when:

  • Config round-trip and listing merge covered by unit tests.
  • Optional exclude globs for sensitive filenames (see edge cases).

Phase 2 — Prompt integration (optional / gated)

Goal: Model may see filenames without calling list_* first.

  • Compact [NEARBY FILES] in panel.py or via get_document_context_for_chat — cap separately from 8k doc context (e.g. 50 files / 2000 chars); consider chat_context_length config.
  • Guidance in constants.py templates, gated by config flag default off.
  • Re-evaluate open question #1 after measuring token use.

Done when:

  • Prompt snapshot tests; flag off by default.

Phase 3 — Hidden open hardening

Goal: Reliable in-process opens without disturbing the user.

  • Weakref cache of hidden models per path; close after read or idle timeout.
  • Size heuristic: small files in-process; threshold for “large” (log/warn only until Phase 4).
  • UNO tests: Writer, Calc, Draw/Impress samples.
  • Verify ToolContext.doc scoping between active and opened models (not DocumentCache — inactive in codebase).

Done when:

  • UNO tests for cache lifecycle and multi-doc isolation.

Phase 4 — Headless / separate process (deferred)

Goal: Large files do not block the UI LO instance.

  • Persistent or one-shot soffice --headless with separate user profile (--env:UserInstallation=...).
  • Worker script or UNO bridge; AsyncProcess for lifecycle.
  • Default to headless for non-small docs once stable; fallback to in-process hidden open on failure.
  • Inner sub-agent API unchanged; transport only.

Done when:

  • Large-file scenario does not block main LO UI thread.

Phase 5 — UI: @ mentions and pickers (deferred)

Goal: Discoverability; not required for core behavior.

Done when:

  • Manual UI checklist completed.

Status surfacing in chat

Shipped (open-only): When the outer document_research sub-agent calls delegate_read_document, the sidebar chat shows a short status block (Tool: delegate_read_document + open preview line — same shape as web research’s Tool: web_search + query preview). Formatting lives in document_research_chat.py; the outer delegate’s tool_call_handler in specialized_base.py appends via chat_append_callback (wired from tool_loop.py for domain=document_research). Status only — no HITL approval.

Deferred (not implemented):

  • list_nearby_files preview blocks when the outer agent lists siblings.
  • Inner read-tool steps (read_cell_range, get_document_content, …) via chat_append_callback on the inner ToolContext in document_research_specialized.py — useful when step-by-step cross-doc read visibility is desired (similar to “Show search thinking” for web research).
  • Optional config gate (e.g. chatbot.show_document_research_steps or tie to an existing thinking toggle) if chat noise becomes an issue.
  • MCP parity: mcp_protocol.py ToolContext has no chat sink today; hosts would need a stream hook if they want the same status lines.

Phase 6 — Polish

Goal: Performance, MCP clarity, optional integrations.

  • Metadata cache (title, headings, sheet names) keyed by path + mtime — SQLite optional.
  • MCP (see below).
  • Recent LO documents list — if worth the coupling.
  • Drag-and-drop onto chat.

Done when:

  • MCP cross-doc path documented and tested.

MCP (Phase 6)

  • Active document: unchanged — X-Document-URL on mcp_protocol.py.
  • Nearby reads: path or url in tool args / delegate task; do not retarget X-Document-URL to sibling files.
  • Tool surface: same as sidebar — document_research via delegate_to_specialized_*_toolset(domain="document_research", …), not dozens of new MCP tool names (mcp-protocol.md specialized-tier policy).
  • Test: test_mcp_server.py — document_research delegate with header pointing at active Writer doc.

Phase 7 — Advanced Workflows (Future)

Goal: Move beyond simple reading to powerful cross-document aggregation and provenance.

7.1 Cross-Document Search ("Office Grep") — shipped

  • Concept: A dedicated tool for the outer document_research sub-agent to find keywords across multiple files without manually delegating to each one sequentially.
  • Implementation: grep_nearby_files in plugin/doc/document_research_grep.py. Args: pattern, optional file_subset (basename token such as budget*budget*.od* via list_nearby_files), caps (max_files, max_results_per_file, max_total_results). Opens each candidate hidden+read-only, searches by type (Writer paragraphs with automatic 2-paragraph context, Calc findAll all sheets, Draw shape text with shape cap), returns snippet previews, closes hidden opens, calls processEventsToIdle between files. Prefer grep before delegate_read_document when locating which file contains a keyword (hint in specialized_base.py).
  • Tests: tests/doc/test_document_research_grep.py, tests/doc/test_document_research_grep_uno.py.

7.2 Metadata Caching ("Librarian's Index")

  • Concept: "Remember" the structure of nearby files so the agent can answer "Which document has the Q3 numbers?" instantly.
  • Implementation: Expand the Phase 6 polish concept. Create plugin/doc/nearby_cache.py (using SQLite or JSON). When open_document_for_read is used, extract and cache the get_document_tree (Writer) or get_sheet_summary (Calc) keyed by (path, st_mtime). Modify list_nearby_files to optionally return this cached structural summary in the FileEntry so the outer agent sees what's inside files without opening them.

7.3 Automatic Citations & Provenance

  • Concept: When facts are pulled from sibling files, automatically cite the source.
  • Implementation: This is primarily a prompt engineering and main-agent workflow update.
    • Update DELEGATION_USER_FILE_DATA_HINT in plugin/framework/constants.py to instruct the main agent: "When inserting information extracted from nearby files, use footnotes or explicit text references (e.g., 'Source: Budget_2026.ods') to cite the origin."
    • Ensure the inner agent always returns the path and name alongside the extracted facts (currently DelegateReadDocument returns { status, path, doc_type, result }).

7.4 Multi-File Comparison (Native LO Compare)

  • Concept: Compare versions of documents (e.g., "What changed between v1 and v2?") using LibreOffice's built-in comparison engine.
  • Implementation:
    • Create a specialized tool CompareDocuments(path_a, path_b) in plugin/doc/nearby_tools.py.
    • Logic:
      1. Open path_b (the "newer" version) hidden.
      2. Use XDispatchHelper to execute .uno:CompareDocuments on the frame of path_b, passing the URL of path_a as the "original" document.
      3. LibreOffice automatically generates Tracked Changes (Redlines) in path_b.
      4. The sub-agent then uses the existing get_redline_summary (Writer) or get_change_track (Calc) tools to extract a high-level summary of exactly what was added, deleted, or changed.
    • Benefit: Provides a far more accurate and "office-native" understanding of changes than a raw text diff, including formatting and structural changes. This saves the LLM from having to hold two massive documents in its context window simultaneously.

Calc-specific scenarios (acceptance checks)

Use these to validate design and tests (Phase 0 targets #1–#5):

  1. Active Calc, read sibling Calc: main delegates → outer lists/opens budget → inner read_cell_range → outer returns figures → main write_formula_range on active sheet.
  2. Active Calc, read sibling Writer: inner get_document_content on brief → main insert_cell_html / write_formula_range.
  3. Active Writer, read sibling Calc: inner get_sheet_summary + read_cell_range → main apply_document_content.
  4. Multiple budgets: outer list_nearby_files(filter="budget"), disambiguate, optionally run inner twice before returning to main.
  5. Formula semantics: main respects Calc ; separator rules (CALC_FORMULA_SYNTAX) when writing active sheet only.

Writer / Draw scenarios (acceptance checks)

  1. Outline-first: get_document_tree on nearby .odt before full read for long docs.
  2. Search: search_in_document on nearby file for “Q4” instead of full HTML dump.
  3. Draw/Impress: Nearby .odplist_pages + text extraction; insert into active Writer/Calc via main write tools.

Files and entry points

Phase 0 (shipped)

Area Path
Catalog + hidden open plugin/doc/document_research.py
Outer tools plugin/doc/document_research_tools.pylist_nearby_files, grep_nearby_files
Grep library plugin/doc/document_research_grep.py
Inner + delegate_read_document plugin/doc/document_research_specialized.py
Module auto_discover plugin/doc/__init__.py
Domain enum markers plugin/writer/specialized_base.py (ToolWriterDocumentResearchBase), plugin/calc/base.py, plugin/draw/base.py
Delegate gateway + document_research guard plugin/doc/specialized_base.py
read_only_target + registry guard plugin/framework/tool.py
Unit tests tests/doc/test_document_research.py, test_document_research_specialized.py, test_document_research_grep.py
UNO tests tests/doc/test_document_research_uno.py, test_document_research_grep_uno.py

Later phases

Area Path
Config keys (Phase 1+) plugin/framework/config.py, manifest_registry.py
Document path / URL plugin/doc/document_helpers.py
Chat context / prompts (Phase 2) plugin/chatbot/panel.py, plugin/framework/constants.py
Tool loop plugin/chatbot/tool_loop.py
MCP (Phase 6) plugin/mcp/mcp_protocol.py

Test strategy

Phase Unit UNO / integration
0 test_nearby.py: scandir, extensions, sort, exclude self, filter, truncate, Work-path fallback test_nearby_uno.py: hidden+read-only open; list; sibling read_cell_range
0 test_nearby_specialized.py: document_research on all delegates; mock inner; read-only guard; no gateway recurse
1 Config merge, dedupe
2 Prompt snapshot / max length
3 Cache, close-on-idle, Writer/Calc/Draw samples; ToolContext.doc isolation
4 Optional subprocess mocks
5 Manual UI checklist
6 Metadata cache MCP document_research delegate test

Per AGENTS.md: matching test_*.py names; run make test before calling a phase done.


Related docs


Changelog

Date Phase / change PR / notes
2026-05-20 Phase 7.1 shipped: grep_nearby_files + document_research_grep.py; shared Writer/Calc search helpers; CPU caps + processEventsToIdle between files
2026-05-19 Chat status (open-only): tool + preview blocks for delegate_read_document via chat_append_callbackdocument_research_chat.py, specialized_base.py, tool_loop.py
2026-05-17 Phase 0 shipped: nearby.py, nearby_tools.py, nearby_specialized.py; document_research on Writer/Calc/Draw delegates; two-tier smol (outer list/delegate_read, inner READ_TOOLS_BY_DOC_TYPE); ToolContext.read_only_target + READ_ONLY_TARGET; untitled → Work path then open-docs fallback; tests in tests/doc/test_nearby*.py
(prior) Plan refresh: Phase 0 = full two-tier delegation; phases renumbered 0–6; data contracts, threading, edge cases