Skip to content

improvements: cap read_file at 10 MiB to avoid multi-hundred-MB string round-trips#167

Merged
jmaxdev merged 1 commit intoTrixtyAI:mainfrom
matiaspalmac:improvements/read-file-size-cap
Apr 21, 2026
Merged

improvements: cap read_file at 10 MiB to avoid multi-hundred-MB string round-trips#167
jmaxdev merged 1 commit intoTrixtyAI:mainfrom
matiaspalmac:improvements/read-file-size-cap

Conversation

@matiaspalmac
Copy link
Copy Markdown
Contributor

[Improvement]: Cap read_file at 10 MiB to avoid multi-hundred-MB string round-trips

Description

read_file returned the entire file body as a String with no size
gate. Opening a 200 MB log pipes ~500 MB+ through the pipeline: disk
read → owned String allocation → serde_json encode → IPC → JS string
decode → Monaco buffer. That is enough to OOM the renderer on
low-memory machines and causes a visible freeze on anything
substantially larger than the available RAM.

Change

apps/desktop/src-tauri/src/lib.rsread_file now stats the file
first and rejects up front when metadata.len() exceeds
READ_FILE_MAX_BYTES = 10 * 1024 * 1024 (10 MiB). The error message
includes the actual file size and the limit so the frontend can
surface an actionable message to the user instead of a generic I/O
error.

const READ_FILE_MAX_BYTES: u64 = 10 * 1024 * 1024;

#[tauri::command]
fn read_file(path: String) -> Result<String, String> {
    let metadata = fs::metadata(&path)…?;
    if metadata.len() > READ_FILE_MAX_BYTES {
        return Err(format!(
            "File {path} is {actual} bytes, which exceeds the {limit}-byte read_file limit; use a streaming reader for files this large",));
    }
    fs::read_to_string(&path)}

Trade-offs

  • Why 10 MiB. Monaco, the AI-chat context window and the
    git-explorer diff viewer are all the realistic consumers of
    read_file today; none of them handle files larger than a few MiB
    well in practice. 10 MiB is comfortably above every legitimate
    source file / package.json / skill-MD / project file in the tree
    and still an order of magnitude below the "renderer falls over"
    region.
  • Streaming path deferred. Point 2 of the issue (chunked streaming
    via Tauri v2 channels) requires a new command, a new frontend
    consumer UI, and teaching Monaco to render chunks incrementally —
    all bigger than this PR. Consumers that genuinely need large files
    (log tailer, binary viewer) should use a dedicated command rather
    than lift this cap.
  • User-visible setting deferred. Point 3 (expose the limit as a
    setting) needs a settings-schema addition and a docs update. The
    current behaviour — fail fast with a clear number in the error —
    is the part that matters for the memory-safety fix; making it
    tunable can land separately without reopening the safety question.
  • get_recursive_file_list and get_git_file_diff. The issue
    also mentions both as carrying similar patterns. They are kept out
    of this PR: the first returns an in-memory Vec<String> of paths
    (risk is cardinality, not per-file size — a different shape of
    fix), and the second shells out to git diff, where the right
    bound is on the diff output rather than any single source file.

Verification

  • cargo check and cargo clippy -- -D warnings on src-tauri
    clean.
  • Manual trace:
    • Existing callers (awareness.ts, AgentContext.tsx skill/index
      reads, AiChatComponent.tsx tool path, GitExplorerComponent.tsx
      diff/search file open) all pass small text paths → unaffected.
    • Path to a non-existent file → now returns Failed to stat file
      instead of Failed to read file. Same Result::Err branch for
      the frontend, friendlier message.
    • Path to a >10 MiB file → File … is N bytes, which exceeds the 10485760-byte read_file limit; use a streaming reader….

Related Issue

Closes #87

Checklist

  • I have checked for existing feature requests.
  • I have followed the project's coding guidelines.
  • Documentation has been updated (if applicable).
  • My changes generate no new warnings or errors.

Copilot AI review requested due to automatic review settings April 21, 2026 02:34
@github-actions github-actions bot added bug Something isn't working documentation Improvements or additions to documentation enhancement New feature or request question Further information is requested labels Apr 21, 2026
@github-actions
Copy link
Copy Markdown

Thanks for the contribution! I'll review it as soon as possible. If you have still changes, please mark this PR as draft and all reviews will be cancelled. Tests reviews will be re-run only when the PR is marked as ready for review.

Copy link
Copy Markdown

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR adds a size limit to the Tauri read_file command to prevent large file reads from being fully buffered into a String and sent over IPC, which can lead to renderer freezes/OOMs in the desktop app.

Changes:

  • Introduce a READ_FILE_MAX_BYTES constant (10 MiB) for read_file.
  • read_file now stats the target first and returns a clear error when the file exceeds the limit.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread apps/desktop/src-tauri/src/lib.rs
@jmaxdev jmaxdev force-pushed the improvements/read-file-size-cap branch 2 times, most recently from 23730d1 to 5d19f7c Compare April 21, 2026 03:30
@matiaspalmac matiaspalmac force-pushed the improvements/read-file-size-cap branch from 5d19f7c to 789d846 Compare April 21, 2026 03:54
…g round-trips

read_file returned the entire file body as a String with no size gate.
Opening a 200 MB log pipes ~500+ MB through: disk read → owned String
allocation → serde_json encode → IPC → JS string decode → Monaco
buffer, which is enough to OOM the renderer on low-memory machines and
causes a visible freeze on anything bigger.

Fix: stat the file first and reject up front when the length exceeds
10 MiB (READ_FILE_MAX_BYTES). 10 MiB is well above any source file the
Monaco editor, AI chat context window, or git-explorer diff viewer
actually handle well, and every in-tree caller reads small text
(package.json, skill/index MD, workspace files opened from the tree),
so legitimate usage is unchanged. Consumers that genuinely need large
files (future streaming viewer, log tailer) are expected to use a
separate chunked Tauri command rather than lifting this cap.

The error message includes the actual file size and the limit so the
frontend can surface an actionable message to the user.
@jmaxdev jmaxdev force-pushed the improvements/read-file-size-cap branch from 789d846 to fabcd4e Compare April 21, 2026 04:15
@jmaxdev jmaxdev enabled auto-merge (squash) April 21, 2026 04:16
@jmaxdev jmaxdev merged commit ea6ef5c into TrixtyAI:main Apr 21, 2026
1 check passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug Something isn't working documentation Improvements or additions to documentation enhancement New feature or request question Further information is requested

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Improvement]: Enforce read_file maximum size or stream large files via Tauri channels

3 participants