Skip to content

render: parallelize frame capture across a Chrome tab pool (frames are pure functions of t) #3

Description

@darioalessandro

Summary

Frame capture is strictly sequential today, but the engine's own contract makes every frame independent: templates are pure functions of (data, t) (enforced by design — no CSS animations, no timers, __sceneSeek(t) is idempotent). That makes rendering embarrassingly parallel, and on an M-class laptop we're leaving most of the cores idle during the slowest part of the pipeline.

Where the time goes

crates/videoeditor/src/render.rs (render_web_scene):

for i in 0..total_frames {
    let t_ms = i as f64 / ep.meta.fps as f64 * 1000.0;
    chrome.seek(t_ms)?;                                   // Runtime.evaluate over one WS
    chrome.screenshot(&frames_dir.join(...))?;            // Page.captureScreenshot over the same WS
}

One Chrome (one process, one page target, one synchronous tungstenite WebSocketcrates/videoeditor-chrome/src/lib.rs) does seek → screenshot → base64-decode → fs::write for every frame, one at a time. A 1080×1920 scene at 30fps means hundreds of round-trips with zero overlap: while Chrome rasterizes + PNG-encodes, the Rust side is blocked on the socket; while Rust decodes base64 and writes the PNG, Chrome is idle. Scenes are also rendered one after another (run() loop).

Proposal: a small tab pool, frames fanned out across it

  1. Keep one Chrome process (launch is already guarded against profile races), but open N page targets via /json/new — each with its own WebSocket. Chrome gives each target its own renderer process, so rasterization genuinely parallelizes.
  2. Each worker tab does the same navigate(template) + init_scene(data_json) once, then pulls frame indices from a shared queue (work-stealing not needed — a chunked AtomicUsize counter is enough since frames cost roughly the same).
  3. rayon fits naturally on the Rust side (par_iter over frame indices with a per-thread tab via a small pool), or a plain std::thread::scope with min(available_parallelism, jobs) workers — either way the base64-decode + fs::write also moves off the single thread. Chrome is !Sync (stateful WS), so it's one tab handle per worker, never shared.
  4. Scene-level parallelism composes on top for videoeditor render <dir> with many scenes (each scene claims a tab), but frame-level is the win that also helps the common --scene X iteration loop.

Determinism is untouched: same (data, t) → same pixels, and output filenames f_{i:05}.png are index-derived, so capture order is irrelevant. encode_frames still runs once after the scene completes.

Details to keep in mind

  • sceneWarnings() should run once (on one tab), not N times, or warnings print N-fold.
  • Progress line (\r frames: i/total) needs an atomic counter instead of loop order.
  • Pool size: cap it (--jobs, default something like min(cores - 2, 8)) — each renderer target costs real memory at 1080×1920, and past ~8 tabs Page.captureScreenshot throughput tends to flatten before memory does. Worth a quick benchmark table in the PR.
  • Multiple Chrome processes (instead of tabs) would need the profile-dir nonce fixed first: the temp profile is keyed on std::process::id() of videoeditor (videoeditor-chrome/src/lib.rs), so two Chrome::launches from one run would collide. Tabs avoid this entirely; if processes ever become preferable, add a per-launch nonce.
  • video-clip scenes already bypass Chrome (ffmpeg) and are unaffected.

Expected impact

Anecdatum from a real episode (M4 Max, 1080×1920@30): a 5-scene render is a coffee break today, with 14+ performance cores never breaking a sweat. Even a conservative 4–6× on frame capture changes the director loop from "render, wait, review" to near-interactive.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Fields

    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions