Summary
Frame capture is strictly sequential today, but the engine's own contract makes every frame independent: templates are pure functions of (data, t) (enforced by design — no CSS animations, no timers, __sceneSeek(t) is idempotent). That makes rendering embarrassingly parallel, and on an M-class laptop we're leaving most of the cores idle during the slowest part of the pipeline.
Where the time goes
crates/videoeditor/src/render.rs (render_web_scene):
for i in 0..total_frames {
let t_ms = i as f64 / ep.meta.fps as f64 * 1000.0;
chrome.seek(t_ms)?; // Runtime.evaluate over one WS
chrome.screenshot(&frames_dir.join(...))?; // Page.captureScreenshot over the same WS
}
One Chrome (one process, one page target, one synchronous tungstenite WebSocket — crates/videoeditor-chrome/src/lib.rs) does seek → screenshot → base64-decode → fs::write for every frame, one at a time. A 1080×1920 scene at 30fps means hundreds of round-trips with zero overlap: while Chrome rasterizes + PNG-encodes, the Rust side is blocked on the socket; while Rust decodes base64 and writes the PNG, Chrome is idle. Scenes are also rendered one after another (run() loop).
Proposal: a small tab pool, frames fanned out across it
- Keep one Chrome process (launch is already guarded against profile races), but open N page targets via
/json/new — each with its own WebSocket. Chrome gives each target its own renderer process, so rasterization genuinely parallelizes.
- Each worker tab does the same
navigate(template) + init_scene(data_json) once, then pulls frame indices from a shared queue (work-stealing not needed — a chunked AtomicUsize counter is enough since frames cost roughly the same).
rayon fits naturally on the Rust side (par_iter over frame indices with a per-thread tab via a small pool), or a plain std::thread::scope with min(available_parallelism, jobs) workers — either way the base64-decode + fs::write also moves off the single thread. Chrome is !Sync (stateful WS), so it's one tab handle per worker, never shared.
- Scene-level parallelism composes on top for
videoeditor render <dir> with many scenes (each scene claims a tab), but frame-level is the win that also helps the common --scene X iteration loop.
Determinism is untouched: same (data, t) → same pixels, and output filenames f_{i:05}.png are index-derived, so capture order is irrelevant. encode_frames still runs once after the scene completes.
Details to keep in mind
sceneWarnings() should run once (on one tab), not N times, or warnings print N-fold.
- Progress line (
\r frames: i/total) needs an atomic counter instead of loop order.
- Pool size: cap it (
--jobs, default something like min(cores - 2, 8)) — each renderer target costs real memory at 1080×1920, and past ~8 tabs Page.captureScreenshot throughput tends to flatten before memory does. Worth a quick benchmark table in the PR.
- Multiple Chrome processes (instead of tabs) would need the profile-dir nonce fixed first: the temp profile is keyed on
std::process::id() of videoeditor (videoeditor-chrome/src/lib.rs), so two Chrome::launches from one run would collide. Tabs avoid this entirely; if processes ever become preferable, add a per-launch nonce.
video-clip scenes already bypass Chrome (ffmpeg) and are unaffected.
Expected impact
Anecdatum from a real episode (M4 Max, 1080×1920@30): a 5-scene render is a coffee break today, with 14+ performance cores never breaking a sweat. Even a conservative 4–6× on frame capture changes the director loop from "render, wait, review" to near-interactive.
Summary
Frame capture is strictly sequential today, but the engine's own contract makes every frame independent: templates are pure functions of
(data, t)(enforced by design — no CSS animations, no timers,__sceneSeek(t)is idempotent). That makes rendering embarrassingly parallel, and on an M-class laptop we're leaving most of the cores idle during the slowest part of the pipeline.Where the time goes
crates/videoeditor/src/render.rs(render_web_scene):One
Chrome(one process, one page target, one synchronous tungstenite WebSocket —crates/videoeditor-chrome/src/lib.rs) doesseek → screenshot → base64-decode → fs::writefor every frame, one at a time. A 1080×1920 scene at 30fps means hundreds of round-trips with zero overlap: while Chrome rasterizes + PNG-encodes, the Rust side is blocked on the socket; while Rust decodes base64 and writes the PNG, Chrome is idle. Scenes are also rendered one after another (run()loop).Proposal: a small tab pool, frames fanned out across it
/json/new— each with its own WebSocket. Chrome gives each target its own renderer process, so rasterization genuinely parallelizes.navigate(template)+init_scene(data_json)once, then pulls frame indices from a shared queue (work-stealing not needed — a chunkedAtomicUsizecounter is enough since frames cost roughly the same).rayonfits naturally on the Rust side (par_iterover frame indices with a per-thread tab via a small pool), or a plainstd::thread::scopewithmin(available_parallelism, jobs)workers — either way the base64-decode +fs::writealso moves off the single thread.Chromeis!Sync(stateful WS), so it's one tab handle per worker, never shared.videoeditor render <dir>with many scenes (each scene claims a tab), but frame-level is the win that also helps the common--scene Xiteration loop.Determinism is untouched: same
(data, t)→ same pixels, and output filenamesf_{i:05}.pngare index-derived, so capture order is irrelevant.encode_framesstill runs once after the scene completes.Details to keep in mind
sceneWarnings()should run once (on one tab), not N times, or warnings print N-fold.\r frames: i/total) needs an atomic counter instead of loop order.--jobs, default something likemin(cores - 2, 8)) — each renderer target costs real memory at 1080×1920, and past ~8 tabsPage.captureScreenshotthroughput tends to flatten before memory does. Worth a quick benchmark table in the PR.std::process::id()of videoeditor (videoeditor-chrome/src/lib.rs), so twoChrome::launches from one run would collide. Tabs avoid this entirely; if processes ever become preferable, add a per-launch nonce.video-clipscenes already bypass Chrome (ffmpeg) and are unaffected.Expected impact
Anecdatum from a real episode (M4 Max, 1080×1920@30): a 5-scene render is a coffee break today, with 14+ performance cores never breaking a sweat. Even a conservative 4–6× on frame capture changes the director loop from "render, wait, review" to near-interactive.