[Draft!] Add higher fpm + ability to edit videos

[anscg]

tl;dr: added higher frame per minute with uploading videos instead of screenshots per minute, and added the ability to edit timelapses before they're complete. these changes are 100% backwards compatible and doesn't have meaningful performance impact, in fact, compiling are now faster :D

this pr is very much incomplete and still need a bit of polish to make sure it doesn't break anything

how it looks on the desktop app:

combined_small.mp4

tested this on lapse too :)

3e09bac1-8a28-4226-8e7b-82a94d5657d1.mp4

also some qol: including
this selector thing with the program start url thing

the recording screen now remembers what screen you chose last time!
the filter thing no longer lags your computer

REDIRECT HOOKS! so a program, e.g. lapse, would be able to redirect user to a url after a recording is completed on desktop

among other things...

i got claude to summarise this pr and i think it did a pretty good job:

1. Clips — 15 frames per minute through the same once-a-minute pipeline

Today a session is one JPEG per minute and the compiled timelapse is a
slideshow. Clips replaces that JPEG with a ~60 s video holding ~15 frames
captured 4 s apart. Same request cadence, same credit math, same everything
downstream, because a clip is still one capture unit: one upload-url, one
PUT, one confirm per minute. RATE_LIMIT_PER_MINUTE,
MAX_SCREENSHOTS_PER_SESSION and MAX_UPLOAD_REQUESTS_PER_SESSION are
untouched.

The opt-in, and why it's shaped this way

POST /api/internal/sessions takes "clips": true →
sessions.clips_enabled (0015_clips.sql). Default off, set only at
creation — no route anywhere mutates it, because a session's capture
character must never change mid-recording.

The gate is enforced per upload, not per session. upload-url?format=…
is validated against CAPTURE_FORMATS, and a clip request on a non-clips
session is silently downgraded — HTTP 200 with format: "jpeg" and a
.jpg key, not an error:

const format = requestedFormat !== "jpeg" && !session.clipsEnabled
  ? "jpeg" : requestedFormat;

A conforming client falls back without an error round-trip, and a stale or
hostile client asking for webm every minute simply gets JPEG presigns.

The granted format is law. The presigned PUT is signed with
CAPTURE_FORMAT_CONTENT_TYPES[format], so uploading anything else fails the
signature, and confirm re-validates via HeadObject against the format
stored on the row (not the request) plus the right size cap — 2 MiB for JPEG,
MAX_CLIP_BYTES = 8 MiB for clips.

Cadence is server-authoritative. frameIntervalMs and clipsEnabled ride
on both GET /api/sessions/:token and every upload-url response, and no
client exposes an override — frame rate and bitrate decide storage cost and
text legibility for everyone, so they aren't an embedder's knob. Discovery
happens on the session fetch clients already make before recording, so the
very first minute of a clips session is already a clip.

Formats may mix inside one session and this is load-bearing, not a
tolerated accident: any clip failure (encoder error, oversize output) falls
back to a plain JPEG for that minute, and an older client pointed at a clips
session just keeps uploading JPEGs. That's what makes it safe to turn clips on
before every client has shipped.

Encoding: hardware H.264 on the desktop

clients/desktop/src-tauri/src/clips.rs (new, ~840 lines) records
format=mp4 on the GPU:

Platform	Encoder
macOS	AVAssetWriter → VideoToolbox, muxes MP4 itself
Windows	Media Foundation sink writer (hardware MFT when available)
Linux	GStreamer appsrc, trying vah264enc → vaapih264enc → x264enc → openh264enc

Frames come from the existing capture loop through one redaction-aware path
that also feeds the focus-gated live preview, so a clip costs one grab, not
two. Frames are normalized to even dimensions for 4:2:0, with a fast RGBA→BGRA
swizzle when dimensions already match and a resize-and-pillarbox slow path for
a mid-clip display change.

Why the GOP is pinned by hand on all three platforms. Frames sit seconds
apart in media time, so any keyframe interval expressed in seconds makes the
encoder emit an all-keyframe clip — rationing the bit budget across ~20 I-frames
and producing uniformly soft ~20 KB/frame output. So: exactly one IDR per clip,
no frame reordering, and a rate-control hint that the source is ~1 fps
(AVVideoMaxKeyFrameIntervalKey = 1200 + AVVideoAllowFrameReorderingKey = false + AVVideoExpectedSourceFrameRateKey = 1;
MF_MT_MAX_KEYFRAME_SPACING = 10_000 + MF_MT_FRAME_RATE = 1/1;
speed-preset=superfast tune=zerolatency on x264, whose default medium
preset burned 5–10× the CPU needed). An in-crate test pushes 5 synthetic frames
through the real OS encoder and asserts via ffprobe that the output is
h264, has 5 packets, and exactly 1 keyframe — the compile pipeline depends
on that.

libvpx was measured and rejected: VP8 ran ~41 ms/frame against ~1.9 ms for
VideoToolbox.

Capability discovery is fail-safe: fetch_clip_capabilities GETs the session at
loop start, and any failure or non-2xx falls back to Default — clips off,
bit-for-bit legacy behaviour. The server's frameIntervalMs wins, clamped to
500..=30_000 against a misbehaving server.

Uploads run concurrently with capture (a2d3980). Serially awaiting
finalize+upload (~4–5 s) stalled frame collection at every tick — a hole after
each cut that compounded to minutes of lost screen time per hour. The tick now
cuts the clip and spawns the upload as a background task; collection for the
next clip resumes immediately, and the wait loop gained a third select arm
that applies the confirm when it lands (tray sync, nextExpectedAt
refinement, error recovery). Strictly one upload in flight, so capturedAt
monotonicity and the rate-limit assumptions hold; a pending upload at stop
detaches and finishes in the background so the final minute still lands.

Encoding: the browser needed a completely different number

clients/react/src/hooks/clipRecorder.ts (new) records via MediaRecorder,
probing H.264/MP4 first (avc1.640028 → 4d0028 → 42e01e →
video/mp4) with VP9/VP8 WebM as the Firefox fallback. Measured at matched
output size on Chromium 148 at 1080p, H.264 beat VP9 by 8–10 dB PSNR for the
same bytes (~115 KB/frame: 30.9 dB vs 22.3 dB), and distributed it evenly,
where VP9 spent nearly everything on the keyframe. VP8's rate control is inert
below ~10 Mbps. Same conclusion the desktop reached about libvpx.

Then the subtle one, and the reason there are now two bitrate constants:

CLIP_VIDEO_BITS_PER_SECOND (800 kbps, native) and
CLIP_WEB_VIDEO_BITS_PER_SECOND (40 Mbps, browser) are not comparable
numbers — they're denominated in different things.

A native encoder gets each frame's true presentation timestamp 4 s apart plus
an explicit ~1 fps rate-control hint, so 800 kbps really does buy ~400 KB per
frame — JPEG-q85-class keyframes at 1080p, the bar the old pipeline set.
MediaRecorder ignores wall-clock frame spacing entirely: recording the same
frames 4000 ms apart and 125 ms apart produces byte-identical output.
There is no "× 60 seconds" budget to spend. At 800 kbps the browser encoder
sat pinned at its maximum quantizer and still overshot the request — which
is why raising the shared constant 400k → 800k sharpened the desktop and did
nothing whatsoever for the web (768dcf4).

Measured sweep, worst-case dense-text content, PSNR vs source:

bitrate	KB/frame	PSNR
0.8 M	53.7	25.8 dB ← the old setting
5 M	114.6	30.9 dB
20 M	192.0	34.3 dB
40 M	335.4	38.6 dB ← knee, ~parity with native
80 M	582.3	43.7 dB (worst case exceeds the 8 MB cap)

Over a full 15-frame clip against the cap: busy screen 0.89 MB / 23.8 dB →
3.24 MB / 43.3 dB; typical screen 0.37 MB → 2.46 MB. Even incompressible
content sits at ~40 % of the cap, and if a clip ever does exceed it
ClipRecorder halves its bitrate (floored at the native rate), warns, and
returns null so the tick falls back to a JPEG — a server rejection would
cost the whole minute, and an engine with different rate-control semantics
self-corrects instead of failing every upload.

Other details worth knowing: captureStream(0) with explicit
requestFrame() (falling back to auto-capture on older Firefox), canvas fixed
to even dimensions for the clip's life, imageSmoothingQuality = "high"
before every drawImage because downscaled 1080p UI text aliases badly, and
cut() races the encoder's onstop against a 10 s timeout so a wedged
encoder can't stall the capture loop.

Cost and performance

The question everyone asks first, so: clips are close to free.

• CPU: averages ~1.3 % over a minute on a low-end Windows laptop with
  clips on. The encode is on the GPU, the frame grab is the one the preview
  already needed, and the upload is off the capture path.
• Bandwidth: only slightly above the single-JPEG pipeline, because
  compression is doing the work — 15 frames of a mostly-static screen 4 s apart
  are nearly identical, so you pay for roughly one good keyframe plus deltas
  rather than 15 independent JPEGs. Static content undershoots the VBR ceiling
  heavily; the 8 MB/min cap is for the pathological case, not the normal one.
• The capture loop got faster on the way here too: SIMD downscale via
  fast_image_resize (29 ms → 7 ms for 5K→1080p), redaction applied after
  the downscale, and Bytes buffers so an upload retry is a refcount bump
  instead of a full frame copy.

Compiling clips into a timelapse

packages/worker/src/segments.ts (new, 333 lines, deliberately free of DB/R2
imports so the ffmpeg contracts are unit-testable) normalizes every capture
unit — JPEG still, VP8 WebM, H.264 MP4, any resolution — into exactly one
second of 30 fps H.264 in an MPEG-TS segment, then stream-copy concatenates
them.

The old pipeline downloaded every JPEG, waited on a barrier, renumbered
survivors into a contiguous sequence, and ran one whole-session x264
encode at CRF 28. That's gone. Now:

• One ffmpeg pass per unit. For a clip: setpts=N/(frames*TB) spreads the N
  real demuxed frames evenly across [0, 1 s), then fps=30, then
  tpad=stop_mode=clone:stop=-1 clone-extends the last frame so PTS rounding
  can never come up short, then -frames:v 30. fa22053 removed the
  intermediate JPEG round-trip the first draft had — one fewer lossy
  generation per frame and ~40 % less per-clip work.
• The client's reported frame_count is never trusted — ffprobe -count_packets counts. Clips are VFR and a static screen legitimately emits
  fewer frames than nominal, so FRAMES_PER_CLIP = 15 is informational only.
  verifySegmentFrameCount is then strict equality, because a short
  segment would silently desync every later minute of the video.
• Downloads are pipelined into segment builds with no barrier, so early
  units encode while later ones are still downloading, with
  SEGMENT_CONCURRENCY = 8 and -threads 1 per ffmpeg (parallelism lives in
  the pool, not in x264).
• Segment CRF went 28 → 18 (f977898): the compile step must not be a quality
  event — the clip bitrate is the only intended quality dial, and CRF 28 added
  a visible second generation of loss on top of already-compressed clips.
• A corrupt unit is caught and skipped per-minute instead of poisoning the
  whole encode; wall clock now scales with units / SEGMENT_CONCURRENCY.

The mixed case needs no special handling at all — because normalization is
per-unit and uniform, a session can interleave legacy JPEGs, VP8 WebM and
H.264 MP4 at different resolutions and the assembly is still -c copy.
There's a test for exactly that ordering.

And the piece everything else hangs off: every segment is encoded with a
closed GOP of exactly 30 frames starting on an IDR —
-g 30 -keyint_min 30 -sc_threshold 0 -x264-params open-gop=0, plus pinned
-profile:v high -level:v 4.0 -pix_fmt yuv420p so segments stay
bit-compatible for copy-concat. Three separate encode paths (segment build,
assembly fallback, cut re-encode fallback) must keep those args in lockstep;
the code says so and so does the mixed-session test.

Seed-capture fix (d46a5e7, segments.ts#dropSeedUnit). A session's first
capture opens the recording rather than closing a recorded minute, and
credits 0 tracked seconds in both tracking modes — yet it was getting an equal
one-second segment. Two visible consequences: the video ran one second longer
than the tracked minute count (so the editor labelled a 1-minute recording
"2 min"), and in clips mode the opening clip is deliberately cut after ~8 s so
the session activates quickly, so that second played at ~8× against the rest
of the video's 60×. Measured on a real compile: second 0 held 9 unique frames
spanning ~8 real seconds, second 1 held 16 spanning 60 — a slow-motion lurch
at the head of every timelapse. The seed is now excluded, giving one uniform
rule: a capture earns video time exactly when it earns tracked time. The
timelapse still opens on motion because the first shown unit is a full
~15-frame clip. A single-unit session keeps its one unit (a zero-length video
is worse than an imprecise one), and the seed's DB row is preserved — only its
R2 object is cleaned up with the other unsampled captures — so /timings and
the credit history are untouched.

What program authors have to do

Nothing. trackedSeconds, screenshotCount, /timings (still one timestamp
per minute, so Hackatime forwarding is unchanged), videoUrl and every
response shape are identical between clips and non-clips sessions. The flag is
per session, so it can be enabled for a fraction of new sessions and compared.
Documented in docs/integration.md.

---

2. Edits — cutting footage, before anything downstream sees it

Full design in docs/edit-feature-plan.md.

recording ──Stop──> ┌ Keep recording ────────────────────> (back to recording)
                    ├ Stop & save ───> compile ──────────> complete
                    └ Edit & save ──> stop {edit:true}
                                        │ compile runs, session is HELD
                                        │ (status stays "stopped", unpublished)
                                        ├ user cuts → publish ─> compiling ─> complete
                                        ├ "publish as recorded" ──────────> complete
                                        └ lease lapses (~2 min) ──────────> complete

Editing lives inside the stop flow, never after complete. complete is
the signal programs act on — forwarding heartbeats to Hackatime, accepting a
submission, firing the redirect hook. Editing a published session would mutate
numbers someone already consumed. So a session reaches complete exactly
once, with its cuts already applied, and post-publication editing does not
exist (editable: false, PUT /cuts → 409).

The hold is a lease, not a countdown. Any surface representing active
editing renews it every EDIT_HEARTBEAT_SECONDS (30 s) via
POST /:token/editing; the server holds the session EDIT_LEASE_SECONDS
(120 s) past the last renewal, bounded by EDIT_HOLD_MAX_MINUTES (120) from
the stop. A fixed deadline was wrong in both directions — it cut off someone
carefully trimming a long recording, and left an abandoned session unpublished
for half an hour. A lease has neither failure, and renewal deliberately works
through a term that lapsed seconds ago (a network stall shouldn't end an
edit) — safe only because the write is guarded on
status IN ('stopped','compiling'), so a published session can never be
pulled back.

A lapsed lease publishes. lib/timeouts.ts#publishExpiredHolds runs on the
existing every-minute cron and publishes anything whose lease expired or that
passed the ceiling. The hold can delay publication, never cancel it — which is
what makes offering the edit step safe at all.

Cuts are lossless, and there's a test that proves it

The clips invariant pays off here. One capture unit = one real-world minute =
exactly one second of output, and every second boundary is an IDR frame in a
closed GOP. So:

1. Video-second i is capture unit i, whose capturedAt is known.
   sessions.video_units is that map, and it's built from the segment list,
   not the sampled rows, so build-failure holes can never desync it.
2. Each kept range is extracted with an input seek to its IDR (-ss) plus an
   exact copied packet count (-frames:v n×30) into a TS intermediate, then
   the ranges stream-copy concat. Not the concat demuxer's
   inpoint/outpoint — outpoint is dts-based and B-frame dts offsets leak
   ~2 frames of the cut region past each boundary.
3. A cut-compile is I/O-bound. Seconds even for a 12-hour session, and zero
   added generations of loss. It never downloads capture units either, so it
   works fine after the 7-day screenshot purge.

"Does cutting affect quality" deserved evidence, not an assertion (46a1011).
The worker test hashes every decoded frame of the original and of the
edited output with ffmpeg -f framemd5 (keeping only the hash column, since a
cut restarts timestamps at zero by design) and asserts the kept frames are
pixel-identical — toEqual, tolerance zero. A companion test asserts the
re-encode fallback is not bit-exact, which is what keeps the first test
honest: two re-encodes compared against each other would still pass.

That fallback runs only when video_copy_aligned = false or the copy path
throws, which shouldn't happen now that every encode pins its GOP. It used to
be a console.warn lost among other warnings; it's now a loud console.error
saying the timelapse just lost a generation of quality and what to
investigate.

Derived effects, all settled before publication

Consumer	Effect
Published video	Kept ranges of original.mp4, stream-copy concatenated into edited.mp4
GET /timings	Cut timestamps excluded by default, so existing Hackatime forwarders respect edits with zero code changes; ?includeCut=true returns them as cutTimestamps
trackedSeconds	raw − cutSeconds everywhere, via one read-side helper; raw preserved as uncutTrackedSeconds

trackedSeconds shrinking is deliberate: /timings and trackedSeconds must
tell the same story, and cutting can only reduce the number, so there's no
fraud vector. sessions.tracked_seconds stays raw as an audit trail.

One shared membership rule — a unit is cut iff
coalesce(captured_at, requested_at) ∈ [start, end) — lives once in
packages/shared/src/cuts.ts alongside normalizeCuts, computeKeptRanges,
countCutUnits and computeCutSeconds, and is used identically by the
server's preview, the worker's authoritative write, and the editor's footer.
Cuts are absolute wall-clock intervals, never "video offset + duration",
because one list has to drive three derived views consistently. Granularity is
whole minutes, which is also heartbeat granularity.

API

Token-authenticated, each with its own rate-limit bucket:

• POST /:token/stop — optional { edit: true }. The body schema is
  deliberately additionalProperties: true: this route accepted and ignored
  any body before edit existed, so rejecting unknown fields would turn a
  working custom client into a 400 for no benefit (848c768). No body → no
  hold, byte-identical to today.
• GET /:token/units (10/min) — units, cuts, editable,
  editableReason, editHoldUntil, expectedUnits, originalVideoUrl,
  recompilesRemaining. The presigned GET (1 h) is minted only when editable,
  and deliberately isn't the public media URL — after an edit, cut content
  exists only in the original, which must stay reachable through the secret
  token alone. expectedUnits = screenshotCount − 1 so the waiting-room copy
  doesn't promise a minute the video won't hold.
• PUT /:token/cuts (20/min) — full replace, idempotent, [] clears.
  Validates ISO dates and end > start, clamps to the session window ±5 min,
  sorts and merges overlapping and adjacent intervals, caps at
  MAX_CUT_INTERVALS (120), and rejects a list that would cut every unit.
  Written under status = 'stopped' AND edit_hold_until > now(), so a session
  that published mid-edit can never be mutated.
• POST /:token/compile (5/min) — publish. No cuts → publishHeldSession,
  200 instant: true, no worker round-trip and no recompile consumed.
  With cuts → claim stopped → compiling, clear the hold, bump
  recompile_count (capped at MAX_USER_RECOMPILES = 5), enqueue. Already
  complete → 200 instant: true; already compiling → 202; mid-build
  ("publish as recorded" before the original exists) → drop the hold and let
  the in-flight compile publish normally. Every path is designed so a client
  retry is never a spurious failure.
• POST /:token/editing (20/min) — renew; returns held: false once the
  session is out so clients stop polling.

lib/publish.ts#publishHeldSession is the single atomic guard shared by the
user's publish call and the expiry cron, so the two racing each other publish
exactly once and the loser's endpoint still returns 200 — the timelapse is out
either way.

lib/timeouts.ts#purgeEditedOriginals is the retention backstop on the daily
cron for originals orphaned by a crashed publish; the normal path deletes cut
content immediately on an edited publish, and only after the edited video
is uploaded and the row committed, because flipping that order would leave a
crash pointing a session at bytes that no longer exist.

Migrations 0018_session_edits / 0019_edit_hold add cuts, cut_seconds,
video_units, original_video_r2_key, video_copy_aligned,
recompile_count, last_edit_compile_at, edit_hold_until. No
screenshots changes — membership is computed from the interval list, never
denormalized onto rows.

The editor

Built once in @lookout/react (<TimelapseEditor>) and used by all three
surfaces. Region-based cutting, as asked for: drag on the timeline creates a
cut region in one gesture, and regions are first-class objects with edge
handles, selection, and delete-key removal — not a brushed mask, because a cut
needs boundaries you can inspect and adjust. Regions are integer [start, end)
unit indices, so they're snapped to capture boundaries by construction. Plain
click seeks, the ruler lane owns scrubbing, playback skips cut regions while
scrubbing passes through them so edges can be judged, and dragging an edge
seeks the preview to that boundary frame.

The filmstrip is generated client-side by seeking a hidden video and drawing to
canvas — no new endpoints, identical for JPEG and clip sessions.

Notable fixes on the way, most of which were "the editor and the server
disagreed":

• Cut intervals assumed a 60 s stride, so the server over-counted
  (6b7110f). Real captures jitter — the server credits anything within
  ±30 s — so ending an interval at lastCutUnit + 60 s swallowed a neighbour
  landing at +57 s. Selecting 2 of 3 minutes was rejected as "would remove the
  entire timelapse" while the footer read "1m kept". Intervals now anchor the
  exclusive end to the next kept capture's actual timestamp (still capped at
  one interval so a cut before a long pause can't swallow it), and the
  footer runs the serialized intervals through the same shared countCutUnits
  the server uses — two implementations of one question will always drift, so
  now there's one. Nine tests, including the reported case at 57 s spacing and
  a sweep of gaps from 40 s to 75 s.
• The editor opens before the preview exists (d7dfdc7). The compile
  starts at stop and runs for tens of seconds, so preparing is the normal
  opening state, not an error; it also had to stop reporting a compiling
  session as un-editable, since the worker flips a held session to compiling
  within a second of the stop and "Edit & save" failed immediately for every
  user.
• Waiting for the preview 429'd on its own poll (52fc8b6). /units
  presigns a URL and is limited to 10/min; the editor polled it every 1.5 s, so
  any compile longer than ~15 s hit the limit and reported a rate-limit error.
  The wait broke exactly when there was something worth waiting for. It now
  polls /status (60/min, already carries editable) and touches /units
  only at the edges.
• The progress ring restarted at 0 on every poll (b4cd87b). It's now
  anchored to a single start ref, eases asymptotically
  (1 − exp(−2.2·elapsed/estimate)), and is clamped to 99 % so it can never
  sit at 100 % while the user waits.
• Thumbnails failed silently when the preview lacked CORS headers
  (8d2114a) — now a two-strategy probe: try crossOrigin="anonymous" and
  keep the canvas clean, else fetch to a same-origin blob: URL (on desktop
  that fetch goes through Tauri's HTTP plugin and isn't subject to browser
  CORS). This replaced a server-side CORS change, which is why bb012d1
  reverts it.
• The filmstrip tiles whole frames sized by the track at device resolution
  (e966b16) — one tile per minute meant 19 px smears on a 48-minute session.
• Cuts could never stay selected (3211d4f): the drag's pointer-up no
  longer clears selection, and re-finds the region after normalizeRegions
  may have merged and reindexed it, so "Remove cut" is reachable.
• The ruler labelled time of day, which read as a duration (f396a19) —
  "1:29 … 1:45" on a 17-minute timelapse. It's elapsed recording time now.
  Same commit dropped the hover-preview card and its second video decoder
  (9254ccd): the filmstrip already shows the footage, and a card chasing the
  cursor was noise on top of it.

<StopChoiceModal> is where editing is offered — keep recording / stop & save
/ edit & save — and it's the only entry point. <SessionDetail> shows a review
panel for any session with a live editHoldUntil, holds the lease while it's
visible (reading the panel for two minutes must not publish underneath the
reader), and suppresses the compile spinner there, since "processing" under
"ready to review" would contradict itself.

---

3. Embedders get all of this for free

Both dialogs portal into document.body, so the editor opens as a modal
overlay sized to the viewport rather than to whatever width the host gave the
recorder (b8db804, 03591a3) — position: fixed resolves against a
transformed ancestor, so an embedder with a transform up the tree used to get
a modal pinned inside their card. Any program that redirects to the hosted
recorder or embeds <LookoutRecorder> gets the stop dialog and the editor with
no code change and no version to adopt.

• Opt out with ?edit=false on the recorder URL or
  <LookoutRecorder editing={false} /> — stopping stays one click.
• <LookoutProvider accentColor="#16a34a" accentTextColor="#fff"> replaces
  Lookout's blue on primary buttons, focus rings and progress (6bba123).
  Applied to document.documentElement, not a wrapper, precisely because the
  overlays portal out.
• Only a program driving the headless useLookout() hook with its own stop
  button has to opt in, by passing stop({ edit: true }) and rendering
  <TimelapseEditor>.

New exports: TimelapseEditor, StopChoiceModal, useEditLease,
ClipRecorder, regionsToCuts / cutsToRegions / normalizeRegions /
gapIndices, useSessionTimerState, deriveDisplaySeconds,
computeBestTrackedSeconds, setAccentColor, Overlay, ProgressRing,
MinutesFlow, UploadPayload, CutInterval.

clients/playground (new workspace, port 5199) is a harness for exercising
the SDK against a real server. Its own header states the reason: most of the
bugs in this feature were disagreements between the client and the server. So
it has a "server truth" panel printing editable/editableReason next to the
editor, and a "verify against server" button that dry-runs the editor's cut
list through PUT /cuts and prints unitsCut — which must equal what the
footer says was removed. It also renders the editor in a resizable box with
presets for the shapes that previously broke layout, including the desktop
window's actual minimum.

---

4. Desktop app

The editor window

The main window is a fixed 480×640, far too small to scrub a multi-hour
timeline, so editing opens a dedicated 940×660 resizable WebviewWindow
(min 620×480, EditorWindow.tsx, route #/editor?token=…, capability window
editor-*). macOS draws the window title (fc55288) so it can't drift out of
alignment with the traffic lights the way the hand-placed label did, and the
session name rides along as Edit — <name>.

While it's open the main window steps aside with "Edit your timelapse in
the edit window" (click to bring it forward) rather than two live views of one
session competing for attention. It learns the editor opened from an event and
then polls isEditorWindowOpen every second — the poll is what guarantees it
can never get stuck behind the placeholder if the editor is force-quit.

Closing the window is itself the decision (3211d4f, d61d29f):
onCloseRequested confirms ("Closing publishes this timelapse with N cuts
applied…"), then publishes and emits the edited event fire-and-forget, guarded
by a ref so saving from inside the editor can't double-publish. Closing focuses
main first — closing the frontmost window otherwise drops the user behind
another app — and on failure falls back to destroy() instead of the old
swallowed .catch(() => {}), which left people staring at a window that said
it saved and wouldn't go away.

Native chrome

• Add-menu (AddMenu.swift, native_menu.rs): a Raycast-style
  non-activating NSPanel with SwiftUI content over NSVisualEffectView, full
  keyboard handling, and program icons served from a main-thread cache warmed by
  prefetch_add_menu_icons so they're on the first frame rather than popping
  in. AddMenuPopup.tsx is the DOM replica for Windows/Linux, documented as
  mirroring the Swift visual constants. Gallery's onAdd now passes an
  AddAnchor rect so either popup can anchor to the + button.
• Menu-bar item (Tray.swift, native_tray.rs): macOS gets a real
  NSStatusItem hosting a SwiftUI view, so the time digits roll with
  contentTransition(.numericText()) instead of snapping. Windows doesn't
  render tray titles at all, so it gets a tooltip (Lookout — 12m recorded)
  with a ⏸ prefix while paused.
• Program icons + redirect URLs. programs.icon_url (0017) and
  sessions.redirect_url (0016), both manageable from /admin. The app
  opens the redirect URL in the default browser the moment a session it's
  watching flips to complete — de-duped, and never when re-opening an
  already-finished session. javascript:, file:// and bare strings are
  rejected at creation, with tests.

Updates and launch

useAppUpdate.ts + UpdatePill.tsx replace UpdateBanner, useUpdateCheck
and useBackgroundUpdate — and the blocking full-screen launch gate that
used to return early before any UI rendered ("Checking for updates…"). Checks
now run in the background every 30 min and surface as a quiet titlebar pill
with a progress ring; installing is strictly behind a click. A relaunch
timestamp with a 60 s cooldown means a bad manifest can't relaunch-loop. On
Windows/Linux there's no overlay titlebar, so the pill floats bottom-left —
top-right overlapped the gallery header (1cd06ad).

Instant launch: permission grants are cached (lookout-perm-screen /
-camera, written only on a real grant, never on Skip) and seeded into state
synchronously, then re-verified in the background and re-gated if revoked.

The clock, in three places, derived once

The menu-bar title and the tray popup both tick their own clock while the main
WebView is throttled, so they go through the exported deriveDisplaySeconds
(mirrored as tray_display_seconds in Rust, with 6 tests pinning the mirror
against the JS constant). Two bugs this closes: the pushed payload now carries
the interpolation anchor rather than displaySeconds, so the popup stops
extrapolating on top of an extrapolation; and the tray's tracked seconds
ratchet with fetch_max, re-anchoring only on a genuine advance — an
idempotent confirm retry can return a lower value, and storing it made the menu
bar jump backwards. API.md documents both rules for anyone else building a
ticking surface.

Performance, beyond the capture loop

• One shared reqwest::Client in a OnceLock. Every 60 s tick used to build a
  fresh one — new pool, new TLS config, new handshake.
• No base64 round-tripping: capture returns raw JPEG bytes, base64 runs exactly
  once for the JS preview, and the upload body is Bytes.
• The tray ticker caches its last title and only touches the native tray when
  the string actually changes (minute granularity against a 1 s ticker); the
  per-second tray-timer-tick emit that nothing listened to is gone.
• macOS App Nap / idle-sleep assertion is now scoped to the tray-timer
  lifetime instead of leaked for the process lifetime, so an idle Lookout
  sitting on the gallery no longer prevents the Mac from sleeping.
• New capture-preview-frame event, emitted only while the main window is
  focused, at ≤854×480 / q65 through a borrowing resize so the full-res frame
  stays intact for the clip encoder — IPC gets ~5× lighter and an unfocused
  tick in non-clips mode skips the grab entirely. The tray sync went from 3 IPC
  calls plus a broadcast every second to an effect keyed on the values that
  actually changed (the old rate also drowned the 200-entry debug buffer).
• Settings no longer polls running apps every 5 s. list_running_apps became
  list_installed_apps + get_app_icon, both async and both memoized, with
  icons loaded lazily through one IntersectionObserver. Rows order pinned →
  open → other from a snapshot taken when the page opened, so toggling a filter
  never reshuffles rows under the cursor.
• Gallery paints instantly from a localStorage summary cache while a
  background refresh runs; the old 45-minute presigned-thumbnail cache is gone
  because the thumbnail endpoint now streams bytes with ETag +
  Cache-Control: max-age=86400 and 304 handling instead of redirecting to a
  URL that changed every request and defeated the browser cache entirely.

Configurable server

serverConfig.ts normalizes and validates a user-supplied origin (HTTPS only,
plus localhost for dev), and the new Advanced settings subpage puts a
confirm speed bump and a live probe of /api/programs in front of it, so a
typo fails there rather than silently during the next recording. An active
override is surfaced on the settings menu row and behind a persistent warning
card. Settings itself was restructured into a menu with subpages to hold it.

[socket-security]

Review the following changes in direct dependencies. Learn more about Socket for GitHub.

Diff	Package	Supply Chain Security	Vulnerability	Quality	Maintenance	License
	npm/​@​number-flow/​react@​0.6.0 ⏵ 0.6.2				+8
	npm/​vitest@​4.1.6 ⏵ 4.1.10	+1			+2
	npm/​@​phosphor-icons/​react@​2.1.10
	cargo/​fast_image_resize@​5.5.0
	npm/​@​lookout/​react@​0.2.11 ⏵ 0.3.3	N/A	N/A	N/A	N/A	N/A
	npm/​@​lookout/​shared@​0.2.11 ⏵ 0.3.3	N/A	N/A	N/A	N/A	N/A

View full report