Headless media capture: screenshots/video of the app + skill

.claude/skills/capturing-voltius-media/SKILL.md

@@ -0,0 +1,112 @@
+---
+name: capturing-voltius-media
+description: Use when producing screenshots or videos of the Voltius app UI for docs, blog posts, changelog, or demos — capturing or recording the live headless dev build and editing the result (crop, zoom, loop) with ffmpeg.
+---
+
+# Capturing Voltius Media (screenshots & video)
+
+Produce real screenshot and video assets of the Voltius desktop app by driving the
+**headless dev build** and screen-grabbing its Xvfb framebuffer. Same stack as
+[[iterating-on-voltius-ui]] (`compose.headless.yml`: `tauri-headless` = app + Xvfb +
+tauri-driver on 4444; `ssh-host-1` = throwaway SSH host `voltius`/`voltius`).
+
+**Path note:** the screenshot pipeline and brand framing live in the **docs sibling repo**,
+`../docs/tools/screenshots/` — one level *up* from the voltius repo (there is no `docs/`
+inside voltius). All references below are to that sibling path.
+
+## ⚠️ The one thing that will silently ruin captures
+
+**WebKitGTK renders opacity/transform animations (CSS transitions, `animate-fadeIn`,
+dropdown/modal open) on GPU compositor layers. Under Xvfb's software path those layers
+are NOT flushed to the X framebuffer that a screen grab reads** — so recordings show
+**instant jumps with no transitions**, and a screenshot taken mid-animation shows the
+pre-animation state. There is no error; it just looks wrong.
+
+**Fix: run the app with `WEBKIT_DISABLE_COMPOSITING_MODE=1`.** It's baked into
+`compose.headless.yml` (`tauri-headless` `environment:`). Verify before trusting a capture:
+
+```bash
+docker exec tauri-headless sh -c 'pid=$(pgrep -f target/debug/voltius|head -1); \
+  tr "\0" "\n" </proc/$pid/environ | grep WEBKIT_DISABLE_COMPOSITING_MODE' || echo "MISSING — recompose"
+```
+
+**Always confirm a transition actually recorded** (this check is manual — the scripts
+don't run it for you). Sample per-frame luminance with `signalstats` YAVG and require a
+*ramp* over several frames, not a single step:
+- **Crop over the element that animates** (the modal / dropdown / palette) — NOT the
+  backdrop. The dimming backdrop often fades in ~1 frame and reads as a step even when the
+  element ramps, so a backdrop crop gives a false negative.
+- Find the moment `T` from the timeline your scene logs: `T ≈ event_epoch −
+  record_start_epoch (+ ~1.0s warm-up − any head trim)`. Or just scan all frames.
+- A constant value across the whole clip ⇒ compositing was NOT disabled. Command in
+  `record-video.sh` / `edit-video.sh`.
+
+## Screenshots (stills)
+
+Use the existing pipeline — do not reinvent it: `../docs/tools/screenshots/`
+(`run.sh`, `shots.json` manifest, `capture.mjs`, `frame.py`). It seeds state, dismisses the
+dev banner, drives tauri-driver, frames the raw PNG (brand gradient), and injects into docs.
+From the docs repo: `./tools/screenshots/run.sh [shot-id ...]`. Add a new shot = add an
+entry to `shots.json`.
+
+## Video
+
+Three moving parts, all inside the container: **record** (ffmpeg x11grab) + **drive**
+(WebDriver) + **edit** (ffmpeg). Templates: `record-video.sh`, `edit-video.sh` (copy into
+`/tmp`, adapt the scene).
+
+1. **Driving = same WebDriver as the screenshot pipeline.** Reuse the verb vocabulary in
+   `../docs/tools/screenshots/capture.mjs` (`clickAt`, `clickText`, `evalJs`, `setVal`,
+   `keyCombo` for chords like Ctrl+K, `seedHost`, …). tauri-driver allows **one** session:
+   - Reuse the live one via the id in `/tmp/wd_sid`. **If that file is absent**, a prior
+     run may have left it in `/tmp/wd_sid*` (e.g. `wd_sid2`) — check those; else create a
+     fresh session (capture.mjs's `ensureSession` does this and writes `/tmp/wd_sid`).
+     There is no list-sessions endpoint — `/status` only says whether one exists, not its id.
+   - If the `tauri-docker` MCP holds it (`{"ready":false,"message":"A session already
+     exists"}`), stop the MCP (`pkill -f mcp-tauri-automation`) — the session and app
+     survive; then reuse its id. You do NOT need the MCP in your Claude session; drive
+     `tauri-driver` directly over HTTP.
+   - `record-video.sh` runs the scene as `node /tmp/<scene>` — put the scene .mjs directly
+     in `/tmp` and pass a **bare filename**, not a path.
+2. **Recording.** ffmpeg `x11grab` on the app's display. Derive `DISPLAY`/`XAUTHORITY`
+   from the running app process (the Xvfb auth path regenerates per boot). Grab the exact
+   window region (set the window to a known size first, e.g. `1200x800@0,0`). See
+   `record-video.sh` — it also logs an `event → epoch` **timeline** so edits can key zooms
+   to real moments.
+3. **Editing = ffmpeg only.** `edit-video.sh` has the recipes: crop, **smooth keyed zoom
+   via `zoompan`** (NOT `crop` — `crop` evaluates `w`/`h` once at config time, so it can't
+   animate; `zoompan` can), supersample-then-downscale to kill zoom shimmer, clean loop,
+   and web encode (`-pix_fmt yuv420p -movflags +faststart`, silent).
+
+Blog `<video>` tags want silent looping H.264 at the app's native window size.
+
+## Framing (brand gradient — match the docs media)
+
+Docs/blog assets sit on the brand look: navy→lift vertical gradient + a faint top-left
+accent glow, the app as a rounded floating window with a drop shadow, 1600px wide. For
+stills this is `../docs/tools/screenshots/frame.py`; **reuse it — don't reinvent the
+colors** (it's the single source of truth; brand tweaks there should flow to video too).
+
+For video, `frame-video.py` calls `frame.py` to render the gradient background + shadow +
+rounded-window mask at the clip's size, then `frame-video.sh` ffmpeg-overlays the
+corner-rounded video onto it. Needs `python3-pil` + `ffmpeg` (both in the Dockerfile) and
+`frame.py` copied alongside `frame_video.py` in `/tmp`. Produces the identical framed look
+to the screenshots, in motion.
+
+## Cursor
+
+`draw_mouse 1` captures the X cursor, but **synthetic WebDriver clicks don't move the OS
+cursor** — it sits wherever it last was. For static demos use `draw_mouse 0` (cleaner).
+For drag/drop/paste demos where pointer motion IS the story, animate a synthetic cursor in
+post along the click path (you already log click coordinates + timestamps in the timeline).
+
+## Common mistakes
+
+| Mistake | Fix |
+|---|---|
+| Animations missing / instant in the video | App not launched with `WEBKIT_DISABLE_COMPOSITING_MODE=1`. Verify env on the *app process*, not just the container. |
+| `crop` "Error when evaluating the expression" with `t` | `crop` can't animate w/h. Use `zoompan` for time-varying zoom. |
+| `ffmpeg: command not found` after a rebuild | ffmpeg is in `Dockerfile.tauri-headless`; a container *recreate* without *rebuild* can lag it — `apt-get install -y ffmpeg` in-container as a stopgap. |
+| `"A session already exists"` when creating a session | The MCP (or a prior run) holds the single session. Stop it / reuse `/tmp/wd_sid`. |
+| Black bars / wrong region | Set the window rect to a known size at 0,0 first; grab that exact region. |
+| Blank frames | App failed to launch (splash) — see [[iterating-on-voltius-ui]] (needs `seccomp=unconfined`). |

.claude/skills/capturing-voltius-media/edit-video.sh

@@ -0,0 +1,32 @@
+#!/usr/bin/env bash
+# ffmpeg editing recipes for headless captures. All run in-container (ffmpeg is there).
+# These are reference recipes — copy the one you need; don't run this file wholesale.
+set -e
+IN="${1:-raw.mp4}"
+
+# ── Crop to a sub-region (e.g. drop a panel out of frame) ─────────────────────────────
+# ffmpeg -y -i "$IN" -vf "crop=W:H:X:Y,format=yuv420p" -c:v libx264 -crf 21 -movflags +faststart -an crop.mp4
+
+# ── Clean web encode (silent, loop-friendly) ─────────────────────────────────────────
+# ffmpeg -y -i "$IN" -vf format=yuv420p -c:v libx264 -crf 21 -preset slow -movflags +faststart -an clean.mp4
+
+# ── Smooth KEYED ZOOM (punch-in on a UI element, hold, ease out) ─────────────────────
+# Use zoompan, NOT crop: crop evaluates w/h once at config time and CANNOT animate with t.
+# Supersample 2x first so the moving crop doesn't shimmer. t = on/30 (output frame / fps).
+# Center on the ORIGINAL point (CX,CY); at 2x that's (2*CX, 2*CY). ZMAX = peak zoom.
+# Ease in over [T_IN, T_IN+D], hold, ease out over [T_OUT, T_OUT+D] (smoothstep).
+CX=780; CY=450; ZMAX=1.6; TIN=5.3; TOUT=12.9; D=0.9
+UIN="clip((on/30-$TIN)/$D\,0\,1)"; UOUT="clip((on/30-$TOUT)/$D\,0\,1)"
+Z="1+($ZMAX-1)*($UIN*$UIN*(3-2*$UIN))-($ZMAX-1)*($UOUT*$UOUT*(3-2*$UOUT))"
+# ffmpeg -y -i "$IN" -filter_complex \
+# "scale=2400:1600:flags=bicubic,\
+# zoompan=z='$Z':x='clip($((2*CX))-(iw/zoom)/2\,0\,iw-iw/zoom)':y='clip($((2*CY))-(ih/zoom)/2\,0\,ih-ih/zoom)':d=1:fps=30:s=1200x800,\
+# format=yuv420p" -c:v libx264 -crf 20 -preset medium -movflags +faststart -an zoom.mp4
+
+# ── Trim ─────────────────────────────────────────────────────────────────────────────
+# ffmpeg -y -ss START -to END -i "$IN" -c copy trimmed.mp4   # or re-encode if cuts must be frame-exact
+
+# ── Verify an animation is present (luminance must ramp, not step once) ───────────────
+# ffmpeg -ss T -t 1.4 -i "$IN" -vf "crop=W:H:X:Y,signalstats,metadata=print:key=lavfi.signalstats.YAVG:file=-" -f null - \
+#   2>/dev/null | grep -o "YAVG=[0-9.]*" | uniq
+echo "reference recipes — see comments; adapt CX/CY/ZMAX/TIN/TOUT for zoom"

.claude/skills/capturing-voltius-media/frame-video.sh

@@ -0,0 +1,24 @@
+#!/usr/bin/env bash
+# Frame a captured clip with the Voltius brand gradient — matches the docs screenshots
+# (navy->lift gradient + top-left glow, rounded floating window + drop shadow, 1600px wide).
+# Reuses the docs frame.py as the single source of truth for the brand look.
+#
+# Runs IN-CONTAINER (needs ffmpeg + python3-pil, both in Dockerfile.tauri-headless).
+# Prereq — copy the brand-framing files into /tmp first:
+#   docker cp ../docs/tools/screenshots/frame.py tauri-headless:/tmp/frame.py
+#   docker cp .claude/skills/capturing-voltius-media/frame_video.py tauri-headless:/tmp/
+#   docker cp .claude/skills/capturing-voltius-media/frame-video.sh  tauri-headless:/tmp/
+#   docker exec tauri-headless bash /tmp/frame-video.sh /app/screenshots/video/in.mp4 out.mp4
+set -e
+IN="${1:?input mp4}"; OUT="${2:-framed.mp4}"
+read W H < <(ffprobe -v error -select_streams v:0 -show_entries stream=width,height -of csv=p=0:s=' ' "$IN")
+META=$(cd /tmp && python3 frame_video.py "$W" "$H" /tmp)          # writes /tmp/frame_bg.png + /tmp/frame_mask.png
+PAD=$(echo "$META" | python3 -c 'import sys,json;print(json.load(sys.stdin)["pad"])')
+TW=$(echo "$META"  | python3 -c 'import sys,json;print(json.load(sys.stdin)["target_w"])')
+# Round the video's corners (alphamerge with the mask), overlay onto the brand background,
+# scale to target width. shortest=1 so the looped bg/mask images end with the video.
+ffmpeg -y -loglevel error -i "$IN" -loop 1 -i /tmp/frame_bg.png -loop 1 -i /tmp/frame_mask.png \
+ -filter_complex "[2:v]format=gray[m];[0:v]format=rgba[v];[v][m]alphamerge[rv];\
+[1:v][rv]overlay=${PAD}:${PAD}:shortest=1[c];[c]scale=${TW}:-2,format=yuv420p" \
+ -c:v libx264 -crf 20 -preset medium -movflags +faststart -an "$OUT"
+echo "framed -> $OUT"

.claude/skills/capturing-voltius-media/frame_video.py

@@ -0,0 +1,29 @@
+# Generate the brand-gradient background + shadow + rounded-window mask for framing a
+# video, reusing the real docs frame.py (single source of truth for the brand look).
+# Usage: python3 frame_video.py <video_w> <video_h> <out_dir>   (frame.py must be importable)
+import sys, json
+from PIL import Image, ImageDraw, ImageFilter
+import frame  # sibling copy of docs/tools/screenshots/frame.py
+
+W, H, OUT = int(sys.argv[1]), int(sys.argv[2]), sys.argv[3]
+pad = int(W * frame.PAD_RATIO)
+radius = max(10, int(W * 0.013))
+cw, ch = W + pad * 2, H + pad * 2
+
+canvas = frame._gradient(cw, ch)  # brand navy->lift gradient + top-left accent glow
+
+# rounded-window mask (drives both the shadow silhouette and the video's alpha)
+mask = Image.new("L", (W, H), 0)
+ImageDraw.Draw(mask).rounded_rectangle([0, 0, W - 1, H - 1], radius=radius, fill=255)
+
+# drop shadow — identical params to frame.frame_image()
+shadow = Image.new("RGBA", (cw, ch), (0, 0, 0, 0))
+shaped = Image.new("RGBA", (W, H), (0, 0, 0, 0))
+shaped.paste(Image.new("RGBA", (W, H), (0, 0, 0, 150)), (0, 0), mask)
+shadow.paste(shaped, (pad, pad + int(pad * 0.18)), shaped)
+shadow = shadow.filter(ImageFilter.GaussianBlur(int(pad * 0.4)))
+canvas = Image.alpha_composite(canvas, shadow)
+
+canvas.convert("RGB").save(f"{OUT}/frame_bg.png")
+mask.save(f"{OUT}/frame_mask.png")
+print(json.dumps({"pad": pad, "radius": radius, "cw": cw, "ch": ch, "target_w": 1600}))

.claude/skills/capturing-voltius-media/record-video.sh

@@ -0,0 +1,42 @@
+#!/usr/bin/env bash
+# Record a video of the headless Voltius app while a WebDriver "scene" script drives it.
+# Runs INSIDE the tauri-headless container. Copy to /tmp, point SCENE at your scene .mjs.
+#
+#   docker cp record-video.sh tauri-headless:/tmp/  &&  docker cp scene.mjs tauri-headless:/tmp/
+#   docker exec tauri-headless bash /tmp/record-video.sh scene.mjs out.mp4
+#
+# The scene .mjs must: reuse the session in $WDSID against localhost:$WDPORT, set the
+# window to $W x $H at 0,0, and drive the UI. Emit an event->epoch timeline for keyed edits.
+# See docs/tools/screenshots/capture.mjs for the WebDriver verb vocabulary to reuse.
+set -e
+SCENE="${1:?scene .mjs required}"
+OUTNAME="${2:-out.mp4}"
+: "${WDPORT:=4444}" "${WDSID:=/tmp/wd_sid}"
+W=1200; H=800
+OUT=/app/screenshots/video; mkdir -p "$OUT"; cd "$OUT"
+
+# Derive DISPLAY/XAUTHORITY from the running app (Xvfb auth path regenerates each boot).
+APPPID=$(pgrep -f target/debug/voltius | head -1)
+export $(tr '\0' '\n' </proc/"$APPPID"/environ | grep -E '^(DISPLAY|XAUTHORITY)=')
+
+# CRITICAL: verify compositing is disabled, else animations won't be captured.
+tr '\0' '\n' </proc/"$APPPID"/environ | grep -q WEBKIT_DISABLE_COMPOSITING_MODE \
+  || { echo "FATAL: app lacks WEBKIT_DISABLE_COMPOSITING_MODE=1 — animations will not record"; exit 1; }
+
+# Record the exact window region. draw_mouse 0 (synthetic clicks don't move the cursor).
+ffmpeg -y -loglevel error -f x11grab -draw_mouse 0 -video_size ${W}x${H} -framerate 30 \
+  -i "${DISPLAY}+0,0" -c:v libx264 -preset ultrafast -qp 0 -pix_fmt yuv444p raw.mp4 &
+FF=$!
+sleep 1.0                          # let ffmpeg establish before the scene acts
+WDPORT=$WDPORT WDSID=$WDSID node "/tmp/$SCENE"
+sleep 0.5
+kill -INT "$FF" 2>/dev/null || true; wait "$FF" 2>/dev/null || true
+
+# Encode a clean web-friendly clip (trim 0.4s warm-up head). Silent, faststart.
+ffmpeg -y -loglevel error -ss 0.4 -i raw.mp4 -vf format=yuv420p \
+  -c:v libx264 -crf 21 -preset medium -movflags +faststart -an "$OUTNAME"
+echo "wrote $OUT/$OUTNAME"
+
+# Verify an animation actually recorded: luminance of a region must RAMP over several
+# frames, not step once. Constant value across the whole clip => compositing not disabled.
+#   ffmpeg -ss <t> -i "$OUTNAME" -vf "crop=W:H:X:Y,signalstats,metadata=print:key=lavfi.signalstats.YAVG:file=-" -f null -

Dockerfile.tauri-headless

@@ -3,7 +3,8 @@ FROM ubuntu:24.04
 ENV DEBIAN_FRONTEND=noninteractive
 
 # System dependencies: Xvfb + WebKit + compilers + mold (fast linker) + keyutils
-# (the app's OS keychain talks to the kernel keyutils keyring at startup).
+# (the app's OS keychain talks to the kernel keyutils keyring at startup) + ffmpeg
+# (records the Xvfb framebuffer for headless UI screen-capture / video).
 RUN apt-get update && apt-get install -y \
     git \
     xvfb \
@@ -16,6 +17,8 @@ RUN apt-get update && apt-get install -y \
     libgtk-3-dev \
     mold \
     keyutils \
+    ffmpeg \
+    python3-pil \
     && rm -rf /var/lib/apt/lists/*
 
 # Install Node.js and enable pnpm

compose.headless.yml

@@ -29,6 +29,13 @@ services:
     container_name: tauri-headless
     security_opt:
       - seccomp=unconfined
+    environment:
+      # WebKitGTK renders opacity/transform animations on GPU compositor layers that
+      # Xvfb's software path never flushes to the X framebuffer — so screen-grabbed
+      # captures (ffmpeg x11grab / driver screenshots) miss every CSS transition and
+      # animation. Forcing compositing off paints everything into the readable window
+      # buffer, so captured video/screenshots show animations faithfully.
+      - WEBKIT_DISABLE_COMPOSITING_MODE=1
     volumes:
       - .:/app
     networks: