diff --git a/CLAUDE.md b/CLAUDE.md index d4778bef55..7c895db16d 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -38,7 +38,7 @@ Atomic capabilities the creation workflows compose against — pull one when you - `/hyperframes-animation` — all animation knowledge: atomic motion rules, scene blueprints, transitions, runtime adapters (GSAP default, plus Lottie / Three.js / Anime.js / CSS / WAAPI / TypeGPU). - `/hyperframes-keyframes` — seek-safe keyframe authoring across runtimes: GSAP timelines, CSS keyframes, Anime.js, WAAPI, FLIP, paths, masks, SVG morph/draw, text trails, 3D depth; plus `hyperframes keyframes` diagnostics for surfacing and verifying rendered motion. - `/hyperframes-creative` — non-animation creative direction: `frame.md` / `design.md` handling, palettes, typography, narration, beat planning, audio-reactive visuals, composition patterns. -- `/media-use` — the media OS: resolve any media need (BGM, SFX, image, icon, logo, voice) into a frozen local file + ledger record; generate via TTS / music / image models when the catalog misses; transcribe, caption, remove backgrounds, and reuse assets across projects. One shared `scripts/audio.mjs` engine + manifest tracking; keeps search noise on disk. +- `/media-use` — the media OS: resolve any media need (BGM, SFX, image, icon, logo, voice, color grade, LUT) into a frozen local file or paste-ready block + ledger record; generate via TTS / music / image models when the catalog misses; transcribe, caption, remove backgrounds, and reuse assets across projects. One shared `scripts/audio.mjs` engine + manifest tracking; keeps search noise on disk. - `/hyperframes-cli` — CLI dev loop: `init`, `add`, `lint`, `validate`, `inspect`, `preview`, `render`, `publish`, `doctor`, `lambda` (AWS Lambda cloud rendering). - `/hyperframes-registry` — install and wire registry blocks and components into compositions via `hyperframes add`. Covers authoring a new block or component to contribute upstream. - `/figma` — import Figma assets, tokens, components, and storyboard sections → reconstructed motion (frames read as states, not slides) (REST/CLI) plus Motion animations (MCP) and shaders (MCP source / native export) into a composition. diff --git a/README.md b/README.md index 4a224041f3..e6b2a4a7bb 100644 --- a/README.md +++ b/README.md @@ -83,16 +83,16 @@ Installs stay lean after that: `npx hyperframes init` keeps the **core set** fre Atomic capabilities the creation workflows compose against — pull one when you need that specific layer. -| Skill | Covers | -| ------------------------ | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -| `/hyperframes-core` | The composition contract — `data-*` timing attributes, `class="clip"`, tracks, sub-compositions, variables, framework-owned media playback, determinism rules. | -| `/hyperframes-animation` | All animation knowledge — atomic motion rules, scene blueprints, transitions, runtime adapters (GSAP / Lottie / Three.js / Anime.js / CSS / WAAPI / TypeGPU). | -| `/hyperframes-keyframes` | Seek-safe keyframe authoring across runtimes — GSAP timelines, CSS keyframes, Anime.js, WAAPI, FLIP, paths, masks, SVG morph/draw, 3D depth — plus `hyperframes keyframes` diagnostics for rendered motion. | -| `/hyperframes-creative` | Non-animation creative direction — `frame.md` / `design.md`, palettes, typography, narration, beat planning, audio-reactive visuals, composition patterns. | -| `/media-use` | The media OS — resolve any media need (BGM, SFX, image, icon, logo, voice) into a frozen local file + ledger record, generate via TTS/music/image models when the catalog misses, transcribe, caption, remove backgrounds, and reuse assets across projects. One shared audio engine + manifest tracking. | -| `/hyperframes-cli` | CLI dev loop — `init`, `lint`, `validate`, `inspect`, `preview`, `render`, `publish`, `doctor`, plus AWS Lambda cloud rendering (`lambda deploy / render / progress`). | -| `/hyperframes-registry` | Install and wire registry blocks and components into compositions via `hyperframes add`. Authoring a new block or component to contribute upstream. | -| `/figma` | Import Figma assets, tokens, components, and storyboard sections → reconstructed motion (frames read as states, not slides) (REST/CLI) plus Motion animations (MCP) and shaders (MCP source / native export) into a composition. | +| Skill | Covers | +| ------------------------ | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | +| `/hyperframes-core` | The composition contract — `data-*` timing attributes, `class="clip"`, tracks, sub-compositions, variables, framework-owned media playback, determinism rules. | +| `/hyperframes-animation` | All animation knowledge — atomic motion rules, scene blueprints, transitions, runtime adapters (GSAP / Lottie / Three.js / Anime.js / CSS / WAAPI / TypeGPU). | +| `/hyperframes-keyframes` | Seek-safe keyframe authoring across runtimes — GSAP timelines, CSS keyframes, Anime.js, WAAPI, FLIP, paths, masks, SVG morph/draw, 3D depth — plus `hyperframes keyframes` diagnostics for rendered motion. | +| `/hyperframes-creative` | Non-animation creative direction — `frame.md` / `design.md`, palettes, typography, narration, beat planning, audio-reactive visuals, composition patterns. | +| `/media-use` | The media OS — resolve any media need (BGM, SFX, image, icon, logo, voice, color grade, LUT) into a frozen local file or paste-ready block + ledger record, generate via TTS/music/image models when the catalog misses, transcribe, caption, remove backgrounds, and reuse assets across projects. One shared audio engine + manifest tracking. | +| `/hyperframes-cli` | CLI dev loop — `init`, `lint`, `validate`, `inspect`, `preview`, `render`, `publish`, `doctor`, plus AWS Lambda cloud rendering (`lambda deploy / render / progress`). | +| `/hyperframes-registry` | Install and wire registry blocks and components into compositions via `hyperframes add`. Authoring a new block or component to contribute upstream. | +| `/figma` | Import Figma assets, tokens, components, and storyboard sections → reconstructed motion (frames read as states, not slides) (REST/CLI) plus Motion animations (MCP) and shaders (MCP source / native export) into a composition. | For visual design handoff workflows, see the [Claude Design guide](https://hyperframes.heygen.com/guides/claude-design) and [Open Design guide](https://hyperframes.heygen.com/guides/open-design). diff --git a/docs/guides/skills.mdx b/docs/guides/skills.mdx index 394a572f9a..9579f7df62 100644 --- a/docs/guides/skills.mdx +++ b/docs/guides/skills.mdx @@ -98,7 +98,7 @@ Atomic capabilities the creation workflows compose against — pull one when you | `/hyperframes-animation` | All animation knowledge — atomic motion rules, scene blueprints, transitions, runtime adapters (GSAP / Lottie / Three.js / Anime.js / CSS / WAAPI / TypeGPU). | | `/hyperframes-keyframes` | Seek-safe keyframe authoring across runtimes — GSAP timelines, CSS keyframes, Anime.js, WAAPI, FLIP, paths, masks, SVG morph/draw, 3D depth — plus `hyperframes keyframes` diagnostics for rendered motion. | | `/hyperframes-creative` | Non-animation creative direction — `frame.md` / `design.md`, palettes, typography, narration, beat planning, audio-reactive visuals, composition patterns. | -| `/media-use` | The media OS — resolve any media need (BGM, SFX, image, icon, logo, voice) into a frozen local file + ledger record, generate via TTS/music/image models when the catalog misses, transcribe, caption, remove backgrounds, and reuse assets across projects. One shared audio engine + manifest tracking. | +| `/media-use` | The media OS — resolve any media need (BGM, SFX, image, icon, logo, voice, color grade, LUT) into a frozen local file or paste-ready block + ledger record, generate via TTS/music/image models when the catalog misses, transcribe, caption, remove backgrounds, and reuse assets across projects. One shared audio engine + manifest tracking. | | `/hyperframes-cli` | CLI dev loop — `init`, `lint`, `validate`, `inspect`, `preview`, `render`, `publish`, `doctor`, plus AWS Lambda cloud rendering (`lambda deploy / render / progress / destroy / policies`). | | `/hyperframes-registry` | Install and wire registry blocks and components into compositions via `hyperframes add`. Authoring a new block or component to contribute upstream. | | `/figma` | Import Figma assets, tokens, components, and storyboard sections → reconstructed motion (frames read as states, not slides) (REST/CLI) plus Motion animations (MCP) and shaders (MCP source / native export) into a composition. | diff --git a/packages/cli/src/capture/captureCompositionFrame.test.ts b/packages/cli/src/capture/captureCompositionFrame.test.ts new file mode 100644 index 0000000000..cf942565c9 --- /dev/null +++ b/packages/cli/src/capture/captureCompositionFrame.test.ts @@ -0,0 +1,39 @@ +import { mkdtempSync, rmSync, writeFileSync } from "node:fs"; +import { tmpdir } from "node:os"; +import { join } from "node:path"; +import { describe, expect, it } from "vitest"; +import { runFfmpegOnce } from "./captureCompositionFrame.js"; + +function tempDir(): string { + return mkdtempSync(join(tmpdir(), "hf-capture-frame-test-")); +} + +describe("runFfmpegOnce", () => { + it("returns the process exit code and collected stderr", async () => { + const dir = tempDir(); + try { + const script = join(dir, "fail.cjs"); + writeFileSync(script, 'process.stderr.write("ffmpeg failed"); process.exit(3);\n'); + + const result = await runFfmpegOnce(process.execPath, [script], 1000); + + expect(result).toEqual({ code: 3, stderr: "ffmpeg failed", timedOut: false }); + } finally { + rmSync(dir, { recursive: true, force: true }); + } + }); + + it("terminates the process when the timeout elapses", async () => { + const dir = tempDir(); + try { + const script = join(dir, "hang.cjs"); + writeFileSync(script, "setTimeout(() => {}, 10000);\n"); + + const result = await runFfmpegOnce(process.execPath, [script], 50); + + expect(result.timedOut).toBe(true); + } finally { + rmSync(dir, { recursive: true, force: true }); + } + }); +}); diff --git a/packages/cli/src/capture/captureCompositionFrame.ts b/packages/cli/src/capture/captureCompositionFrame.ts new file mode 100644 index 0000000000..fbc15331a5 --- /dev/null +++ b/packages/cli/src/capture/captureCompositionFrame.ts @@ -0,0 +1,175 @@ +import { spawn } from "node:child_process"; +import type { Browser, Page } from "puppeteer-core"; +import { c } from "../ui/colors.js"; +import { resolveCompositionViewportFromHtml } from "../utils/compositionViewport.js"; + +const CHROME_LAUNCH_ARGS = [ + "--no-sandbox", + "--disable-gpu", + "--disable-dev-shm-usage", + "--enable-webgl", + "--use-gl=angle", + "--use-angle=swiftshader", +]; + +const SHADER_TRANSITIONS_TIMEOUT_MS = 90_000; +const CAPTURE_SETTLE_MS = 1500; + +export interface SettledCompositionPage { + browser: Browser; + page: Page; + // True when the runtime never signaled __renderReady within the timeout — the + // capture proceeds anyway (possibly mid-animation), so callers can surface it. + renderReadyTimedOut: boolean; +} + +export interface OpenSettledCompositionPageOptions { + renderReadyTimeoutMs: number; + renderReadyWarningSuffix: string; +} + +export interface FfmpegRunResult { + code: number | null; + stderr: string; + timedOut: boolean; +} + +function compositionRuntimeReadyInBrowser(): boolean { + return Boolean(Reflect.get(window, "__renderReady")); +} + +function shaderTransitionsReadyInBrowser(): boolean { + function shaderTransitionRegistryReady(): boolean | undefined { + const hf = Reflect.get(window, "__hf"); + if (typeof hf !== "object" || hf === null) return undefined; + + const shaderTransitions = Reflect.get(hf, "shaderTransitions"); + if (typeof shaderTransitions !== "object" || shaderTransitions === null) return undefined; + + for (const key of Object.keys(shaderTransitions)) { + const entry = Reflect.get(shaderTransitions, key); + if (typeof entry !== "object" || entry === null) return false; + if (Reflect.get(entry, "ready") !== true) return false; + } + return true; + } + + function shaderLoadingOverlayReady(): boolean { + const overlay = document.querySelector("[data-hyper-shader-loading]"); + if (!overlay) return true; + if (!(overlay instanceof HTMLElement)) return true; + return window.getComputedStyle(overlay).display === "none"; + } + + return shaderTransitionRegistryReady() ?? shaderLoadingOverlayReady(); +} + +async function waitForCompositionSettle( + page: Page, + options: OpenSettledCompositionPageOptions, +): Promise { + const runtimeReady = await page + .waitForFunction(compositionRuntimeReadyInBrowser, { timeout: options.renderReadyTimeoutMs }) + .then(() => true) + .catch(() => false); + + if (!runtimeReady) { + console.warn( + `\n ${c.warn("⚠")} Runtime did not become render-ready within ${options.renderReadyTimeoutMs}ms — ${options.renderReadyWarningSuffix}`, + ); + } + + await page + .waitForFunction(shaderTransitionsReadyInBrowser, { + timeout: SHADER_TRANSITIONS_TIMEOUT_MS, + }) + .catch(() => { + console.warn(` ${c.warn("⚠")} Shader transitions did not finish pre-rendering`); + }); + + await page.evaluate(() => document.fonts.ready).catch(() => {}); + await new Promise((resolveSettle) => setTimeout(resolveSettle, CAPTURE_SETTLE_MS)); + return runtimeReady; +} + +export async function openSettledCompositionPage( + html: string, + url: string, + options: OpenSettledCompositionPageOptions, +): Promise { + const { ensureBrowser } = await import("../browser/manager.js"); + const browser = await ensureBrowser(); + const puppeteer = await import("puppeteer-core"); + + let chromeBrowser: Browser | undefined; + try { + chromeBrowser = await puppeteer.default.launch({ + headless: true, + executablePath: browser.executablePath, + args: CHROME_LAUNCH_ARGS, + }); + + const page = await chromeBrowser.newPage(); + await page.setViewport(resolveCompositionViewportFromHtml(html)); + await page.goto(url, { waitUntil: "domcontentloaded", timeout: 10000 }); + const renderReadyTimedOut = !(await waitForCompositionSettle(page, options)); + return { browser: chromeBrowser, page, renderReadyTimedOut }; + } catch (err) { + await chromeBrowser?.close().catch(() => {}); + throw err; + } +} + +export async function seekCompositionTimeline( + page: Pick, + timeSeconds: number, +): Promise { + await page.evaluate((t: number) => { + const player = (window as any).__player; + if (!player) return; + const safe = Math.max(0, Number(t) || 0); + if (typeof player.renderSeek === "function") { + player.renderSeek(safe); + } else if (typeof player.seek === "function") { + player.seek(safe); + } + if ((window as any).gsap?.ticker?.tick) { + (window as any).gsap.ticker.tick(); + } + }, timeSeconds); + + await page.evaluate(`new Promise(function(r) { + var settled = false; + function finish() { if (settled) return; settled = true; r(); } + window.setTimeout(finish, 100); + requestAnimationFrame(function() { requestAnimationFrame(finish); }); + })`); +} + +export async function runFfmpegOnce( + ffmpegPath: string, + args: readonly string[], + timeoutMs: number, +): Promise { + return await new Promise((resolvePromise) => { + const ff = spawn(ffmpegPath, args); + let stderr = ""; + let timedOut = false; + const timer = setTimeout(() => { + timedOut = true; + ff.kill("SIGTERM"); + }, timeoutMs); + + ff.stderr.on("data", (data: Buffer) => { + stderr += data.toString(); + }); + ff.on("close", (code) => { + clearTimeout(timer); + resolvePromise({ code, stderr, timedOut }); + }); + ff.on("error", () => { + clearTimeout(timer); + resolvePromise({ code: null, stderr: "ffmpeg spawn failed", timedOut }); + }); + }); +} diff --git a/packages/cli/src/capture/contactSheet.test.ts b/packages/cli/src/capture/contactSheet.test.ts new file mode 100644 index 0000000000..2c1b5808d8 --- /dev/null +++ b/packages/cli/src/capture/contactSheet.test.ts @@ -0,0 +1,52 @@ +import { mkdtempSync, rmSync } from "node:fs"; +import { tmpdir } from "node:os"; +import { join } from "node:path"; +import sharp from "sharp"; +import { describe, expect, it } from "vitest"; +import { createContactSheet } from "./contactSheet.js"; + +function tempDir(): string { + return mkdtempSync(join(tmpdir(), "hf-contact-sheet-test-")); +} + +describe("createContactSheet", () => { + it("writes PNG output when the output path uses a .png extension", async () => { + const dir = tempDir(); + try { + const a = join(dir, "a.png"); + const b = join(dir, "b.png"); + const out = join(dir, "sheet.png"); + await sharp({ + create: { + width: 16, + height: 9, + channels: 3, + background: { r: 255, g: 0, b: 0 }, + }, + }) + .png() + .toFile(a); + await sharp({ + create: { + width: 16, + height: 9, + channels: 3, + background: { r: 0, g: 255, b: 0 }, + }, + }) + .png() + .toFile(b); + + await createContactSheet([a, b], out, { + cols: 2, + labelMode: "custom", + labels: ["A", "B"], + maxImages: 2, + }); + + await expect(sharp(out).metadata()).resolves.toMatchObject({ format: "png" }); + } finally { + rmSync(dir, { recursive: true, force: true }); + } + }); +}); diff --git a/packages/cli/src/capture/contactSheet.ts b/packages/cli/src/capture/contactSheet.ts index 0f9d002078..8468c107b7 100644 --- a/packages/cli/src/capture/contactSheet.ts +++ b/packages/cli/src/capture/contactSheet.ts @@ -93,17 +93,20 @@ export async function createContactSheet( overlays.push({ input: labelSvg, left: x, top: y }); } - await sharp({ + const sheet = sharp({ create: { width: totalW, height: totalH, channels: 3, background: { r: 26, g: 26, b: 26 }, }, - }) - .composite(overlays) - .jpeg({ quality }) - .toFile(outputPath); + }).composite(overlays); + + if (extname(outputPath).toLowerCase() === ".png") { + await sheet.png().toFile(outputPath); + } else { + await sheet.jpeg({ quality }).toFile(outputPath); + } return outputPath; } @@ -263,6 +266,7 @@ export async function createAssetContactSheet( * parent assets/ root (for external SVGs downloaded as ). * Files are deduplicated by basename so duplicates across dirs are collapsed. */ +// fallow-ignore-next-line complexity export async function createSvgContactSheet( svgsDir: string, outputPath: string, diff --git a/packages/cli/src/cli.ts b/packages/cli/src/cli.ts index 2c2726e79a..b98ad6d445 100644 --- a/packages/cli/src/cli.ts +++ b/packages/cli/src/cli.ts @@ -139,6 +139,8 @@ const commandLoaders = { events: () => import("./commands/events.js").then((m) => m.default), validate: () => import("./commands/validate.js").then((m) => m.default), snapshot: () => import("./commands/snapshot.js").then((m) => m.default), + "grade-compare": () => import("./commands/grade-compare.js").then((m) => m.default), + compare: () => import("./commands/compare.js").then((m) => m.default), capture: () => import("./commands/capture.js").then((m) => m.default), lambda: () => import("./commands/lambda.js").then((m) => m.default), cloudrun: () => import("./commands/cloudrun.js").then((m) => m.default), diff --git a/packages/cli/src/commands/compare.test.ts b/packages/cli/src/commands/compare.test.ts new file mode 100644 index 0000000000..f5381449ed --- /dev/null +++ b/packages/cli/src/commands/compare.test.ts @@ -0,0 +1,141 @@ +import { existsSync, mkdirSync, mkdtempSync, readFileSync, rmSync, writeFileSync } from "node:fs"; +import { tmpdir } from "node:os"; +import { join } from "node:path"; +import { describe, expect, it, vi } from "vitest"; +import { + buildCompareSuccessPayload, + capCompareVariants, + parseCompareArgs, + prepareCompareVariantProjects, +} from "./compare.js"; + +function tempDir(): string { + return mkdtempSync(join(tmpdir(), "hf-compare-test-")); +} + +describe("parseCompareArgs", () => { + it("requires at least two composition paths", () => { + expect(() => parseCompareArgs({ _: ["variant-a"] }, "/tmp")).toThrow( + "need 2+ paths to compare", + ); + }); + + it("rejects --labels when the count does not match the path count", () => { + expect(() => + parseCompareArgs({ _: ["variant-a", "variant-b"], labels: "one,two,three" }, "/tmp"), + ).toThrow("--labels count (3) must match path count (2)"); + }); + + it("derives default labels from directory basenames and html filenames", () => { + const parsed = parseCompareArgs( + { _: ["./looks/warm", "./looks/cool.html", "/tmp/hero.alt.html"] }, + "/work/project", + ); + + // Labels are the subject here — derived cross-platform via path.basename. + // (inputPath/displayPath are absolute/relative resolutions that differ by OS + // — separators + drive letter on Windows — and are covered by the resolution + // tests; asserting them literally here made this a POSIX-only test.) + expect(parsed.variants.map((v) => v.label)).toEqual(["warm", "cool", "hero.alt"]); + }); +}); + +describe("capCompareVariants", () => { + it("truncates over-cap variants and exposes truncation metadata for JSON output", () => { + const errorSpy = vi.spyOn(console, "error").mockImplementation(() => {}); + const logSpy = vi.spyOn(console, "log").mockImplementation(() => {}); + try { + const variants = Array.from({ length: 20 }, (_, index) => ({ + label: `variant ${index + 1}`, + inputPath: `/tmp/variant-${index + 1}`, + displayPath: `variant-${index + 1}`, + })); + + const capped = capCompareVariants(variants); + + expect(capped.variants).toHaveLength(16); + expect(capped.variants.at(0)?.label).toBe("variant 1"); + expect(capped.variants.at(15)?.label).toBe("variant 16"); + expect(capped.truncated).toBe(true); + expect(capped.total).toBe(20); + expect(buildCompareSuccessPayload("/tmp/compare.png", capped.variants, capped)).toMatchObject( + { + ok: true, + sheet: "/tmp/compare.png", + rendered: 16, + truncated: true, + total: 20, + }, + ); + expect(errorSpy).toHaveBeenCalledWith( + expect.stringContaining("Warning: 20 compare variants exceed the 16-variant cap"), + ); + expect(logSpy).not.toHaveBeenCalled(); + } finally { + vi.restoreAllMocks(); + } + }); +}); + +describe("prepareCompareVariantProjects", () => { + it("uses project directories directly and stages standalone html files as index.html", () => { + const dir = tempDir(); + try { + const projectDir = join(dir, "variant-a"); + const htmlDir = join(dir, "variant-b"); + const projectIndex = join(projectDir, "index.html"); + const htmlFile = join(htmlDir, "candidate.html"); + mkdirSync(projectDir, { recursive: true }); + mkdirSync(htmlDir, { recursive: true }); + writeFileSync(projectIndex, "A", { flag: "wx" }); + writeFileSync(htmlFile, "B", { flag: "wx" }); + writeFileSync(join(htmlDir, "asset.txt"), "asset"); + + const prepared = prepareCompareVariantProjects([ + { label: "A", inputPath: projectDir, displayPath: "variant-a" }, + { label: "B", inputPath: htmlFile, displayPath: "variant-b/candidate.html" }, + ]); + + try { + expect(prepared).toHaveLength(2); + expect(prepared[0]).toMatchObject({ + label: "A", + inputPath: projectDir, + displayPath: "variant-a", + projectDir, + }); + expect(prepared[0]?.stagedDir).toBeUndefined(); + expect(prepared[1]?.projectDir).not.toBe(htmlDir); + expect(prepared[1]?.stagedDir).toBe(prepared[1]?.projectDir); + expect(readFileSync(join(prepared[1]!.projectDir, "index.html"), "utf-8")).toContain( + "B", + ); + expect(readFileSync(join(prepared[1]!.projectDir, "asset.txt"), "utf-8")).toBe("asset"); + expect(existsSync(join(prepared[1]!.projectDir, "candidate.html"))).toBe(false); + } finally { + for (const variant of prepared) { + if (variant.stagedDir) rmSync(variant.stagedDir, { recursive: true, force: true }); + } + } + } finally { + rmSync(dir, { recursive: true, force: true }); + } + }); + + it("errors clearly for inputs that are not composition projects or html files", () => { + const dir = tempDir(); + try { + const badFile = join(dir, "notes.txt"); + writeFileSync(badFile, "not a composition"); + + expect(() => + prepareCompareVariantProjects([ + { label: "notes", inputPath: badFile, displayPath: "notes.txt" }, + { label: "other", inputPath: join(dir, "missing"), displayPath: "missing" }, + ]), + ).toThrow(/not a composition input.*notes\.txt/); + } finally { + rmSync(dir, { recursive: true, force: true }); + } + }); +}); diff --git a/packages/cli/src/commands/compare.ts b/packages/cli/src/commands/compare.ts new file mode 100644 index 0000000000..a55fd168e3 --- /dev/null +++ b/packages/cli/src/commands/compare.ts @@ -0,0 +1,413 @@ +import { cpSync, existsSync, mkdirSync, mkdtempSync, renameSync, rmSync, statSync } from "node:fs"; +import { tmpdir } from "node:os"; +import { basename, dirname, extname, join } from "node:path"; +import { defineCommand } from "citty"; +import { createContactSheet } from "../capture/contactSheet.js"; +import { + openSettledCompositionPage, + seekCompositionTimeline, +} from "../capture/captureCompositionFrame.js"; +import { c } from "../ui/colors.js"; +import { normalizeErrorMessage } from "../utils/errorMessage.js"; +import { displayPathFromInput, readOptionalString, resolveFromBase } from "../utils/pathArgs.js"; +import { trackCompareSheet } from "../telemetry/events.js"; +import { serveStaticProjectHtml } from "../utils/staticProjectServer.js"; +import { withMeta } from "../utils/updateCheck.js"; +import type { Example } from "./_examples.js"; + +const MAX_COMPARE_VARIANTS = 16; +const MAX_COLUMNS = 4; +const DEFAULT_RENDER_READY_TIMEOUT_MS = 5000; + +export interface CompareVariantSpec { + label: string; + inputPath: string; + displayPath: string; +} + +export interface PreparedCompareVariant extends CompareVariantSpec { + projectDir: string; + stagedDir?: string; +} + +export interface ParsedCompareArgs { + variants: CompareVariantSpec[]; + outPath: string; + atSeconds: number; + cols?: number; + json: boolean; + timeoutMs: number; +} + +export interface CompareVariantCapResult { + variants: CompareVariantSpec[]; + truncated: boolean; + total: number; +} + +export interface CompareSuccessPayload { + ok: true; + sheet: string; + variants: { label: string; path: string }[]; + rendered: number; + truncated?: true; + total?: number; +} + +export const examples: Example[] = [ + [ + "Compare two agent-authored composition variants", + "hyperframes compare ./variants/a ./variants/b --out compare.png", + ], + [ + "Compare three variants at a specific timeline time", + "hyperframes compare ./a ./b ./c --at 2.5 --labels classic,bold,quiet --json", + ], +]; + +function defaultLabelForPath(input: string): string { + const name = basename(input); + return extname(name).toLowerCase() === ".html" ? basename(name, extname(name)) : name; +} + +function parsePathArgs(args: { _?: readonly unknown[] }): string[] { + return (args._ ?? []) + .filter((value): value is string => typeof value === "string") + .map((value) => value.trim()) + .filter(Boolean); +} + +function parseLabels(value: unknown, pathCount: number): string[] | undefined { + const raw = readOptionalString(value); + if (!raw) return undefined; + const labels = raw.split(",").map((part) => part.trim()); + if (labels.some((label) => label.length === 0)) { + throw new Error("--labels entries must be non-empty"); + } + if (labels.length !== pathCount) { + throw new Error(`--labels count (${labels.length}) must match path count (${pathCount})`); + } + return labels; +} + +function parseAtSeconds(value: unknown): number { + const raw = readOptionalString(value); + if (!raw) return 0; + const parsed = Number(raw); + if (!Number.isFinite(parsed) || parsed < 0) { + throw new Error("--at must be a non-negative number of seconds"); + } + return parsed; +} + +function parseColumns(value: unknown): number | undefined { + const raw = readOptionalString(value); + if (!raw) return undefined; + const parsed = Number(raw); + if (!Number.isInteger(parsed) || parsed < 1) { + throw new Error("--cols must be a positive integer"); + } + return parsed; +} + +function defaultCompareCols(cellCount: number): number { + return Math.max(1, Math.min(MAX_COLUMNS, Math.ceil(Math.sqrt(cellCount)))); +} + +export function parseCompareArgs( + args: { + _?: readonly unknown[]; + labels?: unknown; + out?: unknown; + at?: unknown; + cols?: unknown; + json?: unknown; + timeout?: unknown; + }, + cwd = process.cwd(), +): ParsedCompareArgs { + const paths = parsePathArgs(args); + if (paths.length < 2) { + throw new Error("need 2+ paths to compare"); + } + + const labels = parseLabels(args.labels, paths.length); + const variants = paths.map((input, index) => ({ + label: labels?.[index] ?? defaultLabelForPath(input), + inputPath: resolveFromBase(cwd, input), + displayPath: displayPathFromInput(cwd, input), + })); + + return { + variants, + outPath: resolveFromBase(cwd, readOptionalString(args.out) ?? "compare.png"), + atSeconds: parseAtSeconds(args.at), + cols: parseColumns(args.cols), + json: args.json === true, + timeoutMs: + Number.parseInt(readOptionalString(args.timeout) ?? "", 10) || + DEFAULT_RENDER_READY_TIMEOUT_MS, + }; +} + +export function capCompareVariants( + variants: readonly CompareVariantSpec[], +): CompareVariantCapResult { + const total = variants.length; + if (total <= MAX_COMPARE_VARIANTS) { + return { variants: [...variants], truncated: false, total }; + } + console.error( + c.warn( + `Warning: ${total} compare variants exceed the ${MAX_COMPARE_VARIANTS}-variant cap — rendering the first ${MAX_COMPARE_VARIANTS} of ${total}; re-run with fewer variants or split into multiple runs.`, + ), + ); + return { + variants: variants.slice(0, MAX_COMPARE_VARIANTS), + truncated: true, + total, + }; +} + +export function buildCompareSuccessPayload( + sheet: string, + variants: readonly CompareVariantSpec[], + capResult: CompareVariantCapResult, +): CompareSuccessPayload { + const payload: CompareSuccessPayload = { + ok: true, + sheet, + variants: variants.map((variant) => ({ + label: variant.label, + path: variant.displayPath, + })), + rendered: variants.length, + }; + if (capResult.truncated) { + payload.truncated = true; + payload.total = capResult.total; + } + return payload; +} + +function inputError(variant: CompareVariantSpec): Error { + return new Error( + `Variant "${variant.label}" is not a composition input (${variant.displayPath}): expected a directory containing index.html or a .html file`, + ); +} + +function stageHtmlVariant(variant: CompareVariantSpec): PreparedCompareVariant { + const stagedDir = mkdtempSync(join(tmpdir(), "hf-compare-variant-")); + try { + // Copy the composition's sibling files but skip heavy/irrelevant trees — a + // variant sitting next to node_modules or .git shouldn't drag them into tmp. + cpSync(dirname(variant.inputPath), stagedDir, { + recursive: true, + filter: (src) => { + const base = basename(src); + return base !== "node_modules" && base !== ".git"; + }, + }); + const sourceName = basename(variant.inputPath); + if (sourceName !== "index.html") { + renameSync(join(stagedDir, sourceName), join(stagedDir, "index.html")); + } + return { + ...variant, + projectDir: stagedDir, + stagedDir, + }; + } catch (err) { + rmSync(stagedDir, { recursive: true, force: true }); + throw err; + } +} + +export function prepareCompareVariantProjects( + variants: readonly CompareVariantSpec[], +): PreparedCompareVariant[] { + const prepared: PreparedCompareVariant[] = []; + try { + for (const variant of variants) { + if (!existsSync(variant.inputPath)) { + throw inputError(variant); + } + const stat = statSync(variant.inputPath); + if (stat.isDirectory() && existsSync(join(variant.inputPath, "index.html"))) { + prepared.push({ + ...variant, + projectDir: variant.inputPath, + }); + continue; + } + if (stat.isFile() && extname(variant.inputPath).toLowerCase() === ".html") { + prepared.push(stageHtmlVariant(variant)); + continue; + } + throw inputError(variant); + } + return prepared; + } catch (err) { + for (const variant of prepared) { + if (variant.stagedDir) { + rmSync(variant.stagedDir, { recursive: true, force: true }); + } + } + throw err; + } +} + +function cleanupPreparedCompareVariants(variants: readonly PreparedCompareVariant[]): void { + for (const variant of variants) { + if (variant.stagedDir) { + rmSync(variant.stagedDir, { recursive: true, force: true }); + } + } +} + +async function renderCompareVariant( + variant: PreparedCompareVariant, + opts: { atSeconds: number; framePath: string; timeoutMs: number }, +): Promise<{ framePath: string; renderReadyTimedOut: boolean }> { + try { + const { bundleToSingleHtml } = await import("@hyperframes/core/compiler"); + const html = await bundleToSingleHtml(variant.projectDir); + const server = await serveStaticProjectHtml(variant.projectDir, html); + try { + const { + browser: chromeBrowser, + page, + renderReadyTimedOut, + } = await openSettledCompositionPage(html, server.url, { + renderReadyTimeoutMs: opts.timeoutMs, + renderReadyWarningSuffix: `comparison variant "${variant.label}" may be inaccurate`, + }); + try { + if (opts.atSeconds > 0) { + await seekCompositionTimeline(page, opts.atSeconds); + } + await page.screenshot({ path: opts.framePath, type: "png" }); + return { framePath: opts.framePath, renderReadyTimedOut }; + } finally { + await chromeBrowser.close(); + } + } finally { + await server.close(); + } + } catch (err) { + throw new Error( + `Render failed for variant "${variant.label}" (${variant.displayPath}): ${normalizeErrorMessage(err)}`, + ); + } +} + +async function renderCompareSheet(parsed: ParsedCompareArgs): Promise { + const capResult = capCompareVariants(parsed.variants); + const variants = capResult.variants; + const prepared = prepareCompareVariantProjects(variants); + const frameDir = mkdtempSync(join(tmpdir(), "hf-compare-frames-")); + const framePaths: string[] = []; + + try { + let renderReadyTimedOut = false; + for (let i = 0; i < prepared.length; i++) { + const variant = prepared[i]!; + const framePath = join(frameDir, `variant-${String(i + 1).padStart(2, "0")}.png`); + const rendered = await renderCompareVariant(variant, { + atSeconds: parsed.atSeconds, + framePath, + timeoutMs: parsed.timeoutMs, + }); + framePaths.push(rendered.framePath); + renderReadyTimedOut = renderReadyTimedOut || rendered.renderReadyTimedOut; + } + + mkdirSync(dirname(parsed.outPath), { recursive: true }); + await createContactSheet(framePaths, parsed.outPath, { + cols: parsed.cols ?? defaultCompareCols(framePaths.length), + maxImages: framePaths.length, + labelMode: "custom", + labels: variants.map((variant) => variant.label), + }); + + trackCompareSheet({ + command: "compare", + cells: variants.length, + truncated: capResult.truncated, + total: capResult.total, + renderReadyTimedOut, + }); + return buildCompareSuccessPayload(parsed.outPath, variants, capResult); + } finally { + cleanupPreparedCompareVariants(prepared); + rmSync(frameDir, { recursive: true, force: true }); + } +} + +function printJson(payload: object): void { + console.log(JSON.stringify(withMeta(payload), null, 2)); +} + +export default defineCommand({ + meta: { + name: "compare", + description: "Render independent composition variants into one labeled comparison sheet", + }, + args: { + path: { + type: "positional", + description: "Composition project directory or .html file (pass 2+ paths)", + required: false, + }, + at: { + type: "string", + description: "Timeline time in seconds to seek before screenshotting each variant", + }, + labels: { + type: "string", + description: "Comma-separated labels matching the variant path count", + }, + out: { + type: "string", + description: "Output comparison sheet path (default: ./compare.png)", + }, + cols: { + type: "string", + description: "Grid columns (default: sqrt heuristic, capped at 4)", + }, + timeout: { + type: "string", + description: "Render-ready timeout in ms per variant (default: 5000)", + }, + json: { + type: "boolean", + description: "Output result as JSON", + default: false, + }, + }, + async run({ args }) { + const jsonRequested = args.json === true; + try { + const parsed = parseCompareArgs(args); + if (!parsed.json) { + console.log( + `${c.accent("◆")} Rendering ${Math.min(parsed.variants.length, MAX_COMPARE_VARIANTS)} of ${parsed.variants.length} composition variants`, + ); + } + const payload = await renderCompareSheet(parsed); + if (parsed.json) { + printJson(payload); + } else { + console.log(`\n${c.success("◇")} Comparison sheet saved to ${payload.sheet}`); + } + } catch (err) { + const message = normalizeErrorMessage(err); + if (jsonRequested) { + printJson({ ok: false, error: message }); + } else { + console.error(`\n${c.error("✗")} Compare failed: ${message}`); + } + process.exit(1); + } + }, +}); diff --git a/packages/cli/src/commands/grade-compare.test.ts b/packages/cli/src/commands/grade-compare.test.ts new file mode 100644 index 0000000000..a321a859ad --- /dev/null +++ b/packages/cli/src/commands/grade-compare.test.ts @@ -0,0 +1,253 @@ +import { readFileSync, writeFileSync } from "node:fs"; +import { mkdtempSync, rmSync } from "node:fs"; +import { tmpdir } from "node:os"; +import { basename, join } from "node:path"; +import { HF_COLOR_GRADING_ATTR, serializeHfColorGrading } from "@hyperframes/core"; +import { describe, expect, it, vi } from "vitest"; +import { + buildGradeCompareHtml, + buildGradeCompareSuccessPayload, + capCandidateCells, + parseGradeCompareArgs, + parseGradesFile, + prepareGradeCompareTempProject, + prependBaselineCell, + resolveLutCells, + warnInactiveGradingCells, +} from "./grade-compare.js"; + +function tempDir(): string { + return mkdtempSync(join(tmpdir(), "hf-grade-compare-test-")); +} + +function validCubeLut(): string { + return `LUT_3D_SIZE 2 +0 0 0 +1 0 0 +0 1 0 +1 1 0 +0 0 1 +1 0 1 +0 1 1 +1 1 1 +`; +} + +describe("parseGradesFile", () => { + it("parses a valid grades array into labeled cells", () => { + const dir = tempDir(); + try { + const file = join(dir, "grades.json"); + writeFileSync( + file, + JSON.stringify([ + { label: "warm", grading: { preset: "warm-daylight" } }, + { label: "punch", grading: { adjust: { exposure: 0.5, contrast: 0.4 } } }, + ]), + ); + + expect(parseGradesFile(file)).toEqual([ + { label: "warm", grading: { preset: "warm-daylight" } }, + { label: "punch", grading: { adjust: { exposure: 0.5, contrast: 0.4 } } }, + ]); + } finally { + rmSync(dir, { recursive: true, force: true }); + } + }); + + it("errors clearly for missing or invalid grades files", () => { + const dir = tempDir(); + try { + expect(() => parseGradesFile(join(dir, "missing.json"))).toThrow( + /Grades file not found: .*missing\.json/, + ); + + const invalid = join(dir, "invalid.json"); + writeFileSync(invalid, JSON.stringify({ label: "not an array" })); + expect(() => parseGradesFile(invalid)).toThrow( + "Grades file must be a JSON array of { label, grading } objects", + ); + } finally { + rmSync(dir, { recursive: true, force: true }); + } + }); +}); + +describe("prependBaselineCell", () => { + it("prepends an ungraded 'original' reference cell", () => { + const candidates = resolveLutCells("/tmp/luts/film.cube"); + const withBaseline = prependBaselineCell(candidates); + expect(withBaseline).toHaveLength(candidates.length + 1); + // empty grading (no preset / no lut) → normalizes to inactive → renders the + // source frame untouched, giving a reference to judge candidates against + expect(withBaseline.at(0)).toEqual({ label: "original", grading: {} }); + expect(withBaseline.slice(1)).toEqual(candidates); + }); +}); + +describe("resolveLutCells", () => { + it("expands comma-separated LUT paths into labeled grading cells", () => { + expect(resolveLutCells("/tmp/luts/film.cube,./cool.look.cube")).toEqual([ + { label: "film", grading: { lut: { src: "/tmp/luts/film.cube" } } }, + { label: "cool.look", grading: { lut: { src: "./cool.look.cube" } } }, + ]); + }); +}); + +describe("parseGradeCompareArgs", () => { + it("requires exactly one grade source", () => { + expect(() => parseGradeCompareArgs({ for: "frame.png" })).toThrow( + "Exactly one of --grades or --luts is required", + ); + expect(() => + parseGradeCompareArgs({ for: "frame.png", grades: "grades.json", luts: "a.cube" }), + ).toThrow("Exactly one of --grades or --luts is required"); + }); +}); + +describe("buildGradeCompareHtml", () => { + it("renders one labeled color-graded image per cell with composition metadata", () => { + const cells = [ + { label: "warm ", grading: { preset: "warm-daylight" } }, + { label: "cool", grading: { adjust: { temperature: -0.8 } } }, + { label: "punchy", grading: { adjust: { exposure: 0.5, contrast: 0.4 } } }, + ]; + + const html = buildGradeCompareHtml({ + cells, + frameSrc: "frame.png", + frameWidth: 640, + frameHeight: 360, + }); + + const escapedAttr = HF_COLOR_GRADING_ATTR.replace(/[-/\\^$*+?.()|[\]{}]/g, "\\$&"); + const attrPattern = new RegExp(`]+${escapedAttr}=`, "g"); + expect(html.match(attrPattern)).toHaveLength(cells.length); + expect(html).toContain('data-composition-id="grade-compare"'); + expect(html).toContain('data-width="1168"'); + expect(html).toContain('data-height="742"'); + expect(html).toContain('data-duration="1"'); + expect(html).toContain("warm <daylight>"); + for (const cell of cells) { + expect(html).toContain(`${HF_COLOR_GRADING_ATTR}='${serializeHfColorGrading(cell.grading)}'`); + } + }); +}); + +describe("prepareGradeCompareTempProject", () => { + it("copies the frame and LUTs into a temp project and rewrites LUT src values", async () => { + const dir = tempDir(); + try { + const framePath = join(dir, "frame.png"); + const lutPath = join(dir, "look.cube"); + writeFileSync(framePath, "fake-png"); + writeFileSync(lutPath, validCubeLut()); + + const prepared = await prepareGradeCompareTempProject({ + projectDir: dir, + framePath, + frameBuffer: Buffer.from("fake-png"), + cells: [ + { label: "film", grading: { lut: { src: basename(lutPath), intensity: 0.7 } } }, + { label: "warm", grading: { preset: "warm-daylight" } }, + ], + frameWidth: 640, + frameHeight: 360, + }); + + try { + expect(readFileSync(join(prepared.tempDir, "frame.png"), "utf-8")).toBe("fake-png"); + expect(readFileSync(join(prepared.tempDir, "lut-0.cube"), "utf-8")).toBe(validCubeLut()); + expect(readFileSync(join(prepared.tempDir, "index.html"), "utf-8")).toContain( + `${HF_COLOR_GRADING_ATTR}='${serializeHfColorGrading({ + lut: { src: "lut-0.cube", intensity: 0.7 }, + })}'`, + ); + } finally { + rmSync(prepared.tempDir, { recursive: true, force: true }); + } + } finally { + rmSync(dir, { recursive: true, force: true }); + } + }); + + it("rejects an existing LUT file whose .cube content cannot be parsed", async () => { + const dir = tempDir(); + try { + const framePath = join(dir, "frame.png"); + const lutPath = join(dir, "broken.cube"); + writeFileSync(framePath, "fake-png"); + writeFileSync(lutPath, "plain text, not cube data\n"); + + await expect( + prepareGradeCompareTempProject({ + projectDir: dir, + framePath, + frameBuffer: Buffer.from("fake-png"), + cells: [{ label: "broken look", grading: { lut: { src: basename(lutPath) } } }], + frameWidth: 640, + frameHeight: 360, + }), + ).rejects.toThrow(/LUT for "broken look" is not a valid \.cube:/); + } finally { + rmSync(dir, { recursive: true, force: true }); + } + }); +}); + +describe("warnInactiveGradingCells", () => { + it("warns to stderr for normalized-but-inactive candidate grades", () => { + const errorSpy = vi.spyOn(console, "error").mockImplementation(() => {}); + const logSpy = vi.spyOn(console, "log").mockImplementation(() => {}); + try { + expect(() => + warnInactiveGradingCells([{ label: "numeric lut", grading: { lut: 12345 } }]), + ).not.toThrow(); + + expect(errorSpy).toHaveBeenCalledWith( + expect.stringContaining( + 'Warning: grading for "numeric lut" is inactive/no-op — it will render ungraded', + ), + ); + expect(logSpy).not.toHaveBeenCalled(); + } finally { + vi.restoreAllMocks(); + } + }); +}); + +describe("capCandidateCells", () => { + it("truncates over-cap candidates and exposes truncation metadata for JSON output", () => { + const errorSpy = vi.spyOn(console, "error").mockImplementation(() => {}); + const logSpy = vi.spyOn(console, "log").mockImplementation(() => {}); + try { + const cells = Array.from({ length: 18 }, (_, index) => ({ + label: `candidate ${index + 1}`, + grading: { adjust: { exposure: index / 10 } }, + })); + + const capped = capCandidateCells(cells); + + expect(capped.cells).toHaveLength(16); + expect(capped.cells.at(0)?.label).toBe("candidate 1"); + expect(capped.cells.at(15)?.label).toBe("candidate 16"); + expect(capped.truncated).toBe(true); + expect(capped.total).toBe(18); + expect(buildGradeCompareSuccessPayload("grade-compare.png", 17, capped)).toEqual({ + ok: true, + sheet: "grade-compare.png", + cells: 17, + truncated: true, + total: 18, + }); + expect(errorSpy).toHaveBeenCalledWith( + expect.stringContaining( + "Warning: 18 candidate grades exceed the 16-cell cap — rendering the first 16 of 18; re-run with fewer grades or split into multiple runs.", + ), + ); + expect(logSpy).not.toHaveBeenCalled(); + } finally { + vi.restoreAllMocks(); + } + }); +}); diff --git a/packages/cli/src/commands/grade-compare.ts b/packages/cli/src/commands/grade-compare.ts new file mode 100644 index 0000000000..d62c172b87 --- /dev/null +++ b/packages/cli/src/commands/grade-compare.ts @@ -0,0 +1,698 @@ +import { + copyFileSync, + existsSync, + mkdirSync, + mkdtempSync, + readFileSync, + rmSync, + writeFileSync, +} from "node:fs"; +import { tmpdir } from "node:os"; +import { basename, dirname, extname, join, resolve } from "node:path"; +import { + HF_COLOR_GRADING_ATTR, + isHfColorGradingActive, + normalizeHfColorGrading, + parseCubeLut, + serializeHfColorGrading, +} from "@hyperframes/core"; +import { defineCommand } from "citty"; +import sharp from "sharp"; +import type { Example } from "./_examples.js"; +import { openSettledCompositionPage, runFfmpegOnce } from "../capture/captureCompositionFrame.js"; +import { findFFmpeg } from "../browser/ffmpeg.js"; +import { c } from "../ui/colors.js"; +import { normalizeErrorMessage } from "../utils/errorMessage.js"; +import { displayPathFromBase, readOptionalString, resolveFromBase } from "../utils/pathArgs.js"; +import { trackCompareSheet } from "../telemetry/events.js"; +import { serveStaticProjectHtml } from "../utils/staticProjectServer.js"; +import { withMeta } from "../utils/updateCheck.js"; + +const COMPOSITION_ID = "grade-compare"; +const COMPOSITION_DURATION = "1"; +const DEFAULT_CELL_WIDTH = 560; +const MAX_CANDIDATE_CELLS = 16; +const MAX_COLUMNS = 4; +const GRID_PADDING = 16; +const LABEL_HEIGHT = 32; +const FFMPEG_EXTRACT_TIMEOUT_MS = 30_000; + +export interface GradeCompareCell { + label: string; + grading: unknown; +} + +interface ParsedGradeCompareArgs { + framePath: string; + projectDir: string; + outPath: string; + source: { kind: "grades"; path: string } | { kind: "luts"; value: string }; + json: boolean; + timeoutMs: number; +} + +interface PreparedGradeCompareProject { + tempDir: string; + html: string; + cells: GradeCompareCell[]; +} + +export interface CandidateCellCapResult { + cells: GradeCompareCell[]; + truncated: boolean; + total: number; +} + +export interface GradeCompareSuccessPayload { + ok: true; + sheet: string; + cells: number; + truncated?: true; + total?: number; +} + +interface GradeCompareHtmlOptions { + cells: readonly GradeCompareCell[]; + frameSrc: string; + frameWidth: number; + frameHeight: number; +} + +interface ReferenceFrame { + buffer: Buffer; + width: number; + height: number; + stagedName: string; +} + +export const examples: Example[] = [ + [ + "Compare grade presets on one reference frame", + "hyperframes grade-compare --for frame.png --grades grades.json", + ], + [ + "Compare LUT files and print agent-friendly JSON", + "hyperframes grade-compare --for frame.png --luts looks/a.cube,looks/b.cube --json", + ], +]; + +function isRecord(value: unknown): value is Record { + return typeof value === "object" && value !== null && !Array.isArray(value); +} + +function hasOwn(record: Record, key: string): boolean { + return Object.prototype.hasOwnProperty.call(record, key); +} + +function cloneRecord(record: Record): Record { + const next: Record = {}; + for (const [key, value] of Object.entries(record)) { + next[key] = value; + } + return next; +} + +function escapeXml(value: string): string { + return value + .replace(/&/g, "&") + .replace(//g, ">") + .replace(/"/g, """) + .replace(/'/g, "'"); +} + +function escapeSingleQuotedAttr(value: string): string { + return value + .replace(/&/g, "&") + .replace(//g, ">") + .replace(/'/g, "'"); +} + +function validateCell(label: string, grading: unknown): GradeCompareCell { + if (!normalizeHfColorGrading(grading)) { + throw new Error(`Invalid color grading for cell "${label}"`); + } + return { label, grading }; +} + +export function warnInactiveGradingCells(cells: readonly GradeCompareCell[]): void { + for (const cell of cells) { + const normalized = normalizeHfColorGrading(cell.grading); + if (!isHfColorGradingActive(normalized)) { + console.error( + c.warn(`Warning: grading for "${cell.label}" is inactive/no-op — it will render ungraded`), + ); + } + } +} + +export function capCandidateCells(cells: readonly GradeCompareCell[]): CandidateCellCapResult { + const total = cells.length; + if (total <= MAX_CANDIDATE_CELLS) { + return { cells: [...cells], truncated: false, total }; + } + console.error( + c.warn( + `Warning: ${total} candidate grades exceed the ${MAX_CANDIDATE_CELLS}-cell cap — rendering the first ${MAX_CANDIDATE_CELLS} of ${total}; re-run with fewer grades or split into multiple runs.`, + ), + ); + return { cells: cells.slice(0, MAX_CANDIDATE_CELLS), truncated: true, total }; +} + +export function buildGradeCompareSuccessPayload( + sheet: string, + cells: number, + capResult: CandidateCellCapResult, +): GradeCompareSuccessPayload { + if (!capResult.truncated) { + return { ok: true, sheet, cells }; + } + return { ok: true, sheet, cells, truncated: true, total: capResult.total }; +} + +function serializedGradingForCell(cell: GradeCompareCell): string { + const normalized = normalizeHfColorGrading(cell.grading); + if (!normalized) { + throw new Error(`Invalid color grading for cell "${cell.label}"`); + } + return serializeHfColorGrading(normalized); +} + +function lutSrcFromGrading(grading: unknown): string | null { + if (!isRecord(grading) || !hasOwn(grading, "lut")) return null; + const lut = grading.lut; + if (typeof lut === "string" && lut.trim()) return lut.trim(); + if (!isRecord(lut)) return null; + const src = lut.src; + return typeof src === "string" && src.trim() ? src.trim() : null; +} + +function rewriteGradingLutSrc(grading: unknown, src: string): unknown { + if (!isRecord(grading) || !hasOwn(grading, "lut")) return grading; + const lut = grading.lut; + const next = cloneRecord(grading); + if (typeof lut === "string") { + next.lut = { src }; + return next; + } + if (isRecord(lut)) { + const nextLut = cloneRecord(lut); + nextLut.src = src; + next.lut = nextLut; + } + return next; +} + +export function parseGradesFile(filePath: string): GradeCompareCell[] { + if (!existsSync(filePath)) { + throw new Error(`Grades file not found: ${filePath}`); + } + + let parsed: unknown; + try { + parsed = JSON.parse(readFileSync(filePath, "utf-8")); + } catch (err) { + throw new Error(`Could not parse grades JSON: ${normalizeErrorMessage(err)}`); + } + + if (!Array.isArray(parsed)) { + throw new Error("Grades file must be a JSON array of { label, grading } objects"); + } + + return parsed.map((entry, index) => { + if (!isRecord(entry)) { + throw new Error(`Grade entry ${index + 1} must be an object with label and grading`); + } + const label = entry.label; + if (typeof label !== "string" || !label.trim()) { + throw new Error(`Grade entry ${index + 1} must have a non-empty string label`); + } + if (!hasOwn(entry, "grading")) { + throw new Error(`Grade entry "${label}" must include a grading value`); + } + return validateCell(label, entry.grading); + }); +} + +export function resolveLutCells(luts: string): GradeCompareCell[] { + const paths = luts + .split(",") + .map((part) => part.trim()) + .filter(Boolean); + if (paths.length === 0) { + throw new Error("--luts must include at least one LUT path"); + } + return paths.map((lutPath) => + validateCell(basename(lutPath, extname(lutPath)), { lut: { src: lutPath } }), + ); +} + +// The ungraded frame as a leading reference cell. An empty grading object +// normalizes to inactive, so the runtime renders the source image untouched — +// giving the agent a baseline to judge every candidate look against. +export function prependBaselineCell(cells: GradeCompareCell[]): GradeCompareCell[] { + return [validateCell("original", {}), ...cells]; +} + +export function parseGradeCompareArgs(args: { + for?: unknown; + grades?: unknown; + luts?: unknown; + project?: unknown; + out?: unknown; + json?: unknown; + timeout?: unknown; +}): ParsedGradeCompareArgs { + const frameArg = readOptionalString(args.for); + if (!frameArg) throw new Error("--for is required"); + + const gradesArg = readOptionalString(args.grades); + const lutsArg = readOptionalString(args.luts); + if (Boolean(gradesArg) === Boolean(lutsArg)) { + throw new Error("Exactly one of --grades or --luts is required"); + } + + const projectDir = resolve(readOptionalString(args.project) ?? process.cwd()); + const framePath = resolveFromBase(projectDir, frameArg); + const outPath = resolveFromBase(projectDir, readOptionalString(args.out) ?? "grade-compare.png"); + + return { + framePath, + projectDir, + outPath, + source: gradesArg + ? { kind: "grades", path: resolveFromBase(projectDir, gradesArg) } + : { kind: "luts", value: lutsArg ?? "" }, + json: args.json === true, + timeoutMs: Number.parseInt(readOptionalString(args.timeout) ?? "", 10) || 5000, + }; +} + +function gridMetrics( + cellCount: number, + frameWidth: number, + frameHeight: number, +): { + columns: number; + rows: number; + cellImageWidth: number; + cellImageHeight: number; + width: number; + height: number; +} { + const columns = Math.max(1, Math.min(MAX_COLUMNS, Math.ceil(Math.sqrt(cellCount)))); + const rows = Math.ceil(cellCount / columns); + const cellImageWidth = DEFAULT_CELL_WIDTH; + const aspect = frameHeight > 0 && frameWidth > 0 ? frameHeight / frameWidth : 9 / 16; + const cellImageHeight = Math.max(1, Math.round(cellImageWidth * aspect)); + return { + columns, + rows, + cellImageWidth, + cellImageHeight, + width: columns * cellImageWidth + (columns + 1) * GRID_PADDING, + height: rows * (cellImageHeight + LABEL_HEIGHT) + (rows + 1) * GRID_PADDING, + }; +} + +export function buildGradeCompareHtml(options: GradeCompareHtmlOptions): string { + if (options.cells.length === 0) { + throw new Error("At least one grade cell is required"); + } + const metrics = gridMetrics(options.cells.length, options.frameWidth, options.frameHeight); + + const cellHtml = options.cells + .map((cell, index) => { + const serialized = escapeSingleQuotedAttr(serializedGradingForCell(cell)); + const label = escapeXml(cell.label); + const row = Math.floor(index / metrics.columns); + const col = index % metrics.columns; + const left = GRID_PADDING + col * (metrics.cellImageWidth + GRID_PADDING); + const top = GRID_PADDING + row * (metrics.cellImageHeight + LABEL_HEIGHT + GRID_PADDING); + return `
+
${label}
+ ${label} +
`; + }) + .join("\n"); + + return ` + + + + + HyperFrames Grade Compare + + + + +
+${cellHtml} + +
+ + +`; +} + +function frameFileNameForPath(framePath: string): string { + const ext = extname(framePath).toLowerCase(); + if (ext === ".jpg" || ext === ".jpeg") return `frame${ext}`; + return "frame.png"; +} + +export async function prepareGradeCompareTempProject(opts: { + projectDir: string; + framePath: string; + frameBuffer: Buffer; + cells: readonly GradeCompareCell[]; + frameWidth: number; + frameHeight: number; + frameFileName?: string; +}): Promise { + const tempDir = mkdtempSync(join(tmpdir(), "hf-grade-compare-")); + try { + const frameFileName = opts.frameFileName ?? frameFileNameForPath(opts.framePath); + writeFileSync(join(tempDir, frameFileName), opts.frameBuffer); + + let lutIndex = 0; + const stagedCells = opts.cells.map((cell) => { + const lutSrc = lutSrcFromGrading(cell.grading); + if (!lutSrc) return cell; + + const sourcePath = resolveFromBase(opts.projectDir, lutSrc); + if (!existsSync(sourcePath)) { + throw new Error(`LUT file not found for "${cell.label}": ${sourcePath}`); + } + const lutText = readFileSync(sourcePath, "utf-8"); + try { + parseCubeLut(lutText, { maxSize: 64 }); + } catch (err) { + throw new Error( + `LUT for "${cell.label}" is not a valid .cube: ${normalizeErrorMessage(err)}`, + ); + } + const lutExt = extname(sourcePath) || ".cube"; + const stagedName = `lut-${lutIndex}${lutExt}`; + lutIndex += 1; + copyFileSync(sourcePath, join(tempDir, stagedName)); + return { + label: cell.label, + grading: rewriteGradingLutSrc(cell.grading, stagedName), + }; + }); + + const html = buildGradeCompareHtml({ + cells: stagedCells, + frameSrc: frameFileName, + frameWidth: opts.frameWidth, + frameHeight: opts.frameHeight, + }); + writeFileSync(join(tempDir, "index.html"), html); + return { tempDir, html, cells: stagedCells }; + } catch (err) { + rmSync(tempDir, { recursive: true, force: true }); + throw err; + } +} + +function isVideoPath(filePath: string): boolean { + const ext = extname(filePath).toLowerCase(); + return [".mp4", ".mov", ".m4v", ".webm", ".mkv", ".avi", ".mpeg", ".mpg", ".ogv"].includes(ext); +} + +async function extractVideoFrameToBuffer(videoPath: string): Promise { + const tmp = mkdtempSync(join(tmpdir(), "hf-grade-compare-frame-")); + const outPath = join(tmp, "frame.png"); + try { + const ffmpegPath = findFFmpeg(); + if (!ffmpegPath) return null; + const args = [ + "-hide_banner", + "-loglevel", + "error", + "-ss", + "0", + "-i", + videoPath, + "-frames:v", + "1", + "-q:v", + "2", + "-y", + outPath, + ]; + const result = await runFfmpegOnce(ffmpegPath, args, FFMPEG_EXTRACT_TIMEOUT_MS); + if (result.timedOut) { + throw new Error(`ffmpeg timed out extracting first frame from ${videoPath}`); + } + if (result.code !== 0 || !existsSync(outPath)) { + const detail = result.stderr.trim() ? `: ${result.stderr.trim()}` : ""; + throw new Error(`ffmpeg could not extract first frame from ${videoPath}${detail}`); + } + return readFileSync(outPath); + } finally { + rmSync(tmp, { recursive: true, force: true }); + } +} + +async function loadReferenceFrame(framePath: string): Promise { + if (!existsSync(framePath)) { + throw new Error(`Reference frame not found: ${framePath}`); + } + + const buffer = isVideoPath(framePath) + ? await extractVideoFrameToBuffer(framePath) + : readFileSync(framePath); + if (!buffer) { + throw new Error(`Could not extract a frame from video: ${framePath}`); + } + + const metadata = await sharp(buffer).metadata(); + if (!metadata.width || !metadata.height) { + throw new Error(`Could not read reference frame dimensions: ${framePath}`); + } + + return { + buffer, + width: metadata.width, + height: metadata.height, + stagedName: isVideoPath(framePath) ? "frame.png" : frameFileNameForPath(framePath), + }; +} + +async function captureGradeCompareSheet( + projectDir: string, + timeoutMs: number, +): Promise<{ sheetPath: string; renderReadyTimedOut: boolean }> { + const { bundleToSingleHtml } = await import("@hyperframes/core/compiler"); + + const html = await bundleToSingleHtml(projectDir); + const server = await serveStaticProjectHtml(projectDir, html); + const sheetPath = join(projectDir, "grade-compare.png"); + + try { + const { + browser: chromeBrowser, + page, + renderReadyTimedOut, + } = await openSettledCompositionPage(html, server.url, { + renderReadyTimeoutMs: timeoutMs, + renderReadyWarningSuffix: "grade comparison may be inaccurate", + }); + + try { + await page.screenshot({ path: sheetPath, type: "png" }); + return { sheetPath, renderReadyTimedOut }; + } finally { + await chromeBrowser.close(); + } + } finally { + await server.close(); + } +} + +function printJson(payload: object): void { + console.log(JSON.stringify(withMeta(payload), null, 2)); +} + +export default defineCommand({ + meta: { + name: "grade-compare", + description: "Render candidate color grades onto a reference frame as one comparison PNG", + }, + args: { + for: { + type: "string", + description: "Reference image path, or a video path to sample at t=0", + required: true, + }, + grades: { + type: "string", + description: "JSON array of { label, grading } candidate grades", + }, + luts: { + type: "string", + description: "Comma-separated .cube LUT paths to compare", + }, + project: { + type: "string", + description: "Base directory for relative --for, --grades, and LUT paths (default: cwd)", + }, + out: { + type: "string", + description: "Output PNG path (default: /grade-compare.png)", + }, + json: { + type: "boolean", + description: "Output result as JSON", + default: false, + }, + timeout: { + type: "string", + description: "Render-ready timeout in ms before capture (default: 5000)", + }, + baseline: { + type: "boolean", + description: + "Prepend the ungraded frame as an 'original' reference cell (--no-baseline to omit)", + default: true, + }, + }, + async run({ args }) { + const jsonRequested = args.json === true; + let preparedDir: string | null = null; + try { + const parsed = parseGradeCompareArgs({ + for: args.for, + grades: args.grades, + luts: args.luts, + project: args.project, + out: args.out, + json: args.json, + timeout: args.timeout, + }); + let cells = + parsed.source.kind === "grades" + ? parseGradesFile(parsed.source.path) + : resolveLutCells(parsed.source.value); + if (cells.length === 0) { + throw new Error("At least one grade candidate is required"); + } + const capResult = capCandidateCells(cells); + cells = capResult.cells; + warnInactiveGradingCells(cells); + if (args.baseline !== false) { + cells = prependBaselineCell(cells); + } + + if (!parsed.json) { + console.log( + `${c.accent("◆")} Rendering ${cells.length} grade candidates from ${c.accent(basename(parsed.framePath))}`, + ); + } + + const frame = await loadReferenceFrame(parsed.framePath); + const prepared = await prepareGradeCompareTempProject({ + projectDir: parsed.projectDir, + framePath: parsed.framePath, + frameBuffer: frame.buffer, + frameWidth: frame.width, + frameHeight: frame.height, + frameFileName: frame.stagedName, + cells, + }); + preparedDir = prepared.tempDir; + + const { sheetPath: tempSheet, renderReadyTimedOut } = await captureGradeCompareSheet( + prepared.tempDir, + parsed.timeoutMs, + ); + mkdirSync(dirname(parsed.outPath), { recursive: true }); + copyFileSync(tempSheet, parsed.outPath); + trackCompareSheet({ + command: "grade-compare", + cells: prepared.cells.length, + truncated: capResult.truncated, + total: capResult.total, + renderReadyTimedOut, + }); + + const sheet = displayPathFromBase(parsed.projectDir, parsed.outPath); + if (parsed.json) { + printJson(buildGradeCompareSuccessPayload(sheet, prepared.cells.length, capResult)); + } else { + console.log(`\n${c.success("◇")} Grade comparison saved to ${sheet}`); + } + } catch (err) { + const message = normalizeErrorMessage(err); + if (jsonRequested) { + printJson({ ok: false, error: message }); + } else { + console.error(`\n${c.error("✗")} Grade compare failed: ${message}`); + } + process.exit(1); + } finally { + if (preparedDir) { + rmSync(preparedDir, { recursive: true, force: true }); + } + } + }, +}); diff --git a/packages/cli/src/commands/snapshot.ts b/packages/cli/src/commands/snapshot.ts index 42d408659f..51e9f3c7eb 100644 --- a/packages/cli/src/commands/snapshot.ts +++ b/packages/cli/src/commands/snapshot.ts @@ -1,12 +1,15 @@ // fallow-ignore-file complexity -import { spawn } from "node:child_process"; import { defineCommand } from "citty"; import { existsSync, mkdtempSync, readFileSync, mkdirSync, rmSync, writeFileSync } from "node:fs"; import { tmpdir } from "node:os"; import { resolve, join, relative, isAbsolute, basename } from "node:path"; +import { + openSettledCompositionPage, + runFfmpegOnce, + seekCompositionTimeline, +} from "../capture/captureCompositionFrame.js"; import { resolveProject } from "../utils/project.js"; import { normalizeErrorMessage } from "../utils/errorMessage.js"; -import { resolveCompositionViewportFromHtml } from "../utils/compositionViewport.js"; import { serveStaticProjectHtml } from "../utils/staticProjectServer.js"; import { c } from "../ui/colors.js"; import { findFFmpeg } from "../browser/ffmpeg.js"; @@ -67,46 +70,25 @@ async function extractVideoFrameToBuffer( try { const ffmpegPath = findFFmpeg(); if (!ffmpegPath) return null; - const result = await new Promise<{ code: number | null; stderr: string; timedOut: boolean }>( - (resolvePromise) => { - // `-ss` before `-i` performs a fast keyframe seek; adequate for snapshot accuracy - // (±1 frame) and orders of magnitude faster than the decode-and-scan alternative. - const args = ["-hide_banner", "-loglevel", "error"]; - if (useVp9AlphaDecoder) { - args.push("-c:v", "libvpx-vp9"); - } - args.push( - "-ss", - String(Math.max(0, timeSeconds)), - "-i", - videoPath, - "-frames:v", - "1", - "-q:v", - "2", - "-y", - outPath, - ); - const ff = spawn(ffmpegPath, args); - let stderr = ""; - let timedOut = false; - const timer = setTimeout(() => { - timedOut = true; - ff.kill("SIGTERM"); - }, FFMPEG_EXTRACT_TIMEOUT_MS); - ff.stderr.on("data", (d: Buffer) => { - stderr += d.toString(); - }); - ff.on("close", (code) => { - clearTimeout(timer); - resolvePromise({ code, stderr, timedOut }); - }); - ff.on("error", () => { - clearTimeout(timer); - resolvePromise({ code: null, stderr: "ffmpeg spawn failed", timedOut }); - }); - }, + // `-ss` before `-i` performs a fast keyframe seek; adequate for snapshot accuracy + // (±1 frame) and orders of magnitude faster than the decode-and-scan alternative. + const args = ["-hide_banner", "-loglevel", "error"]; + if (useVp9AlphaDecoder) { + args.push("-c:v", "libvpx-vp9"); + } + args.push( + "-ss", + String(Math.max(0, timeSeconds)), + "-i", + videoPath, + "-frames:v", + "1", + "-q:v", + "2", + "-y", + outPath, ); + const result = await runFfmpegOnce(ffmpegPath, args, FFMPEG_EXTRACT_TIMEOUT_MS); if (result.code !== 0 || result.timedOut || !existsSync(outPath)) return null; return readFileSync(outPath); } finally { @@ -193,7 +175,6 @@ async function captureSnapshots( }, ): Promise { const { bundleToSingleHtml } = await import("@hyperframes/core/compiler"); - const { ensureBrowser } = await import("../browser/manager.js"); const numFrames = opts.frames ?? 5; @@ -203,75 +184,12 @@ async function captureSnapshots( const savedPaths: string[] = []; try { - const browser = await ensureBrowser(); - const puppeteer = await import("puppeteer-core"); - const chromeBrowser = await puppeteer.default.launch({ - headless: true, - executablePath: browser.executablePath, - args: [ - "--no-sandbox", - "--disable-gpu", - "--disable-dev-shm-usage", - "--enable-webgl", - "--use-gl=angle", - "--use-angle=swiftshader", - ], + const { browser: chromeBrowser, page } = await openSettledCompositionPage(html, server.url, { + renderReadyTimeoutMs: opts.timeout ?? 5000, + renderReadyWarningSuffix: "snapshots may be inaccurate", }); try { - const page = await chromeBrowser.newPage(); - await page.setViewport(resolveCompositionViewportFromHtml(html)); - - await page.goto(server.url, { - waitUntil: "domcontentloaded", - timeout: 10000, - }); - - // __renderReady is set after the player is constructed AND the root - // timeline is bound — waiting for it guarantees renderSeek will work. - const timeoutMs = opts.timeout ?? 5000; - const runtimeReady = await page - .waitForFunction(() => !!(window as any).__renderReady, { timeout: timeoutMs }) - .then(() => true) - .catch(() => false); - - if (!runtimeReady) { - console.warn( - `\n ${c.warn("⚠")} Runtime did not become render-ready within ${timeoutMs}ms — snapshots may be inaccurate`, - ); - } - - // Wait for shader transition pre-rendering (HyperShader IndexedDB hydration). - // Uses the ready state flag as primary signal, with the loading overlay - // display:none as a fallback for older builds. - await page - .waitForFunction( - () => { - const win = window as unknown as { - __hf?: { shaderTransitions?: Record }; - }; - const shaderTransitions = win.__hf?.shaderTransitions; - if (shaderTransitions !== undefined) { - return Object.values(shaderTransitions).every((s) => s.ready === true); - } - const overlay = document.querySelector( - "[data-hyper-shader-loading]", - ) as HTMLElement | null; - if (!overlay) return true; - return window.getComputedStyle(overlay).display === "none"; - }, - { timeout: 90_000 }, - ) - .catch(() => { - console.warn(` ${c.warn("⚠")} Shader transitions did not finish pre-rendering`); - }); - - // Wait for fonts to finish loading before capturing - await page.evaluate(() => document.fonts.ready).catch(() => {}); - - // Extra settle time for media and animations to initialize - await new Promise((r) => setTimeout(r, 1500)); - // Font verification — split into loaded / errored / unused. Only status // "error" is a real failure; a face still "unloaded"/"loading" after // document.fonts.ready + the settle wait was simply never requested by any @@ -406,26 +324,7 @@ async function captureSnapshots( for (let i = 0; i < positions.length; i++) { const time = positions[i]!; - await page.evaluate((t: number) => { - const player = (window as any).__player; - if (!player) return; - const safe = Math.max(0, Number(t) || 0); - if (typeof player.renderSeek === "function") { - player.renderSeek(safe); - } else if (typeof player.seek === "function") { - player.seek(safe); - } - if ((window as any).gsap?.ticker?.tick) { - (window as any).gsap.ticker.tick(); - } - }, time); - - await page.evaluate(`new Promise(function(r) { - var settled = false; - function finish() { if (settled) return; settled = true; r(); } - window.setTimeout(finish, 100); - requestAnimationFrame(function() { requestAnimationFrame(finish); }); - })`); + await seekCompositionTimeline(page, time); if (cameraExpr) await page.evaluate(cameraExpr); diff --git a/packages/cli/src/help.ts b/packages/cli/src/help.ts index 41f22a6acf..11f0e15ded 100644 --- a/packages/cli/src/help.ts +++ b/packages/cli/src/help.ts @@ -41,6 +41,11 @@ const GROUPS: Group[] = [ ["inspect", "Inspect rendered visual layout across the timeline"], ["keyframes", "Inspect keyframes and render onion-shot diagnostics"], ["snapshot", "Capture key frames as PNG screenshots for visual verification"], + [ + "grade-compare", + "Render candidate color grades onto a reference frame as one labeled comparison PNG", + ], + ["compare", "Render composition variants into one labeled comparison sheet"], ["info", "Print project metadata"], ["compositions", "List all compositions in a project"], ["docs", "View inline documentation in the terminal"], diff --git a/packages/cli/src/telemetry/events.ts b/packages/cli/src/telemetry/events.ts index a6d0176767..c1d65564c8 100644 --- a/packages/cli/src/telemetry/events.ts +++ b/packages/cli/src/telemetry/events.ts @@ -488,6 +488,25 @@ export function trackTranscribeUnavailable(props: { optional: boolean }): void { trackEvent("transcribe_unavailable", { optional: props.optional }); } +// grade-compare / compare stand up headless Chrome and render up to 16 cells. +// Cell count, truncation-cap hits, and whether the render-ready timeout fired +// are the signals needed before safely lifting the cap. Low-cardinality only. +export function trackCompareSheet(props: { + command: "grade-compare" | "compare"; + cells: number; + truncated: boolean; + total: number; + renderReadyTimedOut: boolean; +}): void { + trackEvent("media_use_compare", { + command: props.command, + cells: props.cells, + truncated: props.truncated, + total: props.total, + render_ready_timed_out: props.renderReadyTimedOut, + }); +} + // A skills install was skipped because a required prerequisite binary is // absent from PATH (e.g. git on a fresh Windows box). Best-effort callers // (init) skip cleanly rather than crash, so the skip is otherwise invisible; diff --git a/packages/cli/src/utils/pathArgs.ts b/packages/cli/src/utils/pathArgs.ts new file mode 100644 index 0000000000..ce6764ce7c --- /dev/null +++ b/packages/cli/src/utils/pathArgs.ts @@ -0,0 +1,21 @@ +import { isAbsolute, relative, resolve } from "node:path"; + +export function readOptionalString(value: unknown): string | undefined { + if (typeof value !== "string") return undefined; + const trimmed = value.trim(); + return trimmed ? trimmed : undefined; +} + +export function resolveFromBase(baseDir: string, input: string): string { + return isAbsolute(input) ? input : resolve(baseDir, input); +} + +export function displayPathFromBase(baseDir: string, filePath: string): string { + const rel = relative(baseDir, filePath); + if (rel && !rel.startsWith("..") && !isAbsolute(rel)) return rel; + return filePath; +} + +export function displayPathFromInput(baseDir: string, input: string): string { + return isAbsolute(input) ? displayPathFromBase(baseDir, input) : input; +} diff --git a/packages/core/src/index.ts b/packages/core/src/index.ts index fec8f76524..0a527d9a32 100644 --- a/packages/core/src/index.ts +++ b/packages/core/src/index.ts @@ -188,6 +188,7 @@ export { type HfColorGradingVariableMap, type NormalizedHfColorGrading, } from "./colorGrading"; +export { parseCubeLut, CubeLutParseError, type ParseCubeLutOptions } from "./colorLuts"; // Inline scripts export { diff --git a/skills-manifest.json b/skills-manifest.json index e1e6ae374a..49bbd956be 100644 --- a/skills-manifest.json +++ b/skills-manifest.json @@ -18,7 +18,7 @@ "files": 1 }, "hyperframes": { - "hash": "24ec436b52957fcc", + "hash": "7ddbb928a3674e5e", "files": 1 }, "hyperframes-animation": { @@ -26,7 +26,7 @@ "files": 116 }, "hyperframes-cli": { - "hash": "493e59cafaab3d10", + "hash": "8ca3bef87f169ba1", "files": 7 }, "hyperframes-core": { @@ -46,8 +46,8 @@ "files": 10 }, "media-use": { - "hash": "22d8da57c4af6ddd", - "files": 103 + "hash": "1bea0c917a66f085", + "files": 113 }, "motion-graphics": { "hash": "96ed2f7d8051b009", diff --git a/skills/hyperframes-cli/SKILL.md b/skills/hyperframes-cli/SKILL.md index 5cd24cfced..a05b06895b 100644 --- a/skills/hyperframes-cli/SKILL.md +++ b/skills/hyperframes-cli/SKILL.md @@ -1,6 +1,6 @@ --- name: hyperframes-cli -description: HyperFrames CLI dev loop. Use when running npx hyperframes init, add, catalog, capture, lint, validate, inspect, layout, snapshot, preview, play, render, publish, feedback, lambda, doctor, browser, info, upgrade, skills, compositions, docs, benchmark, telemetry, transcribe, tts, or remove-background, or when troubleshooting the HyperFrames build/render environment. Entry point for AWS Lambda cloud rendering (`hyperframes lambda deploy / render / progress / destroy / policies / sites`). +description: HyperFrames CLI dev loop. Use when running npx hyperframes init, add, catalog, capture, lint, validate, inspect, layout, snapshot, compare, grade-compare, preview, play, render, publish, feedback, lambda, doctor, browser, info, upgrade, skills, compositions, docs, benchmark, telemetry, transcribe, tts, or remove-background, or when troubleshooting the HyperFrames build/render environment. Entry point for AWS Lambda cloud rendering (`hyperframes lambda deploy / render / progress / destroy / policies / sites`). --- # HyperFrames CLI @@ -24,7 +24,7 @@ Everything runs through `npx hyperframes` unless project instructions specify a Run lint, validate, and inspect before preview. `lint` catches missing `data-composition-id`, overlapping tracks, and unregistered timelines. `validate` loads the composition in headless Chrome and reports runtime console errors plus WCAG contrast issues. `inspect` seeks through the timeline and reports text spilling out of bubbles/containers or off the canvas — and, when a `*.motion.json` sidecar is present, verifies motion intent (entrances firing under seek, stagger order, in-frame, liveness) against that same seeked timeline. -For motion-heavy work, prefer snapshot-driven iteration and a `*.motion.json` sidecar — see `references/lint-validate-inspect.md` for the discipline and motion-verification spec. +For motion-heavy work, prefer snapshot-driven iteration and a `*.motion.json` sidecar — see `references/lint-validate-inspect.md` for the discipline and motion-verification spec. To compare agent-authored candidate variants, use `npx hyperframes compare [--at ] [--labels a,b,c] [--out compare.png] [--cols n] [--json]` to render each composition through its own runtime, assemble one labeled sheet, inspect it side by side, and choose. For color-grade selection, use the color-specific sibling `npx hyperframes grade-compare --for --grades grades.json` (or `--luts a.cube,b.cube`) to render every grading candidate through the real WebGL grading runtime into one labeled PNG before choosing the winner. ## Agent Conventions diff --git a/skills/hyperframes/SKILL.md b/skills/hyperframes/SKILL.md index 5a674a6539..b80d7e1cc0 100644 --- a/skills/hyperframes/SKILL.md +++ b/skills/hyperframes/SKILL.md @@ -25,16 +25,16 @@ Below: a **capability map** (the domain skills, loaded on demand) and the **inte Atomic capabilities you load **on demand** — not full workflows; they never own the end-to-end task. For "make me a…" intent, use the intent router below. -| You want to… | Skill | -| ----------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------ | -| **Author / edit an HTML composition** — the `data-*` contract, clips, tracks, sub-compositions, variables | `/hyperframes-core` | -| **Animate** — atomic motion, scene blueprints, transitions, runtime adapters (GSAP / Lottie / Three.js / Anime.js / CSS / WAAPI / TypeGPU) | `/hyperframes-animation` | -| **Author seek-safe keyframes** — GSAP timelines, CSS keyframes, Anime.js, WAAPI, FLIP, paths, masks, SVG morph/draw, 3D depth, plus `hyperframes keyframes` diagnostics | `/hyperframes-keyframes` | -| **Creative direction** — `frame.md` / `design.md`, palettes, typography, narration, beat planning, audio-reactive | `/hyperframes-creative` | -| **Media** — resolve/generate BGM, SFX, image, icon, brand logo, voice; TTS voiceover, transcription, background removal, captions; cross-project reuse | `/media-use` | -| **CLI dev loop** — init, lint, validate, inspect, preview, render, publish, doctor | `/hyperframes-cli` | -| **Install registry blocks / components** (`hyperframes add`) | `/hyperframes-registry` | -| **Import Figma content** — assets, tokens, components, storyboards→reconstructed motion (REST/CLI); Motion (MCP), shaders (MCP source / native export) | `/figma` | +| You want to… | Skill | +| ------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | ------------------------ | +| **Author / edit an HTML composition** — the `data-*` contract, clips, tracks, sub-compositions, variables | `/hyperframes-core` | +| **Animate** — atomic motion, scene blueprints, transitions, runtime adapters (GSAP / Lottie / Three.js / Anime.js / CSS / WAAPI / TypeGPU) | `/hyperframes-animation` | +| **Author seek-safe keyframes** — GSAP timelines, CSS keyframes, Anime.js, WAAPI, FLIP, paths, masks, SVG morph/draw, 3D depth, plus `hyperframes keyframes` diagnostics | `/hyperframes-keyframes` | +| **Creative direction** — `frame.md` / `design.md`, palettes, typography, narration, beat planning, audio-reactive | `/hyperframes-creative` | +| **Media** — resolve/generate BGM, SFX, image, icon, brand logo, voice, color grade, LUT; TTS voiceover, transcription, background removal, captions; cross-project reuse | `/media-use` | +| **CLI dev loop** — init, lint, validate, inspect, preview, render, publish, doctor | `/hyperframes-cli` | +| **Install registry blocks / components** (`hyperframes add`) | `/hyperframes-registry` | +| **Import Figma content** — assets, tokens, components, storyboards→reconstructed motion (REST/CLI); Motion (MCP), shaders (MCP source / native export) | `/figma` | --- diff --git a/skills/media-use/SKILL.md b/skills/media-use/SKILL.md index 40e94fcc74..0eaf69a59b 100644 --- a/skills/media-use/SKILL.md +++ b/skills/media-use/SKILL.md @@ -1,6 +1,6 @@ --- name: media-use -description: Agent Media OS, the single skill for every media need in a HyperFrames project. Resolve BGM, SFX, image, icon, brand logo, or voice into a frozen local file + ledger record (one verb, `resolve`); generate via TTS / music / image models when the catalog misses; produce voiceover, transcription, captions, and background removal through one shared audio engine; operate on media (cut / reframe / transform); and reuse assets across projects. Keeps search noise on disk, hands the agent a path. Use for any audio, image, icon, voiceover, caption, or media-asset need. +description: Agent Media OS, the single skill for every media need in a HyperFrames project. Resolve BGM, SFX, image, icon, brand logo, voice, color grade, or LUT into a frozen local file or paste-ready block + ledger record (one verb, `resolve`); generate via TTS / music / image models when the catalog misses; produce voiceover, transcription, captions, and background removal through one shared audio engine; operate on media (cut / reframe / transform); and reuse assets across projects. Keeps search noise on disk, hands the agent one path or block. Use for any audio, image, icon, logo, voiceover, caption, color-grading, or media-asset need. --- # media-use @@ -21,13 +21,36 @@ HyperFrames owns media _playback_; media-use owns everything else. Each row is e | No transcript-driven cutting | `scripts/transcript-cut.mjs` compiles word-timestamp edits into cut lists | | No auto-duck / publish loudness | `scripts/audio-duck.mjs` + `references/operations.md` loudnorm/sidechain recipes | | No cross-project memory | global content-addressed cache + auto-promote (`~/.media`) | +| No color-grade authoring | `resolve --type grade` emits a paste-ready `data-color-grading` block; `resolve --type lut` freezes validated `.cube` files | | No image generation | RAM-graded local mflux (FLUX) via `scripts/lib/mflux-provider.mjs`, codex `image_gen` upsell (`scripts/lib/codex-provider.mjs`) | | No video generation | spec-gated local LTX (`videogen` in `scripts/lib/local-models.mjs`); `heygen video create` avatar upsell | | Weak local-model defaults | free-usage HeyGen first (TTS, bg-removal) via the `heygen` CLI; local open-source only as an opt-out fallback (`scripts/lib/local-run.mjs`) | ## When to use -Call `resolve` whenever a composition needs media: background music, sound effects, images, icons, brand logos, or voice. For voiceover / TTS, music, SFX, and caption timing, use the **audio engine** (below); background removal is delegated to the `hyperframes` CLI; transcription defaults to Parakeet (better than whisper.cpp: 6.05% vs 7.44% WER, 5-10x faster) via `scripts/transcribe.mjs`, with whisper.cpp auto-fallback (see `references/operations.md`). For cutting / reframing / transforming existing media, see `references/operations.md`. media-use searches the HeyGen catalog first, freezes the best match locally, registers it in a manifest, and hands the agent one line; all search noise stays on disk. +Call `resolve` whenever a composition needs media: background music, sound effects, images, icons, brand logos, voice, a color grade, or a LUT. For voiceover / TTS, music, SFX, and caption timing, use the **audio engine** (below); background removal is delegated to the `hyperframes` CLI; transcription defaults to Parakeet (better than whisper.cpp: 6.05% vs 7.44% WER, 5-10x faster) via `scripts/transcribe.mjs`, with whisper.cpp auto-fallback (see `references/operations.md`). For cutting / reframing / transforming existing media, see `references/operations.md`. media-use searches the HeyGen catalog first for media files, resolves official logos through the logo cascade, uses local deterministic color grading for `grade`/`lut`, freezes the best match locally when a file is needed, registers it in a manifest, and hands the agent one line; all search noise stays on disk. + +## Be proactive — run a media opportunity pass + +The human usually can't tell which media would lift the piece. You can. When you build or review a composition, do **one** grounded scan and then **ask once** — don't silently add, and don't nag per asset. + +Surface an opportunity only when a concrete signal is present: + +| Signal detected | Offer | +| ------------------------------------------------------ | ------------------------------------------------------------------------------------------- | +| On-screen text / a script with no voiceover | TTS voiceover (audio engine) | +| Emoji or a `
` styled as an icon | resolve real `icon`s | +| Image that is a placeholder, tiny, or upscaled-looking | a better `image` (and/or upscale — see `references/operations.md`) | +| Hard scene cuts / transitions with no sound | transition `sfx` | +| A piece over ~10s with no music bed | `bgm` | +| Footage that reads under/over-exposed or color-cast | a corrective `grade` (analyze with `grade --for`, preview with `hyperframes grade-compare`) | + +Rules that keep this a help, not nagware: + +- **Grounded, not generic.** No signal → no suggestion. Never open with "want better images?". +- **Opinionated + concrete.** Propose the specific fix ("add a VO from your script, swap 3 emoji for real icons, replace the 400×400 hero, whooshes on the 4 cuts"), with defaults chosen — the human just approves **all / some / none**. +- **Once per project.** One consolidated ask, top few highest-value items. Respect "leave it" and don't re-raise. +- **Surface, never silently mutate.** Color grades especially: propose and preview, never auto-apply — a gray-world "correction" ruins an intentional sunset or neon look. ## Resolve @@ -39,14 +62,16 @@ Returns one line: `resolved → (, )` ### Types -| Type | What it finds | Provider | -| ------- | -------------------- | -------------------------------------------------------- | -| `bgm` | Background music | HeyGen audio catalog (10k+ tracks) | -| `sfx` | Sound effects | Bundled 19-file library + HeyGen catalog | -| `image` | Photos, backgrounds | HeyGen asset search (75k+ vectors) | -| `icon` | Icons, symbols | HeyGen asset search (type=icon) | -| `logo` | Official brand marks | svgl → simple-icons → GitHub org avatar → domain favicon | -| `voice` | TTS voiceover | Local Kokoro (free); HeyGen TTS upsell | +| Type | What it finds | Provider / cascade | +| ------- | -------------------------------- | ------------------------------------------------------------ | +| `bgm` | Background music | HeyGen audio catalog (10k+ tracks) | +| `sfx` | Sound effects | Bundled 19-file library + HeyGen catalog | +| `image` | Photos, backgrounds | HeyGen asset search (75k+ vectors) | +| `icon` | Icons, symbols | HeyGen asset search (type=icon) | +| `logo` | Official brand marks | svgl → simple-icons → GitHub org avatar → domain favicon | +| `voice` | TTS voiceover | Local Kokoro (free); HeyGen TTS upsell | +| `grade` | HyperFrames color-grading blocks | Core preset → look index params/CDN LUT → deterministic cube | +| `lut` | Reusable `.cube` LUT files | Look index params/CDN LUT → deterministic cube | ### Examples @@ -70,19 +95,28 @@ node /scripts/resolve.mjs --type icon --intent "rocket" --project . # Brand logo (official mark — never redrawn by hand) node /scripts/resolve.mjs --type logo --entity linkedin --intent "LinkedIn logo" --project . # → resolved logo_001 → .media/images/logo_001.svg (logo, official mark) + +# Color grade block +node /scripts/resolve.mjs --type grade --intent "warm daylight" --project . --json +# → {"ok":true,"preset":"warm-daylight","grading":{"preset":"warm-daylight","intensity":1},...} + +# LUT file +node /scripts/resolve.mjs --type lut --intent "teal orange blockbuster" --project . +# → resolved lut_001 → .media/luts/lut_001.cube (lut) ``` ### Flags | Flag | Description | | --------------- | ------------------------------------------------------------------------------------ | -| `--type, -t` | Media type: bgm, sfx, image, icon, voice | +| `--type, -t` | Media type: bgm, sfx, image, icon, logo, voice, grade, lut | | `--intent, -i` | What you need (natural language) | | `--entity, -e` | Entity name for cache matching (optional) | | `--project, -p` | Project directory (default: .) | | `--candidates` | List reusable assets (project + global cache) for `--type`; no download, no mutation | | `--reuse ` | Import a specific global-cache asset (by content sha/prefix, from `--candidates`) | | `--from` | Freeze a local file or direct public URL (ingest) | +| `--for` | Analyze a local image/video and add measured adjust suggestions (`grade` only) | | `--local-only` | Offline: skip every network provider (cache + local only) | | `--provider` | Force one generator (e.g. `codex`, `mflux`, `kokoro`, `heygen`) | | `--adopt` | Bulk-import existing assets/ into manifest | @@ -90,7 +124,7 @@ node /scripts/resolve.mjs --type logo --entity linkedin --intent "Lin ## Reuse before you resolve -Before resolving bgm/sfx/image/icon, **check what already exists and reuse it when it fits.** media-use does not semantically match for you — you are the judge. It surfaces candidates; you decide. +Before resolving bgm/sfx/image/icon/logo/grade/lut, **check what already exists and reuse it when it fits.** media-use does not semantically match for you — you are the judge. It surfaces candidates; you decide. ```bash node /scripts/resolve.mjs --type bgm --intent "upbeat tech launch" --candidates --project . @@ -110,6 +144,84 @@ Read the list and judge semantic fit yourself — "upbeat tech launch" ≈ "ener The deterministic floor still runs automatically: an identical (case/whitespace-insensitive) repeat auto-reuses with no `--candidates` step. `--candidates` is only for the semantic layer above that floor — and a fuzzy match is **never** auto-applied; reuse is always your explicit call. On a resolve that misses the floor and is about to fetch, media-use prints a one-line stderr hint when similar cached assets exist, pointing you back here. +## Color grading + +Use `grade` when you need the actual HyperFrames `data-color-grading` value to paste onto an `` or `