Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion CLAUDE.md
Original file line number Diff line number Diff line change
Expand Up @@ -38,7 +38,7 @@ Atomic capabilities the creation workflows compose against — pull one when you
- `/hyperframes-animation` — all animation knowledge: atomic motion rules, scene blueprints, transitions, runtime adapters (GSAP default, plus Lottie / Three.js / Anime.js / CSS / WAAPI / TypeGPU).
- `/hyperframes-keyframes` — seek-safe keyframe authoring across runtimes: GSAP timelines, CSS keyframes, Anime.js, WAAPI, FLIP, paths, masks, SVG morph/draw, text trails, 3D depth; plus `hyperframes keyframes` diagnostics for surfacing and verifying rendered motion.
- `/hyperframes-creative` — non-animation creative direction: `frame.md` / `design.md` handling, palettes, typography, narration, beat planning, audio-reactive visuals, composition patterns.
- `/media-use` — the media OS: resolve any media need (BGM, SFX, image, icon, voice) into a frozen local file + ledger record; generate via TTS / music / image models when the catalog misses; transcribe, caption, remove backgrounds, and reuse assets across projects. One shared `scripts/audio.mjs` engine + manifest tracking; keeps search noise on disk.
- `/media-use` — the media OS: resolve any media need (BGM, SFX, image, icon, voice, color grade, LUT) into a frozen local file or paste-ready block + ledger record; generate via TTS / music / image models when the catalog misses; transcribe, caption, remove backgrounds, and reuse assets across projects. One shared `scripts/audio.mjs` engine + manifest tracking; keeps search noise on disk.
- `/hyperframes-cli` — CLI dev loop: `init`, `add`, `lint`, `validate`, `inspect`, `preview`, `render`, `publish`, `doctor`, `lambda` (AWS Lambda cloud rendering).
- `/hyperframes-registry` — install and wire registry blocks and components into compositions via `hyperframes add`. Covers authoring a new block or component to contribute upstream.
- `/figma` — import Figma assets, tokens, components, and storyboard sections → reconstructed motion (frames read as states, not slides) (REST/CLI) plus Motion animations (MCP) and shaders (MCP source / native export) into a composition.
Expand Down
20 changes: 10 additions & 10 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -81,16 +81,16 @@ Run `npx skills add heygen-com/hyperframes --full-depth` for the interactive pic

Atomic capabilities the creation workflows compose against — pull one when you need that specific layer.

| Skill | Covers |
| ------------------------ | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `/hyperframes-core` | The composition contract — `data-*` timing attributes, `class="clip"`, tracks, sub-compositions, variables, framework-owned media playback, determinism rules. |
| `/hyperframes-animation` | All animation knowledge — atomic motion rules, scene blueprints, transitions, runtime adapters (GSAP / Lottie / Three.js / Anime.js / CSS / WAAPI / TypeGPU). |
| `/hyperframes-keyframes` | Seek-safe keyframe authoring across runtimes — GSAP timelines, CSS keyframes, Anime.js, WAAPI, FLIP, paths, masks, SVG morph/draw, 3D depth — plus `hyperframes keyframes` diagnostics for rendered motion. |
| `/hyperframes-creative` | Non-animation creative direction — `frame.md` / `design.md`, palettes, typography, narration, beat planning, audio-reactive visuals, composition patterns. |
| `/media-use` | The media OS — resolve any media need (BGM, SFX, image, icon, voice) into a frozen local file + ledger record, generate via TTS/music/image models when the catalog misses, transcribe, caption, remove backgrounds, and reuse assets across projects. One shared audio engine + manifest tracking. |
| `/hyperframes-cli` | CLI dev loop — `init`, `lint`, `validate`, `inspect`, `preview`, `render`, `publish`, `doctor`, plus AWS Lambda cloud rendering (`lambda deploy / render / progress`). |
| `/hyperframes-registry` | Install and wire registry blocks and components into compositions via `hyperframes add`. Authoring a new block or component to contribute upstream. |
| `/figma` | Import Figma assets, tokens, components, and storyboard sections → reconstructed motion (frames read as states, not slides) (REST/CLI) plus Motion animations (MCP) and shaders (MCP source / native export) into a composition. |
| Skill | Covers |
| ------------------------ | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ |
| `/hyperframes-core` | The composition contract — `data-*` timing attributes, `class="clip"`, tracks, sub-compositions, variables, framework-owned media playback, determinism rules. |
| `/hyperframes-animation` | All animation knowledge — atomic motion rules, scene blueprints, transitions, runtime adapters (GSAP / Lottie / Three.js / Anime.js / CSS / WAAPI / TypeGPU). |
| `/hyperframes-keyframes` | Seek-safe keyframe authoring across runtimes — GSAP timelines, CSS keyframes, Anime.js, WAAPI, FLIP, paths, masks, SVG morph/draw, 3D depth — plus `hyperframes keyframes` diagnostics for rendered motion. |
| `/hyperframes-creative` | Non-animation creative direction — `frame.md` / `design.md`, palettes, typography, narration, beat planning, audio-reactive visuals, composition patterns. |
| `/media-use` | The media OS — resolve any media need (BGM, SFX, image, icon, voice, color grade, LUT) into a frozen local file or paste-ready block + ledger record, generate via TTS/music/image models when the catalog misses, transcribe, caption, remove backgrounds, and reuse assets across projects. One shared audio engine + manifest tracking. |
| `/hyperframes-cli` | CLI dev loop — `init`, `lint`, `validate`, `inspect`, `preview`, `render`, `publish`, `doctor`, plus AWS Lambda cloud rendering (`lambda deploy / render / progress`). |
| `/hyperframes-registry` | Install and wire registry blocks and components into compositions via `hyperframes add`. Authoring a new block or component to contribute upstream. |
| `/figma` | Import Figma assets, tokens, components, and storyboard sections → reconstructed motion (frames read as states, not slides) (REST/CLI) plus Motion animations (MCP) and shaders (MCP source / native export) into a composition. |

For visual design handoff workflows, see the [Claude Design guide](https://hyperframes.heygen.com/guides/claude-design) and [Open Design guide](https://hyperframes.heygen.com/guides/open-design).

Expand Down
2 changes: 1 addition & 1 deletion docs/guides/skills.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -80,7 +80,7 @@ Atomic capabilities the creation workflows compose against — pull one when you
| `/hyperframes-animation` | All animation knowledge — atomic motion rules, scene blueprints, transitions, runtime adapters (GSAP / Lottie / Three.js / Anime.js / CSS / WAAPI / TypeGPU). |
| `/hyperframes-keyframes` | Seek-safe keyframe authoring across runtimes — GSAP timelines, CSS keyframes, Anime.js, WAAPI, FLIP, paths, masks, SVG morph/draw, 3D depth — plus `hyperframes keyframes` diagnostics for rendered motion. |
| `/hyperframes-creative` | Non-animation creative direction — `frame.md` / `design.md`, palettes, typography, narration, beat planning, audio-reactive visuals, composition patterns. |
| `/media-use` | The media OS — resolve any media need (BGM, SFX, image, icon, voice) into a frozen local file + ledger record, generate via TTS/music/image models when the catalog misses, transcribe, caption, remove backgrounds, and reuse assets across projects. One shared audio engine + manifest tracking. |
| `/media-use` | The media OS — resolve any media need (BGM, SFX, image, icon, voice, color grade, LUT) into a frozen local file or paste-ready block + ledger record, generate via TTS/music/image models when the catalog misses, transcribe, caption, remove backgrounds, and reuse assets across projects. One shared audio engine + manifest tracking. |
| `/hyperframes-cli` | CLI dev loop — `init`, `lint`, `validate`, `inspect`, `preview`, `render`, `publish`, `doctor`, plus AWS Lambda cloud rendering (`lambda deploy / render / progress / destroy / policies`). |
| `/hyperframes-registry` | Install and wire registry blocks and components into compositions via `hyperframes add`. Authoring a new block or component to contribute upstream. |
| `/figma` | Import Figma assets, tokens, components, and storyboard sections → reconstructed motion (frames read as states, not slides) (REST/CLI) plus Motion animations (MCP) and shaders (MCP source / native export) into a composition. |
Expand Down
39 changes: 39 additions & 0 deletions packages/cli/src/capture/captureCompositionFrame.test.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,39 @@
import { mkdtempSync, rmSync, writeFileSync } from "node:fs";
import { tmpdir } from "node:os";
import { join } from "node:path";
import { describe, expect, it } from "vitest";
import { runFfmpegOnce } from "./captureCompositionFrame.js";

function tempDir(): string {
return mkdtempSync(join(tmpdir(), "hf-capture-frame-test-"));
}

describe("runFfmpegOnce", () => {
it("returns the process exit code and collected stderr", async () => {
const dir = tempDir();
try {
const script = join(dir, "fail.cjs");
writeFileSync(script, 'process.stderr.write("ffmpeg failed"); process.exit(3);\n');

const result = await runFfmpegOnce(process.execPath, [script], 1000);

expect(result).toEqual({ code: 3, stderr: "ffmpeg failed", timedOut: false });
} finally {
rmSync(dir, { recursive: true, force: true });
}
});

it("terminates the process when the timeout elapses", async () => {
const dir = tempDir();
try {
const script = join(dir, "hang.cjs");
writeFileSync(script, "setTimeout(() => {}, 10000);\n");

const result = await runFfmpegOnce(process.execPath, [script], 50);

expect(result.timedOut).toBe(true);
} finally {
rmSync(dir, { recursive: true, force: true });
}
});
});
171 changes: 171 additions & 0 deletions packages/cli/src/capture/captureCompositionFrame.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,171 @@
import { spawn } from "node:child_process";
import type { Browser, Page } from "puppeteer-core";
import { c } from "../ui/colors.js";
import { resolveCompositionViewportFromHtml } from "../utils/compositionViewport.js";

const CHROME_LAUNCH_ARGS = [
"--no-sandbox",
"--disable-gpu",
"--disable-dev-shm-usage",
"--enable-webgl",
"--use-gl=angle",
"--use-angle=swiftshader",
];

const SHADER_TRANSITIONS_TIMEOUT_MS = 90_000;
const CAPTURE_SETTLE_MS = 1500;

export interface SettledCompositionPage {
browser: Browser;
page: Page;
}

export interface OpenSettledCompositionPageOptions {
renderReadyTimeoutMs: number;
renderReadyWarningSuffix: string;
}

export interface FfmpegRunResult {
code: number | null;
stderr: string;
timedOut: boolean;
}

function compositionRuntimeReadyInBrowser(): boolean {
return Boolean(Reflect.get(window, "__renderReady"));
}

function shaderTransitionsReadyInBrowser(): boolean {
function shaderTransitionRegistryReady(): boolean | undefined {
const hf = Reflect.get(window, "__hf");
if (typeof hf !== "object" || hf === null) return undefined;

const shaderTransitions = Reflect.get(hf, "shaderTransitions");
if (typeof shaderTransitions !== "object" || shaderTransitions === null) return undefined;

for (const key of Object.keys(shaderTransitions)) {
const entry = Reflect.get(shaderTransitions, key);
if (typeof entry !== "object" || entry === null) return false;
if (Reflect.get(entry, "ready") !== true) return false;
}
return true;
}

function shaderLoadingOverlayReady(): boolean {
const overlay = document.querySelector("[data-hyper-shader-loading]");
if (!overlay) return true;
if (!(overlay instanceof HTMLElement)) return true;
return window.getComputedStyle(overlay).display === "none";
}

return shaderTransitionRegistryReady() ?? shaderLoadingOverlayReady();
}

async function waitForCompositionSettle(
page: Page,
options: OpenSettledCompositionPageOptions,
): Promise<void> {
const runtimeReady = await page
.waitForFunction(compositionRuntimeReadyInBrowser, { timeout: options.renderReadyTimeoutMs })
.then(() => true)
.catch(() => false);

if (!runtimeReady) {
console.warn(
`\n ${c.warn("⚠")} Runtime did not become render-ready within ${options.renderReadyTimeoutMs}ms — ${options.renderReadyWarningSuffix}`,
);
}

await page
.waitForFunction(shaderTransitionsReadyInBrowser, {
timeout: SHADER_TRANSITIONS_TIMEOUT_MS,
})
.catch(() => {
console.warn(` ${c.warn("⚠")} Shader transitions did not finish pre-rendering`);
});

await page.evaluate(() => document.fonts.ready).catch(() => {});
await new Promise((resolveSettle) => setTimeout(resolveSettle, CAPTURE_SETTLE_MS));
}

export async function openSettledCompositionPage(
html: string,
url: string,
options: OpenSettledCompositionPageOptions,
): Promise<SettledCompositionPage> {
const { ensureBrowser } = await import("../browser/manager.js");
const browser = await ensureBrowser();
const puppeteer = await import("puppeteer-core");

let chromeBrowser: Browser | undefined;
try {
chromeBrowser = await puppeteer.default.launch({
headless: true,
executablePath: browser.executablePath,
args: CHROME_LAUNCH_ARGS,
});

const page = await chromeBrowser.newPage();
await page.setViewport(resolveCompositionViewportFromHtml(html));
await page.goto(url, { waitUntil: "domcontentloaded", timeout: 10000 });
await waitForCompositionSettle(page, options);
return { browser: chromeBrowser, page };
} catch (err) {
await chromeBrowser?.close().catch(() => {});
throw err;
}
}

export async function seekCompositionTimeline(
page: Pick<Page, "evaluate">,
timeSeconds: number,
): Promise<void> {
await page.evaluate((t: number) => {
const player = (window as any).__player;
if (!player) return;
const safe = Math.max(0, Number(t) || 0);
if (typeof player.renderSeek === "function") {
player.renderSeek(safe);
} else if (typeof player.seek === "function") {
player.seek(safe);
}
if ((window as any).gsap?.ticker?.tick) {
(window as any).gsap.ticker.tick();
}
}, timeSeconds);

await page.evaluate(`new Promise(function(r) {
var settled = false;
function finish() { if (settled) return; settled = true; r(); }
window.setTimeout(finish, 100);
requestAnimationFrame(function() { requestAnimationFrame(finish); });
})`);
}

export async function runFfmpegOnce(
ffmpegPath: string,
args: readonly string[],
timeoutMs: number,
): Promise<FfmpegRunResult> {
return await new Promise((resolvePromise) => {
const ff = spawn(ffmpegPath, args);
let stderr = "";
let timedOut = false;
const timer = setTimeout(() => {
timedOut = true;
ff.kill("SIGTERM");
}, timeoutMs);

ff.stderr.on("data", (data: Buffer) => {
stderr += data.toString();
});
ff.on("close", (code) => {
clearTimeout(timer);
resolvePromise({ code, stderr, timedOut });
});
ff.on("error", () => {
clearTimeout(timer);
resolvePromise({ code: null, stderr: "ffmpeg spawn failed", timedOut });
});
});
}
52 changes: 52 additions & 0 deletions packages/cli/src/capture/contactSheet.test.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,52 @@
import { mkdtempSync, rmSync } from "node:fs";
import { tmpdir } from "node:os";
import { join } from "node:path";
import sharp from "sharp";
import { describe, expect, it } from "vitest";
import { createContactSheet } from "./contactSheet.js";

function tempDir(): string {
return mkdtempSync(join(tmpdir(), "hf-contact-sheet-test-"));
}

describe("createContactSheet", () => {
it("writes PNG output when the output path uses a .png extension", async () => {
const dir = tempDir();
try {
const a = join(dir, "a.png");
const b = join(dir, "b.png");
const out = join(dir, "sheet.png");
await sharp({
create: {
width: 16,
height: 9,
channels: 3,
background: { r: 255, g: 0, b: 0 },
},
})
.png()
.toFile(a);
await sharp({
create: {
width: 16,
height: 9,
channels: 3,
background: { r: 0, g: 255, b: 0 },
},
})
.png()
.toFile(b);

await createContactSheet([a, b], out, {
cols: 2,
labelMode: "custom",
labels: ["A", "B"],
maxImages: 2,
});

await expect(sharp(out).metadata()).resolves.toMatchObject({ format: "png" });
} finally {
rmSync(dir, { recursive: true, force: true });
}
});
});
Loading