Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
32 changes: 23 additions & 9 deletions docs/ui-review-recipe-contract.md
Original file line number Diff line number Diff line change
Expand Up @@ -44,11 +44,12 @@ drive your app.
4. **Report** — posts a head-pinned **pending** PR review and produces a
self-contained HTML artifact.
5. **Teardown** — stops the booted app and, only with explicit confirmation,
removes the worktree (the working destructive step). Dev-DB row-drop currently
**fails closed** — the tagged-drop seam is not wired, so a confirmed manifest
is recorded as a drop failure and untagged rows may remain (a real tagged drop
is a tracked follow-up). A side-effect ledger is ALWAYS emitted, enumerating
what was done vs. left behind.
removes the worktree (the working destructive step) and prunes the Stage-4
hosting gist (`gh gist delete`, when a gist id is recorded in the report
result). Dev-DB row-drop currently **fails closed** — the tagged-drop seam is
not wired, so a confirmed manifest is recorded as a drop failure and untagged
rows may remain (a real tagged drop is a tracked follow-up). A side-effect
ledger is ALWAYS emitted, enumerating what was done vs. left behind.

## Guardrails (non-negotiable)

Expand Down Expand Up @@ -231,10 +232,23 @@ no external resources).
- On the **Claude Code** harness the stage emits a publishable directive and the
artifact is published via **Claude Code Artifacts** — a zero-setup hosted link
the review body links to. The module never calls an Artifacts tool itself; the
orchestrating agent publishes.
- On **any other harness** hosting **fails closed with a stated reason**: the
review body states the artifact is unhosted and why, and never blocks on
hosting. A GitHub-native fallback publisher is a **tracked follow-up**.
orchestrating agent publishes. This is an enhancement layered on top of the
portable default below, not a dependency of the core report flow.
- On **any other harness** the portable **GitHub-native default** publishes the
self-contained HTML as a **secret GitHub Gist** (`gh gist create`) — a real
per-run URL with zero repo pollution. The review body links it. Two honest
caveats: a gist **renders HTML as source, not a live page**, so the review body
links the gist and points at its **raw** file (the plain-text/download view); and
a gist accretes one secret entry per run, so **Stage 5 teardown prunes it**
(`gh gist delete`) when the run records the gist id in the report result and
teardown is confirmed. If gist creation does not yield a URL the stage **fails
closed with a stated reason** — the review body states the artifact is unhosted
and why, and never links a fabricated URL.

Setup a consuming repo must provide: an authenticated **`gh`** with the **`gist`**
scope (`gh auth login`/`gh auth refresh -s gist`). No `.devloops` config key is
involved — hosting rides on the same `gh` auth the loop already uses for reviews.
An explicit `--hosted-url` overrides the gist path when you host the HTML yourself.

## Config key reference

Expand Down
36 changes: 19 additions & 17 deletions packages/core/src/loop/ui-review-report.mjs
Original file line number Diff line number Diff line change
Expand Up @@ -12,8 +12,9 @@
* - a self-contained, CSP-safe HTML artifact string (ranked findings + inline
* screenshot evidence), and
* - a harness-aware hosting directive (Claude Code -> a publishable Artifacts
* directive for the orchestrator; any other harness -> fail closed with a
* stated reason and a follow-up marker — no hosted link this stage).
* directive for the orchestrator; any other harness -> a GitHub-native
* gist-publish directive the CLI executes, which yields a real per-run URL
* or fails closed with a stated reason — never a fake link).
*
* All IO (reading the diagnose output + the screenshot bytes, writing the HTML,
* invoking the poster) lives in the thin CLI. This module reads only its inputs.
Expand All @@ -22,9 +23,6 @@
import { isClaudeHarness } from "./run-context.mjs";
import { sanitizeCopilotSummonTokens } from "../github/copilot-helpers.mjs";

/** Follow-up marker for the descoped GitHub-native hosting fallback. */
export const HOSTING_FOLLOWUP = "#1285";

/** Findings past this cap are dropped from the artifact and the drop is logged. */
export const ARTIFACT_MAX_FINDINGS = 100;

Expand Down Expand Up @@ -105,38 +103,42 @@ export function severityToEvent({ findings = [], submitAuthorized = false } = {}
/**
* Harness-aware hosting directive (pure). Claude Code -> a publishable Artifacts
* directive for the orchestrator to host (this module never calls an agent tool
* itself). Any other harness / Artifacts unavailable -> fail closed with a
* stated reason and the follow-up marker. The self-contained HTML is produced
* regardless; only this link step is harness-aware.
* itself). Any other harness -> the portable GitHub-native default: publish the
* self-contained HTML as a secret GitHub Gist (a real per-run URL, zero repo
* pollution). This module decides the STRATEGY only; the CLI performs the gist
* publish IO and fails closed with a stated reason if it does not yield a URL.
* The self-contained HTML is produced regardless; only this link step differs.
*
* @param {{htmlPath: string, env?: Record<string,string|undefined>}} input
*/
export function decideHosting({ htmlPath, env = process.env } = {}) {
if (isClaudeHarness(env)) {
return { hosting: "claude-artifact", publishable: true, htmlPath: htmlPath ?? null };
}
return {
hosting: "unavailable",
publishable: false,
htmlPath: htmlPath ?? null,
reason: "no hosted-artifact publisher on this harness; GitHub-native fallback is deferred",
followup: HOSTING_FOLLOWUP,
};
return { hosting: "github-gist", publishable: true, htmlPath: htmlPath ?? null };
}

/** One review-body line describing where the screenshot artifact lives. Links a
* real hosted URL when one exists; otherwise states the harness-aware status so
* the review never blocks on hosting. */
function artifactBodyLine({ hosting, hostedUrl }) {
if (typeof hostedUrl === "string" && hostedUrl.length > 0) {
// Only an ACTUALLY-published gist gets the source-rendered caveat: an explicit
// --hosted-url override leaves the strategy as github-gist but sets no gist, so
// a self-hosted (maybe live-rendered) URL falls through to the neutral line.
const rawUrl = hosting?.gist?.rawUrl;
if (rawUrl) {
// A gist renders HTML as source, not a live page; the raw file is the
// download/plain-text view — surface it so "open raw" is actually actionable.
return `Screenshot artifact (GitHub Gist — renders as source; open the raw file to view/download the HTML): ${hostedUrl} (raw: ${rawUrl})`;
}
return `Screenshot artifact: ${hostedUrl}`;
}
if (hosting?.hosting === "claude-artifact") {
return "Screenshot artifact prepared for Claude Artifacts hosting (published by the harness; see run output).";
}
const reason = hosting?.reason ? ` (${hosting.reason})` : "";
const followup = hosting?.followup ? ` [follow-up ${hosting.followup}]` : "";
return `Screenshot artifact is unhosted this stage${reason}${followup}. Findings are included below.`;
return `Screenshot artifact is unhosted this stage${reason}. Findings are included below.`;
}

/** A finding is inlineable ONLY with a complete anchor buildDraftReviewPayload
Expand Down
49 changes: 45 additions & 4 deletions packages/core/src/loop/ui-review-teardown.mjs
Original file line number Diff line number Diff line change
Expand Up @@ -60,6 +60,13 @@ const PROCESS_STATUS = Object.freeze({
SKIPPED: "skipped",
});

const GIST_STATUS = Object.freeze({
DELETED: "deleted",
DELETE_FAILED: "delete-failed",
SKIPPED_UNCONFIRMED: "skipped-unconfirmed",
NONE: "none",
});

/**
* Did the Stage-2 drive potentially create dev-DB rows? Without row tagging this
* is a coarse but honest signal: a drive that actually walked steps exercised
Expand All @@ -82,8 +89,12 @@ function driveMayHaveCreatedRows(driveResult) {
* signal (whether the drive walked mutating steps). Null when no drive ran.
* @param {Array<object>|null} [input.rowManifest] - Explicit rows to drop, when
* a session tag/manifest is available. Absent/empty => untagged fallback.
* @param {{id?:string|null,url?:string|null}|null} [input.gist] - The Stage-4
* GitHub-native hosting artifact (a secret gist) to prune, when one was
* published off-Claude. Absent => nothing to prune.
* @param {boolean} [input.confirm] - Explicit authorization for the destructive
* steps (row drop, worktree removal). Fail-safe: absent means NOT confirmed.
* steps (row drop, worktree removal, gist deletion). Fail-safe: absent means
* NOT confirmed.
* @param {boolean} [input.stopApp] - Stop the Stage-1 app (default true). This is
* a clean shutdown, NOT gated on confirmation.
* @param {object} seams
Expand All @@ -92,12 +103,14 @@ function driveMayHaveCreatedRows(driveResult) {
* signalling is unsupported) — mapped to MAY_BE_RUNNING (non-fatal), not KILL_FAILED.
* @param {(a:{rows:Array<object>})=>Promise<{ok:boolean,dropped:number,detail:string}>} seams.dropRows
* @param {(a:{worktreePath:string})=>Promise<{removed:string|null,ok:boolean,detail:string}>} seams.removeWorktree
* @param {(a:{id:string})=>Promise<{ok:boolean,detail:string}>} [seams.deleteGist] - Prune the
* Stage-4 hosting gist. Only invoked when a gist id is present AND confirmed.
* @param {(msg:string)=>void} [seams.log]
* @returns {Promise<{ok:boolean,confirmed:boolean,ledger:object,errors:string[],logs:string[]}>}
*/
export async function teardown(
{ provisionResult, driveResult = null, rowManifest = null, confirm = false, stopApp = true },
{ killProcess, dropRows, removeWorktree, log = () => {} } = {},
{ provisionResult, driveResult = null, rowManifest = null, gist = null, confirm = false, stopApp = true },
{ killProcess, dropRows, removeWorktree, deleteGist, log = () => {} } = {},
) {
const logs = [];
const errors = [];
Expand Down Expand Up @@ -229,6 +242,33 @@ export async function teardown(
}
}

// 4. Prune the Stage-4 hosting gist — DESTRUCTIVE, confirmation-gated. A gist
// accretes one secret entry per run; deleting it keeps the hosting target
// from piling up. Only ever acts on an explicit gist id from the report
// result; a missing id is NONE (nothing published, or Claude-hosted).
const gistId = typeof gist?.id === "string" && gist.id.trim().length > 0 ? gist.id.trim() : null;
let gistLedger;
if (!gistId) {
gistLedger = { id: null, url: gist?.url ?? null, deleted: false, status: GIST_STATUS.NONE, detail: "no hosting gist to prune" };
} else if (!confirm) {
gistLedger = { id: gistId, url: gist?.url ?? null, deleted: false, status: GIST_STATUS.SKIPPED_UNCONFIRMED, detail: "hosting gist retained: teardown not confirmed" };
record(`gist prune skipped (not confirmed): ${gistId} retained`);
} else {
try {
const del = await deleteGist({ id: gistId });
if (del.ok) {
gistLedger = { id: gistId, url: gist?.url ?? null, deleted: true, status: GIST_STATUS.DELETED, detail: del.detail };
record(`hosting gist deleted: ${gistId} (${del.detail})`);
} else {
gistLedger = { id: gistId, url: gist?.url ?? null, deleted: false, status: GIST_STATUS.DELETE_FAILED, detail: del.detail };
fail(`hosting gist delete FAILED: ${gistId} (${del.detail})`);
}
} catch (err) {
gistLedger = { id: gistId, url: gist?.url ?? null, deleted: false, status: GIST_STATUS.DELETE_FAILED, detail: `deleteGist seam threw: ${err?.message ?? err}` };
fail(`hosting gist delete FAILED: ${gistId} (deleteGist seam threw: ${err?.message ?? err})`);
}
}

// The ledger is ALWAYS emitted (every case), enumerating every known side
// effect. Migrations are recorded as applied-not-reverted by design.
const ledger = {
Expand All @@ -241,10 +281,11 @@ export async function teardown(
},
rows: rowsLedger,
worktree: worktreeLedger,
gist: gistLedger,
process: processLedger,
};

return { ok: errors.length === 0, confirmed: confirm, ledger, errors, logs };
}

export { ROW_STATUS, WORKTREE_STATUS, PROCESS_STATUS };
export { ROW_STATUS, WORKTREE_STATUS, PROCESS_STATUS, GIST_STATUS };
56 changes: 50 additions & 6 deletions scripts/loop/ui-review-report.mjs
Original file line number Diff line number Diff line change
Expand Up @@ -20,7 +20,7 @@ import path from "node:path";
import { fileURLToPath } from "node:url";
import { parseArgs } from "node:util";
import { buildParseError, formatCliError, isDirectCliRun } from "../_core-helpers.mjs";
import { requireTokenValue, parsePositiveInteger } from "../_cli-primitives.mjs";
import { requireTokenValue, parsePositiveInteger, runChild } from "../_cli-primitives.mjs";
import { detectRepoSlug, normalizeRepoSlug } from "@dev-loops/core/github/repo-slug";
import {
buildArtifactHtml,
Expand Down Expand Up @@ -161,7 +161,34 @@ function postPendingReview({ repo, pr, reviewFile, cwd }) {
return { reviewId: parsed.reviewId ?? null, reviewUrl: parsed.reviewUrl ?? null, commitSha: parsed.commitSha ?? null };
}

export async function runCli(argv = process.argv.slice(2), { stdout = process.stdout, stderr = process.stderr, env = process.env } = {}) {
/** Publish the self-contained HTML as a secret GitHub Gist and return its URL +
* id. This is the GitHub-native, harness-agnostic hosting default off-Claude: a
* real per-run URL with zero repo pollution, deletable by Stage-5 teardown (which
* consumes the recorded id). A gist renders HTML as source, so the raw file URL
* is the plain-text/download view. Fails closed: a non-zero gh exit or missing
* URL throws — the caller states the reason and never links a fake URL. */
export async function publishGist({ htmlPath, pr, run = runChild, ghCommand = "gh", env = process.env }) {
const desc = `UI review findings — PR #${pr} (self-contained HTML; renders as source on GitHub Gist)`;
const result = await run(ghCommand, ["gist", "create", htmlPath, "--desc", desc], env);
if (result.code !== 0) {
throw new Error(`gh gist create failed: ${result.stderr.trim() || `exit code ${result.code}`}`);
}
const url = result.stdout
.split(/\r?\n/u)
.map((line) => line.trim())
.filter((line) => line.length > 0)
.pop() ?? null;
if (url === null || !/^https?:\/\//u.test(url)) {
throw new Error(`gh gist create did not return a gist URL (got: ${result.stdout.trim() || "<empty>"})`);
}
const id = url.split("/").filter((seg) => seg.length > 0).pop() ?? null;
return { url, id, rawUrl: `${url}/raw` };
}

export async function runCli(
argv = process.argv.slice(2),
{ stdout = process.stdout, stderr = process.stderr, env = process.env, run = runChild, loadLiveHead = loadLiveHeadSha, postReview = postPendingReview } = {},
) {
const options = parseUiReviewReportCliArgs(argv);
if (options.help) { stdout.write(`${USAGE}\n`); return; }

Expand All @@ -181,7 +208,7 @@ export async function runCli(argv = process.argv.slice(2), { stdout = process.st

// Head pinning: the inline anchors bind to the reviewed commit. Fail closed
// when the diagnose head is missing or the live head has advanced since.
const liveHeadSha = loadLiveHeadSha(options.pr, repo, cwd);
const liveHeadSha = loadLiveHead(options.pr, repo, cwd);
if (!diagnosedHeadSha) {
throw parseError("Stage-3 diagnose output has no pr.headSha; cannot head-pin the review. Failing closed.");
}
Expand All @@ -203,14 +230,31 @@ export async function runCli(argv = process.argv.slice(2), { stdout = process.st
mkdirSync(path.dirname(path.resolve(options.htmlOutput)), { recursive: true });
writeFileSync(options.htmlOutput, html, "utf8");

const hosting = decideHosting({ htmlPath: path.resolve(options.htmlOutput), env });
let hosting = decideHosting({ htmlPath: path.resolve(options.htmlOutput), env });
const policy = severityToEvent({ findings, submitAuthorized: options.submitAuthorized });

// Off-Claude default: publish the HTML as a secret gist to yield a real hosted
// URL. An explicit --hosted-url wins (caller already hosted it); --dry-run
// skips the side effect. A gist-publish failure fails closed here — hosting
// becomes `unavailable` with the stated reason, never a fabricated link.
let hostedUrl = options.hostedUrl ?? null;
if (!hostedUrl && hosting.hosting === "github-gist" && !options.dryRun) {
try {
const gist = await publishGist({ htmlPath: path.resolve(options.htmlOutput), pr: options.pr, run, env });
hostedUrl = gist.url;
hosting = { ...hosting, gist: { id: gist.id, url: gist.url, rawUrl: gist.rawUrl } };
} catch (err) {
const reason = `GitHub-native gist publish failed: ${err?.message ?? err}`;
hosting = { hosting: "unavailable", publishable: false, htmlPath: hosting.htmlPath, reason };
caps.push(`hosting: ${reason}`);
}
}

const reviewInput = buildReviewInput({
findings,
headSha: diagnosedHeadSha,
hosting,
hostedUrl: options.hostedUrl ?? null,
hostedUrl,
});
const reviewFile = options.reviewFile
? path.resolve(options.reviewFile)
Expand All @@ -220,7 +264,7 @@ export async function runCli(argv = process.argv.slice(2), { stdout = process.st

for (const cap of caps) stderr.write(`[ui-review-report] cap: ${cap}\n`);

const review = options.dryRun ? null : postPendingReview({ repo, pr: options.pr, reviewFile, cwd });
const review = options.dryRun ? null : postReview({ repo, pr: options.pr, reviewFile, cwd });

const result = {
ok: true,
Expand Down
Loading