Skip to content

Epic: strengthen the UI review sub-loop inputs and shape #1067

Description

@mfittko

Epic: strengthen the UI review sub-loop inputs and shape

Umbrella follow-on to #97. The designer/vision review loop (#122), UI smoke
harness (#124), and artifact contract (#125) are in place and working, and the
v0.9 /loop-review-ui live-browser flagship (#1114, Stages 0-6) ships on top of
the same shared primitives — captureNamedUiState() and the named-state
artifact contract. This epic addresses a structural gap: the review loop is
screenshot-first, but the strongest and cheapest review signal — rendered
semantics — is not captured or consumed. It also aligns the loop with the
fan-out/converge shape used elsewhere.

Problem

  • state.json carries provenance only; no semantic/a11y snapshot of the render.
  • ui-vision-review.md asks the model to judge contrast from pixels — a
    computable fact delegated to an unreliable proxy, contradicting the
    "an assertion must be able to fail on the defect" doctrine in
    skills/docs/ui-e2e-scoping-step.md.
  • No console/network error capture per named state. (The live /loop-review-ui
    drive captures response/requestfailed/pageerror + a server-log tail, but
    only at the walk level — errors are not attributed to the named state they
    occurred in, and the shared fixture-backed named-state path captures nothing.)
  • designer vs vision is an either/or route, not parallel lenses.
  • Named states are single-viewport render states; interaction states
    (focus/hover/validation) and mobile/desktop pairing aren't first-class.
  • Findings don't reference which acceptance criterion they map to, so
    ui_review_satisfied is a gestalt call rather than auditable coverage.

Acceptance criteria

"Strengthened" means each of these is true and independently verifiable:

  • Each named-state artifact bundle carries, alongside screenshot.png +
    state.json, a semantic snapshot.json (accessibility tree) emitted by
    captureNamedUiState(), and both review modes accept it as a first-class
    input. Defined in docs/ui-artifact-contract.md, not redefined per stage.
  • Each named state carries a computed axe.json (from
    @axe-core/playwright); ui-vision-review.md no longer lists contrast
    (or any computable a11y fact) among its visual-judgment items, and a11y
    findings cite axe.json evidence instead of eyeballing pixels.
  • Console errors and failed requests are captured per named state and
    surfaced as a review input and a fail-closed signal (a captured error is
    not silently dropped).
  • A named state's identity encodes viewport and interaction-state
    (focus/hover/error) in its deterministic slug, so responsive and stateful
    defects are distinct, reviewable states.
  • The review runs a11y / layout-geometry / visual / interaction as parallel
    lenses over one bundle and converges to a deduped findings set, preserving
    the existing outcome enum and fail-closed contract.
  • Every finding carries an acceptanceCriterionRef, and ui_review_satisfied
    requires per-criterion coverage rather than a gestalt call.
  • Every stage ships a fail-closed validation seam (as in
    scripts/loop/ui-designer-review-contract.mjs) and extends the relevant
    contract doc rather than redefining the artifact shape.

Definition of done

Non-goals

Unresolved questions

Operator/RFC calls needed before slicing (RFC team: lead dev, specialized dev,
systems architect):

  1. Per-state capture seam (Stage 3). Does per-state console/network capture
    attach to the shared captureNamedUiState() (so the fixture-backed smoke
    path gains it too), or only re-attribute the live drive's existing walk-level
    capture per state? This decides whether Stage 3 touches
    test/playwright/harness/webkit-smoke-harness.mjs or only the drive.
  2. axe severity policy (Stage 2). Is any axe violation a fail-closed
    must-fix, or only serious/critical impact levels? Drives the a11y lens
    severity mapping.
  3. Lens execution model (Stage 5). Sequential passes over one captured
    bundle vs truly parallel model calls, and whether converge/dedupe is a new
    contract module or an extension of the existing report/designer-review route.
  4. AC-ref source + coverage bar (Stage 6). acceptanceCriterionRef draws
    from the AC list validateUiDesignerReviewInput already requires — but what
    is the coverage bar for ui_review_satisfied: every AC touched by >=1 lens,
    or every must-fix AC backed by a passing/covered named state?
  5. Slug change = migration? (Stage 4). The current path contract normalizes
    sliceId + stateName only; encoding viewport/interaction changes
    deterministic paths — confirm this is an additive schemaVersion bump with
    no back-compat obligation for already-emitted artifacts.

Proposed stages

Six independently-shippable, one-PR stages, dependency-ordered. Stages 1-4 each
enrich the shared named-state artifact and can gate/merge on their own; Stage 5
consumes the enriched bundle; Stage 6 layers traceability on the findings.

Stage 1 — Semantic snapshot artifact

captureNamedUiState() emits a sibling snapshot.json (accessibility tree)
next to screenshot.png + state.json; the artifact contract is extended and
both review modes accept it as input. Depends on: none (foundation).

  • Mini-AC: snapshot.json is emitted for every named state and its path is
    deterministic per the contract.
  • Mini-AC: docs/ui-artifact-contract.md documents the artifact; schemaVersion
    bumped.
  • Mini-AC: fail-closed seam rejects a bundle missing/malformed snapshot.json
    where it is required.

Stage 2 — Computed a11y facts, not eyeballed

Add @axe-core/playwright; emit axe.json per named state; remove contrast (and
other computable a11y facts) from the vision template's visual-judgment list; a11y
findings cite axe evidence. Depends on: Stage 1 (shared artifact contract
already extended).

  • Mini-AC: axe.json is emitted per named state and validated fail-closed.
  • Mini-AC: ui-vision-review.md no longer instructs contrast judgment; the
    designer-review-loop doc reflects the axe-evidence rule.
  • Mini-AC: axe severity->finding-severity mapping is documented and tested.

Stage 3 — Console + network capture per named state

Record per-state console errors and failed requests into the bundle as a review
input and a fail-closed signal; reconcile with the live drive's existing
walk-level response/requestfailed/pageerror + server-log capture.
Depends on: Stage 1.

  • Mini-AC: each named state carries its captured console/network errors.
  • Mini-AC: a captured error is a fail-closed signal, never silently dropped.
  • Mini-AC: no double-reporting between per-state capture and the drive's
    walk-level classifier.

Stage 4 — Interaction states + viewport in the contract

Bake viewport and interaction-state (focus/hover/error) into the deterministic
state slug so responsive and stateful defects are distinct, reviewable states.
Depends on: Stage 1 (path/slug contract).

  • Mini-AC: the slug encodes viewport + interaction-state; the path contract doc
    is updated.
  • Mini-AC: two states differing only by viewport or interaction-state produce
    distinct artifact directories.
  • Mini-AC: fail-closed seam rejects a malformed/duplicate slug.

Stage 5 — Lens fan-out + converge

Run a11y / layout-geometry / visual / interaction lenses in parallel over one
enriched bundle and converge to a deduped findings set, keeping the existing
outcome enum and fail-closed contract. Depends on: Stages 1-4 (needs the
semantic, axe, console, and viewport/interaction inputs to be worth splitting).

  • Mini-AC: the four lenses run against one bundle and converge to a deduped set.
  • Mini-AC: the outcome enum
    (continue_ui_fix_loop/ui_review_satisfied/blocked_needs_human_decision)
    and fail-closed contract are unchanged.
  • Mini-AC: dedupe/converge logic is a pure, tested seam.

Stage 6 — Acceptance-criterion traceability

Add acceptanceCriterionRef to findings and require per-criterion coverage
before ui_review_satisfied. Depends on: Stage 5 (converged finding shape).

  • Mini-AC: every finding carries acceptanceCriterionRef.
  • Mini-AC: ui_review_satisfied is blocked until per-criterion coverage is met;
    the bar is enforced in the fail-closed seam.
  • Mini-AC: coverage is auditable from the emitted result (which AC each finding
    maps to, and which ACs are covered).

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions