Skip to content

feat(loop-grill): visual grilling — screenshots and Playwright navigation as external resources #1034

Description

@mfittko

Context

Follows from #1030 (loop-grill skill). The external resources step (step 5a) covers links, other repos, and API access. This issue extends it to visual resources: screenshots and programmatic UI navigation.

Problem

Design-level grilling often requires visual understanding. A spec for a UI change is underspecified without seeing the current screen. Text descriptions of "click Settings → Profile → Edit" are ambiguous and error-prone. Grilling without visual context produces low-confidence answers on UX-facing questions.

Proposed solution

Extend the loop-grill external resources step to accept:

  1. Screenshots — static image files or URLs attached to the issue/plan. The grill reads them as visual context when filling gaps that involve UI or design.

  2. Playwright navigation descriptors — a structured description of how to reach a screen (e.g. "go to /settings, click Edit Profile"). In --auto mode, the grill can drive a headless browser to capture the current state of that screen and use the screenshot as context, rather than consuming tokens on a prose re-description.

The key constraint from #1030: no raw token consumption for resource fetching — route through bounded wrapper commands/tools (the Playwright runner would be a wrapper, not inline browser code).

Acceptance criteria

  • Issue/plan can reference a screenshot file path or URL as a visual resource; the grill includes it as context when answering design-related gaps
  • Issue/plan can reference a Playwright navigation descriptor; in --auto mode the grill invokes a bounded wrapper to capture the screen
  • Visual resources are surfaced before the Q&A starts (consistent with the external resources step in feat(skills): add standalone loop-grill skill for issue/plan clarification #1030)
  • When a screenshot/navigation descriptor is inaccessible or the Playwright runner is unavailable, the gap is flagged unresolved (same degradation rule as other external resources)
  • Interactive mode asks the operator to provide screenshots or confirm navigation steps when visual gaps are detected

Non-goals

  • Full visual regression testing
  • Replacing Playwright tooling — this wraps an existing runner
  • Automatically detecting that a spec needs visual context (follow-on)

References

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Projects

    Status
    Done

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions