Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
42 changes: 42 additions & 0 deletions .claude/skills/winapp-ui-automation/SKILL.md
Original file line number Diff line number Diff line change
Expand Up @@ -200,6 +200,27 @@ winapp ui wait-for btn-submit-a1b2 -a myapp --timeout 5000
winapp ui wait-for itm-status-c3d4 -a myapp --value "Complete" --timeout 5000
```

### Audit accessibility & contrast
```powershell
# Audit the whole window (all areas, basic level) — exits non-zero if any FAIL is found (CI gate)
winapp ui audit -a myapp

# Machine-readable report ({ summary, issues }); write it to a file too
winapp ui audit -a myapp --json -o audit.json

# Scope to specific areas (repeatable). Areas: names, keyboard, screen-reader, contrast, roles
winapp ui audit -a myapp --area names --area keyboard

# Go deeper: 'thorough' adds heuristic rules (e.g. tab-order coherence) and applies WCAG AAA contrast
winapp ui audit -a myapp --level thorough

# Contrast only, at the default basic level (WCAG AA thresholds)
winapp ui audit -a myapp --area contrast
```
- `--area` selects one or more audit areas (default: all). Contrast requires a window pixel capture; it is measured only when the `contrast` area is selected.
- `--level basic` (default) runs fast essential rules with WCAG **AA** contrast thresholds (normal 4.5, large 3.0); `--level thorough` adds heuristic/deeper rules (e.g. keyboard tab-order) and applies WCAG **AAA** contrast thresholds (normal 7.0, large 4.5).
- Non-client chrome (title-bar caption buttons, scrollbar parts) is suppressed, and the same defect surfaced by multiple areas is de-duplicated, so counts aren't inflated. Elements on a different window/HWND than the captured one are reported as "not measured" for contrast rather than mis-scored.

## Tips
- Use `--interactive` with `inspect` as your first command — it shows only what you can click
- Chain commands with `;` to reduce round-trips (see note below on why not `&&`)
Expand Down Expand Up @@ -596,3 +617,24 @@ Show the element that currently has keyboard focus in the target app.
| `--app` | Target app (process name, window title, or PID). Lists windows if ambiguous. | (none) |
| `--json` | Format output as JSON | (none) |
| `--window` | Target window by HWND (stable handle from list output). Takes precedence over --app. | (none) |

### `winapp ui audit`

Audit the currently visible view of a running app for accessibility and contrast issues. Walks the element tree and evaluates modular audit areas (names, keyboard, screen-reader, contrast, roles) at a chosen level (basic/thorough). Audits one view at a time — it does not navigate; drive the other ui commands (invoke, send-keys) to move through other pages/tabs/states and audit each. Exits non-zero when any fail-severity issue is found, so it can gate CI.

#### Arguments
<!-- auto-generated from cli-schema.json -->
| Argument | Required | Description |
|----------|----------|-------------|
| `<selector>` | No | Semantic slug (e.g., btn-minimize-d1a0) or text to search by name/automationId |

#### Options
<!-- auto-generated from cli-schema.json -->
| Option | Description | Default |
|--------|-------------|---------|
| `--app` | Target app (process name, window title, or PID). Lists windows if ambiguous. | (none) |
| `--area` | Accessibility area(s) to audit (repeatable). Allowed: names, keyboard, screen-reader, contrast, roles, all. Default: all. | (none) |
| `--json` | Format output as JSON | (none) |
| `--level` | Audit depth: basic (essential rules + WCAG AA contrast thresholds) or thorough (deeper rules + WCAG AAA contrast thresholds). Default: basic. | `basic` |
| `--output` | Save output to file path (e.g., screenshot) | (none) |
| `--window` | Target window by HWND (stable handle from list output). Takes precedence over --app. | (none) |
42 changes: 42 additions & 0 deletions .github/plugin/skills/winapp-cli/ui-automation/SKILL.md
Original file line number Diff line number Diff line change
Expand Up @@ -200,6 +200,27 @@ winapp ui wait-for btn-submit-a1b2 -a myapp --timeout 5000
winapp ui wait-for itm-status-c3d4 -a myapp --value "Complete" --timeout 5000
```

### Audit accessibility & contrast
```powershell
# Audit the whole window (all areas, basic level) — exits non-zero if any FAIL is found (CI gate)
winapp ui audit -a myapp

# Machine-readable report ({ summary, issues }); write it to a file too
winapp ui audit -a myapp --json -o audit.json

# Scope to specific areas (repeatable). Areas: names, keyboard, screen-reader, contrast, roles
winapp ui audit -a myapp --area names --area keyboard

# Go deeper: 'thorough' adds heuristic rules (e.g. tab-order coherence) and applies WCAG AAA contrast
winapp ui audit -a myapp --level thorough

# Contrast only, at the default basic level (WCAG AA thresholds)
winapp ui audit -a myapp --area contrast
```
- `--area` selects one or more audit areas (default: all). Contrast requires a window pixel capture; it is measured only when the `contrast` area is selected.
- `--level basic` (default) runs fast essential rules with WCAG **AA** contrast thresholds (normal 4.5, large 3.0); `--level thorough` adds heuristic/deeper rules (e.g. keyboard tab-order) and applies WCAG **AAA** contrast thresholds (normal 7.0, large 4.5).
- Non-client chrome (title-bar caption buttons, scrollbar parts) is suppressed, and the same defect surfaced by multiple areas is de-duplicated, so counts aren't inflated. Elements on a different window/HWND than the captured one are reported as "not measured" for contrast rather than mis-scored.

## Tips
- Use `--interactive` with `inspect` as your first command — it shows only what you can click
- Chain commands with `;` to reduce round-trips (see note below on why not `&&`)
Expand Down Expand Up @@ -596,3 +617,24 @@ Show the element that currently has keyboard focus in the target app.
| `--app` | Target app (process name, window title, or PID). Lists windows if ambiguous. | (none) |
| `--json` | Format output as JSON | (none) |
| `--window` | Target window by HWND (stable handle from list output). Takes precedence over --app. | (none) |

### `winapp ui audit`

Audit the currently visible view of a running app for accessibility and contrast issues. Walks the element tree and evaluates modular audit areas (names, keyboard, screen-reader, contrast, roles) at a chosen level (basic/thorough). Audits one view at a time — it does not navigate; drive the other ui commands (invoke, send-keys) to move through other pages/tabs/states and audit each. Exits non-zero when any fail-severity issue is found, so it can gate CI.

#### Arguments
<!-- auto-generated from cli-schema.json -->
| Argument | Required | Description |
|----------|----------|-------------|
| `<selector>` | No | Semantic slug (e.g., btn-minimize-d1a0) or text to search by name/automationId |

#### Options
<!-- auto-generated from cli-schema.json -->
| Option | Description | Default |
|--------|-------------|---------|
| `--app` | Target app (process name, window title, or PID). Lists windows if ambiguous. | (none) |
| `--area` | Accessibility area(s) to audit (repeatable). Allowed: names, keyboard, screen-reader, contrast, roles, all. Default: all. | (none) |
| `--json` | Format output as JSON | (none) |
| `--level` | Audit depth: basic (essential rules + WCAG AA contrast thresholds) or thorough (deeper rules + WCAG AAA contrast thresholds). Default: basic. | `basic` |
| `--output` | Save output to file path (e.g., screenshot) | (none) |
| `--window` | Target window by HWND (stable handle from list output). Takes precedence over --app. | (none) |
5 changes: 4 additions & 1 deletion .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -225,4 +225,7 @@ devcert.pfx
/src/winapp-NuGet/tools

# Do not ignore the targets and props files in the build folder, which are needed for consuming the NuGet package
!/src/winapp-NuGet/build/
!/src/winapp-NuGet/build/
# Local scratch / validation working dirs (PR5 UI audit) — never commit
/scratch/
/validation/
133 changes: 133 additions & 0 deletions docs/cli-schema.json
Original file line number Diff line number Diff line change
Expand Up @@ -1678,6 +1678,139 @@
"description": "Inspect and interact with any running Windows app using UI Automation (UIA). Works with WPF, WinForms, Win32, Electron, and WinUI 3 apps.",
"hidden": false,
"subcommands": {
"audit": {
"description": "Audit the currently visible view of a running app for accessibility and contrast issues. Walks the element tree and evaluates modular audit areas (names, keyboard, screen-reader, contrast, roles) at a chosen level (basic/thorough). Audits one view at a time — it does not navigate; drive the other ui commands (invoke, send-keys) to move through other pages/tabs/states and audit each. Exits non-zero when any fail-severity issue is found, so it can gate CI.",
"hidden": false,
"arguments": {
"selector": {
"description": "Semantic slug (e.g., btn-minimize-d1a0) or text to search by name/automationId",
"order": 0,
"hidden": false,
"valueType": "System.String",
"hasDefaultValue": false,
"arity": {
"minimum": 0,
"maximum": 1
}
}
},
"options": {
"--app": {
"description": "Target app (process name, window title, or PID). Lists windows if ambiguous.",
"hidden": false,
"aliases": [
"-a"
],
"valueType": "System.String",
"hasDefaultValue": false,
"arity": {
"minimum": 1,
"maximum": 1
},
"required": false,
"recursive": false
},
"--area": {
"description": "Accessibility area(s) to audit (repeatable). Allowed: names, keyboard, screen-reader, contrast, roles, all. Default: all.",
"hidden": false,
"valueType": "System.String[]",
"hasDefaultValue": false,
"arity": {
"minimum": 0
},
"required": false,
"recursive": false
},
"--json": {
"description": "Format output as JSON",
"hidden": false,
"valueType": "System.Boolean",
"hasDefaultValue": true,
"defaultValue": false,
"arity": {
"minimum": 0,
"maximum": 1
},
"required": false,
"recursive": false
},
"--level": {
"description": "Audit depth: basic (essential rules + WCAG AA contrast thresholds) or thorough (deeper rules + WCAG AAA contrast thresholds). Default: basic.",
"hidden": false,
"valueType": "System.String",
"hasDefaultValue": true,
"defaultValue": "basic",
"arity": {
"minimum": 1,
"maximum": 1
},
"required": false,
"recursive": false
},
"--output": {
"description": "Save output to file path (e.g., screenshot)",
"hidden": false,
"aliases": [
"-o"
],
"valueType": "System.String",
"hasDefaultValue": false,
"arity": {
"minimum": 1,
"maximum": 1
},
"required": false,
"recursive": false
},
"--quiet": {
"description": "Suppress progress messages",
"hidden": false,
"aliases": [
"-q"
],
"valueType": "System.Boolean",
"hasDefaultValue": true,
"defaultValue": false,
"arity": {
"minimum": 0,
"maximum": 1
},
"required": false,
"recursive": false
},
"--verbose": {
"description": "Enable verbose output",
"hidden": false,
"aliases": [
"-v"
],
"valueType": "System.Boolean",
"hasDefaultValue": true,
"defaultValue": false,
"arity": {
"minimum": 0,
"maximum": 1
},
"required": false,
"recursive": false
},
"--window": {
"description": "Target window by HWND (stable handle from list output). Takes precedence over --app.",
"hidden": false,
"aliases": [
"-w"
],
"valueType": "System.Nullable<System.Int64>",
"hasDefaultValue": false,
"arity": {
"minimum": 1,
"maximum": 1
},
"required": false,
"recursive": false
}
}
},
"click": {
"description": "Click an element by slug or text search using mouse simulation. Works on elements that don't support InvokePattern (e.g., column headers, list items). Use --double for double-click, --right for right-click.",
"hidden": false,
Expand Down
21 changes: 21 additions & 0 deletions docs/fragments/skills/winapp-cli/ui-automation.md
Original file line number Diff line number Diff line change
Expand Up @@ -195,6 +195,27 @@ winapp ui wait-for btn-submit-a1b2 -a myapp --timeout 5000
winapp ui wait-for itm-status-c3d4 -a myapp --value "Complete" --timeout 5000
```

### Audit accessibility & contrast
```powershell
# Audit the whole window (all areas, basic level) — exits non-zero if any FAIL is found (CI gate)
winapp ui audit -a myapp

# Machine-readable report ({ summary, issues }); write it to a file too
winapp ui audit -a myapp --json -o audit.json

# Scope to specific areas (repeatable). Areas: names, keyboard, screen-reader, contrast, roles
winapp ui audit -a myapp --area names --area keyboard

# Go deeper: 'thorough' adds heuristic rules (e.g. tab-order coherence) and applies WCAG AAA contrast
winapp ui audit -a myapp --level thorough

# Contrast only, at the default basic level (WCAG AA thresholds)
winapp ui audit -a myapp --area contrast
```
- `--area` selects one or more audit areas (default: all). Contrast requires a window pixel capture; it is measured only when the `contrast` area is selected.
- `--level basic` (default) runs fast essential rules with WCAG **AA** contrast thresholds (normal 4.5, large 3.0); `--level thorough` adds heuristic/deeper rules (e.g. keyboard tab-order) and applies WCAG **AAA** contrast thresholds (normal 7.0, large 4.5).
- Non-client chrome (title-bar caption buttons, scrollbar parts) is suppressed, and the same defect surfaced by multiple areas is de-duplicated, so counts aren't inflated. Elements on a different window/HWND than the captured one are reported as "not measured" for contrast rather than mis-scored.

## Tips
- Use `--interactive` with `inspect` as your first command — it shows only what you can click
- Chain commands with `;` to reduce round-trips (see note below on why not `&&`)
Expand Down
19 changes: 19 additions & 0 deletions docs/usage.md
Original file line number Diff line number Diff line change
Expand Up @@ -1091,9 +1091,28 @@ winapp ui [command] [options]
- `wait-for` - Wait for element state
- `list-windows` - List all windows for an app
- `get-focused` - Report the currently focused element
- `audit` - Audit the UI for accessibility and contrast issues (names, keyboard, screen-reader, contrast, roles); exits non-zero on any failure so it can gate CI

**Options:**
- `-a, --app <app>` - Target app (name, title, or PID)
- `-w, --window <hwnd>` - Target window by HWND (stable)

**`ui audit` options:**
- `--area <area>` - Accessibility area(s) to audit (repeatable). Allowed: `names`, `keyboard`, `screen-reader`, `contrast`, `roles`, `all`. Default: all.
- `--level <level>` - Audit depth: `basic` (essential rules + WCAG **AA** contrast thresholds: normal 4.5, large 3.0) or `thorough` (deeper rules such as keyboard tab-order + WCAG **AAA** contrast thresholds: normal 7.0, large 4.5). Default: `basic`.
- `-o, --output <path>` - Write the report (text, or JSON with `--json`) to a file.

```powershell
# Gate CI on accessibility failures (non-zero exit when any FAIL is found)
winapp ui audit -a myapp

# Contrast only, machine-readable report to a file
winapp ui audit -a myapp --area contrast --json -o audit.json

# Deeper sweep across all areas (adds tab-order, applies AAA contrast thresholds)
winapp ui audit -a myapp --level thorough
```

Contrast is measured only when the `contrast` area is selected; it captures the target window's pixels and samples each text element's bounds. Elements belonging to a different window/HWND than the captured one are reported as "not measured" rather than sampled against the wrong pixels. Non-client chrome (title-bar caption buttons, scrollbar parts) is suppressed, and a defect surfaced by multiple areas is de-duplicated so counts aren't inflated.

For full documentation, see [docs/ui-automation.md](ui-automation.md).
2 changes: 1 addition & 1 deletion scripts/generate-llm-docs.ps1
Original file line number Diff line number Diff line change
Expand Up @@ -104,7 +104,7 @@ $SkillCommandMap = @{
"manifest" = @("manifest generate", "manifest update-assets", "manifest add-alias")
"troubleshoot" = @("get-winapp-path", "tool", "store")
"frameworks" = @() # No auto-generated command sections — links to guides
"ui-automation" = @("ui status", "ui inspect", "ui search", "ui get-property", "ui get-value", "ui screenshot", "ui invoke", "ui click", "ui drag", "ui hover", "ui send-keys", "ui set-value", "ui focus", "ui scroll-into-view", "ui scroll", "ui wait-for", "ui list-windows", "ui get-focused")
"ui-automation" = @("ui status", "ui inspect", "ui search", "ui get-property", "ui get-value", "ui screenshot", "ui invoke", "ui click", "ui drag", "ui hover", "ui send-keys", "ui set-value", "ui focus", "ui scroll-into-view", "ui scroll", "ui wait-for", "ui list-windows", "ui get-focused", "ui audit")
}

# Validate that all CLI commands are covered by at least one skill
Expand Down
15 changes: 15 additions & 0 deletions src/winapp-CLI/WinApp.Cli.Tests/FakeUiServices.cs
Original file line number Diff line number Diff line change
Expand Up @@ -37,6 +37,12 @@ internal class FakeUiAutomationService : IUiAutomationService
public Dictionary<string, object?> PropertiesResult { get; set; } = [];
public string InvokeResult { get; set; } = "InvokePattern";
public (byte[] Pixels, int Width, int Height) ScreenshotResult { get; set; } = (new byte[4], 1, 1);

/// <summary>Configurable full-window capture returned by <see cref="CaptureWindowAsync"/> (used by the audit's contrast checks).</summary>
public (byte[] Pixels, int Width, int Height, int OriginX, int OriginY) WindowCaptureResult { get; set; } = (new byte[4], 1, 1, 0, 0);

/// <summary>When set, <see cref="CaptureWindowAsync"/> throws to simulate an unavailable capture.</summary>
public Exception? WindowCaptureException { get; set; }
public List<(nint Hwnd, int Pid, string Title)> WindowsByTitleResult { get; set; } = [];
public List<(nint Hwnd, int Pid, string Title)> WindowsByPidResult { get; set; } = [];

Expand Down Expand Up @@ -88,6 +94,15 @@ public Task<UiElement[]> SearchAsync(UiSessionInfo session, SelectorExpression s
public Task<(byte[] Pixels, int Width, int Height)> ScreenshotAsync(UiSessionInfo session, string? elementId, bool captureScreen, bool focus, CancellationToken ct)
=> Task.FromResult(ScreenshotResult);

public Task<(byte[] Pixels, int Width, int Height, int OriginX, int OriginY)> CaptureWindowAsync(UiSessionInfo session, CancellationToken ct)
{
if (WindowCaptureException is not null)
{
throw WindowCaptureException;
}
return Task.FromResult(WindowCaptureResult);
}

public Task<string> InvokeAsync(UiSessionInfo session, UiElement element, CancellationToken ct)
=> Task.FromResult(InvokeResult);

Expand Down
Loading
Loading