feat(ios-qa): adaptive XCUITest QA — reaches nested controls, verifies taps

[time-attack]

What

Adds the adaptive iOS QA interaction layer that the /qa skill needs for real physical-device testing: a JSON-driven XCUITest runner, SwiftUI/UIKit fixture apps, and a standalone execution-plan module (ios-qa/executor).

Why

The prior physical-device QA drove apps with coordinate-synthesized taps. That approach reports "success" even when the intended semantic control never received the action — it misses nested controls and cannot tell a tap that landed from one that did nothing. In the blind benchmark this produced a false High "defect" on a working SwiftUI Toggle: the tap hit the full-width row instead of the switch, the value stayed 0, and the runner couldn't distinguish "I tapped the wrong spot" from "the app is broken."

The fix

Two changes at one choke point (GStackFlowRunnerUITests.swift):

1. Reaches nested controls. SwiftUI composes a Toggle as an outer row wrapping a native switch. The runner resolves the actual switch, and when a center tap misses the full-width row it retries on the control's trailing edge via an element-anchored normalized offset — adaptive, resolved from the element's live frame, not a raw screen coordinate.
2. Verifies the tap took. After toggling a switch it confirms the value actually changed. A switch tap that does nothing is now reported as an interaction failure, not a confirmed product defect.

Evidence

Ran the same Favorite-toggle flow against the fixture app on two surfaces:

• Simulator: center tap missed → trailing-edge retry flipped the switch 0 → 1 → verified. Passed.
• Physical iPhone (iPhone17,1): built + signed + installed + driven by the same runner. Passed.

Before the trailing-edge refinement the runner correctly failed loudly ("switch tap did not change its value") instead of silently passing — the honesty half of the fix.

Scope

Targets codex/gstack-2 (where /qa lives). Includes only the device-proven external XCUITest path. Deliberately excludes the in-app DebugBridge semantic-action rewrite (a separate mechanism not exercised in this validation) and all benchmark output artifacts. VERSION/CHANGELOG are deferred to GStack 2's own ship to main.

🤖 Generated with Claude Code

---

Summary by cubic

Adds an adaptive, JSON-driven XCUITest QA runner and planner that uses semantic selectors to reach nested controls and verifies taps, enabling reliable physical-device testing without coordinate taps. Includes SwiftUI/UIKit fixture apps and an ios-qa/executor module that emits a safe xcodebuild plan for simulator or device.

• New Features
  • ios-qa/executor: plan builder + CLI (bun ios-qa/executor/cli.ts) that validates flows and produces xcodebuild args and env; no coordinate taps; covered by unit tests.
  • XCUITest runner (GStackFlowRunnerUITests.swift): resolves nested controls (e.g., SwiftUI Toggle → native switch), retries on trailing edge when center tap misses, and confirms value changes after tap.
  • Post-action verification (exists, notExists, labelEquals) and hittability waits for launch, tap, typeText, swipe, wait.
  • Fixture apps: SwiftUI and UIKit galleries with stable accessibility identifiers to exercise lists, sheets, alerts, text input, and nested scroll containers.
  • Project wiring (project.yml) to pass flow via env and run the same JSON flow on simulator or physical device.

^{Written for commit c2ac1f3. Summary will update on new commits.}

[cursor]

Bugbot is not enabled for your account, so this pull request was not reviewed.

Enable Bugbot in the Cursor dashboard to get automatic reviews on future PRs.

[cubic-dev-ai]

18 issues found across 13 files

Prompt for AI agents (unresolved issues)

Check if these issues are valid — if so, understand the root cause of each and fix them. If appropriate, use sub-agents to investigate and fix each issue separately.


<file name="ios-qa/executor/cli.ts">

<violation number="1" location="ios-qa/executor/cli.ts:22">
P2: Malformed JSON or an unexpected input shape produces a `blocked` object without the required `remediation`, so consumers cannot safely treat all CLI output as `PlanResult`. Include remediation in this fallback result to preserve the executor contract.</violation>
</file>

<file name="ios-qa/executor/README.md">

<violation number="1" location="ios-qa/executor/README.md:14">
P2: Runner consumers cannot receive the documented `blocked`/`unsupported` step result; the checked-in Swift runner reports these as XCTest failures. Document the actual failure channel, or add a structured result artifact before promising this contract.</violation>

<violation number="2" location="ios-qa/executor/README.md:22">
P2: Clients following this output contract may assume fields that are absent for JSON/file errors and flow-version or target-kind failures. Describe `remediation` and `stepId` as conditional, matching the actual result variants.</violation>
</file>

<file name="ios-qa/executor/contract.ts">

<violation number="1" location="ios-qa/executor/contract.ts:25">
P2: `labelEquals.timeoutMs` does not wait for the label to reach the expected value; it only bounds element lookup. Post-action checks can therefore fail on normal asynchronous UI updates, so the runner should poll the label until the timeout.</violation>

<violation number="2" location="ios-qa/executor/contract.ts:31">
P3: Selector-based swipes cannot configure their resolution timeout even though the runner supports it. Adding `timeoutMs?: number` keeps swipe behavior consistent with the other selector-based actions.</violation>

<violation number="3" location="ios-qa/executor/contract.ts:37">
P1: A flow without `bundleIdentifier` is accepted as `ready` but the XCUITest runner always blocks it at execution time. Make this field required and validate malformed JSON in `buildXCUITestPlan()` so readiness reflects executability.</violation>
</file>

<file name="ios-qa/executor/xcuitest-plan.ts">

<violation number="1" location="ios-qa/executor/xcuitest-plan.ts:20">
P3: `selectorCandidates` is dead in production, while the Swift runner duplicates its ordering independently, so the two contracts can silently drift. Remove the unused export/documentation claim or add a shared conformance mechanism that actually governs runner behavior.</violation>

<violation number="2" location="ios-qa/executor/xcuitest-plan.ts:53">
P0: Physical-device plans now bypass the repository's required DebugBridge/CoreDevice harness by routing `device` targets through XCUITest. Limit this backend to simulators and route physical devices through the existing harness.</violation>

<violation number="3" location="ios-qa/executor/xcuitest-plan.ts:56">
P1: `buildXCUITestPlan` crashes with a `TypeError` instead of returning a clean validation error when the JSON input is missing `flow.name`, `target.udid`, `config.scheme`, or `config.testIdentifier`. The function calls `.trim()` on these values assuming they're strings, but `JSON.parse` can produce `undefined` or `null` for missing/omitted fields. Since the CLI reads JSON files directly, a malformed flow, target, or runner config will throw an uncaught runtime exception instead of surfacing a useful `blocked`/`unsupported` result.

Guard each `.trim()` call with a truthiness check (e.g. `if (!flow.name || !flow.name.trim())`) so the function returns a structured error instead of crashing.</violation>

<violation number="4" location="ios-qa/executor/xcuitest-plan.ts:59">
P3: A whitespace-only project or workspace path is treated as configured, producing a `ready` plan that `xcodebuild` cannot use. Apply the same trimmed-empty validation used for the other required strings.</violation>

<violation number="5" location="ios-qa/executor/xcuitest-plan.ts:67">
P2: Swipe step `direction` values are not validated. A JSON flow like `{"id":"s","action":"swipe","direction":"diagonal"}` passes `buildXCUITestPlan` without error and is embedded in the plan. The downstream XCUITest runner would receive an unhandled direction value. Since the flow is JSON-loaded, there's no compile-time protection against typos like `"diagonal"`, `"vertical"`, or `""`.

Add a check inside the step-validation loop for swipe steps: if `step.action === 'swipe'`, verify `step.direction` is one of the four allowed values and return `unsupported` if not.</violation>

<violation number="6" location="ios-qa/executor/xcuitest-plan.ts:71">
P1: Malformed JSON can escape the structured `PlanResult` contract or be marked `ready` because required action/verification fields are trusted from TypeScript types after unchecked JSON casts. Add runtime schema validation before accessing fields or constructing the plan.</violation>

<violation number="7" location="ios-qa/executor/xcuitest-plan.ts:89">
P1: Test credentials can be exposed in process arguments and plan logs because the complete flow, including `typeText` values, is only base64-encoded. Use a secret-safe runtime transport or references resolved at execution, and redact payloads from emitted plans.</violation>
</file>

<file name="test/fixtures/ios-qa/FixtureApp/project.yml">

<violation number="1" location="test/fixtures/ios-qa/FixtureApp/project.yml:66">
P1: The adaptive scheme disables code signing for both apps and the UI-test bundle, so selecting a physical-device destination cannot install or execute these fixtures. The scheme needs to retain automatic/device signing (and use the available team/provisioning setup) while leaving simulator-only unsigned builds as an explicit override if desired.</violation>
</file>

<file name="test/fixtures/ios-qa/FixtureApp/Tests/AdaptiveFixtureUITests/GStackFlowRunnerUITests.swift">

<violation number="1" location="test/fixtures/ios-qa/FixtureApp/Tests/AdaptiveFixtureUITests/GStackFlowRunnerUITests.swift:132">
P2: Selector-based swipes do not honor the runner's hittability wait, so nested/off-screen scroll actions can fail even when the element becomes interactable within `timeoutMs`. Waiting for the resolved element before dispatching the direction-specific swipe would make swipe behavior consistent with tap and typeText.</violation>

<violation number="2" location="test/fixtures/ios-qa/FixtureApp/Tests/AdaptiveFixtureUITests/GStackFlowRunnerUITests.swift:189">
P2: An invalid JSON role is silently treated as `any`, which can make a label-based selector tap the wrong accessibility element. Reject roles outside the contract allowlist as unsupported rather than widening the query.</violation>

<violation number="3" location="test/fixtures/ios-qa/FixtureApp/Tests/AdaptiveFixtureUITests/GStackFlowRunnerUITests.swift:202">
P2: The central nested-switch interaction path is not covered by the new fixtures: neither fixture exposes a switch and no UI test invokes `tapSwitch`. Adding a SwiftUI full-width Toggle plus a flow/test that asserts its value changes would make the advertised adaptive retry and false-positive protection regression-tested.</violation>

<violation number="4" location="test/fixtures/ios-qa/FixtureApp/Tests/AdaptiveFixtureUITests/GStackFlowRunnerUITests.swift:203">
P2: A switch with an unavailable accessibility value is treated as a successful tap because verification is skipped when `before` is nil. The runner should fail the step as unverifiable (or otherwise require an observable value) instead of reporting success without checking the outcome.</violation>
</file>

<sub>Tip: instead of fixing issues one by one fix them all with cubic

Re-trigger cubic</sub>

[cubic-dev-ai]

P0: Physical-device plans now bypass the repository's required DebugBridge/CoreDevice harness by routing device targets through XCUITest. Limit this backend to simulators and route physical devices through the existing harness.

Prompt for AI agents

Check if this issue is valid — if so, understand the root cause and fix it. At ios-qa/executor/xcuitest-plan.ts, line 53:

<comment>Physical-device plans now bypass the repository's required DebugBridge/CoreDevice harness by routing `device` targets through XCUITest. Limit this backend to simulators and route physical devices through the existing harness.</comment>

<file context>
@@ -0,0 +1,113 @@
+  planPath: string,
+): PlanResult {
+  if (flow.version !== 1) return { status: 'unsupported', reason: `flow version ${String(flow.version)} is unsupported` };
+  if (target.kind !== 'simulator' && target.kind !== 'device') {
+    return { status: 'unsupported', reason: `target kind ${String(target.kind)} is unsupported` };
+  }
</file context>

[cubic-dev-ai]

P1: A flow without bundleIdentifier is accepted as ready but the XCUITest runner always blocks it at execution time. Make this field required and validate malformed JSON in buildXCUITestPlan() so readiness reflects executability.

Prompt for AI agents

Check if this issue is valid — if so, understand the root cause and fix it. At ios-qa/executor/contract.ts, line 37:

<comment>A flow without `bundleIdentifier` is accepted as `ready` but the XCUITest runner always blocks it at execution time. Make this field required and validate malformed JSON in `buildXCUITestPlan()` so readiness reflects executability.</comment>

<file context>
@@ -0,0 +1,77 @@
+export interface IOSQAFlow {
+  version: 1;
+  name: string;
+  bundleIdentifier?: string;
+  steps: FlowStep[];
+}
</file context>

[cubic-dev-ai]

P1: Test credentials can be exposed in process arguments and plan logs because the complete flow, including typeText values, is only base64-encoded. Use a secret-safe runtime transport or references resolved at execution, and redact payloads from emitted plans.

Prompt for AI agents

Check if this issue is valid — if so, understand the root cause and fix it. At ios-qa/executor/xcuitest-plan.ts, line 89:

<comment>Test credentials can be exposed in process arguments and plan logs because the complete flow, including `typeText` values, is only base64-encoded. Use a secret-safe runtime transport or references resolved at execution, and redact payloads from emitted plans.</comment>

<file context>
@@ -0,0 +1,113 @@
+  args.push('-scheme', config.scheme, '-destination', `platform=${destinationPlatform},id=${target.udid}`, `-only-testing:${config.testIdentifier}`);
+  args.push(
+    `GSTACK_IOS_QA_FLOW_PATH_VALUE=${resolve(planPath)}`,
+    `GSTACK_IOS_QA_FLOW_JSON_BASE64_VALUE=${encodedFlow}`,
+    `GSTACK_IOS_QA_TARGET_KIND_VALUE=${target.kind}`,
+  );
</file context>

[cubic-dev-ai]

P1: Malformed JSON can escape the structured PlanResult contract or be marked ready because required action/verification fields are trusted from TypeScript types after unchecked JSON casts. Add runtime schema validation before accessing fields or constructing the plan.

Prompt for AI agents

Check if this issue is valid — if so, understand the root cause and fix it. At ios-qa/executor/xcuitest-plan.ts, line 71:

<comment>Malformed JSON can escape the structured `PlanResult` contract or be marked `ready` because required action/verification fields are trusted from TypeScript types after unchecked JSON casts. Add runtime schema validation before accessing fields or constructing the plan.</comment>

<file context>
@@ -0,0 +1,113 @@
+      return { status: 'unsupported', reason: `action ${String(step.action)} is unsupported`, stepId: step.id };
+    }
+    const selectors: ElementSelector[] = [];
+    if ('selector' in step && step.selector) selectors.push(step.selector);
+    const verification = 'verify' in step ? step.verify : step.action === 'wait' ? step.verification : undefined;
+    if (verification) selectors.push(verification.selector);
</file context>

[cubic-dev-ai]

P1: The adaptive scheme disables code signing for both apps and the UI-test bundle, so selecting a physical-device destination cannot install or execute these fixtures. The scheme needs to retain automatic/device signing (and use the available team/provisioning setup) while leaving simulator-only unsigned builds as an explicit override if desired.

Prompt for AI agents

Check if this issue is valid — if so, understand the root cause and fix it. At test/fixtures/ios-qa/FixtureApp/project.yml, line 66:

<comment>The adaptive scheme disables code signing for both apps and the UI-test bundle, so selecting a physical-device destination cannot install or execute these fixtures. The scheme needs to retain automatic/device signing (and use the available team/provisioning setup) while leaving simulator-only unsigned builds as an explicit override if desired.</comment>

<file context>
@@ -47,3 +47,73 @@ targets:
+    settings:
+      base:
+        PRODUCT_BUNDLE_IDENTIFIER: com.gstack.iosqa.adaptive.swiftui
+        CODE_SIGNING_ALLOWED: NO
+        TARGETED_DEVICE_FAMILY: "1"
+        SWIFT_VERSION: "5.9"
</file context>

[cubic-dev-ai]

P2: A switch with an unavailable accessibility value is treated as a successful tap because verification is skipped when before is nil. The runner should fail the step as unverifiable (or otherwise require an observable value) instead of reporting success without checking the outcome.

Prompt for AI agents

Check if this issue is valid — if so, understand the root cause and fix it. At test/fixtures/ios-qa/FixtureApp/Tests/AdaptiveFixtureUITests/GStackFlowRunnerUITests.swift, line 203:

<comment>A switch with an unavailable accessibility value is treated as a successful tap because verification is skipped when `before` is nil. The runner should fail the step as unverifiable (or otherwise require an observable value) instead of reporting success without checking the outcome.</comment>

<file context>
@@ -0,0 +1,263 @@
+    /// change, that's an interaction failure, reported as such rather than as a
+    /// confirmed product defect.
+    private func tapSwitch(_ element: XCUIElement, stepID: String) throws {
+        let before = element.value as? String
+        element.tap()
+        if let before, settledValue(of: element) == before {
</file context>

[cubic-dev-ai]

P2: Selector-based swipes do not honor the runner's hittability wait, so nested/off-screen scroll actions can fail even when the element becomes interactable within timeoutMs. Waiting for the resolved element before dispatching the direction-specific swipe would make swipe behavior consistent with tap and typeText.

Prompt for AI agents

Check if this issue is valid — if so, understand the root cause and fix it. At test/fixtures/ios-qa/FixtureApp/Tests/AdaptiveFixtureUITests/GStackFlowRunnerUITests.swift, line 132:

<comment>Selector-based swipes do not honor the runner's hittability wait, so nested/off-screen scroll actions can fail even when the element becomes interactable within `timeoutMs`. Waiting for the resolved element before dispatching the direction-specific swipe would make swipe behavior consistent with tap and typeText.</comment>

<file context>
@@ -0,0 +1,263 @@
+                }
+                element.typeText(text)
+            case "swipe":
+                let element = try step.selector.map { try resolve($0, in: app, timeoutMs: step.timeoutMs) } ?? app
+                switch step.direction {
+                case "up": element.swipeUp()
</file context>

[cubic-dev-ai]

P3: Selector-based swipes cannot configure their resolution timeout even though the runner supports it. Adding timeoutMs?: number keeps swipe behavior consistent with the other selector-based actions.

Prompt for AI agents

Check if this issue is valid — if so, understand the root cause and fix it. At ios-qa/executor/contract.ts, line 31:

<comment>Selector-based swipes cannot configure their resolution timeout even though the runner supports it. Adding `timeoutMs?: number` keeps swipe behavior consistent with the other selector-based actions.</comment>

<file context>
@@ -0,0 +1,77 @@
+  | { id: string; action: 'launch'; arguments?: string[]; environment?: Record<string, string>; verify?: Verification }
+  | { id: string; action: 'tap'; selector: ElementSelector; timeoutMs?: number; verify?: Verification }
+  | { id: string; action: 'typeText'; selector: ElementSelector; text: string; clear?: boolean; timeoutMs?: number; verify?: Verification }
+  | { id: string; action: 'swipe'; direction: 'up' | 'down' | 'left' | 'right'; selector?: ElementSelector; verify?: Verification }
+  | { id: string; action: 'wait'; verification: Verification };
+
</file context>

[cubic-dev-ai]

P3: A whitespace-only project or workspace path is treated as configured, producing a ready plan that xcodebuild cannot use. Apply the same trimmed-empty validation used for the other required strings.

Prompt for AI agents

Check if this issue is valid — if so, understand the root cause and fix it. At ios-qa/executor/xcuitest-plan.ts, line 59:

<comment>A whitespace-only project or workspace path is treated as configured, producing a `ready` plan that `xcodebuild` cannot use. Apply the same trimmed-empty validation used for the other required strings.</comment>

<file context>
@@ -0,0 +1,113 @@
+  if (!flow.name.trim()) return { status: 'blocked', reason: 'flow name is empty', remediation: 'Give the flow a stable name.' };
+  if (!flow.steps.length) return { status: 'blocked', reason: 'flow has no steps', remediation: 'Add at least one semantic action or verification.' };
+  if (!target.udid.trim()) return { status: 'blocked', reason: `${target.kind} UDID is missing`, remediation: `Select a booted ${target.kind} and pass its UDID.` };
+  if (!!config.projectPath === !!config.workspacePath) {
+    return { status: 'blocked', reason: 'runner needs exactly one projectPath or workspacePath', remediation: 'Point to the existing XCUITest runner project or workspace.' };
+  }
</file context>

Suggested change

	if (!!config.projectPath === !!config.workspacePath) {
	if (!!config.projectPath?.trim() === !!config.workspacePath?.trim()) {

[cubic-dev-ai]

P3: selectorCandidates is dead in production, while the Swift runner duplicates its ordering independently, so the two contracts can silently drift. Remove the unused export/documentation claim or add a shared conformance mechanism that actually governs runner behavior.

Prompt for AI agents

Check if this issue is valid — if so, understand the root cause and fix it. At ios-qa/executor/xcuitest-plan.ts, line 20:

<comment>`selectorCandidates` is dead in production, while the Swift runner duplicates its ordering independently, so the two contracts can silently drift. Remove the unused export/documentation claim or add a shared conformance mechanism that actually governs runner behavior.</comment>

<file context>
@@ -0,0 +1,113 @@
+}
+
+/** Ordered queries for the XCUITest runner. Never lets a human label outrank a stable id. */
+export function selectorCandidates(selector: ElementSelector): SelectorCandidate[] {
+  const candidates: SelectorCandidate[] = [];
+  if (selector.identifier) candidates.push({ strategy: 'identifier', value: selector.identifier, role: selector.role });
</file context>