RA11y iOS: games expansion, Crystal Resonance, VoiceOver & screenshot pipeline#1
Merged
Merged
Conversation
Add new art assets (dungeon rooms, enchanter relics, rogue seals, etc.) and corresponding asset catalogs; rename dungeon_room_entrance to dungeon_descent_bg and remove the old file. Remove Fastlane screenshot coverage section from AGENTS.md. Update memlog/requirements to include the new design assets. Refactor iOS app startup and routing: remove the AppRoute.game case and related gameDestination, simplify iOSRootView by owning UserDefaultsStorageComponent directly, keep iOSHubView as the stable NavigationStack root (use a translucent overlay ProgressView while resolving the initial route), and remove the previous hubViewModel init wiring. Update Xcode project to drop the RA11y-tvOS project reference. Misc updates to UI, design tokens, Hub/Quest/VoiceOver files, tests and localization to match these changes.
Add deterministic onboarding control and more robust screenshot extraction. - Add a new launch argument `-screenshotMarkOnboardingComplete` and apply it in @main to set `basicsCompleted = true`, allowing the app to route directly to the hub regardless of simulator state. - Update UI test `testScreenshots_Hub_VORequired` to include the new launch argument so the hub is deterministic even after `testScreenshots_FirstRun` clears onboarding flags. - Harden Fastlane `screenshots` lane: don't abort on a non-zero xcodebuild exit, always attempt to extract an xcresult bundle if produced, validate presence of PNGs, copy found screenshots to docs/screenshots, and emit clearer logs when xcodebuild or extraction partially fail. - Add new screenshot assets (VORequired and FirstRun variants) and update existing Hub PNGs for iPad and iPhone sizes. These changes ensure reliable, repeatable screenshot captures and allow partial test failures to still yield usable screenshots for documentation.
Apply a fixed dark color scheme to the hub and quest card views so semantic adaptive colors (.primary, .secondary, nav title, footer material, etc.) resolve to dark-mode (white-based) values on the app's fixed-dark surfaces. iOSHubView now sets .environment(\.colorScheme, .dark) scoped to the hub subtree (does not affect NavigationStack destinations), and QuestCardButtonStyle forces the same on each card. Comments clarify interaction with Increase Contrast and Reduce Motion. Also updates related screenshots and Xcode user interface state.
Implements The Enchanter's Trial (Find & Focus) as a full M5 feature: adds a new iOSEnchantersTrialView and EnchanterTrialViewModel to manage the 4-level game flow (prologue, attempt, rising, timed), timers, VoiceOver announcements, session lifecycle, and result synthesis. Introduces EnchanterRelic model, shared subviews (relic button, timer HUD, prompt/status rows), and UI polish for L0–L3. Updates navigation: adds AppRoute.enchantersTrial, threads an optional gameSpecificAnnouncement into AppRoute.gameResult, and wires restartGame to push the correct game route after viewing results. Adds a debug-only screenshot direct-route hook (applyScreenshotDirectRouteIfNeeded) to support fastlane screenshot passes. Also updates core scoring-related types/tests and adds new screenshots for the Enchanter prologue across device sizes. Misc: accessibility improvements, deterministic UI-testing behavior for relic selection, and various supporting view/layout adjustments.
Add a new Dungeon Descent (Scroll Hunt) starter implementation for iOS: UI, assets, scroll-practice zone, room rows, target reachability, session lifecycle, activation handlers and result routing. Wire the new route into AppRoute, iOSRootView and the Hub so the game can be launched from the hub and from screenshot args. Improve Enchanter's Trial view model and views: announce target prompt on view appear, refactor timer/task lifecycle and coordinator monitoring for safer cancellation, apply time-bucket mistake recording on L3 completion, and adjust RelicImage rendering to blend sprites on a dark surface. Update Localizable.xcstrings formatting and add accessibility/localization strings; update tests/snapshots. Update Gemfile.lock and other minor housekeeping changes.
Introduce The Rogue's Gauntlet (M6): add iOSRogueGauntletView with a full View + RogueGauntletViewModel (timers, session/coordinator, VO announcements, seal sets, L0–L3 flows). Wire up routing and screenshot CLI flags in iOSAppRouter/iOSRootView to navigate to the new rogue-gauntlet game and push game result routes. Polish and bugfixes across existing games/UI: - DungeonDescent: track visibleRect, fix reachable calculation via onScrollGeometryChange, adjust row layout (line limits/fixedSize), padding/frame tweaks. - Enchanter views: add accessibility identifiers for levels and relics, replace translucent materials with darker color backgrounds, and unify foreground colors for better contrast; tweak RelicImage (background, padding, overlay stroke). Also update tests, screenshots/docs, Fastlane/script changes and a few minor project metadata edits.
Move Dungeon Descent logic into a DungeonDescentViewModel and split the large view into smaller components (DungeonPrologueView, DungeonPlayView, DungeonTimerHUD, etc.). Add robust timer, VoiceOver monitoring, GameSession lifecycle handling, scroll observability, and room/level state management in the view model; adapt navigation to push results and interstitials from model state. Update iOSRootView to pre-populate the navigation router for fastlane screenshot modes (handle -screenshotDirectTo* args in @State init and remove runtime push helper). Rename coordinate space / preference keys, improve accessibility labels/hints and localized strings, and apply various UI/layout tweaks for the prologue and play screens. Also add Observation/os imports and adjust related test/screenshot assets and metadata.
Introduce a deterministic screenshot contract and boot path for fastlane/UI tests: add iOSScreenshotScene, iOSScreenshotRootView, a validation script, and a route catalog. Update App routing to include GameKind for gameResult and propagate screenshotScene into game container views and view models so automated screenshots render stable, sample results. Implement deterministic applyScreenshotScene logic and onDisappear handlers to stop timers/abandon sessions to avoid persistence during capture. Add design tokens and numerous UI/accessibility improvements: adaptive layouts, Dynamic Type support, @ScaledMetric sizing, semantic colors, localized strings and accessibility labels/hints. Also update tests, fastlane config, screenshots, and related resources to match the new screenshot automation contract.
Replace ZStack backgrounds with .background { } in Enchanter, Dungeon, and Rogue game views to avoid layout bleed from scaled background images and keep background sizing tied to foreground content. Add .preferredColorScheme(.dark) and move navigation/onChange/onDisappear modifiers alongside the foreground. Add .transition(.identity) to each phase case to suppress SwiftUI's default fade (prevents mid-transition screenshots). Update localization for "dungeon.l1.title" and include updated screenshots and workspace UI state.
Introduce Lights Off gameplay support and improve the guided Basics flow and accessibility. Add a blackout ViewModifier (ra11yLightsOffGameplayBlackout) and propagate a Lights Off flag into game containers and view models to randomize layouts, preserve deterministic test launches, and apply VO-friendly grace delays. Enhance the Basics sequence with router flags (isInBasicsSequence, basicsStepCompleted) and popForBasicsContinue() to reliably advance or finish the M4 onboarding flow. Improve root/loading behavior with VoiceOver announcements and a slow-loading announcement task, plus numerous responsive layout and accessibility label/hint refinements across FirstRun, Enchanter, Dungeon, and other views. Update AGENTS.md to persist and reuse last-working simulator UDIDs and add small tooling/scripts and test adjustments.
Introduce BasicsProgressSnapshot and batch storage APIs to reduce cross-actor hops: added basicsProgressSnapshot() and bestResults(for:) to StorageComponent and implemented them in UserDefaults and in-memory test helpers. Update HubViewModel to use the batched bestResults read for a single storage hop. Change startup routing to use the snapshot: iOSAppRouter now resolves initial route from basicsProgressSnapshot, and iOSRootView defers constructing the full hub until the initial route is resolved (shows a lightweight loading view). Make iOSHubView injectable with a storage parameter and shouldRefreshOnAppear flag to avoid refreshing during startup. Update tests to conform to the new APIs. Also include localization reordering, small layout/frame alignment fixes in Dungeon views and Hub, an asset update + backup, and add design/ADR docs for the Dungeon resonance redesign.
Introduce Crystal Resonance (formerly referenced as Dungeon/Scroll Hunt) prototype and supporting infrastructure: add new mockup route & screenshot scene, wire the mockup into root/router views, and reorder the FirstRun basics sequence to hub unlock order. Add a small iOS feedback subsystem (audio renderer, haptic renderer, coordinator, and settings) and new RA11yCore feedback types/reducer/profile with unit tests. Add numerous resonance-related image assets and updated app icon variants, and update Xcode project settings (supported interface orientations and Swift/tools version bump). These changes enable iterative design of the Crystal Resonance experience and deliver semantic haptic/audio feedback plumbing for quests.
Introduce Crystal Resonance UI and alignment system: add iOSLaunchLoadingView and a new iOSDungeonResonanceComponents file with alignment math, presentation bands, assets, and SwiftUI components (orb, reticle, lane elements, backgrounds). Integrate resonance play view into iOSDungeonDescentView (replace legacy DungeonPlayView usage), and update DungeonDescentViewModel to compute resonance deltas, manage QuestFeedbackCoordinator, reset alignment state on phase changes, and emit feedback bands. Remove large legacy play/view plumbing in favor of the new resonance components. Also add agent policies (.claude/settings.local.json and AGENTS.md updates), tweak localizable strings, logging, workspace user state, and miscellaneous docs/memlog updates.
Introduce centralized VoiceOver gating for launching games via iOSAppRouter.pushGame(kind:), with a UI-test (-uiTesting) bypass. Replace ad-hoc route pushes across root and hub views to use pushGame so quests are never entered without VoiceOver in production. Refactor iOSDungeonResonancePlayView to use a VoiceOver scroll-proxy (transparent ScrollView) driving a non-scrolling visual lane, add a PreferenceKey for aim-line midY and alignment logic, remove a root GeometryReader that created extra VoiceOver frames, and improve accessibility ordering, hints and in-game tip copy. Add investigation notes (memlog) and link from ADR-0003 about moving to a UIKit UIScrollView if SwiftUI proves unreliable.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
RA11y iOS: games expansion, Crystal Resonance, VoiceOver & screenshot pipeline
—
Summary
This branch delivers a substantial slice of the iOS product: three training games with updated art and hub integration, Crystal Resonance (scroll-hunt / alignment) gameplay with multimodal feedback, VoiceOver-first routing and messaging, Lights Off and basics-flow support where wired, a deterministic screenshot / UI-test harness (-screenshotScene, -uiTesting), and RA11yCore updates (hub, storage, quest feedback, scoring tests). Repository agent/build docs (AGENTS.md), Fastlane, and utility scripts are aligned with dynamic simulator selection and screenshot validation.
User-visible behavior
Hub & navigation: Quest cards, routing, and loading behavior updated for current game set and flows.
Games: Enchanter’s Trial, Rogue’s Gauntlet, and Dungeon Descent / Crystal Resonance with resonance alignment (orb / reticle, lane, seal when aligned), timers on later levels, and result flow.
VoiceOver: Starting a quest still expects VoiceOver for real users; UI tests and automation can pass -uiTesting so game entry does not require VoiceOver on the simulator.
Crystal Resonance / three-finger scroll: The playable scroll proxy is a UIKit UIScrollView bridged via iOSResonanceVoiceOverScrollProxyRepresentable, replacing the SwiftUI-only scroll proxy so VoiceOver can focus and scroll the lane reliably; programmatic focus uses screenChanged → layoutChanged (scroll view) → announcement.
Screenshots: Regenerated / extended iPad and iPhone doc screenshots; catalog and Fastlane allowlists updated where required.
Technical notes
iOS: Notable files include iOSAppRouter.swift (including -uiTesting bypass for pushGame), iOSDungeonDescentView.swift / iOSDungeonResonancePlayView.swift, iOSResonanceVoiceOverScrollProxyRepresentable.swift, hub/first-run/results/VoiceOver views, Localizable.xcstrings, and asset catalog additions.
RA11yCore: Hub view model, storage batching, game session coordinator, quest feedback types/reducer, logging, scoring tests—see package tests for behavior.
Tooling: utility/build_and_test.sh, utility/validate_screenshot_contract.sh, utility/ExtractScreenshots.swift, utility/ValidateScreenshotContract.swift, Fastlane/Snapfile, resolve_simulator_udid.jq (simulator discovery per AGENTS.md).
Documentation / memlog: DirectoryTree, requirements and design notes, ADR-0003 (resonance design), CrystalResonance-VoiceOverScrollProxy-Investigation.md (history of SwiftUI issues + UIKit follow-up).
Testing / merge checklist
utility/build_and_test.sh (or Core + iOS as you usually run).
If screenshot routes or anchors changed: utility/validate_screenshot_contract.sh.
Smoke VoiceOver on device: hub → Dungeon / Crystal Resonance → three-finger scroll + seal when aligned.
Optional: confirm no unwanted tracked files (e.g. IDE xcuserstate, .bak assets, local .claude config) if those are not meant for the repo.
Follow-ups (not blocking this PR)
UI/copy polish on Crystal Resonance and related strings.
Any screenshot re-runs after final polish.
Continued tightening of navigation / focus order if further VoiceOver tuning is needed outside the scroll proxy.
You can shorten the “Summary” for reviewers who only care about the latest commit by adding a one-line callout at the top:
Latest: VoiceOver scroll for Crystal Resonance now uses a UIKit UIScrollView representable; confirmed working on device.