Stabilize browser test route bootstrap

Author: KSemenenko

AGENTS.md

@@ -115,6 +115,7 @@ Rule format:
 - Shared test-support libraries that contain no runnable test cases must not reference the TUnit engine package directly; keep them on non-engine TUnit packages so solution-level `dotnet test` does not discover zero-test support DLLs as runnable test apps.
 - Every runnable test project must declare `MaxParallelTestsForPipeline : EnvironmentAwareParallelLimitBase` and rely on the base class limits by default; do not add per-suite `LocalLimit` or `CiLimit` overrides unless a specific suite has a documented exception.
 - Browser-suite CI parallelism is user-tunable. When the user wants higher throughput, keep in-suite browser-test CI parallelism in the `6-9` worker range when the harness can sustain it, and prefer extra GitHub Actions job splitting on top of that before reaching for timeout increases; do not pin browser suites back to `2` workers as a blanket fix.
+- Browser UI tests must be data-isolated and storage-isolated by default: each scenario should create or import its own script or workspace data, avoid shared persisted document IDs or browser-local state, and remain parallel-safe when multiple pages or contexts hit the app at the same time.
 - Local regression verification must include solution-level `dotnet test --solution ./PrompterOne.slnx --max-parallel-test-modules 1` so test-project split changes are proven under the real all-tests entrypoint, not only as isolated per-project runs.
 - When the user explicitly asks to validate a test fix in actual GitHub Actions, do not spend more time on local `CI=true` emulation; push the fix and monitor the real CI run instead.
 - Do not use local `CI=true` browser-suite runs as a primary reproduction or validation signal unless the user explicitly asks for that exact experiment; in this repo it mostly changes in-suite parallelism and can hide the real CI failure mode instead of reproducing it.

full-suite-stabilization.plan.md

@@ -88,13 +88,13 @@ Tracked failing tests from the current baseline:
   - the studio suite is hitting the same cold-start bootstrap pressure as `Shell`, but the failures fan out through GoLive/media helpers because those helpers seed state only after the first shell route becomes interactive
   Fix path:
   - reuse the shared-runtime warmup so the first library/settings/go-live routes are already booted before studio scenarios start seeding or asserting
-- [ ] `Release Pipeline` run `24203100281` still reports cold fresh-context bootstrap failures after the shared-runtime warmup
+- [x] `Release Pipeline` run `24203100281` still reports cold fresh-context bootstrap failures after the shared-runtime warmup
   Symptom:
   - `Shell` and `Studio` still fail on the first visible page assertion for a newly created browser context, and editor-only scenarios can still hit context/page closure while opening the first routed page inside a fresh context
   Root cause:
-  - the one-time shared-runtime warmup removed process-level cold start, but each brand-new Playwright browser context on CI still pays its own first-route browser bootstrap cost before the test scenario begins
+  - the earlier hypothesis was incomplete; the added CI-only sacrificial per-context warmup made CI bootstrap behavior diverge from local runs and added extra routed app startup work before every returned test page
   Fix path:
-  - add a CI-only per-context warmup in the browser harness, then reset browser storage again so each returned page is both warmed and isolated
+  - remove the CI-only per-context routed warmup, keep only the one-time shared-runtime warmup, and make every returned test page follow the same bootstrap path locally and on CI
 - [ ] `StandaloneAppFixture` build break: `WarmUpContextPageIfNeededAsync` missing in `StandaloneAppFixture.cs`
   Symptom:
   - compiler reports `CS0103` for `WarmUpContextPageIfNeededAsync` from `tests/PrompterOne.Web.UITests/Infrastructure/StandaloneAppFixture.cs`
@@ -114,6 +114,18 @@ Tracked failing tests from the current baseline:
   Fix path:
   - preserve and inspect CI screenshot artifacts for each failing run, then harden the fresh-context bootstrap so local and CI both receive the same blank primed page
   - remove mutable library/settings seeding from `AddInitScript`, keep explicit isolated reset+seed only, and make blocked IndexedDB reset fail fast instead of silently succeeding
+- [ ] `Release Pipeline` run `24218745809` fails in `Shell` (`27` tests) and `Studio` (`5` tests) after the direct `main` push on commit `4bea2cd`
+  Symptom:
+  - `Shell` now fails broadly on the very first routed page visibility checks for `library-page` and `settings-page`, with screenshots showing the shell UI eventually rendered after the assertion already timed out
+  - `Studio` fails on initial `library-page` visibility in go-live setup plus a smaller cluster of route-return clicks that stay on `/go-live?id=...`
+  Root cause:
+  - the CI-only sacrificial per-context warmup still makes every new context pay extra routed bootstrap work that local runs never execute
+  - `Shell` still opens many routes with raw `GotoAsync(...)` plus one-shot `ToBeVisibleAsync(...)` checks instead of the shared retrying route driver, so slower CI bootstrap falls straight into suite-wide flakes
+  - `Studio` route-open and go-live-return flows still rely on root-page readiness alone instead of a stronger page-ready contract before clicking or waiting for downstream session state
+  Fix path:
+  - remove the per-context routed warmup so local and CI use the same isolated-page bootstrap path
+  - introduce shared shell route-open helpers and migrate raw shell route opens onto `BrowserRouteDriver`
+  - strengthen go-live route readiness in the studio helpers before interaction-heavy assertions
 - [x] `Release Pipeline` run `24199073318` reports a `Reader` bootstrap cluster (`163` failing tests)
   Symptom:
   - learn, teleprompter, responsive, and route-visibility tests all fail at the initial `learn-page`, `teleprompter-page`, `settings-page`, `go-live-page`, `editor-page`, or `library-page` visibility gate
@@ -156,6 +168,13 @@ Tracked failing tests from the current baseline:
   - the test mixed a fixed wall-clock sleep with a CSS-delay contract, so slower solution-level execution could consume enough time between `HoverAsync` and the assertion for the tooltip fade-in to have already started
   Fix path:
   - assert the low-opacity state immediately after hover, then rely on the existing tooltip driver to wait for the real visible state instead of stacking extra fixed settle sleeps
+- [x] `Reader` route bootstrap/readiness hardening after the `24218745809` CI failures
+  Symptom:
+  - remote reader failures clustered around first-route `learn-page` visibility timeouts while the same commit stayed green locally
+  Root cause:
+  - the shared reader route helper treated the routed page root as "ready" too early, so slower CI could advance into shortcut/playback assertions before the interactive learn or teleprompter controls were actually present
+  Fix path:
+  - strengthen `ReaderRouteDriver` to wait for production-owned interactive sentinels (`learn` progress/play controls, teleprompter stage/play toggle, settings title) instead of stopping at the top-level page shell
 
 ## Ordered Plan
 
@@ -189,6 +208,10 @@ Tracked failing tests from the current baseline:
   - local build passes
   - local full test pass count is green across all projects
   - formatting completes cleanly
+  Result:
+  - `dotnet format ./PrompterOne.slnx` passed
+  - `dotnet build ./PrompterOne.slnx -warnaserror` passed
+  - `dotnet test @./tests/dotnet-test-progress.rsp --solution ./PrompterOne.slnx --max-parallel-test-modules 1` passed with `1162/1162` green in `7m 42.943s`
 
 - [ ] Step 5. Publish directly to `main`.
   Actions:

tests/PrompterOne.Web.UITests.Reader/Infrastructure/ReaderRouteDriver.cs

@@ -1,27 +1,37 @@
 using System.Runtime.CompilerServices;
 using Microsoft.Playwright;
 using PrompterOne.Shared.Contracts;
+using static Microsoft.Playwright.Assertions;
 
 namespace PrompterOne.Web.UITests;
 
 internal static class ReaderRouteDriver
 {
-    internal static Task OpenLearnAsync(
+    internal static async Task OpenLearnAsync(
         IPage page,
         string route,
-        [CallerMemberName] string scenarioName = "") =>
-        OpenAsync(page, route, UiTestIds.Learn.Page, scenarioName);
+        [CallerMemberName] string scenarioName = "")
+    {
+        await OpenAsync(page, route, UiTestIds.Learn.Page, scenarioName);
+        await WaitForLearnReadyAsync(page);
+    }
 
-    internal static Task OpenTeleprompterAsync(
+    internal static async Task OpenTeleprompterAsync(
         IPage page,
         string route,
-        [CallerMemberName] string scenarioName = "") =>
-        OpenAsync(page, route, UiTestIds.Teleprompter.Page, scenarioName);
+        [CallerMemberName] string scenarioName = "")
+    {
+        await OpenAsync(page, route, UiTestIds.Teleprompter.Page, scenarioName);
+        await WaitForTeleprompterReadyAsync(page);
+    }
 
-    internal static Task OpenSettingsAsync(
+    internal static async Task OpenSettingsAsync(
         IPage page,
-        [CallerMemberName] string scenarioName = "") =>
-        OpenAsync(page, BrowserTestConstants.Routes.Settings, UiTestIds.Settings.Page, scenarioName);
+        [CallerMemberName] string scenarioName = "")
+    {
+        await OpenAsync(page, BrowserTestConstants.Routes.Settings, UiTestIds.Settings.Page, scenarioName);
+        await WaitForSettingsReadyAsync(page);
+    }
 
     private static Task OpenAsync(
         IPage page,
@@ -33,4 +43,24 @@ private static Task OpenAsync(
             route,
             pageTestId,
             $"{scenarioName}-{pageTestId}");
+
+    private static async Task WaitForLearnReadyAsync(IPage page)
+    {
+        await Expect(page.GetByTestId(UiTestIds.Learn.Page)).ToBeVisibleAsync();
+        await Expect(page.GetByTestId(UiTestIds.Learn.ProgressLabel)).ToBeVisibleAsync();
+        await Expect(page.GetByTestId(UiTestIds.Learn.PlayToggle)).ToBeVisibleAsync();
+    }
+
+    private static async Task WaitForTeleprompterReadyAsync(IPage page)
+    {
+        await Expect(page.GetByTestId(UiTestIds.Teleprompter.Page)).ToBeVisibleAsync();
+        await Expect(page.GetByTestId(UiTestIds.Teleprompter.Stage)).ToBeVisibleAsync();
+        await Expect(page.GetByTestId(UiTestIds.Teleprompter.PlayToggle)).ToBeVisibleAsync();
+    }
+
+    private static async Task WaitForSettingsReadyAsync(IPage page)
+    {
+        await Expect(page.GetByTestId(UiTestIds.Settings.Page)).ToBeVisibleAsync();
+        await Expect(page.GetByTestId(UiTestIds.Settings.Title)).ToBeVisibleAsync();
+    }
 }

tests/PrompterOne.Web.UITests.Shell/AppShell/HeaderPlaybackLaunchFlowTests.cs

@@ -17,11 +17,7 @@ await page.SetViewportSizeAsync(
                 BrowserTestConstants.ResponsiveLayout.IphoneMediumWidth,
                 BrowserTestConstants.ResponsiveLayout.IphoneMediumHeight);
 
-            await BrowserRouteDriver.OpenPageAsync(
-                page,
-                BrowserTestConstants.Routes.Library,
-                UiTestIds.Library.Page,
-                nameof(LibraryCardPlaybackActions_OpenScopedReaderRoutes_AndHeaderBackReturnsToEditor));
+            await ShellRouteDriver.OpenLibraryAsync(page);
 
             await page.GetByTestId(UiTestIds.Library.CardLearn(BrowserTestConstants.Scripts.QuantumId)).ClickAsync();
             await BrowserRouteDriver.WaitForRouteAsync(page, BrowserTestConstants.Routes.LearnQuantum);
@@ -44,8 +40,7 @@ await UiScenarioArtifacts.CapturePageAsync(
                 BrowserTestConstants.AppShellFlow.LearnBackStep);
 
             await page.GetByTestId(UiTestIds.Header.Home).ClickAsync();
-            await BrowserRouteDriver.WaitForRouteAsync(page, AppRoutes.Library);
-            await Expect(page.GetByTestId(UiTestIds.Library.Page)).ToBeVisibleAsync();
+            await ShellRouteDriver.WaitForLibraryReadyAsync(page, AppRoutes.Library);
 
             await page.GetByTestId(UiTestIds.Library.CardRead(BrowserTestConstants.Scripts.QuantumId)).ClickAsync();
             await BrowserRouteDriver.WaitForRouteAsync(page, BrowserTestConstants.Routes.TeleprompterQuantum);
@@ -78,8 +73,7 @@ await page.SetViewportSizeAsync(
                 BrowserTestConstants.ResponsiveLayout.IphoneMediumWidth,
                 BrowserTestConstants.ResponsiveLayout.IphoneMediumHeight);
 
-            await page.GotoAsync(BrowserTestConstants.Routes.GoLiveDemo);
-            await Expect(page.GetByTestId(UiTestIds.GoLive.Page)).ToBeVisibleAsync();
+            await ShellRouteDriver.OpenGoLiveRouteAsync(page, BrowserTestConstants.Routes.GoLiveDemo);
 
             await page.GetByTestId(UiTestIds.GoLive.OpenSettings).ClickAsync();
             await BrowserRouteDriver.WaitForRouteAsync(page, BrowserTestConstants.Routes.Settings);
@@ -92,8 +86,7 @@ await UiScenarioArtifacts.CapturePageAsync(
                 BrowserTestConstants.AppShellFlow.SettingsLaunchStep);
 
             await page.GetByTestId(UiTestIds.Header.Back).ClickAsync();
-            await BrowserRouteDriver.WaitForRouteAsync(page, BrowserTestConstants.Routes.GoLiveDemo);
-            await Expect(page.GetByTestId(UiTestIds.GoLive.Page)).ToBeVisibleAsync();
+            await ShellRouteDriver.WaitForGoLiveReadyAsync(page, BrowserTestConstants.Routes.GoLiveDemo);
             await UiScenarioArtifacts.CapturePageAsync(
                 page,
                 BrowserTestConstants.AppShellFlow.SettingsOriginScenario,

tests/PrompterOne.Web.UITests.Shell/AppShell/NavigationFlowTests.cs

@@ -19,15 +19,13 @@ public async Task ShellHeader_OpensGoLive_FromLibraryAndSettings()
 
         try
         {
-            await page.GotoAsync(BrowserTestConstants.Routes.Library);
-            await Expect(page.GetByTestId(UiTestIds.Library.Page)).ToBeVisibleAsync();
+            await ShellRouteDriver.OpenLibraryAsync(page);
 
             await page.GetByTestId(UiTestIds.Header.GoLive).ClickAsync();
             await BrowserRouteDriver.WaitForRouteAsync(page, AppRoutes.GoLive);
             await Expect(page.GetByTestId(UiTestIds.GoLive.Page)).ToBeVisibleAsync();
 
-            await page.GotoAsync(BrowserTestConstants.Routes.Settings);
-            await Expect(page.GetByTestId(UiTestIds.Settings.Page)).ToBeVisibleAsync();
+            await ShellRouteDriver.OpenSettingsAsync(page);
 
             await page.GetByTestId(UiTestIds.Header.GoLive).ClickAsync();
             await BrowserRouteDriver.WaitForRouteAsync(page, AppRoutes.GoLive);
@@ -47,9 +45,10 @@ public async Task ScreenNavigation_UsesSpaRoutingWithoutReloadingBrowserContext(
 
         try
         {
-            await page.GotoAsync(BrowserTestConstants.Routes.EditorQuantum);
-            await Expect(page.GetByTestId(UiTestIds.Editor.Page))
-                .ToBeVisibleAsync(new() { Timeout = BrowserTestConstants.Timing.ExtendedVisibleTimeoutMs });
+            await EditorRouteDriver.OpenReadyAsync(
+                page,
+                BrowserTestConstants.Routes.EditorQuantum,
+                nameof(ScreenNavigation_UsesSpaRoutingWithoutReloadingBrowserContext));
 
             await page.EvaluateAsync("value => window.__prompterSpaNonce = value", nonce);
 
@@ -58,8 +57,10 @@ await Expect(page.GetByTestId(UiTestIds.Editor.Page))
             await Expect(page.GetByTestId(UiTestIds.Learn.Page)).ToBeVisibleAsync();
             await Assert.That(await page.EvaluateAsync<string>("() => window.__prompterSpaNonce")).IsEqualTo(nonce);
 
-            await page.GotoAsync(BrowserTestConstants.Routes.EditorQuantum);
-            await Expect(page.GetByTestId(UiTestIds.Editor.Page)).ToBeVisibleAsync();
+            await EditorRouteDriver.OpenReadyAsync(
+                page,
+                BrowserTestConstants.Routes.EditorQuantum,
+                $"{nameof(ScreenNavigation_UsesSpaRoutingWithoutReloadingBrowserContext)}-return");
             await page.EvaluateAsync("value => window.__prompterSpaNonce = value", nonce);
 
             await page.GetByTestId(UiTestIds.Header.EditorRead).ClickAsync();
@@ -85,15 +86,13 @@ public async Task ShellHeader_UsesConsistentNeutralGoLiveChrome_OnLibraryAndSett
 
         try
         {
-            await page.GotoAsync(BrowserTestConstants.Routes.Library);
-            await Expect(page.GetByTestId(UiTestIds.Library.Page)).ToBeVisibleAsync();
+            await ShellRouteDriver.OpenLibraryAsync(page);
             await Expect(page.GetByTestId(UiTestIds.Header.GoLive))
                 .ToHaveAttributeAsync(BrowserTestConstants.GoLive.LiveStateAttributeName, GoLiveIndicatorStates.Idle);
 
             var libraryChrome = await ReadGoLiveChromeAsync(page);
 
-            await page.GotoAsync(BrowserTestConstants.Routes.Settings);
-            await Expect(page.GetByTestId(UiTestIds.Settings.Page)).ToBeVisibleAsync();
+            await ShellRouteDriver.OpenSettingsAsync(page);
             await Expect(page.GetByTestId(UiTestIds.Header.GoLive))
                 .ToHaveAttributeAsync(BrowserTestConstants.GoLive.LiveStateAttributeName, GoLiveIndicatorStates.Idle);
 

tests/PrompterOne.Web.UITests.Shell/AppShell/OnboardingFlowTests.cs

@@ -17,7 +17,7 @@ public Task FirstRunOnboarding_WalksAcrossCoreProductRoutes_AndStaysDismissedAft
         {
             await SeedPendingOnboardingAsync(page);
 
-            await page.GotoAsync(BrowserTestConstants.Routes.Library);
+            await ShellRouteDriver.OpenLibraryAsync(page);
 
             await Expect(page.GetByTestId(UiTestIds.Onboarding.Surface)).ToBeVisibleAsync();
             await AssertOnboardingStepAsync(
@@ -73,7 +73,7 @@ public Task FirstRunOnboarding_DismissReturnsToLibrary_AndStaysHiddenAfterReload
         {
             await SeedPendingOnboardingAsync(page);
 
-            await page.GotoAsync(BrowserTestConstants.Routes.Library);
+            await ShellRouteDriver.OpenLibraryAsync(page);
 
             await Expect(page.GetByTestId(UiTestIds.Onboarding.Surface)).ToBeVisibleAsync();
 
@@ -91,7 +91,7 @@ public Task OnboardingReopen_FromSettings_ReturnsToLibraryWithOverlay() =>
         {
             await SeedCompletedOnboardingAsync(page);
 
-            await page.GotoAsync(BrowserTestConstants.Routes.Settings);
+            await ShellRouteDriver.OpenSettingsAsync(page);
             await page.GetByTestId(UiTestIds.Settings.NavAbout).ClickAsync();
 
             await page.GetByTestId(UiTestIds.Settings.AboutOnboardingRestart).ClickAsync();
@@ -113,7 +113,7 @@ public Task FirstRunOnboarding_UsesUkrainianBrowserCulture_ForLocalizedCopy() =>
             await SeedPendingOnboardingAsync(page);
             await page.AddInitScriptAsync(BrowserTestConstants.Localization.BuildNavigatorLanguagesInitScript("uk-UA", "en-US"));
 
-            await page.GotoAsync(BrowserTestConstants.Routes.Library);
+            await ShellRouteDriver.OpenLibraryAsync(page);
 
             await Expect(page.GetByTestId(UiTestIds.Onboarding.Title))
                 .ToHaveTextAsync(BrowserTestConstants.AppShellFlow.OnboardingUkrainianWelcomeTitle);

tests/PrompterOne.Web.UITests.Shell/AppShell/RuntimeTelemetryFlowTests.cs

@@ -2,7 +2,6 @@
 using System.Text.Json;
 using System.Text.Json.Serialization;
 using PrompterOne.Shared.Contracts;
-using static Microsoft.Playwright.Assertions;
 
 namespace PrompterOne.Web.UITests;
 
@@ -80,8 +79,7 @@ await route.FulfillAsync(new()
 
             await page.AddInitScriptAsync(UiTestHostConstants.RuntimeTelemetryAllowVendorLoadsScript);
 
-            await page.GotoAsync(BrowserTestConstants.Routes.Library);
-            await Expect(page.GetByTestId(UiTestIds.Library.Page)).ToBeVisibleAsync();
+            await ShellRouteDriver.OpenLibraryAsync(page);
             await WaitForTelemetryCollectionsAsync(
                 page,
                 BrowserTestConstants.Telemetry.MinimumInitializationCount,