Skip to content

feat(e2e): browser+Docker preflight in global-setup (PP-cao)#1284

Open
timothyfroehlich wants to merge 5 commits intomainfrom
feat/pp-cao-e2e-preflight
Open

feat(e2e): browser+Docker preflight in global-setup (PP-cao)#1284
timothyfroehlich wants to merge 5 commits intomainfrom
feat/pp-cao-e2e-preflight

Conversation

@timothyfroehlich
Copy link
Copy Markdown
Owner

Summary

Adds two fail-fast checks to e2e/global-setup.ts before the existing Supabase/Postgres pre-flight:

  1. checkBrowserBinaries(config) — inspects config.projects to derive needed browsers (chromium/firefox/webkit), verifies each binary exists via existsSync(executablePath()). On miss, throws with Install with: pnpm exec playwright install --with-deps <names>. Only checks browsers the active config actually needs, so CI's chromium-only install isn't penalized.
  2. checkDocker() — runs docker info via spawnSync (no shell, fixed argv) with a 5s timeout. On failure: Start OrbStack: open -a OrbStack.

Both checks run regardless of SKIP_SUPABASE_RESET — that flag only skips DB reset/migration/seed; browsers and the Docker daemon are still required to launch Playwright.

Replaces cryptic mid-run failures like browserType.launch: Executable doesn't exist with clear actionable messages before any spec runs.

Also updates the global-setup unit test to pass a mock FullConfig (empty projects array disables the browser check for tests focused on DB orchestration) and mocks spawnSync for the Docker check.

Inherited CI failures

main is currently red. Inherited failures from main (PP-q9r/PP-e20/PP-jsh/PP-v7g/PP-49m) will appear in CI; not introduced by this PR. See Tim's status comment on PR #1278.

Test plan

  • pnpm run check passes (1013 unit tests, including updated global-setup tests)
  • CI Gate green
  • Manual: rename ~/Library/Caches/ms-playwright/chromium-* and confirm fail-fast with install hint
  • Manual: stop Docker and confirm fail-fast with OrbStack hint

Related

🤖 Generated with Claude Code

Adds two fail-fast checks at the top of e2e/global-setup.ts before the
existing Supabase/Postgres pre-flight:

1. checkBrowserBinaries(config): inspects config.projects, derives the
   set of browsers needed (chromium/firefox/webkit), and verifies each
   binary exists via existsSync(executablePath()). On miss, throws with
   the actionable hint "Install with: pnpm exec playwright install
   --with-deps <names>". Only checks browsers the active config needs,
   so CI's chromium-only install isn't penalized.

2. checkDocker(): runs `docker info` via spawnSync (no shell, fixed argv)
   with a 5s timeout. On failure, throws with "Start OrbStack: open -a
   OrbStack" hint.

Both checks run regardless of SKIP_SUPABASE_RESET — that flag only skips
DB reset/migration/seed; browsers and the Docker daemon are still
required to launch Playwright.

Replaces cryptic mid-run failures like
  browserType.launch: Executable doesn't exist
with clear actionable messages before any spec runs.

Also updates the global-setup unit test to pass a mock FullConfig (empty
projects array disables the browser check for tests focused on DB
orchestration) and mocks spawnSync for the Docker check.

Refs PP-2on (epic).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Copilot AI review requested due to automatic review settings May 5, 2026 01:19
@vercel
Copy link
Copy Markdown

vercel Bot commented May 5, 2026

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated (UTC)
pin-point Ready Ready Preview, Comment May 5, 2026 8:32pm

@supabase
Copy link
Copy Markdown

supabase Bot commented May 5, 2026

This pull request has been ignored for the connected project udhesuizjsgxfeotqybn due to reaching the limit of concurrent preview branches.
Go to Project Integrations Settings ↗︎ if you wish to update this limit.


Preview Branches by Supabase.
Learn more about Supabase Branching ↗︎.

Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Adds fail-fast preflight checks to the Playwright E2E global setup so missing Playwright browser binaries or a down Docker daemon fail immediately with actionable errors (before the existing Supabase/Postgres checks and DB reset/migrations).

Changes:

  • Add checkBrowserBinaries(config) to verify required Playwright browser executables exist for the active config’s projects.
  • Add checkDocker() to verify docker info succeeds before attempting any Supabase/Postgres orchestration.
  • Update unit tests to call globalSetup(config) and mock spawnSync used by the Docker preflight.

Reviewed changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated 2 comments.

File Description
e2e/global-setup.ts Adds browser-binary and Docker-daemon preflight checks; updates globalSetup signature to accept Playwright FullConfig.
src/test/unit/global-setup.test.ts Adjusts unit tests for new globalSetup(config) signature and mocks spawnSync.
Comments suppressed due to low confidence (1)

src/test/unit/global-setup.test.ts:120

  • This test name/intent no longer matches the implementation: globalSetup now always runs the browser + Docker preflights before checking SKIP_SUPABASE_RESET. Either rename the test (and/or assert the preflights ran) so it reflects the current behavior.
  it("skips everything when SKIP_SUPABASE_RESET is set", async () => {
    process.env.SKIP_SUPABASE_RESET = "true";
    const setup = await loadSetup();

    await setup(EMPTY_CONFIG);

    expect(fetchMock).not.toHaveBeenCalled();
    expect(execSyncMock).not.toHaveBeenCalled();
  });

Comment thread src/test/unit/global-setup.test.ts
Comment thread e2e/global-setup.ts
timothyfroehlich and others added 2 commits May 5, 2026 15:02
…P-cao)

Branch checkDocker() error handling: ENOENT (not installed → install
hint) vs other errors (daemon down → start hint). Add spawnSync
mock-call assertion and 4 failure-path tests: Docker not running,
Docker not installed via ENOENT, missing browser binary, and ordering
check (Docker before Supabase).

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
@timothyfroehlich timothyfroehlich added the ready-for-review PR passed CI and has no unresolved review comments label May 5, 2026
Copilot AI review requested due to automatic review settings May 5, 2026 20:21
…-up)

The previous error text recommended `brew install --cask orbstack` and
`open -a OrbStack` which is mac-only guidance. Replace with platform-neutral
messages naming OrbStack/Docker Desktop/Colima/systemctl alternatives, with
the mac brew hint preserved as a sub-line for the most common platform.

- ENOENT (not installed): Lists OrbStack, Docker Desktop, Docker Engine,
  with Mac brew hint as sub-line
- Daemon down: Lists OrbStack, Docker Desktop, Colima, systemctl alternatives
  with "start your Docker daemon" as the lead instruction
- Test assertions still pass: substring matching still matches new messages
  ("Docker is not installed" and "Docker daemon is not running" preserved)

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 2 out of 2 changed files in this pull request and generated no new comments.

Comments suppressed due to low confidence (1)

src/test/unit/global-setup.test.ts:143

  • The test case "skips everything when SKIP_SUPABASE_RESET is set" is now misleading/incomplete: globalSetup still runs the Docker (and potentially browser) preflight checks before returning when SKIP_SUPABASE_RESET=true. Consider renaming the test to reflect what is actually skipped and/or add assertions that spawnSync (and the browser check when applicable) is still invoked under SKIP so this contract can’t regress silently.
  it("skips everything when SKIP_SUPABASE_RESET is set", async () => {
    process.env.SKIP_SUPABASE_RESET = "true";
    const setup = await loadSetup();

    await setup(EMPTY_CONFIG);

    expect(fetchMock).not.toHaveBeenCalled();
    expect(execSyncMock).not.toHaveBeenCalled();
  });

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ready-for-review PR passed CI and has no unresolved review comments

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants