Skip to content

ci: gate on e2e suite against a production build#262

Merged
rootulp merged 1 commit into
mainfrom
ci-e2e
Jul 17, 2026
Merged

ci: gate on e2e suite against a production build#262
rootulp merged 1 commit into
mainfrom
ci-e2e

Conversation

@rootulp

@rootulp rootulp commented Jul 17, 2026

Copy link
Copy Markdown
Owner

Follow-up to #261: wire the Playwright e2e suite into CI so integration/prod-only regressions get caught. The vitest suite only exercises lib/ functions, so recent prod-only bugs — soft-404s (#257, #259), the searchParams 500 (#260), the edge-shard gzip 404 (#256) — slipped past it.

  • New e2e CI job: npm ci → install chromium → npm run buildnext start → run Playwright. Runs against a production bundle, not next dev, since some regressions only reproduce in prod rendering.
  • Timing-sensitive specs tagged @perf (Web Vitals thresholds; the cold-start journey, which needs a deployed BASE_URL) and excluded from the gate via --grep-invert @perf.
  • CI-only Playwright retries (retries: process.env.CI ? 2 : 0) so the one-time ~190MB corpus warmup on a cold server doesn't flake the gate; genuine failures still fail every attempt. Local runs stay strict.
  • Fix a stale assertion in result-share-and-pills.spec.ts: pills now render "Top N%".

Verified locally by building, serving with next start, and running the gated suite (CI=1 --grep-invert @perf) across repeated cold starts — green, with cold-start losers surfacing as flaky-passed rather than failures.

🤖 Generated with Claude Code

https://claude.ai/code/session_01Bytmqtq2bNyot4PRvo8es7

Add an `e2e` CI job that builds the app, serves it with `next start`, and
runs the Playwright suite. It runs against a production bundle rather than
`next dev` because past regressions (e.g. searchParams 500s) only reproduced
in prod rendering.

Timing-sensitive specs are tagged `@perf` (Web Vitals thresholds and the
cold-start journey, which needs a deployed URL) and excluded from the gate
via `--grep-invert @perf`. Add CI-only Playwright retries so the one-time
~190MB corpus warmup on a cold server doesn't flake the correctness gate.
Also fix a stale pills assertion: the pill now renders "Top N%".

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Bytmqtq2bNyot4PRvo8es7
@rootulp rootulp self-assigned this Jul 17, 2026
@vercel

vercel Bot commented Jul 17, 2026

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated (UTC)
tritimes Ready Ready Preview, Comment Jul 17, 2026 10:13pm

@coderabbitai

coderabbitai Bot commented Jul 17, 2026

Copy link
Copy Markdown

Warning

Review limit reached

@rootulp, you've reached your PR review limit, so we couldn't start this review.

Next review available in: 40 minutes

Enable usage-based reviews in Billing to review now. Otherwise, wait until the next included review is available.
You're only billed for reviews past your plan's rate limits ($0.25/file).

How can I continue?

After more reviews become available, a review can be triggered using the @coderabbitai review command as a PR comment. Alternatively, push new commits to this PR.

To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based reviews.

How do review limits work?

CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan review availability.

For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, additional reviews become available more gradually as earlier reviews age out of the rolling window.

Please refer docs for additional details.

Review details
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro Plus

Run ID: 4c7b9697-f702-48ef-876e-6bd1aeceb094

📥 Commits

Reviewing files that changed from the base of the PR and between a39ac91 and a535812.

📒 Files selected for processing (5)
  • .github/workflows/ci.yml
  • app/e2e/journey-performance.spec.ts
  • app/e2e/performance.spec.ts
  • app/e2e/result-share-and-pills.spec.ts
  • app/playwright.config.ts
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch ci-e2e

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@rootulp
rootulp enabled auto-merge (squash) July 17, 2026 22:10
@rootulp
rootulp merged commit a3c0731 into main Jul 17, 2026
8 checks passed
@rootulp
rootulp deleted the ci-e2e branch July 17, 2026 22:13
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant