feat : configure playwright e2e testing and integrate into CI#136
Conversation
📝 WalkthroughWalkthroughAdds Playwright end-to-end smoke test infrastructure: a ChangesPlaywright E2E Smoke Test Setup
Sequence Diagram(s)sequenceDiagram
participant Playwright as Playwright (Chromium)
participant AppPage as /en/app
participant AnalyzeAPI as POST /api/analyze
participant ReactFlow as React Flow Diagram
Playwright->>AppPage: navigate to /en/app
AppPage-->>Playwright: textarea visible, Analyze button enabled
Playwright->>AppPage: click Analyze
AppPage->>AnalyzeAPI: submit default code snippet
AnalyzeAPI-->>AppPage: graph node data
AppPage->>ReactFlow: render nodes
ReactFlow-->>Playwright: assert 6 nodes, labels snippet.py / main / clean
Estimated code review effort🎯 2 (Simple) | ⏱️ ~10 minutes Poem
🚥 Pre-merge checks | ✅ 4 | ❌ 1❌ Failed checks (1 inconclusive)
✅ Passed checks (4 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. ✨ Finishing Touches🧪 Generate unit tests (beta)
Comment |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 2
🧹 Nitpick comments (4)
.github/workflows/ci.yml (2)
31-37: 🧹 Nitpick | 🔵 Trivial | ⚡ Quick winConsider making the e2e job dependent on successful build.
The e2e job runs in parallel with the build job, which means E2E tests could start even if linting, type-checking, or unit tests fail. This wastes CI resources and creates confusing failure scenarios.
⚡ Proposed fix
e2e: runs-on: ubuntu-latest + needs: build env:🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In @.github/workflows/ci.yml around lines 31 - 37, The e2e job runs in parallel with the build job instead of waiting for it to complete, which wastes CI resources if the build fails. Add a job dependency directive to the e2e job definition to make it wait for the build job to finish successfully before starting. This should be added as a property at the same level as runs-on and env in the e2e job configuration.
38-45: 🧹 Nitpick | 🔵 TrivialSecurity: Consider pinning GitHub Actions to commit SHAs.
Static analysis flagged that actions are referenced by tag (e.g.,
@v7) rather than pinned to specific commit hashes. Pinning to SHAs provides stronger supply chain security against tag manipulation.Additionally, consider setting
persist-credentials: falseon the checkout action to prevent credential leakage.🔒 Example hardening
steps: - - uses: actions/checkout@v7 + - uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v7 + with: + persist-credentials: false - - uses: pnpm/action-setup@v6 + - uses: pnpm/action-setup@fe02b34f77f8bc703788d5817da081398fad5dd2 # v6 - - uses: actions/setup-node@v6 + - uses: actions/setup-node@39370e3970a6d050c480ffad4ff0ed4d3fdee5af # v6Note: These concerns may also apply to the existing
buildjob. Consider addressing them repository-wide if they're not already handled.🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In @.github/workflows/ci.yml around lines 38 - 45, Replace the version tag references (v7, v6, v6) with specific commit SHAs for the three GitHub Actions in this workflow: actions/checkout, pnpm/action-setup, and actions/setup-node. Additionally, add a configuration parameter `persist-credentials: false` to the actions/checkout action to prevent credential leakage. This strengthens supply chain security by pinning to immutable commit hashes instead of mutable tags, making the workflow resistant to tag manipulation attacks.Source: Linters/SAST tools
playwright.config.ts (1)
10-23: 🧹 Nitpick | 🔵 Trivial | 💤 Low valueConsider extracting the base URL to a constant.
The URL
http://localhost:3000is repeated in bothwebServer.url(line 12) anduse.baseURL(line 22). Extracting it to a constant would reduce duplication and ensure consistency if the port ever needs to change.♻️ Proposed refactor
+const BASE_URL = "http://localhost:3000"; + export default defineConfig({ testDir: "./e2e", fullyParallel: true, forbidOnly: !!process.env.CI, retries: process.env.CI ? 2 : 0, workers: process.env.CI ? 1 : undefined, reporter: "html", webServer: { command: "pnpm dev", - url: "http://localhost:3000", + url: BASE_URL, reuseExistingServer: !process.env.CI, stdout: "ignore", stderr: "pipe", env: { AUTH_TRUST_HOST: "true", AUTH_SECRET: "some-dummy-secret-for-e2e", }, }, use: { - baseURL: "http://localhost:3000", + baseURL: BASE_URL, trace: "on", },🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@playwright.config.ts` around lines 10 - 23, The base URL string http://localhost:3000 is duplicated in the playwright.config.ts configuration object, appearing in both the webServer.url and use.baseURL fields. Extract this URL to a constant variable defined at the top of the file (before the export statement), and then replace both hardcoded occurrences with references to this constant to eliminate duplication and improve maintainability.e2e/smoke.spec.ts (1)
24-27: 🧹 Nitpick | 🔵 Trivial | 💤 Low valueConsider verifying all function labels for completeness.
The test verifies 3 node labels (snippet.py, main, clean) but the default Python sample defines 5 functions: main, load, transform, clean, and save. Adding assertions for the remaining functions (load, transform, save) would provide more comprehensive smoke test coverage.
♻️ Proposed additional assertions
// 6. Verify key node labels contain our function names and container file await expect(page.locator(".react-flow__node", { hasText: "snippet.py" })).toBeVisible(); await expect(page.locator(".react-flow__node", { hasText: "main" })).toBeVisible(); + await expect(page.locator(".react-flow__node", { hasText: "load" })).toBeVisible(); + await expect(page.locator(".react-flow__node", { hasText: "transform" })).toBeVisible(); await expect(page.locator(".react-flow__node", { hasText: "clean" })).toBeVisible(); + await expect(page.locator(".react-flow__node", { hasText: "save" })).toBeVisible(); });🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@e2e/smoke.spec.ts` around lines 24 - 27, The smoke test in the verification section currently only checks for three node labels (snippet.py, main, and clean) but the default Python sample defines five functions total. Add two additional await expect statements after the existing assertions to verify the remaining function nodes for "load", "transform", and "save" using the same page.locator pattern with the ".react-flow__node" selector and hasText matcher. This will ensure the test comprehensively validates all expected function nodes are rendered in the flow diagram.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In @.github/workflows/ci.yml:
- Around line 31-53: The e2e CI job is missing artifact uploads for Playwright
test results, which means debugging information is lost when tests fail. After
the "Run E2E Smoke Tests" step that executes pnpm test:e2e, add a new step using
the actions/upload-artifact@v4 action to upload the Playwright HTML report and
traces directory. Configure the upload to run even when tests fail by adding a
condition that checks the job status, and specify the appropriate paths where
Playwright stores its report and trace files (typically playwright-report and
test-results directories).
- Line 36: The E2E job configures TURSO_DATABASE_URL to use a local file-based
database but does not initialize the database schema before running tests. The
application does not automatically create tables on startup; instead, it only
validates the environment variable. Add a database migration step by inserting
pnpm db:push command in the CI workflow before the pnpm test:e2e command to
ensure that the schema from drizzle/schema.sql is executed and all required
tables are created before the E2E tests run against the local database.
---
Nitpick comments:
In @.github/workflows/ci.yml:
- Around line 31-37: The e2e job runs in parallel with the build job instead of
waiting for it to complete, which wastes CI resources if the build fails. Add a
job dependency directive to the e2e job definition to make it wait for the build
job to finish successfully before starting. This should be added as a property
at the same level as runs-on and env in the e2e job configuration.
- Around line 38-45: Replace the version tag references (v7, v6, v6) with
specific commit SHAs for the three GitHub Actions in this workflow:
actions/checkout, pnpm/action-setup, and actions/setup-node. Additionally, add a
configuration parameter `persist-credentials: false` to the actions/checkout
action to prevent credential leakage. This strengthens supply chain security by
pinning to immutable commit hashes instead of mutable tags, making the workflow
resistant to tag manipulation attacks.
In `@e2e/smoke.spec.ts`:
- Around line 24-27: The smoke test in the verification section currently only
checks for three node labels (snippet.py, main, and clean) but the default
Python sample defines five functions total. Add two additional await expect
statements after the existing assertions to verify the remaining function nodes
for "load", "transform", and "save" using the same page.locator pattern with the
".react-flow__node" selector and hasText matcher. This will ensure the test
comprehensively validates all expected function nodes are rendered in the flow
diagram.
In `@playwright.config.ts`:
- Around line 10-23: The base URL string http://localhost:3000 is duplicated in
the playwright.config.ts configuration object, appearing in both the
webServer.url and use.baseURL fields. Extract this URL to a constant variable
defined at the top of the file (before the export statement), and then replace
both hardcoded occurrences with references to this constant to eliminate
duplication and improve maintainability.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: defaults
Review profile: CHILL
Plan: Pro Plus
Run ID: cfd0c108-7e82-4f01-b40b-bb4e6302d0fc
⛔ Files ignored due to path filters (1)
pnpm-lock.yamlis excluded by!**/pnpm-lock.yaml
📒 Files selected for processing (5)
.github/workflows/ci.yml.gitignoree2e/smoke.spec.tspackage.jsonplaywright.config.ts
| e2e: | ||
| runs-on: ubuntu-latest | ||
| env: | ||
| AUTH_SECRET: some-dummy-secret-for-ci | ||
| AUTH_TRUST_HOST: "true" | ||
| TURSO_DATABASE_URL: file:local.db | ||
| steps: | ||
| - uses: actions/checkout@v7 | ||
|
|
||
| - uses: pnpm/action-setup@v6 | ||
|
|
||
| - uses: actions/setup-node@v6 | ||
| with: | ||
| node-version: 24 | ||
| cache: pnpm | ||
|
|
||
| - run: pnpm install --frozen-lockfile | ||
|
|
||
| - name: Install Playwright Chromium Browser | ||
| run: pnpm exec playwright install --with-deps chromium | ||
|
|
||
| - name: Run E2E Smoke Tests | ||
| run: pnpm test:e2e |
There was a problem hiding this comment.
🛠️ Refactor suggestion | 🟠 Major | ⚡ Quick win
Add artifact upload for Playwright test results.
When E2E tests fail in CI, the HTML report and traces are essential for debugging. Without artifact upload, these files are lost when the job completes.
📦 Proposed addition
- name: Run E2E Smoke Tests
run: pnpm test:e2e
+
+ - name: Upload Playwright Report
+ if: always()
+ uses: actions/upload-artifact@v4
+ with:
+ name: playwright-report
+ path: playwright-report/
+ retention-days: 30📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
| e2e: | |
| runs-on: ubuntu-latest | |
| env: | |
| AUTH_SECRET: some-dummy-secret-for-ci | |
| AUTH_TRUST_HOST: "true" | |
| TURSO_DATABASE_URL: file:local.db | |
| steps: | |
| - uses: actions/checkout@v7 | |
| - uses: pnpm/action-setup@v6 | |
| - uses: actions/setup-node@v6 | |
| with: | |
| node-version: 24 | |
| cache: pnpm | |
| - run: pnpm install --frozen-lockfile | |
| - name: Install Playwright Chromium Browser | |
| run: pnpm exec playwright install --with-deps chromium | |
| - name: Run E2E Smoke Tests | |
| run: pnpm test:e2e | |
| e2e: | |
| runs-on: ubuntu-latest | |
| env: | |
| AUTH_SECRET: some-dummy-secret-for-ci | |
| AUTH_TRUST_HOST: "true" | |
| TURSO_DATABASE_URL: file:local.db | |
| steps: | |
| - uses: actions/checkout@v7 | |
| - uses: pnpm/action-setup@v6 | |
| - uses: actions/setup-node@v6 | |
| with: | |
| node-version: 24 | |
| cache: pnpm | |
| - run: pnpm install --frozen-lockfile | |
| - name: Install Playwright Chromium Browser | |
| run: pnpm exec playwright install --with-deps chromium | |
| - name: Run E2E Smoke Tests | |
| run: pnpm test:e2e | |
| - name: Upload Playwright Report | |
| if: always() | |
| uses: actions/upload-artifact@v4 | |
| with: | |
| name: playwright-report | |
| path: playwright-report/ | |
| retention-days: 30 |
🧰 Tools
🪛 zizmor (1.25.2)
[warning] 38-38: credential persistence through GitHub Actions artifacts (artipacked): does not set persist-credentials: false
(artipacked)
[warning] 31-54: overly broad permissions (excessive-permissions): default permissions used due to no permissions: block
(excessive-permissions)
[error] 38-38: unpinned action reference (unpinned-uses): action is not pinned to a hash (required by blanket policy)
(unpinned-uses)
[error] 40-40: unpinned action reference (unpinned-uses): action is not pinned to a hash (required by blanket policy)
(unpinned-uses)
[error] 42-42: unpinned action reference (unpinned-uses): action is not pinned to a hash (required by blanket policy)
(unpinned-uses)
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In @.github/workflows/ci.yml around lines 31 - 53, The e2e CI job is missing
artifact uploads for Playwright test results, which means debugging information
is lost when tests fail. After the "Run E2E Smoke Tests" step that executes pnpm
test:e2e, add a new step using the actions/upload-artifact@v4 action to upload
the Playwright HTML report and traces directory. Configure the upload to run
even when tests fail by adding a condition that checks the job status, and
specify the appropriate paths where Playwright stores its report and trace files
(typically playwright-report and test-results directories).
| env: | ||
| AUTH_SECRET: some-dummy-secret-for-ci | ||
| AUTH_TRUST_HOST: "true" | ||
| TURSO_DATABASE_URL: file:local.db |
There was a problem hiding this comment.
🧩 Analysis chain
🏁 Script executed:
#!/bin/bash
# Description: Search for database initialization or migration logic
# Check for Turso/database initialization in the app
rg -n --type=ts --type=js -C3 'TURSO_DATABASE_URL|drizzle.*migrate|prisma.*migrate'
# Check for any db setup in package.json scripts
rg -n 'db.*setup|db.*migrate|db.*push' package.jsonRepository: DataDave-Dev/weftmap
Length of output: 1256
🏁 Script executed:
# Check the full CI workflow E2E job setup
cat -n .github/workflows/ci.yml | head -60Repository: DataDave-Dev/weftmap
Length of output: 1653
🏁 Script executed:
# Search for any startup/middleware that might trigger migrations
rg -n --type=ts --type=js "db:push|migrate|startup|middleware" src/Repository: DataDave-Dev/weftmap
Length of output: 46
🏁 Script executed:
# Check the test:e2e script definition in package.json
rg -n "test:e2e" package.json -A 2Repository: DataDave-Dev/weftmap
Length of output: 228
🏁 Script executed:
# Look for any startup/initialization code that might run migrations
rg -n "db:push|migrat" src/ --type=ts --type=jsRepository: DataDave-Dev/weftmap
Length of output: 46
🏁 Script executed:
# Check if there's a drizzle migration directory or any setup in the app
find . -name "drizzle" -type d -o -name "*migrat*" -type f | head -20Repository: DataDave-Dev/weftmap
Length of output: 2565
🏁 Script executed:
# Check what's in the drizzle directory
ls -la drizzle/ 2>/dev/null | head -20Repository: DataDave-Dev/weftmap
Length of output: 238
🏁 Script executed:
# Read full src/db/index.ts to see initialization logic
cat -n src/db/index.tsRepository: DataDave-Dev/weftmap
Length of output: 834
Add database migration step to E2E job before running tests.
The E2E job sets TURSO_DATABASE_URL: file:local.db but does not run migrations. The application only validates the environment variable at startup and does not automatically initialize the database—the drizzle/schema.sql migration file exists but is never executed in CI. Add pnpm db:push before pnpm test:e2e to ensure tables exist when E2E tests run.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In @.github/workflows/ci.yml at line 36, The E2E job configures
TURSO_DATABASE_URL to use a local file-based database but does not initialize
the database schema before running tests. The application does not automatically
create tables on startup; instead, it only validates the environment variable.
Add a database migration step by inserting pnpm db:push command in the CI
workflow before the pnpm test:e2e command to ensure that the schema from
drizzle/schema.sql is executed and all required tables are created before the
E2E tests run against the local database.
2 participants
Pull Request Description
Title
test: add Playwright E2E smoke test for analyze-and-render flowDescription
Resolves #69
Summary of Changes
This PR adds End-to-End (E2E) smoke testing coverage to
weftmapusing Playwright. It ensures that the core user flow—loading the app, clicking "Analyze", invoking the server-side WASM tree-sitter parsers, and successfully drawing the React Flow call graph—works without regressions.Key Implementations
playwright.config.tsrunning tests against the dev server (pnpm dev) for fast local feedback.e2e/smoke.spec.tswhich asserts the default Python call graph successfully renders 6 canvas nodes (snippet.py+ 5 functions).AUTH_TRUST_HOST=trueandAUTH_SECRETin both Playwright and the CI runner to suppress NextAuth untrusted host warnings.e2eworkflow job in.github/workflows/ci.ymlthat installs browser dependencies and runs the test suite on every PR and push tomain.trace: 'on') to simplify debugging browser actions.How to Test Locally
Install Playwright Browsers:
Run E2E Tests:
Run in UI / Interactive Mode:
Summary by CodeRabbit
Tests