Skip to content

feat: Enable all Playwright tests and improve error handling - Phase 3#67

Merged
aoshimash merged 8 commits into
mainfrom
feature/enable-all-playwright-tests
Jul 27, 2025
Merged

feat: Enable all Playwright tests and improve error handling - Phase 3#67
aoshimash merged 8 commits into
mainfrom
feature/enable-all-playwright-tests

Conversation

@aoshimash

@aoshimash aoshimash commented Jul 26, 2025

Copy link
Copy Markdown
Owner

Summary

  • Enable all Playwright tests in CI environment (Reverted due to missing system dependencies)
  • Implement comprehensive error handling and debugging capabilities
  • Add screenshot capture on test failure (optional feature)

Background

This PR was intended to complete Phase 3 of the Playwright CI integration (#64), following Phase 1 (#62) which validated basic functionality and Phase 2 (#63) which was supposed to optimize performance with caching.

However, we discovered that the CI environment is missing critical system dependencies for Playwright to run properly. This should have been addressed in Phase 2.

Current Status

Due to missing system dependencies in CI (GTK4, GStreamer, multimedia codecs, etc.), we've had to revert the CI enablement while keeping the enhanced error handling features.

Missing Dependencies in CI:

  • libgtk-4.so.1
  • libgraphene-1.0.so.0
  • libwoff2dec.so.1.0.2
  • libvpx.so.9
  • libevent-2.1.so.7
  • libopus.so.0
  • Various GStreamer libraries
  • And many more...

Changes Made

1. Removed CI Skip Conditions (Reverted)

  • Tests still skip in CI due to missing dependencies
  • Tests run successfully in local environments

2. Enhanced Error Handling ✅

  • Created test_helpers.go with debug capture utilities:
    • SetupPageDebugHandlers() - Captures console logs and network activity
    • CaptureScreenshotOnFailure() - Takes screenshots when tests fail
    • LogTestDebugInfo() - Logs comprehensive debug information
    • RetryTest() - Provides retry logic for potentially flaky tests

3. Browser Pool Integration ✅

  • Modified browser_pool.go to automatically capture debug logs when running in test mode
  • Enhanced error messages with console logs and network activity details
  • Improved debugging output when tests fail

Test Results

All Playwright tests pass locally where dependencies are available:

=== RUN   TestBrowserPool_RenderPage
--- PASS: TestBrowserPool_RenderPage (2.51s)
=== RUN   TestJSClient_RenderPage
--- PASS: TestJSClient_RenderPage (2.15s)
=== RUN   TestJSClient_Get
--- PASS: TestJSClient_Get (2.16s)

Next Steps

  1. Phase 2 (feat: Implement Playwright browser binary caching in CI - Phase 2 #63) needs to be properly implemented to install all required system dependencies
  2. Once dependencies are installed, we can remove the CI skip conditions
  3. Consider using the official Playwright Docker image or GitHub Action for better dependency management

Success Criteria (Partially Met)

  • All Playwright tests run and pass in CI environment (blocked by dependencies)
  • Clear error messages when tests fail
  • Debugging information readily available in logs
  • Screenshot capture capability on test failure

Related Issues

Test Plan

  1. Run all tests locally with go test ./...
  2. Verify Playwright tests run without CI skip conditions (blocked)
  3. Check that error handling provides useful debug information ✅
  4. Confirm CI pipeline remains stable (tests skip in CI)

🤖 Generated with Claude Code

aoshimash and others added 3 commits July 27, 2025 01:54
- Remove all CI skip conditions from Playwright tests:
  - TestBrowserPool_RenderPage
  - TestJSClient_RenderPage
  - TestJSClient_Get
  - TestNewUnifiedClient_JSEnabled

- Implement enhanced error handling with debug logging:
  - Add test_helpers.go with debug capture utilities
  - Capture console logs and network activity during tests
  - Log detailed error context when tests fail
  - Add screenshot capture capability on test failure

- Add retry logic helper for potentially flaky tests

- Integrate debug handlers into browser pool for automatic logging
  when running in test mode

This completes Phase 3 of the Playwright CI integration, enabling all
tests to run in CI with comprehensive debugging capabilities.

Fixes #64

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>
- Remove unused init function from test_helpers.go
- Fix file formatting to pass gofmt -s check
…pendencies

The CI environment is missing critical system dependencies for Playwright:
- GTK4 libraries
- GStreamer libraries
- Various multimedia codecs
- And many more

These dependencies should be addressed in Phase 2 (issue #63) before
enabling Playwright tests in CI. For now, we're keeping the enhanced
error handling but skipping tests in CI.

The tests still run locally where dependencies are available.
@aoshimash

Copy link
Copy Markdown
Owner Author

現在の状況

このPRは当初の目的(Phase 3: すべてのPlaywrightテストをCI環境で有効化)を完全には達成できませんでした。

実装できたこと ✅

  • エラーハンドリングの強化(ブラウザログ、ネットワークログの収集)
  • テスト失敗時のスクリーンショット撮影機能
  • デバッグ情報の自動収集機能

実装できなかったこと ❌

  • CI環境でのPlaywrightテストの有効化(システム依存関係の不足により)

次のステップ

  1. issue fix: Install missing system dependencies for Playwright in CI - Phase 2 Fix #68 でPhase 2の実装不備を修正
  2. CI環境に必要なシステム依存関係をインストール
  3. その後、このPRに戻ってCI環境でのテスト有効化を完了

現時点では、このPRはマージせずに保留とし、#68の解決後に再度作業を進めます。

Now that Phase 2 (#68) has been completed with Docker-based CI setup,
we can enable all Playwright tests to run in the CI environment.

- Remove CI skip conditions from all Playwright tests
- Tests now run in Docker container with all dependencies pre-installed
- Enhanced error handling and debugging capabilities remain in place

Completes Phase 3 (#64)
Fix go vet error by removing unused os package imports
Match Docker image version with project's Playwright version to resolve
compatibility issues
The CI Docker environment may have slower network access,
causing timeouts when navigating to external URLs
@aoshimash aoshimash merged commit ace889e into main Jul 27, 2025
4 checks passed
@aoshimash aoshimash deleted the feature/enable-all-playwright-tests branch July 27, 2025 03:33
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant