Add automated UI testing

[wrslatz]

Background

The Private Mirrors App is a Next.js-based application with a web UI that manages the lifecycle of private mirrors and synchronization between public forks and private repositories. Automated UI tests will ensure the functionality of the app. The tests should run against a mock GitHub setup to simplify and speed up the tests and allow them to run locally.

Goal

Implement automated UI testing using Playwright to test the Private Mirrors application UI. These tests should:

1. Run against a mock, offline GitHub setup to validate UI functionality without requiring live GitHub API calls
2. Be executable locally for development and debugging
3. Be integrated into a new GitHub Actions workflow for continuous testing on pull requests and merges
4. Provide reliable and maintainable test coverage for critical user workflows

Scope

Tests to implement:

• Application startup and initialization
• User authentication flow (if applicable in mock setup)
• Mirror creation workflow
• Mirror management (list, view, update)
• Navigation between different UI sections
• Form validation and error handling
• Critical user interactions and workflows

Mock GitHub Setup:

• Design and implement a mock GitHub API server or use existing mocking tools (e.g., Playwright fixtures, MSW - Mock Service Worker)
• Define mock data for repositories, organizations, users, etc.
• Ensure the mock setup can be used both locally and in CI/CD

Local Development:

• Document how developers can run tests locally
• Include test setup instructions in the development documentation
• Ensure fast feedback loop for test failures

GitHub Actions Integration:

• Create a new .github/workflows/ui-tests.yml workflow
• Tests should run on pull requests and on push to main
• Generate test reports/artifacts for visibility
• Consider caching strategies to optimize workflow run time

Technical Details

Technology Stack:

• Test Framework: Playwright
• Test Runner: Playwright's built-in runner (or Jest with Playwright preset if preferred)
• Mock API: Consider Mock Service Worker (MSW) or custom mock server
• Reporting: Playwright HTML report generation

Existing Project Context:

• The project already uses Jest for testing (see jest.config.js)
• Next.js application running on port 3000
• Uses environment variables (.env.example exists for configuration)
• Already has GitHub Actions workflows for CodeQL, Linting, Docker Build, and Tests

Acceptance Criteria

• Playwright is installed and configured in package.json
• Mock GitHub setup allows running tests without external API calls
• At least 5-10 core UI workflows have test coverage
• Tests run successfully locally via npm run test:ui or similar command
• GitHub Actions workflow is created and tests run on PRs/merges to main
• CI workflow runs successfully and provides meaningful feedback
• Documentation is updated in /docs/developing.md with test execution instructions
• Tests pass consistently (no flaky tests)

Implementation Notes

• Start with a small set of critical user workflows to establish the testing pattern
• Ensure mock setup is flexible enough to support future test additions
• Consider test parallelization for faster CI runs
• Make sure tests don't interfere with each other or require specific ordering

References

• Developing documentation
• Playwright documentation
• Existing test setup in jest.config.js and babel.config.testing.js

[m13v]

for the mock GitHub API setup, MSW (Mock Service Worker) with Playwright is a really good combination. we use MSW's request handlers to intercept GitHub API calls at the network level, so the app code doesn't need any changes to support testing. one thing to get right early: define your mock data as factory functions (createMockRepo, createMockOrg) rather than static fixtures, because tests that share fixtures end up coupled and one test's setup breaks another. for the mirror creation workflow test, make sure you test the error path where the upstream repo doesn't exist or permissions are insufficient, those failure modes are where most bugs hide in sync workflows. also worth adding a test for the race condition where a sync starts while a mirror creation is still in progress.

[m13v]

for auto-generating the initial test scaffolding from your app's workflows, we open sourced a tool that crawls apps and generates Playwright specs with self-healing selectors. the test generation engine: https://github.com/assrt-ai/assrt-freestyle/blob/main/src/core/freestyle.ts outputs standard .spec.ts files so no vendor lock-in.