Add A8 Synthetic Canary monitoring system for dashboard health checks

[Copilot]

Overview

This PR implements a comprehensive synthetic monitoring system (A8 Synthetic Canary) for the Sherlock Multi-Agent Data Scientist dashboard. The system runs automated end-to-end tests every 6 hours to continuously verify dashboard health and functionality.

What's New

Automated Monitoring Tests

Created a read-only Playwright test suite that verifies critical dashboard functionality without mutating production data:

• Dashboard loading: Ensures the application loads within 10 seconds
• Navigation: Verifies all 4 agent tabs (Orchestrator, EDA, Refinery, ML) are functional
• Health indicators: Checks system status and agent health displays
• UI components: Validates console panel, workflows panel, datasets panel, and background processes panel
• User interactions: Tests navigation between tabs, console input handling, and page refresh

All tests are read-only and use a dedicated test tenant (synthetic-canary-test) to ensure production data is never affected.

CRON-Scheduled GitHub Actions Workflow

The workflow runs automatically every 6 hours (00:00, 06:00, 12:00, 18:00 UTC) and:

1. Starts backend services and dashboard dev server
2. Executes the Playwright test suite
3. On failure:
   • Uploads artifacts (trace.zip, screenshots, videos, test results) with 30-day retention
   • Sends notifications via Slack webhook (if configured)
   • Supports custom webhooks for PagerDuty, Datadog, or other monitoring systems

Manual triggering is also supported via workflow_dispatch for testing and validation.

Comprehensive Documentation

Integration Guide (/reports/synthetic-integration.md)

Provides complete setup instructions including:

• Slack and custom webhook configuration with example payloads
• Test tenant isolation and environment variables
• Artifact management and debugging with Playwright Trace Viewer
• Troubleshooting procedures for common issues
• Monitoring best practices and metrics tracking

Quick Start Guide (/synthetic/README.md)

Developer-focused documentation covering:

• Local test execution commands
• Test philosophy and design principles
• Adding new synthetic tests
• Common troubleshooting scenarios

Technical Details

Dependencies Added

• @playwright/test@^1.48.0 - End-to-end testing framework
• @types/node@^20.10.0 - TypeScript type definitions
• typescript@^5.3.3 - TypeScript compiler

NPM Scripts

• npm run test:synthetic - Run tests headless
• npm run test:synthetic:ui - Run with interactive UI debugger
• npm run test:synthetic:headed - Run in headed browser mode
• npm run playwright:install - Install Chromium browser

Test Configuration

Playwright is configured to:

• Run tests in the synthetic/checks directory
• Capture traces on first retry for debugging
• Take screenshots and videos on failure
• Generate HTML and JSON reports

Security & Safety

• ✅ Read-only tests - No data creation, modification, or deletion
• ✅ Dedicated test tenant - Isolated from production data
• ✅ Webhook secrets - Uses GitHub Secrets for sensitive URLs
• ✅ No credentials in code - All secrets externalized

Testing

All components have been validated:

• ✅ TypeScript compilation passes
• ✅ YAML syntax validation passes
• ✅ Playwright detects all 4 test cases
• ✅ No mutating operations in test code
• ✅ Test tenant documented throughout

Usage

After merging, users should:

1. Configure SLACK_WEBHOOK_URL secret in repository settings (optional)
2. Configure WEBHOOK_URL secret for custom integrations (optional)
3. Monitor test results in GitHub Actions
4. Review artifacts when failures occur using npx playwright show-trace trace.zip

The synthetic canary will begin running automatically on the configured schedule, providing continuous monitoring of dashboard health and early detection of issues.

Files Changed

.github/workflows/synthetic.yml              # CRON workflow (168 lines)
.gitignore                                   # Artifact exclusions (35 lines)
package.json                                 # Dependencies & scripts (updated)
package-lock.json                            # Dependency lock file (auto-generated)
playwright.config.ts                         # Playwright config (60 lines)
reports/synthetic-integration.md             # Integration guide (500+ lines)
synthetic/README.md                          # Quick start (149 lines)
synthetic/checks/login-view-logout.spec.ts   # Test suite (158 lines)

Total: ~1,100 lines of code and documentation

Original prompt

You are A8 Synthetic Canary.

Create:

• synthetic/checks/login-view-logout.spec.ts (read-only)
• .github/workflows/synthetic.yml on CRON (e.g., every 6h)
• Artifact upload on fail (trace.zip), Slack/webhook placeholder in /reports/synthetic-integration.md

Do not mutate production data; use a dedicated test tenant.

---

✨ Let Copilot coding agent set things up for you — coding agent works faster and does higher quality work when set up for your repo.