feat: enforce performance budgets in CI via Lighthouse CI#71
Merged
Conversation
Adds a Lighthouse CI workflow that runs on PRs touching themes/, asserts against resource budgets + Core Web Vitals + Lighthouse category scores, and uploads HTML reports to lhci's temporary-public-storage for a sharable link in the PR check output. Pipeline: - .github/workflows/lighthouse-ci.yml reuses the visual-regression Docker boot + WooCommerce install + tests/visual/seed.sh for deterministic content, then drives treosh/lighthouse-ci-action@v12. - lighthouserc.json defines URLs (/, /shop/, /product/test-hoodie/, /cart/), assertion thresholds (perf >= 0.70, a11y >= 0.90, CLS <= 0.1), and points budgetsPath at tests/lighthouse/budgets.json. - tests/lighthouse/budgets.json is the editable budget surface — KB resource sizes, ms timing budgets, third-party request counts. - tests/lighthouse/annotate-report.mjs reads .lighthouseci/assertion-results.json post-run and emits ::error / ::warning / ::notice workflow commands so failures show as native check annotations with the URL, audit, expected, and actual values (consistent with how visual regression annotates). Mobile-only this iteration; lhci defaults to the mobile preset. Local workflow added to package.json: - pnpm lighthouse:run — boot lhci end-to-end against the running stack - pnpm lighthouse:collect — capture only (skip assert/upload) - pnpm lighthouse:assert — assert against the most recent collection - pnpm lighthouse:open — open the captured HTML reports @lhci/cli@^0.14.0 added to devDependencies. Soft-fail mode (`continue-on-error: true`) is on while the suite settles — flip once results are stable against main and tighten the starting budgets at the same time. No fixed flip date. Trend tracking deliberately deferred to a future issue: temporary-public-storage gives ~7 days of browseable reports per run; in-repo history can be added later via an LHCI server or a JSON-summary-commit bot. Closes #20 Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adds a Lighthouse CI workflow that runs against
themes/flavian-shop/on every PR that touches theme code. Asserts resource budgets + Core Web Vitals + Lighthouse category scores and uploads HTML reports to lhci's temporary-public-storage for sharable links in the PR check output. Closes #20.Pipeline
tests/visual/seed.sh(no duplication)treosh/lighthouse-ci-action@v12via.github/workflows/lighthouse-ci.ymllighthouserc.jsontests/lighthouse/budgets.jsontests/lighthouse/annotate-report.mjsDecisions locked in (from scoping)
upload.target = "temporary-public-storage"in lighthouserc.json +temporaryPublicStorage: trueon the treosh actiontests/lighthouse/README.mdas a "future enhancement" with options/,/shop/,/product/test-hoodie/,/cart/ci.collect.urlin lighthouserc.jsondesktopoverridecontinue-on-error: truewith prose-only flip guidance in the workflow header and the READMEURLs and runs
4 URLs ×
numberOfRuns: 3= 12 collections per PR. Lighthouse asserts evaluate the median, which matches Google's recommended sample size for stability.skipAudits: ["uses-http2"]is set because the local Docker stack runs HTTP/1.1 — that audit would always fail without telling us anything about the theme.Mobile vs desktop
Lighthouse's mobile preset (4G throttling, 360×640 viewport) is the default. It's the pessimistic case — passing mobile generally implies passing desktop. Adding desktop later is one config block; not worth doubling CI time at v1.
Soft-fail period
continue-on-error: trueis set on the job. Failures do not block merge while the suite settles. The starting budgets inbudgets.jsonare intentionally lenient; tighten them and flipcontinue-on-errortofalseonce you've seen ~2 weeks of stable results againstmain. No date pinned, per your instruction.Trend tracking
Deferred. Temporary-public-storage gives a sharable HTML report URL per run with ~7-day retention — surfaced via
::noticeannotation byannotate-report.mjs. Per-run HTML reports are also uploaded as a workflow artifact for indefinite retention while the run exists. In-repo trend history (LHCI server or a JSON-summary-commit bot) can be added in a follow-up issue when it's a real need; documented intests/lighthouse/README.md.Local debugging
Requires the local Docker stack to be running first (
docker compose up -d wordpress db+ WooCommerce install + seed).Files
.github/workflows/lighthouse-ci.yml— CI workflowlighthouserc.json— lhci config (top-level so it's discoverable)tests/lighthouse/budgets.json— editable resource/timing budgetstests/lighthouse/annotate-report.mjs— checkstyle-equivalent annotatortests/lighthouse/README.md— config docs, debug guide, tuning workflowpackage.json— adds@lhci/cli@^0.14.0+ 4 npm scriptspnpm-lock.yaml— updated (pinned@lhci/cli@0.14.0).gitignore— adds.lighthouseci/scratch dirCONTRIBUTING.md— "Performance budgets" subsection under TestingTest plan
commitlintpasses.lighthouse-ciworkflow runs on this PR and completes (may or may not pass budgets on first run — that's fine while soft-fail is on; the goal here is wiring works)..lighthouseci-*) and prints atemporary-public-storageURL in the log.annotate-report.mjsemits::error/::warningworkflow commands that surface as PR check annotations.package.jsondeps, which they don't read).pnpm exec lhci assert --config=./lighthouserc.jsonparses without config errors (already verified — "Checking assertions against 0 URL(s)" with no schema errors).🤖 Generated with Claude Code