Skip to content

feat: enforce performance budgets in CI via Lighthouse CI#71

Merged
PAMulligan merged 1 commit into
mainfrom
20-add-performance-budget-enforcement
May 20, 2026
Merged

feat: enforce performance budgets in CI via Lighthouse CI#71
PAMulligan merged 1 commit into
mainfrom
20-add-performance-budget-enforcement

Conversation

@PAMulligan
Copy link
Copy Markdown
Collaborator

Summary

Adds a Lighthouse CI workflow that runs against themes/flavian-shop/ on every PR that touches theme code. Asserts resource budgets + Core Web Vitals + Lighthouse category scores and uploads HTML reports to lhci's temporary-public-storage for sharable links in the PR check output. Closes #20.

Pipeline

Stage File
Docker WP boot + WC install + seed reuses tests/visual/seed.sh (no duplication)
Lighthouse CI orchestration treosh/lighthouse-ci-action@v12 via .github/workflows/lighthouse-ci.yml
lhci config (URLs, assertions, upload) lighthouserc.json
Editable resource/timing budgets tests/lighthouse/budgets.json
Native GH check annotations tests/lighthouse/annotate-report.mjs

Decisions locked in (from scoping)

# Choice Implementation
1 Temporary-public-storage for uploads upload.target = "temporary-public-storage" in lighthouserc.json + temporaryPublicStorage: true on the treosh action
2 No trend tracking in v1 Documented in tests/lighthouse/README.md as a "future enhancement" with options
3 URLs: /, /shop/, /product/test-hoodie/, /cart/ ci.collect.url in lighthouserc.json
4 Drafted budget numbers 250KB JS / 80KB CSS / 400KB images / 900KB total; LCP 4.0s / TTI 5.0s / FCP 2.5s; <=10 third-party requests
5 Drafted score thresholds Perf >= 0.70 (error), A11y >= 0.90 (error), BP/SEO >= 0.90 (warn), CLS <= 0.1 (error)
6 Mobile-only Lighthouse's default preset; no desktop override
7 Soft-fail without a fixed flip date continue-on-error: true with prose-only flip guidance in the workflow header and the README

URLs and runs

4 URLs × numberOfRuns: 3 = 12 collections per PR. Lighthouse asserts evaluate the median, which matches Google's recommended sample size for stability.

skipAudits: ["uses-http2"] is set because the local Docker stack runs HTTP/1.1 — that audit would always fail without telling us anything about the theme.

Mobile vs desktop

Lighthouse's mobile preset (4G throttling, 360×640 viewport) is the default. It's the pessimistic case — passing mobile generally implies passing desktop. Adding desktop later is one config block; not worth doubling CI time at v1.

Soft-fail period

continue-on-error: true is set on the job. Failures do not block merge while the suite settles. The starting budgets in budgets.json are intentionally lenient; tighten them and flip continue-on-error to false once you've seen ~2 weeks of stable results against main. No date pinned, per your instruction.

Trend tracking

Deferred. Temporary-public-storage gives a sharable HTML report URL per run with ~7-day retention — surfaced via ::notice annotation by annotate-report.mjs. Per-run HTML reports are also uploaded as a workflow artifact for indefinite retention while the run exists. In-repo trend history (LHCI server or a JSON-summary-commit bot) can be added in a follow-up issue when it's a real need; documented in tests/lighthouse/README.md.

Local debugging

pnpm lighthouse:run      # boot lhci end-to-end against the running stack
pnpm lighthouse:collect  # capture only, skip assert/upload
pnpm lighthouse:assert   # assert against the most recent collection
pnpm lighthouse:open     # open the captured HTML reports

Requires the local Docker stack to be running first (docker compose up -d wordpress db + WooCommerce install + seed).

Files

  • .github/workflows/lighthouse-ci.yml — CI workflow
  • lighthouserc.json — lhci config (top-level so it's discoverable)
  • tests/lighthouse/budgets.json — editable resource/timing budgets
  • tests/lighthouse/annotate-report.mjs — checkstyle-equivalent annotator
  • tests/lighthouse/README.md — config docs, debug guide, tuning workflow
  • package.json — adds @lhci/cli@^0.14.0 + 4 npm scripts
  • pnpm-lock.yaml — updated (pinned @lhci/cli@0.14.0)
  • .gitignore — adds .lighthouseci/ scratch dir
  • CONTRIBUTING.md — "Performance budgets" subsection under Testing

Test plan

  • commitlint passes.
  • lighthouse-ci workflow runs on this PR and completes (may or may not pass budgets on first run — that's fine while soft-fail is on; the goal here is wiring works).
  • Each lhci run uploads HTML reports as an artifact (.lighthouseci-*) and prints a temporary-public-storage URL in the log.
  • On failure, annotate-report.mjs emits ::error / ::warning workflow commands that surface as PR check annotations.
  • The visual-regression and plugin-validation workflows are unaffected (no shared file changes beyond package.json deps, which they don't read).
  • Spot check: locally, pnpm exec lhci assert --config=./lighthouserc.json parses without config errors (already verified — "Checking assertions against 0 URL(s)" with no schema errors).

🤖 Generated with Claude Code

Adds a Lighthouse CI workflow that runs on PRs touching themes/, asserts
against resource budgets + Core Web Vitals + Lighthouse category scores,
and uploads HTML reports to lhci's temporary-public-storage for a
sharable link in the PR check output.

Pipeline:
- .github/workflows/lighthouse-ci.yml reuses the visual-regression Docker
  boot + WooCommerce install + tests/visual/seed.sh for deterministic
  content, then drives treosh/lighthouse-ci-action@v12.
- lighthouserc.json defines URLs (/, /shop/, /product/test-hoodie/,
  /cart/), assertion thresholds (perf >= 0.70, a11y >= 0.90, CLS <= 0.1),
  and points budgetsPath at tests/lighthouse/budgets.json.
- tests/lighthouse/budgets.json is the editable budget surface — KB
  resource sizes, ms timing budgets, third-party request counts.
- tests/lighthouse/annotate-report.mjs reads .lighthouseci/assertion-results.json
  post-run and emits ::error / ::warning / ::notice workflow commands so
  failures show as native check annotations with the URL, audit, expected,
  and actual values (consistent with how visual regression annotates).

Mobile-only this iteration; lhci defaults to the mobile preset.

Local workflow added to package.json:
- pnpm lighthouse:run     — boot lhci end-to-end against the running stack
- pnpm lighthouse:collect — capture only (skip assert/upload)
- pnpm lighthouse:assert  — assert against the most recent collection
- pnpm lighthouse:open    — open the captured HTML reports

@lhci/cli@^0.14.0 added to devDependencies.

Soft-fail mode (`continue-on-error: true`) is on while the suite settles —
flip once results are stable against main and tighten the starting budgets
at the same time. No fixed flip date.

Trend tracking deliberately deferred to a future issue: temporary-public-storage
gives ~7 days of browseable reports per run; in-repo history can be added
later via an LHCI server or a JSON-summary-commit bot.

Closes #20

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@PAMulligan PAMulligan merged commit 08327ee into main May 20, 2026
7 checks passed
@PAMulligan PAMulligan deleted the 20-add-performance-budget-enforcement branch May 20, 2026 15:11
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Add performance budget enforcement in CI

1 participant