Skip to content

docs: tidy conformance table + drop shields.io badges from compliance report #24

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
chad-loder merged 1 commit into from
May 13, 2026
Merged

docs: tidy conformance table + drop shields.io badges from compliance report #24

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
18 changes: 9 additions & 9 deletions README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -49,15 +49,15 @@ WPT corpus is SHA-pinned by [`scripts/fetch_references.sh`](scripts/fetch_refere
to commit [`dd54691`](https://github.com/web-platform-tests/wpt/commit/dd54691426c23a08c6f4a0972b2c40965307e5ce)
(2026-05-11) so the pass count is reproducible at any future date.

| Suite | Source | Cases | Status |
|---|---|---:|:---:|
| `urlpattern.any.js` | WPT &nbsp;·&nbsp; `urlpatterntestdata.json` | 366 | <kbd>✓</kbd> &nbsp; 366 / 366 |
| `urlpattern-constructor.any.js` | WPT *(inline)* | 4 | <kbd>✓</kbd> &nbsp; 4 / 4 |
| `urlpattern-hasregexpgroups.any.js` | WPT &nbsp;·&nbsp; `urlpattern-hasregexpgroups-tests.js` | 55 | <kbd>✓</kbd> &nbsp; 55 / 55 |
| `urlpattern-compare.tentative.any.js` | WPT &nbsp;·&nbsp; `urlpattern-compare-test-data.json` | 25 | <kbd>✓</kbd> &nbsp; 25 / 25 |
| `urlpattern-generate.tentative.any.js` | WPT &nbsp;·&nbsp; `urlpattern-generate-test-data.json` | 19 | <kbd>✓</kbd> &nbsp; 19 / 19 |
| yarlpattern unit tests | this repo &nbsp;·&nbsp; tokenizer / parser / parts / regex / engine / pattern | 130 | <kbd>✓</kbd> &nbsp; 130 / 130 |
| **Total** | | **599** | <kbd>✓</kbd> &nbsp; **599 / 599** |
| Suite | Source | Status |
|---|---|:---|
| `urlpattern.any.js` | WPT &nbsp;·&nbsp; `urlpatterntestdata.json` | ✅ &nbsp; 366 / 366 |
| `urlpattern-constructor.any.js` | WPT *(inline)* | ✅ &nbsp; 4 / 4 |
| `urlpattern-hasregexpgroups.any.js` | WPT &nbsp;·&nbsp; `urlpattern-hasregexpgroups-tests.js` | ✅ &nbsp; 55 / 55 |
| `urlpattern-compare.tentative.any.js` | WPT &nbsp;·&nbsp; `urlpattern-compare-test-data.json` | ✅ &nbsp; 25 / 25 |
| `urlpattern-generate.tentative.any.js` | WPT &nbsp;·&nbsp; `urlpattern-generate-test-data.json` | ✅ &nbsp; 19 / 19 |
| yarlpattern unit tests | this repo &nbsp;·&nbsp; tokenizer / parser / parts / regex / engine / pattern | ✅ &nbsp; 130 / 130 |
| **Total** | | ✅ &nbsp; **599 / 599** |

→ [**Full per-case conformance report**](docs/wpt-compliance.md) (regenerate via `just compliance-report`)
&nbsp;·&nbsp; [**Documented deviations and stricter-than-yarl rules**](SPEC_DEVIATIONS.md)
Expand Down
8 changes: 1 addition & 7 deletions docs/wpt-compliance.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,15 +1,9 @@
# WHATWG URLPattern Conformance Report

Generated by `scripts/generate_compliance_report.py` on **2026-05-13 03:13:47 UTC**, running against [`web-platform-tests/wpt/urlpattern/`](https://github.com/web-platform-tests/wpt/tree/dd54691426c23a08c6f4a0972b2c40965307e5ce/urlpattern) pinned at [`dd54691`](https://github.com/web-platform-tests/wpt/commit/dd54691426c23a08c6f4a0972b2c40965307e5ce) with regex engine **`regex`** (set-operation support: yes). Suite names match the upstream WPT runner basenames.
Generated by `scripts/generate_compliance_report.py` on **2026-05-13 04:06:52 UTC**, running against [`web-platform-tests/wpt/urlpattern/`](https://github.com/web-platform-tests/wpt/tree/dd54691426c23a08c6f4a0972b2c40965307e5ce/urlpattern) pinned at [`dd54691`](https://github.com/web-platform-tests/wpt/commit/dd54691426c23a08c6f4a0972b2c40965307e5ce) with regex engine **`regex`** (set-operation support: yes). Suite names match the upstream WPT runner basenames.

> **Legend.** <kbd>✓</kbd> pass · <kbd>✗</kbd> fail · <kbd>◐</kbd> xfail (known engine gap) · <kbd>◑</kbd> skip · <kbd>⚠</kbd> error.

![urlpattern.any.js 366/366](https://img.shields.io/badge/urlpattern.any.js-366%2F366-2ea043?labelColor=24292f)
![urlpattern-constructor.any.js 4/4](https://img.shields.io/badge/urlpattern-constructor.any.js-4%2F4-2ea043?labelColor=24292f)
![urlpattern-hasregexpgroups.any.js 55/55](https://img.shields.io/badge/urlpattern-hasregexpgroups.any.js-55%2F55-2ea043?labelColor=24292f)
![urlpattern-compare.tentative.any.js 25/25](https://img.shields.io/badge/urlpattern-compare.tentative.any.js-25%2F25-2ea043?labelColor=24292f)
![urlpattern-generate.tentative.any.js 19/19](https://img.shields.io/badge/urlpattern-generate.tentative.any.js-19%2F19-2ea043?labelColor=24292f)

## Summary

| Suite | Total | Pass | XFail | Skip | Fail | Error |
Expand Down
45 changes: 6 additions & 39 deletions scripts/generate_compliance_report.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4,19 +4,17 @@
Standalone script (not packaged in the wheel). Runs every WHATWG
``web-platform-tests/wpt/urlpattern/`` case against :class:`URLPattern`,
captures the outcome of each, and emits a structured Markdown report
with shields.io badges + accessible Unicode-symbol status indicators.
with accessible Unicode-symbol status indicators.

The script reproduces the harness logic from ``tests/test_wpt.py`` and
the auxiliary test files inline — running it does not require pytest.
``just compliance-report`` is the canonical invocation.

The output is meant for human reading on GitHub: collapsible
``<details>`` blocks per suite, summary table at the top, per-case
rows with pattern previews and pass/fail symbols. Colors come from
shields.io (high-contrast green/red/blue/grey palette that's
distinguishable for the common color-vision-deficiency types) and
never carry meaning alone — every status is paired with a symbol and
a word.
rows with pattern previews and pass/fail symbols. Status is conveyed
by a Unicode glyph paired with a status word — color never carries
meaning alone.
"""

from __future__ import annotations
Expand Down Expand Up @@ -63,26 +61,17 @@ def _wpt_ref() -> str:
return "main" # best-effort fallback for a non-git corpus


# -------------------------- symbols + shields.io palette ---------------------
# -------------------------- per-case status symbols --------------------------
#
# Accessibility rule (matches the README): the symbol carries the meaning;
# color is decorative. Every cell pairs a Unicode glyph with a status word
# and (in summary tables) a shields.io badge whose alt text restates the
# result.
# every cell pairs a Unicode glyph with a status word.

SYM_PASS = "✓"
SYM_FAIL = "✗"
SYM_XFAIL = "◐"
SYM_SKIP = "◑"
SYM_ERROR = "⚠"

# Wong-style colorblind-safe palette (paired with shape + text)
COLOR_PASS = "2ea043" # GitHub success-green
COLOR_FAIL = "cf222e" # GitHub danger-red
COLOR_XFAIL = "1f6feb" # informational blue
COLOR_SKIP = "6e7681" # muted grey
COLOR_ERROR = "bf8700" # attention yellow


# -------------------------- result data classes -----------------------------

Expand Down Expand Up @@ -436,13 +425,6 @@ def add(name: str, pattern: dict[str, str], *, expected: bool) -> None:
# -------------------------- rendering ---------------------------------------


def _badge(label: str, value: str, color: str) -> str:
"""One shields.io badge anchored at the suite's section."""
safe_label = label.replace(" ", "%20").replace("/", "%2F")
safe_value = value.replace(" ", "%20").replace("/", "%2F")
return f"![{label} {value}](https://img.shields.io/badge/{safe_label}-{safe_value}-{color}?labelColor=24292f)"


def _status_cell(status: str) -> str:
"""One Unicode-symbol-and-word cell for a single case result."""
return {
Expand All @@ -454,17 +436,6 @@ def _status_cell(status: str) -> str:
}[status]


def _suite_badge(suite: SuiteResult) -> str:
"""Header badge for one suite — picks color from the result mix."""
if suite.is_clean and suite.passing == suite.total:
return _badge(suite.title, f"{suite.passing}/{suite.total}", COLOR_PASS)
if suite.is_clean and suite.skipping == suite.total:
return _badge(suite.title, "skipped (tentative)", COLOR_SKIP)
if suite.is_clean:
return _badge(suite.title, f"{suite.passing}/{suite.total}", COLOR_PASS)
return _badge(suite.title, f"{suite.failing + suite.erroring} failing", COLOR_FAIL)


def _render(suites: list[SuiteResult]) -> str:
out: list[str] = []
engine = get_default_engine()
Expand Down Expand Up @@ -496,10 +467,6 @@ def _render(suites: list[SuiteResult]) -> str:
)
out.append("")

# ---- top-of-page badges
out.extend(_suite_badge(suite) for suite in suites)
out.append("")

# ---- summary table
out.append("## Summary")
out.append("")
Expand Down
Loading