[test][button] Add axe test coverage and docs#48708
Conversation
Deploy previewBundle size
Check out the code infra dashboard for more information about this PR. |
99db26a to
014886f
Compare
|
@mj12albert the accessibility.md is not auto-generated right? I want to get alignment on the plan to make them public on the docs, what's your view on this? |
No ~ it's impossible to auto-generate (at least now) 😅, I kick-started research with AI but eventually had to manually re-do ~80% of it anyway, so for each I gave it my manual notes and initial draft as a start Possibly once we have a few more of these, there will be more reference material that improves the results, but the fundamental issue is that AI is unreliable at "understanding" the WCAG specs (esp when there's no direct code to trace) Possibly we can later automate the aggregation of component level scorecards: into a library-level scorecard
I agree we should do this eventually, but that's a matter of presentation; for now I'm solely focused on scoring components, first so there is a body of content to present. IMO we don't need to worry about this until we have ~20 components scored. Does that make sense? @siriwatknp |
| ## Reports | ||
|
|
||
| - [Button](./Button/accessibility.md) |
There was a problem hiding this comment.
Instead of individual reports I would like a table, where the findings are aggregated into a per level (A / AA) report. This could effectively look similar to this:
| Component | WCAG 2.2 A | WCAG 2.2 AA |
|---|---|---|
| Button | ✅ (8/8 ) |
...
There was a problem hiding this comment.
Yes, the component level ones can eventually be aggregated into being displayed something like this, though x/total AA may not accurately partial/irrelevant criteria, so far I'm just following a somewhat standard formats/terminology
| export const A11Y_RULES: A11yRule[] = [ | ||
| { test: 'docs/data/material/components/buttons/{BasicButtons,ColorButtons}', enabled: true }, | ||
| { | ||
| test: 'docs/data/material/components/buttons/{BasicButtons,TextButtons,ContainedButtons,DisableElevation,OutlinedButtons,ColorButtons,ButtonSizes,IconLabelButtons,InputFileUpload,LoadingButtons,CustomizedButtons,ButtonA11yNonNative,ButtonA11ySemanticStates,ButtonA11yTextSpacing}', |
There was a problem hiding this comment.
Pretty hard to read. Can we maybe use an array of strings here that we define outside of the scope and just spread it here?
There was a problem hiding this comment.
Updated, I think quite soon we can't pack everything in a single file either or it'll get ugly
2edd27f to
7bd80c7
Compare
7bd80c7 to
4654ad4
Compare
fair enough. |
|
|
||
| #### 1.3.3 Sensory Characteristics · A | ||
|
|
||
| `🚩 Unverified` · `✅ Supports` · `○ Author` |
There was a problem hiding this comment.
The Unverified confuses me or at least better to present differently. In my understanding it means "not audited by trusted firms" right?
If yes, I think it's better to be removed and declare globally in accessibility.md instead on how we do audit manually.
Currently there are too many "Unverified" which is expected because we don't plan to do external audit soon.
There was a problem hiding this comment.
Unverified basically means both:
- It isn't covered by tests (because it's not test-able, e.g. resize, reflow), and
- It hasn't been manually audited and recorded – we can audit it ourselves; I've verified most of them but just keeping them flagged for now because I didn't fully cover all OS/browser/SRs (e.g. on the Windows side I only tested NVDA+Edge, but JAWS is most popular SR and I don't have it)
Open to suggestion if you think Unverified sounds bad 😅 but they need to be flagged somehow regardless
I haven't thought in depth about how to document manual audit records, not sure if it should be docs committed into the repo or something else as it will require manual maintenance
There was a problem hiding this comment.
Updated in 49a91ae to separate the flags from the official criteria with a simpler explanation
2332833 to
49a91ae
Compare
Doc: https://github.com/mj12albert/material-ui/blob/docs-button-axe-coverage/packages/mui-material/src/Button/accessibility.md
Expand axe test coverage for
Buttonto cover as much of WCAG as possible, and document the component's current WCAG conformance and testing notes/procedures.