Testing the accessibility of pattern libraries

When assessing how accessible your website or app is, auditors will likely want to look at whole pages. But perhaps most of what you are working on is individual components. I mean, this is how lots of companies work nowadays. Can pattern libraries be tested for accessibility? And if so, what do we test? In this post I will address those questions and look at accessibility testing in different levels of pattern library driven development.

Note that this post is mostly about whether it can be done, as opposed to how, which is a subject all of itself. I may do a post about that later.

A pattern is a reusable piece of your user interface, also known as component or module. It often documents the UI design and is made of code that runs in the browser. Examples of patterns: buttons, messages, accordions, headers, sliders, breadcrumbs and tooltips. A pattern library is a collection of such patterns and slightly more than that. As Alla Kholmotova says in her upcoming book Design systems, it is “a tool to capture, collect and share design patterns and guidelines for their usage.”

With our interface separated into building blocks, it would be great if we can do our accessibility testing pattern by pattern, too. Can we? Well, yes and no.

Let’s start with the bad news. To be able to say something about how accessible your whole site is, you need to assess whole pages, WCAG2 says:

2. Full pages: Conformance (and conformance level) is for full Web page(s) only, and cannot be achieved if part of a Web page is excluded.

But there’s good news, too: there are heaps of accessibility issues that can be spotted on a pattern level, or criteria that can be marked as sufficiently met. Let’s look at how that would work.

What auditors test your site for

When deciding whether something is easy to access for as many people as possible, auditors will make sure four things are the case. They’ll want your site to be:

  • Perceivable
  • Operable
  • Understandable
  • Robust

In WCAG2, the accessibility guidelines, there is a number of criteria categorised under each of these main principles. Knowing those criteria is a great way to learn about making more accessible interfaces. There are quite a few. The actual spec is a good place to read about them and it is also the official norm. There is also the WebAIM checklist, which has slightly more developer-friendly wording and (opinionated) recommendations on how to meet the criteria — it’s a great resource!

Although you cannot formally claim your site’s WCAG2 conformance based on audits of just its components, as we’ve seen above, you can look at many of the criteria on a component level. This can have advantages in comparison with testing whole pages.

Patterns as isolated test cases

At the Patterns Day conference last month, Rachel Andrew mentioned something interesting about patterns. She said that working with reusable interface components, where each one has its own page, made her realise that those work quite well as isolated test cases. I feel this also goes for some accessibility tests: there is a number of criteria where isolation aids testing.

Component-specific guidance is available

Not only can isolating patterns be useful in the testing phase, there is also lots of documentation for accessibility-proofing components specifically. WAI ARIA Authoring Practices describes of common patterns how to get them accessible, with examples of keyboard operability and what should happen to focus. There is also Heydon Pickering’s Inclusive Components Club, which describes in detail what optimisations can be made and how they help whom.

Individual patterns

Let’s look at some things that can be tested when looking just at individual patterns. In order to avoid this to become too high level, I will give some suggested questions for specific, real-world-ish use cases.

Collapsible (“Show/hide”)

Form field

Video player

Assuming your final page only has pattern libary markup, CSS and JS, all of the above things will yield the same test result in isolation and when used in a page. This is what makes them suitable to test on a component level.

I think for most components, there’s a whole lot of accessibility criteria that can be tested. There is also a number of things where we need whole pages, and in some cases, whole pages with the real data. So let’s look at those next.

Whole pages

Some criteria can only be tested in the context of a full page. Some examples:

Headings

  • Are no heading levels skipped?

IDs

Tab order

  • Can we tab through the page and does tabbing make sense?

Whole pages with real data

Lastly, there are also some aspects that can only be tested when pages are composed and they have real content in them. For instance:

Language

Video player

Alternative content

  • Can I find out what’s happening in that graph without seeing it?
  • Does the alt attribute of that image describe what’s in the image (if it is not decorative)?

TL;DR

Although claiming a site meets the WCAG2 criteria has to be based on research of full pages, there are lots of things you can figure out and tackle while working on components with mock data. Above, I have given some examples of such things. I’ve also shown some examples of criteria that would require full and/or finished pages to be properly assessed.

As always, your mileage may vary! I would recommend checking the components you have against the WCAG2 criteria to see which ones can be checked on component level and which require full pages. Hopefully, the examples give an idea of what kind of things can be tested on which level, and when to run (automatic) accessibility tooling through patterns or full pages.

Comments, likes & shares (13)

@leaverou (to give you a few random thoughts)I think it's useful to document for components which WCAG criteria apply to them how and maybe specify manual checks one can do after using (USWDS does a great job at this, see https://designsystem.digital.gov/components/button/accessibility-tests/) Button accessibility tests