This is an unpublished draft preview that might include content that is not yet approved. The published website is at w3.org/WAI/.

Involving Users in Evaluating Web Accessibility

Video: Involving Users in Web Accessibility Overview

This video is also available on a W3C server: Video: Involving Users in Web Accessibility Overview (file format: MP4, file size: 38MB).

Text Transcript with Description of Visuals

Audio Visual
Involving users in web accessibility. Involving users in web accessibility.
Accessibility is about making your websites and applications usable by people with disabilities. That includes your customers, clients, employees, students, and others. Accessibility. A person in front of a computer. Next to the computer the words: customers; clients; employees; and students.
Unfortunately many approach accessibility just as a checklist. This risks missing the real purpose of accessibility - the user experience. A checklist replaces the person. The website on the screen crumbles. 4 people replace the computer and checklist.
Involving people with disabilities throughout your design and development process can be more effective and yield better results:
  • designers and developers learn how people with disabilities use the web, and understand assistive technologies and adaptive strategies they use;
The group of people are surrounded by icons inserted in a development process cycle: pen icon; coding icon; paintbrush icon; and a magnifying glass icon. Smaller versions of these people are integrated in the process cycle.
(continued list)
  • the project team is more motivated when they understand how their work impacts people's lives;
The process icons remain as the people are replaced with a motivation gauge.
(continued list)
  • development is more efficient, and your products work better for more people, with and without disability;
The process icons remain as the gauge is replaced with a graph showing an upward trend in results. The chart is replaced with multiple people figures.
(continued list)
  • your resulting products are more inclusive and reach a bigger audience, have higher customer satisfaction, and demonstrate social responsibility.
The people increase in numbers as the process icons are removed. A 5 star rating appears above the people to get all 5 stars filled. The people transition into a globe with a heart in the middle.
"Involving Users in Web Projects for Better, Easier Accessibility" provides guidance on project planning, and throughout the design and development process. Involving Users in Web Projects for Better, Easier Accessibility. The development process cycle with icons appears.
"Involving Users in Evaluating Web Accessibility" provides more specific guidance on the evaluation stage of the process. Involving Users in Evaluating Web Accessibility. The icons in the process cycle are focused in turn with a magnifying glass.
Together these resources help you focus on accessibility for your website users rather than focusing on technical requirements only. Involving Users in Evaluating Web Accessibility and Involving Users in Web Projects for Better, Easier Accessibility. merge into a person in front of the computer. On the screen, passes and fails are marked.
Web accessibility: essential for some, useful for all Icons around a computer: hand; eye; brain; ear; and mouth with sound waves.
For information on involving users in web accessibility, visit w3.o-r-g/W-A-I/involve-users. Involving Users in Web Accessibility, W3C and Web Accessibility Initiative (WAI) logos.

Introduction

Web accessibility evaluation often focuses on conformance to accessibility standards such as WCAG. While conformance is important, there are many benefits to evaluating with real people. For example, to learn how your website or web tool really works for users and to better understand accessibility issues. Evaluating with users with disabilities and with older users identifies usability issues that are not discovered by conformance evaluation alone.

This page is part of a multi-page Evaluating Web Accessibility resource suite that outlines different aspects of evaluating web accessibility. It is the second of two pages on including users in web projects.

Please read Involving Users in Web Projects for Better, Easier Accessibility, which covers broader issues of including users early in website design, tool development, standards, and other web projects.

Initial Review

A first step in evaluating web accessibility is reviewing the website to check for any obvious accessibility problems. For example, some checks are in Easy Checks – A First Review of Web Accessibility.

Even web developers with little accessibility knowledge can find some accessibility issues through an initial review. An accessibility expert with first-hand experience of how people with different disabilities interact with the web can:

The initial review identifies any significant accessibility barriers to fix before evaluating with users. It also helps define what to focus on for evaluation with users.

Range of User Evaluation

Users with disabilities and older users can be included in a wide range of evaluation activities, from brief consultations to large-scale usability studies. There are many options in between these extremes:

What type of evaluation you do depends on factors such as the stage in your project, for example, initial investigation of design ideas, testing specific areas of prototypes, or reviewing final designs.

Conducting informal evaluations throughout development is more effective than only formal usability testing at the end of a project.

Basics

In most cases, including users in evaluation involves:

See Involving Users in Web Projects for Better, Easier Accessibility for guidance on getting a range of users and users’ experience interacting with the web

Just as with any evaluation with users, whether you include novice, average, or advanced users depends on your target users. For example, if you are developing a web application for accountants inside a company, you probably want advanced assistive technology users. And for a public website to apply for disability benefits, you want novice assistive technology users.

Caution: Carefully consider all input. Avoid assuming that input from one person with a disability applies to all people with disabilities.

A person with a disability does not necessarily know how other people with the same disability interact with the web. They might not know enough about other disabilities to provide valid guidance on other accessibility issues. Getting input from a range of users is best.

Note: In addition to finding accessibility problems, evaluating with users with disabilities usually reveals general usability problems that impact all users, including those without disabilities.

Analyzing Accessibility Issues

Web accessibility depends on several components of web development and interaction working together, including web browsers, assistive technologies (AT), and web content.

Accessibility problems can be caused by one or more different components. For example, if a user who cannot use a mouse has trouble with keyboard access, it could be because:

Combine User Evaluation with Standards

Involving users with disabilities in evaluation has many benefits. Yet that alone cannot determine if a website is accessible. Combine user involvement with evaluating conformance to WCAG to ensure that accessibility is provided to users with a range of disabilities and situations.

Drawing Conclusions and Reporting

Be careful drawing conclusions from limited evaluations or studies. Results from only a couple of people with disabilities cannot be generalized to apply to all people with similar disabilities or people with other disabilities. See the Caution above.

Include in the reports the scope of the study and the evaluation parameters, such as testing methods and user characteristics. For example, if a study included only participants who are blind, the report should clarify that it did not evaluate conformance to accessibility standards and that it does not apply to all people with disabilities. While small studies often provide useful information, they are not robust enough to provide statistical significance.

Note for usability professionals

When specifically exploring accessibility barriers, the protocol is usually different from a general usability test. For example:

Note that it is also important to evaluate other factors for users with disabilities. These factors include: General usability, user satisfaction, and other similar criteria.

The More Information section below includes additional guidance specifically for usability professionals.

More Information and Guidance

This document briefly addresses a few points of a very complex topic. Many resources on other aspects of involving users in evaluation are available on the Web.

Back to Top

This is an unpublished draft preview that might include content that is not yet approved. The published website is at w3.org/WAI/.