Kick Start on Accessibility and Usability Testing

Abstract: This course gives the fundamentals of accessibility, how to evaluate accessibility(Importance of Comprehensive Accessibility Evaluation, Standards Review, Heuristic Evaluation, Design Walkthroughs, Screening Techniques and Usability Testing) ,brief focus on usability testing and how to understand the difference between accessibility and general usability. CONTENTS: 1. Accessibility 1.1. What is Accessibility? 1.2. Evaluating for Accessibility 1.2.1. Importance of Comprehensive Accessibility Evaluation 1.2.2. Standards Review 1.2.3. Heuristic Evaluation 1.2.4. Design Walkthroughs 1.2.5. Screening Techniques 1.2.6. Usability Testing

2. Usability Testing
2.1. Planning Usability Testing 2.1.1. Determining Participant Characteristics 2.1.2. Recruiting Participants with Disabilities 2.1.3. Choosing the Best Location 2.1.4. Scheduling the Right Amount of Time 2.2. Preparing for Usability Testing 2.2.1. Ensuring the Facility is Accessible 2.2.2. Preparing Test Materials 2.2.3. Setting Up and Testing the Participants' Configurations 2.2.4. Becoming Familiar with the Assistive Technology 2.2.5. Conducting Pilot Testing 2.3. Conducting Usability Testing 2.3.1. Setting up the Room 2.3.2. Orienting the Participant

2.3.3. Completing Paperwork 2.3.4. Completing the Tasks 2.3.5. Collecting Data 2.3.6. Providing Compensation

2.4. Reporting Usability Testing 2.5. Checklist for Usability Testing

2.6. Recruiting Screener
3. Understand the difference between accessibility and general usability 3.1.1. Usability Problems 3.1.2. Accessibility Problems 3.2. Distinguish between usability and accessibility issues, as appropriate

1. and understandable for people with a wide range of abilities. speech.2.2.1. What is Accessibility? Accessibility basically means that people with disabilities can use a product. or lose your glasses. Importance of Comprehensive Accessibility Evaluation 1.2. Situational limitations come from circumstances.3. It encompasses all disabilities. in a dark room. More specifically. Importance of Comprehensive Accessibility Evaluation Accessibility evaluation is often limited to assessing conformance to accessibility standards. situational limitations include using the Web on a mobile phone when your eyes are busy (such as driving). Conformance to accessibility standards is important: in some cases it's a legal requirement and in others .2. people without disabilities as well. physical. This includes temporary conditions.4. Standards Review 1. auditory. or functional limitations. in a noisy environment (where you can't hear well). in bright sunlight. accessibility is making user interfaces perceivable.2.6. such as when you break your arm. Thus.2. 1. Accessibility also makes products more usable by people in a wide range of situations.2. and neurological disabilities. Heuristic Evaluation 1. cognitive.1. including visual.2. For example. Usability Testing 1. when your hands are full. it also benefits people without disabilities and organizations that develop accessible products because designing for functional limitations overlaps with designing for situational limitations. and conditions. operable.1. Evaluating for Accessibility Information on incorporating accessibility into the following evaluation methods: 1. and in an emergency (when you may not be thinking clearly).2. environments. while access to people with disabilities is the primary focus of accessibility.5. Design Walkthroughs 1. in a quiet environment (where you don't want it to make noise). Screening Techniques 1. and can affect anybody—that is.

2. Help fix any known accessibility barriers before bringing in users. we probably want to do lots of informal evaluation with them on early design prototypes. In the more common case where it takes more effort to get people with disabilities for evaluation. heuristic evaluation. and Focus usability testing or informal evaluation with users on potential areas of concern.2. An accessibility expert with first-hand experience of how people with different disabilities interact with products can: • • • Evaluate accessibility issues for a broad range of users. Standards Review A standards review in the User-Centered Design process assesses whether a product conforms to specified interface design standard. which might not be found by a few individual users in usability testing. when the focus is only on the technical aspects of accessibility. Accessibility standards and guidelines are available from international standards organizations. industry groups. While each evaluation plan will be different based on resources and other factors. Heuristic Evaluation . screening techniques. Effective accessibility evaluation includes both evaluation expertise and the experience of people with disabilities. national. and usability testing. If we have limited budget. Sometimes the standards are internal style guide recommendations. the human interaction aspect can be lost. Furthermore. Accessibility standards reviews are often more rigorous than typical user interface reviews. especially when conformance to a standard is a legal requirement. 1. and other times they are external standards. such as employees in the same building. state and local governments.2. or we might be able to afford an accessibility specialist. Usability evaluation methods can assess usable accessibility to ensure that your accessibility solutions are usable by people with disabilities. If we have people with disabilities easily available to help with evaluation. we probably want to employ the other evaluation methods first. and individual organizations. user interface issues often overlap with technical issues in accessibility standards reviews.3. 1. we might need to do the evaluations ourself. design walkthroughs.it's just a good way to help check that you've adequately covered the range of accessibility issues. ensure that you employ comprehensive evaluation that includes at least a little of the methods described next: standards review. However.

the design team listens for the team member playing the device to indicate feedback or interaction provided via sound. An example of focusing on specific accessibility issues during regular software walkthroughs is device-independent interaction. To conduct a heuristic evaluation for accessibility. For design walkthroughs of high-fidelity prototypes we can also use Screening Techniques.2. use persons with disabilities and scenarios that include adaptive strategies to complete the task. inexpensive activities to help identify potential accessibility barriers in product designs.5. 1. specialists judge whether each design element conforms to established usability principles. and other interface elements. . The team then checks that all actions triggered with a mouse are also available through the keyboard for people who don't use pointing devices. Design teams use screening techniques to learn about accessibility issues and to evaluate prototypes or existing products.2.In a heuristic evaluation. a person acts as a representative user while a design team member guides her through actual tasks with early prototypes. The design team listens for the acting user to say. changing paper mockups of windows. and by focusing later usability testing with people with disabilities. Ways to incorporate accessibility into design walkthroughs include: • • Focus on specific accessibility issues during regular walkthroughs. Design Walkthroughs The purpose of a design walkthrough is to find potential usability problems by envisioning the user's route through an early concept or prototype. the acting user would be blind and another design team member would play the role of the screen reader. Sometimes another team member plays the computer or device. when it is less expensive to make changes to the product. 1. For example. Another example of a specific accessibility issue to evaluate during design walkthroughs is use of sound. When walking through use of a consumer product. drop-down menus. Conduct walkthroughs specifically for accessibility. Typically.4. indicating an action that is completed with a mouse. accessibility specialists judge whether design elements conform to accessibility principles. I would click on this. pop-up dialog boxes. Screening techniques save time and money by finding barriers early. To conduct walkthroughs specifically for accessibility. Screening Techniques Screening Techniques are simple.

Preparing for Usability Testing covers ensuring the facility is accessible.3 Conducting Usability Testing 2. While usability testing evaluates how usable accessibility solutions are by some people with disabilities. recruiting participants.2 Preparing for Usability Testing 2.4 Reporting Usability Testing 2. We don't have to be usability professional and we don't have to follow formal usability testing protocols to include people with disabilities in evaluation. choosing the best location. it does not evaluate conformance to accessibility standards. preparing test materials. Short informal evaluation can gather valuable feedback from people with disabilities without the rigor of formal usability testing. While usability testing is useful for learning how people use our products and assessing the usability of accessibility solutions. Discussing accessibility issues with them. including users in evaluation involves: • • • • Finding a few people with disabilities. Usability Testing Usability testing provides quantitative and qualitative data from real users performing real tasks with a product. setting up and testing participants' configurations. Observing them interact with the prototype. with a few modifications for including participants with disabilities. The following sections discuss usability testing with participants with disabilities: 2.6. and scheduling the right amount of time. Usability Testing 2.1 Planning Usability Testing 2. In most cases. becoming familiar with the assistive technology.6 Recruiting Screener • • Planning Usability Testing covers determining participant characteristics. Usability professionals can evaluate some aspects of accessibility by using standard usability testing protocols.2. and conducting pilot testing. Asking them to complete tasks on prototypes.5 Checklist for Usability Testing 2.1. . stability testing can't address all accessibility issues and doesn't evaluate conformance to accessibility standards.

Some studies have led to the conclusion that testing a large number of participants does not yield significantly more information than testing fewer participants. Scheduling the Right Amount of Time 2. Determining Participant Characteristics 2. completing the tasks. and reporting usability studies with participants with disabilities. collecting data. Determining Participant Characteristics There is no definitive answer to the questions.2. "How many participants with disabilities should be included in usability testing?" and "What characteristics should they have?" It depends.3.1.1.1.• Conducting Usability Testing covers setting up the room. being careful about categorizations and comparisons. Understanding the challenge The number of participants to include in any usability test is a subject of debate among usability professionals. 2. Recruiting Screener lists question to ask during recruiting. we need to test additional users. When we have several highly distinct groups of users. including relevant study parameters. and so a thorough usability test for accessibility . Recruiting Participants with Disabilities 2.4.1. Some research shows that five participants is sufficient to find 85% of the usability issues when you have comparable users who will be using the product in fairly similar ways. Planning Usability Testing Planning a usability study that includes people with disabilities involves the following considerations: 2. orienting the participant.1. preparing for. completing paperwork. clarifying conclusions.1.1. and writing about people with disabilities. • • Checklist for Usability Testing summarizes the tasks and considerations involved in planning. People with disabilities use products differently. providing compensation. and specific considerations for people with different disabilities. since the first participants will find most of the usability problems. conducting. This section explains factors that can help answer those questions for our particular situation.1. Choosing the Best Location 2. • Reporting Usability Testing covers distinguishing between accessibility and usability issues.

cognitive) are not sufficient for categorizing usability study participant characteristics. one recommendation is to include three users from each category. a young man who recently went totally blind from retinitis pimentos’ and is a new screen reader user. an elderly woman whose sight is deteriorating from macular degeneration yet she doesn't use any assistive technology. and vice versa. people with visual disabilities include a middle-aged woman who has low vision since birth and is very experienced with screen magnification software. However. For example. In deciding the number of participants with disabilities to include. static. Careful with categorization Disabilities are sometimes grouped into four high-level categories: visual. This variability within categories is significant in product design and evaluation. For example. The following considerations can help us make the most of limited resources and focus on which participant characteristics to include in our usability tests. (The next section includes guidance for using fewer than five participants with disabilities. The number of usability test participants with disabilities included in a given usability test is usually determined by limited project resources. or regressive. although visual disabilities are often categorized together. permanent.requires more than five users. cognitive. the four common categories of disability (visual. physical.) When we have different categories of users. Identifying a realistic range of participants Projects rarely have the time and money resources to do thorough usability testing with a wide range of participants with disabilities. and a young boy whose color blindness has not yet been diagnosed. How a person interacts with products is also impacted by differences such as: whether the person was born with the disability. whether the disability is temporary. consider that users with . and yet totally inaccessible and unusable for a person with low vision. Because of this variability within a category. progressive. a product can be accessible and usable for a person who is blind. yet there is vast variability within each category. auditory. Understand the overlap between accessibility and usability. people with disabilities do not fit easily into categories in terms of product interaction. or acquired it when older. and what adaptive strategies and assistive technology the person uses. auditory. Also note that it is common for people to have multiple disabilities across more than one category. physical. acquired the disability at a young age.

Explaining how users with disabilities also address general usability can help us get more time and money budgeted to include users with disabilities in usability testing and throughout our project. including users without disabilities. a usability test for a product that is marketed primarily to seniors should include seniors who have age-related disabilities.2. the longer it takes to recruit.1. anyone can test using a cell phone with one hand. Focus on the target users. Plan for additional recruiting With any usability test. For example. Focus on highest impact. if we are designing an intranet (internal website) and the company standard is a specific screen reader. In some cases we can include participants with different characteristics in each round of testing. You might not need to include participants with certain characteristics because of the nature of the product or the situation. some aspects can’t be effectively evaluated with screening techniques. product designs go through several iterations of usability testing. focus on those relevant characteristics. we don’t have to have a disability to do that. However. For example. we might choose not to actively recruit participants who are hard of hearing. 2. In many cases general usability issues are amplified when testing with participants with disabilities. Some products will impact people with a specific disability more than people with another disability. If our target users include a higher percentage of people with a certain disability. Include different characteristics in different usability tests. making it easier to find issues that impact all users. we might choose not to include participants who use other screen readers.disabilities also address general usability issues that impact all users. For example. if we don’t have much experience with screen readers. For example. and a test for a website containing information on diabetes should include people with visual disabilities. if we are designing a product that produces no sound. it turns out to be fairly easy. Ideally. For example. usability testing with participants with disabilities will identify both accessibility issues and general usability issues. we won’t be able to effectively test our product with one. yet after a little bit of effort. the more specific the participant requirements. rather than trying to fit participants representing all characteristics into every test. Recruiting Participants with Disabilities Often usability specialists think it will be difficult to recruit people with disabilities. Screening techniques can effectively supplement testing with people with disabilities in some cases. Therefore. recruiting doctors age 35-55 who use Macs takes .

etc. Specific questions to add to the screener are included in the Recruiting Screener section. there might be some additional cost . consider conducting pilot tests early in your recruiting efforts and seek out participants who are well-respected. (This is usually not the case when a person with a physical disability has a personal assistant. Additional information to cover in the recruiting screener includes technology use. The same is true of recruiting people with disabilities. they will make the arrangements. Some people with disabilities are part of networks that actively share accessibility information. Add relevant information to recruiting screener. alternative formats for printed materials. If we are conducting usability studies with participants who are deaf. such as a student organization. Word of a positive usability testing experience can spread rapidly and result in potential participants contacting you. Encourage them to spread the word of our recruiting. Although the basic compensation for people with disabilities is the same as for any participant in your test. such as type of disability and use of assistive technology. Adding specific participant requirements. Make key contacts. and pay for sign language interpreters. will likely add time and effort for recruiting. schedule. Arrange for interpreters as needed. senior center. use the same parameters as we would for participants without disabilities. Places to look for participants with disabilities include the following: • • • • • • • • Organizations for specific disabilities or conditions Cross-disability organizations Mailing lists College and university programs for students with disabilities Local disability-related support groups Local or regional government rehabilitation or disability services departments Seniors organizations and local senior centers Independent living organizations Consider pilot tests as a recruiting tool.longer than if the only requirement is that participants are age 35-55. vocal members of a community.) Plan to reimburse participants for necessary expenses. When screening participants with disabilities for a usability test. On the other hand. it's customary for us to find. our recruiting could take less time than usual because of "viral marketing" within disability communities. or disability support group. To take advantage of this word-of-mouth recruiting.

Providing the required assistive technologies in a lab can be costly.3. When testing software or web-based products. complicated. As part of the recruitment process. Most designers don't know how people with disabilities use their products. participants might require different versions of assistive technologies with different configurations. there are additional factors to consider when making the decision of location. We should usually reimburse for additional transportation expenses. When we want face-to-face interaction. For some participants we might need to pay for the time of a personal assistant or interpreter. Conducting usability tests in your lab usually provides the opportunity for more of the project team to observe. Informal sessions with lots of interaction between designers and participants are especially useful. and many people are uncomfortable around people with disabilities. Consider the goals of the usability test. home. such as an accessible taxi van. When conducting usability tests with people with disabilities. field studies are best. remote evaluation is not an option. and timeconsuming. rather than in a formal lab. For example. Additionally. it may be equally effective and much easier to do some evaluation remotely rather than traveling to the same location. Transportation might be more complicated and costly for participants with disabilities.Choosing the Best Location With any usability test there are advantages and disadvantages of conducting the sessions in a usability lab or "in the field" at the participant's work. Changing system configurations between participants . Assistive technologies are a significant factor in where to conduct usability tests. or other location. Some people use remote evaluation where the facilitators and participants are in different locations.considerations. there are cases where remote evaluation is best. and can be conducted wherever is most convenient for the project team. Consider assistive technology needs. However.1. remote evaluation doesn't provide equivalent results as in-person usability testing. if we already have an established relationship with a person who helps us evaluate for accessibility. and for you to record the sessions. Where the best location is depends partly on the goals and potential additional benefits of the test. 2. find out about participant expenses and confirm with participants which you will reimburse. When the goal is for a couple of people to learn more about how people use the product in their own environment.

accessible transportation is often unreliable and participants using such transportation might not be able to meet a tight schedule in a usability lab.can be difficult. It is common for an information technology specialist. get in and out of the testing room. Participants might not know how to configure assistive technologies in your lab to work as they are used to them working in their regular system. and take breaks. and a person with a cognitive disability might need longer to process textual information and instructions. Transportation may be more difficult for some participants with disabilities. rehabilitation specialist. and post-study survey online. they dropped the web and financial surveys from the study. they planned to have participants complete the consent form. "Ensuring the Facility is Accessible" in the Preparing for Usability Testing section provides specific points to consider regarding the location's accessibility. In one of our the studies with people with disabilities. For example. For example. for example. Scheduling the Right Amount of Time Use pilot tests to work out timing.5 hour session to complete the online forms. Evaluate the accessibility of potential locations. In some cases. the time for each usability test participant session will be impacted by a participant's disability and longer or shorter sessions may be more effective. It will take some people with disabilities longer to complete tasks. cursor. might take longer to do basic task steps such as activate a button. including assistive technologies. It may take some participants longer to complete pre-test paperwork. contrast. financial knowledge assessment. Some assistive technologies have many settings. You might also .1.4. However. especially if the product is not highly accessible or if they haven't used products like it before. You may need to schedule additional time between sessions as well. or family member to set up a person's home or office computer system. and more. 2. such as tremor and poor fine motor control. Plan based on specific disability considerations. screen magnification software provides multiple options for zoom. Plan extra time for the pilot tests and record how long each step takes. Thus. color. their first pilot participant with low vision took almost 1 hour of the planned 1. web expertise survey. Many buildings that claim to be accessible could have barriers that make it difficult for people with disabilities to participate in the test. one site that was technically wheelchair accessible had very thick carpeting that made access with a manual wheelchair difficult. people with some physical disabilities. Take into account transportation. Also.

design walkthroughs.2. medications. Plan time at the beginning of each test session for participants to check that any assistive technologies are set up and configured as they want. In one of the studies. the usability testing schedule might include time for the participant to interact with the product before usability testing begins. and the extra effort required to use assistive technology. becoming familiar with the assistive technology. Preparing for Usability Testing The Evaluating for Accessibility page provides guidance on incorporating accessibility into common evaluation methods. Be aware of energy level considerations. and plan session timing accordingly. On the other hand. Some are used to taking longer to accomplish tasks and have more patience and more determination to complete a task successfully.want to plan for an additional staff person to be available. Schedule time to confirm assistive technology setup. They might want extra time. some participants with disabilities have high energy levels and will be effective longer than some people without disabilities. preparing test materials. and informal evaluation with users with disabilities. heuristic evaluation. When the usability goals are for people somewhat familiar with the product. Fatigue tends to be more of an issue with some participants with disabilities due to factors such as the disability itself. Preparing for a usability test that includes people with disabilities involves the following considerations: . Ask participants during recruiting about any time and fatigue considerations. including standards review. Plan time for the participant to become familiar with the product. for example. Preparing for Usability Testing covers ensuring the facility is accessible. The Usability Testing section is an overview of usability testing with participants with disabilities. to help escort participants around the facility. setting up and testing participants' configurations. and conducting pilot testing. Participants might need breaks and might not be comfortable or effective in long sessions. a participant using a screen magnifier told that he gets nauseous after a half-hour of watching the screen and would need a short break every 30 minutes. 2. A sample question is included in the Recruiting Screener.

3. or other format. electronic file. to the test room. Conducting Pilot Testing 2.2. There are things you can do to increase the accessibility of PDF documents for screen reader users. and space in the room for an interpreter to be in the best position. someone from participant screening who didn't meet the participant criteria. Schedule a walkthrough by a person with similar accessibility needs. Depending on your participants. Preparing Test Materials 2. Ensuring the Facility is Accessible Make sure that the usability testing facility is accessible to participants. release form. or guide dog. so you will need to provide the PDF document with the font already in the user's preferred font face and font size. This will be especially important for participants with some types of cognitive disabilities who have difficulty processing information or instructions. and tasks to be completed during the usability test. plain text. People with low vision cannot increase the font and print PDF documents. and to the bathroom. consent form.1. interpreter. 2.5. Ensuring the Facility is Accessible 2. Provide enough space for a wheelchair. assistive technology. Materials include directions to the facility.2.2. personal attendant.) Audio (read live. Preparing Test Materials Write usability test materials in clear and simple language. Alternative formats include: • • • • Large print Braille Electronic (HTML. or provide the document in a format where the user can set the . for example. Setting Up and Testing the Participants' Configurations 2. consider the following: • • Use a checklist to ensure that you have anticipated any potential barriers. etc. check things such as wheelchair access into the building. To help ensure that nothing is overlooked.2.2.1. via e-mail. Be prepared to provide all materials in alternative formats.2. Becoming Familiar with the Assistive Technology 2.2.2. non-disclosure agreement and instructions for the participant.2. CD.2. cassette tape) Note that some people with disabilities don't like PDF format. as needed. parking space for a van with a side wheelchair lift. Consider using modular tables so that the room can be rearranged.4.

Sending the consent form. consider scheduling a little time to work with the interpreter before the test to discuss precise meanings. In most cases. For formal studies where close translation is important. such as the tasks. Only a small percentage of people who are blind read Braille. Label your Braille pages (if they get mixed up you won't be able to sort them out unless you read Braille). 2. and other such documents. Setting Up and Testing the Participants' Configurations As previously mentioned in "Consider assistive technology needs” in the Planning Usability . Provide consent forms and other documents for interpreters and attendants. cannot be sent ahead of time and will be available in the participant's preferred format. non-disclosure agreements. Include alternate format questions in the recruiting screener. you will need sign language interpreters. doesn't mean that the facilitator should read the tasks to all participants. Plan time to have your documents "brailed" if necessary and remember that you can't make last minute changes to your test materials if you have them brailed elsewhere. some read tasks themselves and others hear tasks read to them.font themselves. with their own assistive technologies. Just because one participant chooses to have tasks read to them. Send materials to interpreters ahead of time. non-disclosure agreement. Sign language interpreters can be better prepared if they have at least a rough script of what you will be saying.2. and at their own pace. Test materials in different formats across participants For almost all usability tests. and any others who accompany the participant in the actual test to also sign consent forms.3. Questions about alternative formats to ask participants during recruiting are included in the Recruiting Screener section. Some people may take a long time to read documents and it helps your scheduling if this is done ahead of time. and any other documents to participants before the test lets them read the documents ahead of time in their own environment. Explain to the participant that some materials. Instead. Send consent forms and other documents ahead of time. personal attendants. it is acceptable to have participants use test materials in different formats. provide the tasks and other test materials in the format that is best for each participant. People who are born blind often learn Braille people who go blind later in life usually don't. for example.

There are more things that could go wrong or be new to the test designers and facilitators. 2. such as acquiring older versions of software.2. such as large fonts and alternative color schemes. the AT. People who don't use assistive technologies might use different system settings. If we are new to usability testing with people with disabilities and if you are . ask a regular AT user to answer your questions and help you learn more about how the AT is used with the product.2. such as problems with assistive technologies. the facilitator might not be able to understand the interaction between the participant. • • Practice using the assistive technology with the product. have them use the AT with a product similar to what you'll be testing.Testing section. Starting early leaves time to address complications. When testing software or webbased products. and test the assistive technologies to the participants' configurations. When possible. Becoming Familiar with the Assistive Technology When participants will use assistive technology (AT) in a usability test. Conducting Pilot Testing Pilot testing is especially important when testing with participants with disabilities. After you are somewhat familiar with the AT. it will be more effective if the facilitator is somewhat familiar with the AT. 2.4. Usability test observers. or figuring out that different configuration will not work on a single computer and we will need multiple computers for backto-back participant scheduling. set up. well in advance of the usability test. participants might require different versions of different assistive technologies with different configurations. and the product being tested (as well as being distracted by the novelty of the AT). data analyzers. and others will also benefit from having some familiarity with the AT used in the test. Otherwise. We can gain varying levels of experience with AT in the following ways: • Get an introduction to and demonstration of the AT from an experienced AT trainer or someone who regularly uses the assistive technology— perhaps someone from participant screening who didn't meet the participant criteria. Get experience with the assistive technology as appropriate. Conduct pilot tests early. Acquire. providing the required configurations can be complicated.5. getting different configurations to work on a single computer.

Use pilot tests to work out issues with assistive technologies. Use pilot tests to work out facilitation. and data recording. During a test with participants using screen magnification.5. Assistive technologies behave differently in different configurations. and with participants who use screen readers we probably want to record audio output. observation. Pilot tests help ensure that there are no unforeseen barriers or difficulties for participants in the study. 2. Orienting the Participant 2. For example. completing the tasks. Collecting Data 2. we can use pilot tests to help with recruiting because often when participants with disabilities have a good experience.planning a formal study. orienting the participant.3. Use pilot tests to work out logistics. with participants who are deaf we need to mix the interpreter. in pilot test you may learn that our non-visual directions to our lab can be improved. conduct at least a couple of pilot tests fairly early in the project so you have plenty of time to work out any issues. Completing Paperwork 2. that we need to find a good place for a guide dog to go outside. providing compensation. As mentioned previously.3.3. and users use them in many different ways. some screen recording software may conflict with some assistive technology. For example.6. collecting data. some observers susceptible to motion sickness were unable to stay through the entire test because of the motion on the screen. 2. or that there is a step we hadn't noticed that makes the main path to the lab inaccessible to wheelchairs. If we don't have enough extra participants from our recruiting we might be able to use someone who didn't meet all of the recruiting criteria.3. Also. Additionally. Completing the Tasks 2. completing paperwork.3.2. Providing Compensation .3.4. we can use pilot tests to work out timing. and specific considerations for people with different disabilities. they will tell others. assistive technologies may require adaptations in our data collection. We may need to adjust our facilitation. For example.3. Conducting Usability Testing Conducting Usability Testing covers setting up the room.

cables. A person who is blind might not have rooms well-lit. it might be confusing to watch only on the display. which can be especially hazardous to participants with visual impairments. use a separate audio capture for the participant's voice and for the screen reader. Position audio recording equipment close enough to hear the screen reader. 2. Setting up the Room Check that the area is clear. Consider recording keystrokes. • Record screen reader audio output. Specific considerations for some participants who are blind or visually impaired • Don't move anything without asking first when at the participant's site. People who cannot see might run into a chair or other object that is in a different place than they are used to. If the participant is using a screen reader. Look for power cords. • Watch the keyboard. • Take speakers and lights for testing at the participant's location. If we do move anything (with permission). set up the room so that the facilitator can see the keyboard and hear the screen reader. and other parameters. for example. By watching the keyboard you can better tell whether the participant is navigating by using the tab key. she is not off camera. at the end of the usability test ask if the participant wants you to put things back in their original place. Participants often talk over the screen reader. depending on the participant.3. the product. Participants using a screen reader don't need to stay in one position in front of the monitor. or the arrow keys. • Adjust video for wide-angle capture. we can adjust the volumes when analyzing data and making highlight tapes. Because the focus on the display does not always match what the screen reader is speaking.1. When feasible. or just letting the screen reader read through the entire page. If each is recorded separately. Adjust the video camera before you begin so that if the participant moves around a bit.Some of the considerations in this section might or might not apply to a given usability test. Some screen reader users use headphones and might not have adequate speakers for you to hear the audio. Consider . and wires.

Specific considerations for some participants who have physical impairments Allow enough room for a wheelchair to get in. move around. we probably want to mix both of them. provide room for the interpreter to sit in different positions. at which time they discovered that the light bulb was burned out. he didn't know the light wasn't working. Encourage the participant and interpreter to adjust their seating to make it easy for them to work together. • Be sure the room is well-lit so that a participant who speech reads can easily see the facilitator's lips and facial expressions. such as keyboard. Specific considerations for some participants who are deaf or hearing impaired • Provide seating for an interpreter. especially if we are videotaping.3. allow time for the participant to check the settings and to change them if necessary. Orienting the Participant Encourage the participant to become familiar with the setup of any hardware that will be used in the test. . or other area where the participant will interact with the product. chair.2. mouse. depending on the situation.taking additional lighting. 2. • Record both participant and interpreter audio. we might want to mix only the interpreter. If the participant speaks some. and speakers. for participants who speech read (also known as lip-reading). If an interpreter will be present. • Position seating for a direct line of sight between the participant and the facilitator. such as near the facilitator or across from the participant. As the sun set it became too dark to see our test papers and notes. It is usually not necessary to include the interpreter in the videotape. and be positioned at the computer. table. etc. One of the studies says that one time they were testing at a participant's house. Eventually they asked the participant if they could turn on the room light. Encourage them to adjust the equipment. If the participant doesn't speak at all. so they are comfortable. If assistive technology will be used in the usability test. and when the test started there was enough daylight in the room. Because the participant was blind.

Completing Paperwork Provide documents in the participant's preferred format. Some people read Braille very quickly. . Don't grab the participant's arm. such as when we are beginning to record or changing the videotape. Be prepared to position a signature guide or straight-edged object so that a participant who is blind or has low vision knows where to sign. Most video cameras make a sound when recording starts or stops and the participant might be distracted wondering about the sound. might be uncomfortable if it is out of reach. hand. Be prepared for some people to want to use the Braille themselves and not have you read it. To help manage time. • Describe the setting to the participant briefly. • Remember seating for a personal attendant .Specific considerations for some participants who are blind or visually impaired • Introduce our self as we approach the participant. or cane.3. Introduce others who are with us so that the participant knows who is in the room. Tell the participant when you or others enter or leave the room. including the location of doors and position of the video camera. 2. Be prepared to indicate the place for signature. especially if people will be coming in and out and the video camera will be making noise. • Give directions about where to be seated. such as a walker. Specific considerations for some participants with physical impairments • Don't move mobility aids without asking. Some people who use a mobility aid. a small device that guides them when signing a document. Some participants will have a signature guide. • Offer your elbow to lead the participant. • Explain unusual noises and your activities. Some participants who are blind will have a signature stamp.3. as requested during participant recruiting. Don't assume that the participant will recognize our voice. and some read it very slowly. Generally it is best to give verbal directions and not physically direct a person. you may want to ask participants if we can read the documents aloud and provide the Braille version for reference. We can ask the participant if he would like we to put his hand on the chair.

For example. 2. we might go to another one. Many people who . For usability tests measuring time-on-task.4. Tell participants at the beginning of the session that you might stop tasks before they are complete. For an example of fast screen reading rate. a participant with limited upper body movement might want us to hold it up so that she can sign it with a pen in her mouth. and speak at eye level. “In the interest of time I'm going to stop you there. Instead of. Screen reader users usually set the reading rate fast and most people who aren't used to listening to a screen reader can't understand it that fast. Completing the Tasks Some people with disabilities will be particularly eager to complete tasks without help. you will likely need to ask the participant to slow down the reading rate. "When we have enough information for a task. Specific considerations for some participants who are deaf or hard of hearing  Face the participant while speaking. listen to the end of the Introduction to the Screen Reader video. and might be bothered by being stopped before they are done if they aren't expecting it.Specific considerations for some participants with physical impairments Have a clipboard available to hold documents to be signed. the screen reader should be set to the participant's normal rate. for example. Note that while most screen reader users will be happy to work at a slower pace. When you want to understand specific interaction with the product. Some facilitation techniques might not work with participants with disabilities. it became apparent that my practiced use of eye contact and body language to regulate the flow of discussion was of no use. Facilitating usability tests and focus groups often involves subtle communication. Specific considerations for some participants who are blind or visually impaired Request screen reader speech rate according to usability test protocol. body language won't work with participants who are blind and with some people with autism. even if we haven't completed the task. “Let's stop there and I'll give you something new to do. and if a participant is deaf." Also consider what to say when interrupting a task.” Be prepared to use alternative techniques for facilitating. For example. some might be frustrated working at the slow pace for a long time. Be ready to hold the consent form and other documents in a position where the participant can easily sign it. I had to devise strategies for communicating the same cues verbally.3. we could say something like. When facilitating a focus group with blind participants.” consider something more like. the technique of just not answering a participant's question during a low-interaction session won't work.

If the participant is having trouble hearing your words:  Try rewording what you are saying. Move closer to the participant. If you don't understand the participant. This is helpful for participants who have spent a significant amount of time on the task. Remember not to look down or cover your mouth while speaking. Sometimes a person with a hearing loss might be partially dependent on speech reading because some sounds might not be easily heard even with a hearing aid. Don't pretend that you understand if we don't understand. • Be prepared to offer to use written communication. Make sure that person knows . and ask the participant to repeat for clarification if needed.are deaf or hard of hearing rely at least partially on speech reading and thus need to see your lips. Don't speak too fast. Specific considerations for some participants with speech impairments • Offer to write down what you are saying. seniors who might have short-term memory loss. Since some words are easier to speech read than others.5.   2. while still observing personal space. we can repeat the part that we did understand so he only needs to restate part of it. and participants with cognitive disabilities who have difficulty processing a large amount of information. rephrasing what you said might make it easier for the participant to understand.   Speak clearly. Some people who use hearing aids or cochlear implants are especially sensitive to loudness. ask him to repeat what he said or ask for clarification. Collecting Data Consider debriefing after each task instead of after the entire test. If the participant is seated. sit to speak to him whenever possible. • Be very clear that an attendant should repeat exactly what the participant said. Wait for the participant to finish. That person might be inclined to expound on the participant's comments. Listen carefully. Don't just repeat louder. If it's difficult for the participant to speak. Some participants might prefer to communicate by typing or writing. rather than interrupting or finishing his sentence.3. Sometimes an attendant or other person who is used to the participant's speech will clarify what the participant says.

4. 2. The Usability Testing section is an overview of usability testing with participants with disabilities. 2. Still. there are certain things that . 2. from the earliest prototype screens to a full-blown Website. Specific considerations for some participants who are deaf or hard of hearing Be very clear that an interpreter should repeat just what the participant signs. Here's the basic method for employing a checklist-based user test. Step 1: Preliminary Self-Appraisal No author can view his or her own work with dispassion. Reporting Usability Testing The Evaluating for Accessibility page provides guidance on incorporating accessibility into common evaluation methods.6.the importance of saying just what the participant said. including standards review. Checklist for Usability Testing One of the most effective forms of inspection-based user testing involves the use of a "usability checklist.5." Checklist-based user testing is extremely inexpensive to implement.   Verify the spelling of the participant's name Reporting Usability Testing covers distinguishing between accessibility and usability issues. it can be used at virtually any time throughout the development cycle. and writing about people with disabilities. including relevant study parameters. People with vision impairments often have a particular way to fold each denomination so they can identify it. It's also easy to schedule. design walkthroughs. and requires a surprisingly small number of testers to be effective. Providing compensation Specific considerations for some participants who are blind or visually impaired  State what the currency is as you hand it to the participant if we are paying in cash. and informal evaluation with users with disabilities. clarifying conclusions. heuristic evaluation. being careful about categorizations and comparisons.3. Clarify to the interpreter the importance of translating as accurately as possible and not adding clarifications.

include questions that address the characteristics related to disability and accessibility defined in the usability test protocol. 2. Corporate sites should strongly consider using agency-based temporary employees for user testing. experience level. the more valuable the feedback they can provide. One important consideration is: how many testers are enough? There's no hard-and-fast rule. Recruiting Screener A usability test participant recruiting screener is used to determine if a potential participant matches the user characteristics defined in the usability test protocol. Step 4: Leave In formal usability experiments.6. Additionally. It is therefore vitally important for us to explain to your testers that we need them to make note of any problems they encounter. both for yourself and for your evaluators. etc. Unless we are planning on providing a personally-supervised guided tour of our site to all our readers. our presence will likely as not serve to inhibit your evaluators. and not as a substitute for user testing methods. But unless we are a trained usability professional. A topical site will probably want to enlist the aid of volunteer testers with some interest in the subject of the site. even with a small number of evaluators. if we start with a basic sweep of our site for known usability problems. And a handful (4 or 5) is more than adequate to ensure a generally reader-friendly Website. Distinguishing Between Accessibility and Usability Issues In addition to finding accessibility problems. When recruiting participants with disabilities. however. and thus compromise their ability to test your site. rather than a flaw in the design of the site itself. View this selfappraisal as a preliminary step. ask the usual questions about demographics. We can save considerable time. leave the tester to the business of testing your site. Even a single tester can probably uncover the most common usability problems on your site. Allow the tester sufficient time to test . Step 2: Provide checklists to our testers The more independent and autonomous your testers are. regardless of what they believe the underlying cause to be.inevitably make for an unfriendly Web page. frequency of use. the experimenter typically remains in the room to observe and record testers' behavior. but inspection-based testing methods provide a surprisingly quick payback. usability testing with participants with disabilities . Step 3: Provide some brief instructions Understand that your evaluators will naturally assume that the problems they encounter in using our site are the result of some fault on their part.

that is. environment. and many things are in a gray area where accessibility and usability overlap.will find general usability problems that impact all users. limitations from circumstance. there are times when such a distinction is important. the alt text is so bad that the usability of the site is awful for anyone who relies on alt text. 3. If the site has frustratingly verbose alt text (such as "This image is a line art drawing of a dark green magnifying glass. including users without disabilities.1. When a person with a disability is at a disadvantage relative to a person without a disability. The distinction is further blurred by the fact that features for people with disabilities benefit people without disabilities because of situational limitations (that is. For example. Another point to cloud the distinction is “usable accessibility”—how usable are accessibility solutions. as appropriate. such as when looking at discrimination against people with disabilities and when defining specific accessibility standards. one might say that the site is technically accessible because there is alt text. a person with a disability is not disadvantaged to a greater extent by usability issues than a person without a disability.1.2. Usability problems impact all users equally. When designing products. or device—such as using a mobile phone in bright sunlight with one hand because you're holding a sleeping baby with the other). however. it will take us to the Search page for this Acme Company website" instead of just "Search"). 3. Some things are clearly accessibility. Many of the accessibility guidelines to improve accessibility for people with cognitive disabilities are the same as general usability guidelines.1. it is an accessibility issue. that is. However. In some usability test . if a website uses images for navigation and there's no alt text. The distinction between usability and accessibility is especially difficult to define when considering cognitive and language disabilities. 3. Distinguish between usability and accessibility issues. regardless of ability. the site is clearly not accessible. One way to start looking at the distinction between the two is to categorize interface problems: 3. Accessibility problems decrease access to a product by people with disabilities. some are clearly usability.2. There is not a clear distinction between accessibility for people with disabilities and general usability for all. and accessibility increases general usability. If we click on it. Understand the difference between accessibility and general usability. it's rarely useful to differentiate between usability and accessibility.

----------------------------------------------------End---------------------------------------------------------- . There can be problems when people don't understand the issues around the distinction between usability and accessibility. However. A research study reported results on website accessibility without clearly separating general usability issues not related to accessibility.reporting it may be important to distinguish between accessibility and usability problems. When usability test reports are used internally to improve the usability of the product for all users. it is usually not necessary to distinguish between usability and accessibility issues. the study reported incorrect conclusions about web accessibility guidelines. it can be vital to distinguish between usability and accessibility issues. Because usability issues were mixed with accessibility issues. when usability test reports make statements about accessibility.

Sign up to vote on this title
UsefulNot useful