This action might not be possible to undo. Are you sure you want to continue?
Calendar of Events
• • • • •
01 CFSP Exam Application Deadline 04/15/2010 02 Canadian Forms Institute East 04/19/2010 - 04/20/2010 03 Symposium San Jose 05/03/2010 - 05/06/2010 04 CFSP Exam Period 06/05/2010 - 06/19/2010 05 Master Series: Best Practices in Forms Management 6/10/2010 - 7/15/2010 06 CFSP Study Group 8/2010 07 Forms and Technology Conference 9/13/2010 - 9/15/2010
• • • • • • • • •
Association News Books Certification Chapter Events Expertise Industry News Marketing Council Membership Council Message from the President
• • • • •
Professional Opportunities Program Council Spotlight & Awards Symposium TalkBack
Gold Level Sponsors
IBM Microsoft Adobe, Inc. Exstream Software Kaye-Smith Snap Forms
Silver Level Sponsors
• • • •
Bronze Level Sponsors
• • • • • • • • • • •
AM+A Amgraf Cerenade DocuStep Easel Solutions Essociates Group FormSoft Integriform Mi-Co Resource International Texcel Systems, Inc.
« BFMA Membership and Professional Growth | Main | CANCGSB-6.2-2009 - What It Is and Why You Want to Know »
Usability Series: Overview and Exercise
By Sandra Thwaites Cognitive Systems Engineering This is the first of three articles on usability. The current article includes a basic overview of usability and an exercise in usability testing. The author believes that going through this exercise will give readers a better appreciation for the steps involved in usability and the value that it provides. You can reach Sandra at firstname.lastname@example.org.
WHAT IS USABILITY? Usability can be defined as the ease with which the intended group of users can interact with or use a product to achieve something the product is designed to do. More broadly speaking, usability contributes to the ‘good’ or ‘bad’ user experience of a product. Usability is most typically done when development follows a user-oriented, iterative design approach. The product being developed is anything that has a human interaction component. Instances of such products may include services offered on a website, an insurance company application for approving clients for insurance, an electronic thermometer for infants, a vote ballot, or take-out cup and lid for hot liquids. The term “user-oriented” means there is a belief in the value of including the users’ needs in the product design, along with business needs and technology. The value of usability is easiest to see when there is a strong correlation between business needs and user needs. For example, the business may need to increase sales by 10% (e.g., of cell phones) and users may be more willing to purchase that company’s product if it enables users to ‘easily’ accomplish some core task (e.g., send text messages). Iterative development essentially follows these steps: 1. design 2. usability test 3. refine design if indicated 4. repeat until the usability goal(s) is met. Usability goals should be specified for a product as soon as possible and a priori, that is, before any testing is done and any results are known. Stating usability goals, or destinations, before testing helps to avoid the tendency for people to argue about whether the results of testing indicate the design is 'good enough' to release the product to market. By specifying goals a priori, before testing begins, then the decision to reiterate or evolve product design will be based on fact (testing results) rather than opinion. Usability goals deal with either performance or subjective opinion. Performance ultimately comes down to the speed and errors that result in order to accomplish a task. It may be easiest to set a usability goal in cases where user performance levels with a product will result in life or death for the users or other individuals. In the case of a radiation machine for example, a usability goal may be that an acceptable or good enough design will be one that results in zero patient deaths after a user administers radiation with the machine. (Set Phasers on Stun, by Steven Casey, presents true stories of interface designs with deadly consequences, including one story about a radiation machine: http://www.aegeanpublishing.com/phaser1.html) Most products will not result in life or death but rather will enable users to complete a task ‘quickly’, ‘more slowly’, ‘with a few errors’ or ‘with many errors’. These high level statements of usability goals need to be “operationalized” or stated in a way that makes them measurable. Examples of measurable statements for usability goals include: users will perform all core tasks in 5 minutes or less; users will complete all core tasks at a level of satisfaction that is 2 points higher than the satisfaction rating for the way they currently complete the core tasks. Another example is a goal which states that core task x will be completed in less than 2 minutes while the other core tasks will be completed in less than 5.5 minutes. Subjective usability goals may be of interest instead of or in addition to performance usability goals. Subjective goals relate to ratings on say a 5 point scale Subjects give in response to opinion statements. For example, a subjective usability goal may be that Subjects rate the product 4 or higher on a scale of perceived ease-of-use (e.g., I find the product easy to use). WHAT KIND OF PEOPLE ARE USABILITY PROFESSIONALS? Usability specialists come from a variety of backgrounds but all hold the common desire to contribute to the development of products that are usable; products that enable users to accomplish a specific task(s). This author has a background in Cognitive Psychology as well as several years in a doctoral Human Oriented Technology program within a Psychology Department. Skills the author believes to be invaluable include an understanding of research methods, statistics, limitations and strengths of the human brain, and the ability to be an objective observer. Relevant courses are available at many universities as well as online.
WHEN DOES USABILITY TESTING OCCUR DURING DEVELOPMENT? Usability testing is ideally done early and often. Testing can begin as soon as one has an idea for a product. At the idea or conceptual phase, testing would likely take the form of a focus group where opinions and visions for the product idea are voiced by participants. Beyond the conceptual phase, a user interface designer will be able to provide some rough sketches for the proposed product. Early sketches fall into the category of “low fidelity” and can be tested with potential users to provide invaluable feedback on whether these high level designs make sense to the user. Still later in the process, designers and/or developers will be able to deliver a partially-functioning or highfidelity prototype. Typically, such a prototype will only be functioning for a specific set of core tasks that the product is to support. Testing early means that one can essentially get the design ‘good enough’ before any expensive and costly-to-change programming has begun. WHO ARE THE TEST SUBJECTS? Usability testing can be done on 5 to 7 Subjects who are representative of the target user population. More often than not, Subjects are tested one at a time on a set of tasks that can be completed in approximately one hour. It is important that Subjects understand that it is the product that is being tested rather than them. Perhaps the most difficult part of testing is reminding the Subject to think aloud. The object of the testing is usually to collect qualitative data. The tester needs to know and note what the Subject is doing or trying to do and understand their reasons for anticipating any interaction behaviours. Thinking aloud is a foreign concept to most people and so the tester needs to gently and frequently prod the Subject to continue to think aloud. WHAT IS THE TEST PROTOCOL? The usability specialist needs to devise a protocol which lays out the tasks for the Subject to try and complete with the prototype materials. These tasks should represent the core tasks that the product is designed to do. It is common for a Usability specialist to run one or two ‘pilot’ or practice Subjects to try and gain an appreciation for where common interface issues might be for Subjects. In this way, the specialist can be prepared with a series of hints to try and help the Subject complete a task. Writing everything down in a protocol better ensures that every Subject will be run through the test in the same way. ANALYZING TEST RESULTS For each task, the problems that Subjects encounter during a usability test may be categorized as a Critical, Major, or Minor problem according to the following definitions: Critical: User cannot complete the task because it is not possible with the current interface or because they do not know how. Major: User experiences great difficulty completing the task. They may commit many errors and require several hints to be able to complete the task. Minor: User completes the task but is annoyed along the way with specific elements of the interface as indicated by comments to that effect or facial expressions of annoyance. One wants to focus on common problems found during usability testing. That is, problems which are repeated for almost every Subject. It is necessary to test representative Subjects because it is impossible, even for a seasoned usability professional, to predict 100% accurately where exactly users will have problems with an interface. When testing is done by a usability practitioner alone without any Subjects, it is called a Heuristic Evaluation. While this Heuristic Evaluation is better than doing no Usability testing, it is generally inferior to the results that will be found through testing a representative sample of Subjects. From the analysis, the usability practitioner must pinpoint problems with the current interface design and make recommendations for the way in which that design needs to be changed to better meet the users mental model or understanding of how the application works. THE EXERCISE This exercise is designed to be simplistic and will hopefully provide the reader with an opportunity to perform a usability test on 5 to 7 people.
The material to be usability tested is presented in Figure 1 which is a first pass sketch of a website for a company called GroceriesOnline.com. Assume that this is a new yet to be launched website intended for local users. The owners of the company believe that this design is sufficient to allow their potential customers to order such things as 1. Bananas 2. Bar-b-q tongs 3. Large watermelon Your tasks are to: 1. Set one or more usability goals. 2. Develop a test protocol whereby each Subject must tell you how they would order large bananas, bar-b-que tongs and large watermelon. The test protocol needs to be further specified. 3. Run yourself as a pilot Subject to get a feel for what might happen during actual testing. 4. Run 5 to 7 Subjects through the test. 5. Record & analyze your data. 6. Make at least two key recommendations for creating a better design for the website.
Figure 1: Material for Usability testing (kindly provided by Lise Whitewolf and Hilary Little, User Interface Designers, CBSA, as part of a lunch-and-learn exercise Summer, 2009)
Usability is an art and a science and, like anything, improves with practice. You are bound to learn something by going through this exercise and may even find yourself eager to usability test some of your own designs. The collective results of many usability studies have resulted in the development of standards of best practice for design. Nevertheless we must not forget that the best design for today’s technology will be replaced with a design more suited to tomorrow’s technology. This point is underscored by a recent study in The BBC Magazine. They marked the 30th anniversary of the Sony Walkman by inviting a 13 year-old boy to swap his iPod for a Walkman for a week. Some of the boy’s comments included, “It took me three days to figure out that there was another side to the tape!” The entire article can be found at http://news.bbc.co.uk/2/hi/uk_news/magazine/8117619.stm
The next article in this series will discuss some of the many issues with the initial GrocerersOnline.com website design. Additionally, the second article will discuss some publicly available usability work, noting some key elements to further expand your understanding of usability. SOME CLASSIC BOOKS & A WEBSITE ON USABILITY Bias, Randolph G. & Mayhew, Deorah (Eds.), Cost Justifying Usability, Elsevier, 2005. Krug, Steve & Black, Roger, Don’t Make Me Think: A Common Sense Approach to Web Usability, New Riders Publishing, 2000. Nielsen, Jakob, www.UseIt.com Nielsen, Jakob, www.useit.com/papers/heuristic/heuristic_list.html (10 principles of good interface design) Nielsen, Jakob, Usability Engineering, Morgan Kaufmann, 2001. Norman, Donald, The Design of Everyday Things, The MIT Press, 1998. Rubin, Jeffrey, Handbook of Usability Testing: How to Plan, Design and Conduct Effective Tests, Wiley, 2008.
Posted by BFMA on 09/09/2009 in Expertise | Permalink Digg This | Save to del.icio.us
TrackBack URL for this entry: http://www.typepad.com/services/trackback/6a00e54ef14a7988330120a53ee117970b Listed below are links to weblogs that reference Usability Series: Overview and Exercise:
Comments Verify your Comment Previewing your Comment
Posted by: | This is only a preview. Your comment has not yet been posted.
Top of Form
Bottom of Form
Your comment could not be posted. Error type: Your comment has been posted. Post another comment The letters and numbers you entered did not match the image. Please try again. As a final step before posting your comment, enter the letters and numbers you see in the image below. This prevents automated programs from posting comments. Having trouble reading this image? View an alternate.
Top of Form
Bottom of Form Top of Form
Post a comment
Comment below or sign in with TypePad Facebook Twitter and more... You are currently signed in as (nobody). Sign Out
(URLs automatically linked.) Your Information (Name is required. Email address will not be displayed with the comment.)
Name is required to post a comment
Please enter a valid email address
Web Site URL
Bottom of Form
• • • •
www.bfma.org BFMA International: The Organization for Forms and Business Process Professionals Meet your Board Members Current Newsletter BFMA Newsletter Archive Become a Member of BFMA
• • •
Subscribe to the BFMA Newsletter Receive Information on Educational Events Stay in the Know! Bookmark This Site!
• 1 Fall Forms Institute 2009
• 2 Symposium 2009
• 3 Canadian Forms Institute West 2009
• 4 Fall 2008 Forms Institute & PDF Conference
• 5 Symposium 2008
• 6 Canadian Forms Institute 2008
• 7 Fall 2007 Forms Institute
• 8 Fall 2007 Board Meeting
• 9 Symposium 2007
• • • • • • • • • •
Corin Salnave on New CFSP Recognized Web developer on Have You Submitted Your Accomplishments for 2008? Caroline Jarrett on Usability Pitfalls and a Vision for the Future Caroline Jarrett on BFMA Members Respond to the Loss of Rob Barnett Jessica Enders on San Jose is Looming Large Justiss Boyer on Demonstrating Contribution Carol Romero on Do You Know the Way to San Jose? Joyce Endres on Long Time Member Says Goodbye Carl W. Brannon on Do You Know the Way to San Jose? Oliver Wischmeyer on Do You Know the Way to San Jose?
• • • • • • •
April 2010 March 2010 February 2010 January 2010 December 2009 November 2009 October 2009
• • •
September 2009 August 2009 July 2009
BFMA 3800 Old Cheney Road, Suite 101-285 Lincoln, Nebraska 68516 Ph: 888-367-3078 Fax: 877-204-5979 E-mail: email@example.com www.bfma.org
This action might not be possible to undo. Are you sure you want to continue?
We've moved you to where you read on your other device.
Get the full title to continue reading from where you left off, or restart the preview.