User Experience

Agenda of Today’s Talk

• Usability Testing

• Heuristic Evaluation
What is Usability Testing
Usability Testing is a technique used in UCD to evaluate a product by testing it on users.

Typically, during a test, Participants will try to complete typical tasks while observers watch, listen and takes notes. The goal
is to identify any usability problems, collect qualitative and quantitative data and determine the participant’s satisfaction
with the product.

To run an effective usability test, you need to develop a solid test plan, recruit participants, and then analyze and report
your findings

What is a Usable site

Usable sites allow users to:
• Find what they need
• Recognize and understand what they find
• Use the information to accomplish their goals.
Why Conduct Usability Testing
• Checking to see if the design works (Whether the global design or specific elements of
a design)
• Diagnosing a problem
• Adding new features (Can users get to the new feature ? Can they use the feature ?
How does the new feature interact with the existing interface)
• Verifying that design goals are met

Usability Testing Process

Plan
Formative Test Summative Test
• Conducted throughout product or • Conducted at the end of the
website development development Cycle
• Used to determine if a specific design • Used to Validate if a product has
objective is met accomplished its intended goals

Characters

Participant :- The actual or potential

user
Observer :- Records events as they occur.
Does not interact with the participant

Facilitator / Moderator :- Oversees the entire test process from

planning through conducting & Reporting
Usability Labs
Choose Participants
• Selection of Participants is based on the user profiles or personas that one have developed during data gathering and
market research, It is important to recruit the target group
• Create a screener to enable recruiters to interview and recruit participants based on criteria

How many Users?

• According to research , Five users caught 85% Usability problems, But problem detection rate can go as low as 55% for
group of five.
• On Average a group of 10 ensures that a minimum of 80% usability problems will be detected

Recruiting?
• Recruiting can be done in-house or through agency
• Keep either Backups and Floaters
• Back-ups are people who are scheduled for the end of the day, Just in case some body does not show up
• Floaters are people who are available on call incase some one does not turn up
Types of Test Data
• Qualitative Data :- Obtained by detailed Observation and Understanding
• Quantitative Data :- Measure of Time, Keystrokes, errors etc.

Types of Performance Metrics

• Success Metrics
• Number of successful tasks completed
• Number of successful/ Unsuccessful steps within a task
• Error Metrics
• Number of unproductive paths attempted
• Number of restarts attempted
• Efficiency Metrics
• Time to Complete
• Number of steps required to complete a task
Setup of Test Instruments
• To Conduct a Test
• Hardware – A Computer
• Software or Application
• Internet Connectivity
• Scenarios and tasks
• Facilitator’s Guide
• To Measure and record the observations
• Recording sheets
• Data Logging software
• Audio Video Equipment
• Questionnaire and rating forms

Some tools for helping to record and analyze remote usability testing sessions are Morae, User Zoom, Canvas Flip, Invision,
User Testing , Validately, Lookback
Different Tests
• Early Prototype Test
• Evaluate Mental Model – Expectancy Test
• Evaluate Organization of Content – Reverse Card Sorting Test, Remote Card
Sorting Test
• Evaluate Navigation Design – Performance Test
• Evaluate Affordance – Visual Affordance Test
• Evaluate Brand Message – Brand Definition Test
• Advanced Prototype Test
• Evaluate Actual Usage – Free Exploration Test
• Evaluate Flows, Navigation ,Control & content – Performance Test
• Remote Usability Test
• Moderated
• Un Moderated
Live Site Analysis
You can measure usability while your website is live. There are several technologies to
gather live site data and measure web site usability

• A/B Testing and Multivariate Testing

• Web Analytics
• Web Server Log Files
• Browsers or page tagging files
• Transaction Data
• Direct Feedback
• Web Traffic Analytics
10 Point Check List - Summary
1. Prepare for Kick off Meeting
Obtain the Background of Design, Understand the Objectives, Define User groups and Locations and become familiar with the system

2. Conduct Kick off Meeting

Define usability Issues & Criteria, determine tasks, decide test, determine recruiting criteria, confirm dates, location & Schedule

3. Initiate Recruiting
Develop a Screener, Who will recruit and plan for working with the recruiter

4. Obtain Test Materials

Obtain the prototype for performance tests, cards or sheets or any material required for desired test type

5. Create Protocol and Instruments

Create Protocol and collect Instruments, Remember to Pilot test

6. Organize Test facility and Roles

Organize Observer seating, Set the traffic flow of participants and assign roles

7. Conduct the Test

Plan from Informed consent form, scenarios, performance ratings, task specific questionnaire, Closing questions, Providing
Compensation

8. Analyze the Results

Tabulate the data & notes. Analyze the list findings, Prioritize Findings
9. Create Redesign Recommendations
Focus Initially on High – Priority Findings

10.Share Results
Create a final document or presentation and follow up to ensure that redesign recommendations are carried forward
References
Human Factors International – CUA Track – Usability Testing Documents
https://usabilitygeek.com/usability-testing-mobile-applications/
https://www.usability.gov/how-to-and-tools/methods/usability-testing.html
https://en.wikipedia.org/wiki/Usability_testing
https://www.nngroup.com/videos/ux-without-users-isnt-ux/
https://www.nngroup.com/articles/quant-vs-qual/
https://uxmastery.com/beginners-guide-to-usability-testing/
https://www.uxpin.com/studio/blog/how-to-run-an-insightful-usability-test/
http://www.experienceux.co.uk/faqs/what-is-usability-testing/
http://www.humanfactors.com/whitepapers/global_usability_testing.asp
http://remoteresear.ch/tools/
https://www.interaction-design.org/courses/conducting-usability-testing
https://www.nngroup.com/articles/remote-usability-tests/
Can’t do Usability Testing ? Tough ? No Time?
Perform Heuristic Evaluation

Usability experts review your site’s interface and compare it against accepted usability
principles. The analysis results in a list of potential usability issues.

Requirements to conduct Evaluation

• 3 - 5 Usability Experts
• A Set of Heuristics
• List of User Tasks
• A System to test
• Form for recording notes
Report General Template – Airbnb results page

S No Problem Severity Rating Ease of fixing rating Broad Heuristic Solution

1 No Back Button, If user wants to go back 5 2 Flexibility & efficiency of Use Include cancel & back buttons
from details page, User may click on “Open
New Tab” then even browser can’t help
Jacob Nielsen 10 Heuristics
References – Heuristic Evaluation
https://uxmastery.com/how-to-run-an-heuristic-evaluation/
https://www.usability.gov/how-to-and-tools/methods/heuristic-evaluation.html
https://www.nngroup.com/articles/ten-usability-heuristics/
http://www-personal.umich.edu/~dinoa/portfolio/622/622-HeuristicEval.pdf
http://www.craigmacdonald.com/wp-content/uploads/2014/07/Jessica-
Espejel_J_14368.Espejel-Heuristic-Evaluation-Report.pdf

References – Psychology Principles

https://medium.com/insightdesign/psychology-principles-that-balancing-aesthetic-and-
usability-in-interface-design-997ceed3fb7c
Questions