Activation Journey
Storyboards
Prototyping
Playbacks
Hills
Empathy Map
As-Is-Scenario
Evaluate
To-Be Scenario
User Research
& Personas
Prioritization
Needs Statements
START Playback Theatre
Project Rundowns Big Ideas
ACTIVATION
COMPLETE
©IBM
Evaluation #2: Usability
Testing
INF1602 – Fundamentals of UX
Using a Think-Aloud Protocol
• Think-aloud protocol is the process of having
participants speak what they are thinking as they
complete a task
• Gives you an understanding of why the user is
taking the actions that he or she takes and the
person’s reactions to and thoughts about what
he or she is working with
• Helpful to provide an example to participants
• What they are thinking as they interact with a
product rather than their opinion about a product
Recording and Notetaking
• Take notes
• Use video or audio recording
• Use screen-capture software
• Combine video/audio/screen recording and
notetaking
www.id-book.com ©2015
Moderator role
• Facilitate the test – Ask the questions your team
wants answered, ensure the tasks get completed,
move things along when they get stuck
• Be the prototype – Navigate the user as though
the paper were your solution. For example, flip
the pages when buttons are “pushed”, etc.
• Don’t explain – Don’t demo, justify, or answer the
participant’s question. You’re doing this to get
their responses, not to give yours. If they want to
know what something means or does, ask them
what think it means or does (or just say “Hm… I
don’t know. Good question.”)
©IBM
Moderator role
• Greeter/Flight Attendant
• Safeguard the physical, psychological and
emotional wellbeing of the participant
• Greeting participants, taking coats, offer hot
beverage, etc.
• Monitor for signs of stress
– Read non-verbal behaviour
• Obtain informed consent
Good Moderation
• Let participant know their opinions are valued
• Observe and read the participant’s non-verbal
behaviour – movement, facial expressions, etc.
• Reinforce participant – nodding, paraphrasing
• Take notes, but don’t write all the time
• Reflect back on what you heard and check
understanding
• Ask open ended questions
• Pace your talking speed to participants
• Listen to inconsistencies and probe further
Things not to say…
• Saying “Remember, we’re not testing you” more than 3x...
• Don’t worry the last participant couldn’t do it either
• No-one’s ever done that before!
• HA! HA! HA!
• That’s amazing! I didn’t even know that was possible!
• Could we stop for a moment? Watching you struggle is
tiring
• Yes, it’s normal for observers to laugh during a test...
• Don’t feel bad, many people take 15 or 16 tries...
• Are you sure you’ve used a computer before?
• Wow! Why did you click there?
• Are you stupid? CLICK HERE!
Note taker role
• Watch and listen – You will be responsible for
capturing everything that happens for those who
couldn’t attend (or couldn’t pay attention)
• Categorize your observations – Record
questions, suggestions, successes, and failures.
(In addition to other relevant observations.)
• Don’t react or judge – Usually you aren’t in the
testing room, but even then, it’s sometimes hard
not to laugh, be surprised, or even upset. Do your
best to stay neutral and record what happens
during the test
©IBM
Observer role
• Pay attention – Since you are watching live, you
don’t need to rely on the note taker’s
observations or interpretations… but in order to
make that worth your time, you need to focus
• Process your observations – Consider what you
wanted to learn before the test and think about
that as you observe the user’s reactions and
interactions
• Don’t react or judge – Usually you aren’t in the
testing room, but even then, it’s sometimes hard
not to laugh, be surprised, or even upset. Do your
best to stay focused on what happens during the
©IBM test and be open to what you observe
Participant role
• Stay on task – You were selected to help the
team and your input is very valuable. Be present
when you are in the test and follow the
moderator’s instructions
• Be honest – The best thing you can do is provide
honest feedback. Don’t be concerned about
hurting anyone’s feelings
• Remember you are the tester, not the tested –
You are evaluating the team’s design. They are
not evaluating you. If you don’t understand
something, it’s not your fault it’s the design’s fault
©IBM
Sample User Testing Videos
• Usability Testing of Fruit
– https://www.youtube.com/watch?v=3Qg80qTfzgU
• A shocked designer responding to results
– http://ca.youtube.com/watch?v=J3kTdlV3pZs&NR=1
• Screen capture during user testing with eye tracking
– http://ca.youtube.com/watch?v=Xkl2czgOBC4&featur
e=related
• Usability testing with retrospective think aloud
– http://ca.youtube.com/watch?v=Jpt3qz1gtXI&feature
=related
• Usability testing of a paper prototype
– http://ca.youtube.com/watch?v=ppnRQD06ggY&feat
ure=related
Evaluation Template
©IBM
