+

MDD11
Understanding
Users

CS2002S: Mobile
Development and Design

CSC2002S Melissa Densmore, UCT

In every design cycle, I like to start with understanding users.

You’ve already heard about the importance of empathy in design thinking. And
you’re embarking on doing interviews in order to build empathy for your assignment.
Let’s dig a bit deeper into ways we can try to understand users.

1
 But how do we know what
the users want?

Melissa Densmore, UCT CSC2002S

Design principles are all well and good, but they can only take you so far in the design
of your applications. To design great applications, you need to understand users.
Some pretty lucky designers can do this by intuition – the rest of us use methods to
bolster our intuition. Educated guesses are much better than blind guesses!

2
 +
Exercise:
Quickly develop a user study for your
picture book reader

Melissa Densmore, UCT CSC2002S

(Note: this example is from the 2020 assignment, but many of the notes and
questions still apply for 2021, you can follow along either with the picture book
example or the telling time example)

I want you to get out a piece of paper and take two minutes to think about your
assignment. Imagine you’ve already developed your first prototype. How will you
figure out if it’s a good design or not?

What did you come up with? Who will be a part of your study? What will you ask
them to do?

What questions do you think you can answer?

How confident are you that your findings tell you what you want to know?

3
 Who are your participants?

◼ Consider personas
◼ Parents
◼ Children
◼ Teachers
◼ Others?

◼ Consider context
◼ Where and when will they use the app?

◼ Who can you get access to? How will you recruit them?

Melissa Densmore, UCT CSC2002S

4
+
What do you need to test?

Define Test Activities

What are the tasks? How do you test the
Observe and Test, don’t guess! flow and
implementation of the
Tasks
• Open App app?
• View Book
• View Next/Prev/First/Last

Mixture of easy/hard
•
•
View List of Books
…
… vs. testing the layout
• Navigate Table of Contents
• Download a New Book
Support strange paths…

Melissa Densmore, UCT CSC2002S

5
+
What is your question?
◼ Are buttons or swipe actions more natural
for children using the app?
◼ Does my approach to parental locks
prevent children from accidentally exiting
the app? Or turning the page? Or
downloading a book?
◼ What are user expectations around
navigating a picture book on a mobile
phone?

◼ Is this app more “usable” than alternative

app X?
◼ Is this app “usable”?

◼ Would you use the app?

Melissa Densmore, UCT CSC2002S

6
+
A Rough Approach

1. Observation
2. Inquiry
3. Analysis
4. Prototype
5. Rinse and Repeat
Melissa Densmore, UCT CSC2002S

7
 +
Focus the Study
◼ Who are we interested in? What is the profile (age, socio-
economic background, skills, interests) of our users?

◼ What is the domain? College students? Healthcare?

Education? Leisure? Family reunions?

◼ What broad user-goals are we keen to support?

Collaboration? Diversion/fun?

◼ Are there any technology constraints?

Melissa Densmore, UCT CSC2002S

Whether you are developing for yourself or in response to a client or manager’s

request, it’s important to know the scope of what you are designing. How specific are
the requirements, what constraints are in place?

If you are less focused, you will leave room for inspiring design – looking for new
ideas in a particular domain. But a more focused study lets you gather information
that will help you go in an authoritative direction.

You may find that in earlier iterations you will look more for inspirations and broad
ideas. As you progress, you may test specific interface elements or sequences. By
the end you are testing for look and feel – and small things that can be changed
without altering other parts of your application.

8
 +

Recruiting
Participants

CS2002S: Mobile
Development and Design

CSC2002S Melissa Densmore, UCT

Wait a minute! All these ideas are great, but who will we actually talk to?

I know – the people who are easiest to access – my friends and family, right?

Well, tell me – what are some reasons why your friends and family might not be the
best participants for understanding users of, let’s say, a children’s ebook reader?

• Participant bias – they won’t necessarily give you their honest opinion, wanting
you to feel good about what you’ve developed. Or perhaps worse, they won’t take
you seriously.
• Homogenity – the people around you tend to think like you! They may not give a
good picture of what the general actual target population is like. How average are
you?

• Ok yes they are convenient in a pinch and great for pre-testing your evaluation
approaches. But in an ideal world, you’ll need to go find some users.

9
 +
Find some users

Melissa Densmore, UCT CSC2002S

It’s not easy to 1) find users and 2) find users that will give you useful information.
Your goal should be to find a representative sample – a group that typifies the larger
group of your potential target users.

10
 +
Representative Sample

all possible users

Sampling
approaches
simple random
convenience
snowball
stratified
purposive
…
the ones you actually talk to…

Melissa Densmore, UCT CSC2002S

In science that usually implies finding lots of people, selected in some principled way.
The argument is that with a good sampling, the answers you get from a subset of the
population can represent the answers of the whole population. To make this work
you need to have some percentage of the full target population. More for less
uniform populations.

However it’s not always possible, and you probably only have resources to talk to 5-
15 people.

Image from https://online.stat.psu.edu/stat200/book/export/html/21

11
 +
Key Informants (Millen, 2000)

◼ Field Guides are people who know the lie of the land:
individuals, groups and their dynamics, organizational
politics.

◼ Liminals are accepted by most people and move freely

between groups. They are part of the action, but also can
muse and reflect on what’s going on.

◼ Corporate Informants already spend a lot of time with your

target users and provide insight as well as helping you find
field guides and liminals.

◼ Exceptional Informants are rare gurus that deliver

“disruptive” insight into your topic.

Melissa Densmore, UCT CSC2002S

One approach is to find key informants. These informants can either give you an idea
about ways people in the population might respond, or even assist with recruiting
more participants.

Field guides can act as intermediaries, or be a literal guide into how people in the
target population behave.

So who would be potential key informants in the example of your assignment?

12
 +
Finding People to Study

◼ Off the street

◼ Advertising/recruiting

◼ Existing customers

◼ Colleagues

◼ Referral and Snowball

Sampling

Melissa Densmore, UCT CSC2002S

See table 5.1 in gary’s book

13
+
Now what?

Melissa Densmore, UCT CSC2002S

14
 +
Quantitative
Approaches to
Understanding
Users

CS2002S: Mobile
Development and Design

CSC2002S Melissa Densmore, UCT

Ok, so you now have a set of users.

15
 +
Now what?

Melissa Densmore, UCT CSC2002S

Now what do you do with them?

How do you generate feedback from them that you can turn into design
recommendations?

16
+
Quantitative Measurements
Experiments to determine behavior in controlled
settings
◼ What are your metrics?
◼ Time taken
A/B Testing
◼ Errors
◼ Income generated
Likert Surveys
◼ Data consumed Lab Experiments
◼ User Empowerment Data Logs
◼ …

◼ What’s your setting?

◼ Are there any confounding variables?

◼ Do you have statistical significance?

◼ Depends on variance and number of participants
◼ Are your participants representative of the population?

Melissa Densmore, UCT CSC2002S

17
+
Quantitative data: logging

◼ Instrument software to record interactions

◼ Key presses
◼ Mouse movements
◼ Screen touches / clicks
◼ Reading time
◼ Eye tracking…

◼ Logs of interactions are time stamped

◼ Useful for
◼ Web studies
◼ Testing layouts, displays, interfaces
◼ Understanding reaction time
◼ Measuring features most used, speed, performance
◼ Measuring errors
Melissa Densmore, UCT CSC2002S

18
+
Quantitative data: surveys

◼ Numeric responses
◼ How many times product was purchased
◼ How often one sends SMS

◼ Likert scale responses

◼ Odd (e.g., 3-point, 5-point, 7-point) allow neutrality
◼ Even forces choice

◼ Statistics can be calculated per-question and used in analysis

◼ Mean, median, N, etc.

Melissa Densmore, UCT CSC2002S

19
Melissa Densmore, UCT CSC2002S

20
 +
A/B test example: Button Color
Green
Green connotes ideas like
“natural” and “environment,”
and given its wide use in traffic
lights, suggests the idea of
“Go” or forward movement. It
also matches the company’s
logo color.
Red
Red is often thought to
communicate excitement,
passion, blood, and warning. It
is also used as the color for
stopping at traffic lights. Red is
also known to be eye-catching.
Red, in general, is not used as
a button color nearly as often
as green.

Melissa Densmore, UCT CSC2002S

Consider this A/B test.

A/B tests are ones in which users (typically website users) are presented with one of
two versions of an interface. Metrics such as click-through, purchases, time on page
are then compared. Famously Facebook and other popular websites are constantly
a/b testing us.

In this case, Performable wants to know what color button leads to more people
clicking on the Get Started Now button.
Consider research design. What is the independent variable? Dependent? Who are
the participants, how are they selected?

Give me a hypothesis, which do you think will result in more click-throughs?

21
 +
The new quick and dirty..

The red button

outperformed the
green button by 21%

Over 2000 visits

“a few days” of traffic

https://blog.hubspot.com/blog/tabid/6307/bid/20566/the-button-color-a-b-test-red-beats-green.aspx
Melissa Densmore, UCT CSC2002S

Consider this A/B test.

A/B tests are ones in which users (typically website users) are presented with one of
two versions of an interface. Metrics such as click-through, purchases, time on page
are then compared. Famously Facebook and other popular websites are constantly
a/b testing us.

22
 +
Sample Size

Melissa Densmore, UCT CSC2002S

So how many data points do you get for quantitative studies?

It depends on the distribution and variation of the effect you are measuring in the
population. If you are very confident that the population is relatively homogenous,
then fewer people are fine, but if there’s a large amount of variation in the factors
you are measuring, or even occasional outliers, then you will need more to get
statistically significant results.

You can estimate if you have enough participants using a power-law calculation.

Again I’m not going to teach you stats in this course, but if this is the direction you
will take with your evaluations, then you’ll need to have a good background in stats.

https://www.statisticshowto.com/statistical-power/
https://machinelearningmastery.com/statistical-power-and-power-analysis-in-
python/

Your outcome is only as good as your sample.

23
 +
Statistically Significant Analysis

Melissa Densmore, UCT CSC2002S

Once you have data, you have to analyze and interpret it.

At a high level – reporting descriptive stats, like mean (average), median, and mode,
along with an indication of error such as standard deviation or variance, can be
enough. But in order to be more rigorous you will need to know more about your
sample and to use statistical techniques to match your measures and your sample.

Which brings me to my next point. Your outcome is only as good as your sample. If
you have failed to get a representative sample, or even if you do something more
sophisticated like a stratified or purposive sample, you can’t use these stats the same
way. You can make statements about the subgroups you have selected against, and
perhaps use more specific statistical techniques to analyze the results.

We often use convenience samples and snowball samples. Consider how well these
represent the population. There’s biases inherent in most sampling techniques. You
need to account for this in your analysis and not just assume that because you have
concrete numbers you can make declarative statements about the population as a
whole.

Indeed quant data about a non-representative sample is just qualitative data – you
can use it to identify phenomena, to help describe some things that are happening,
but you can’t use the stats to make statistically significant statements about the

24
population as a whole.

24
+
Resources

◼ Interaction Design textbook – Ch 7&8

◼ Practical Stats for HCI, by Jacob Wobbrock, University of

Washington, in Seattle
http://depts.washington.edu/acelab/proj/ps4hci/index.html

◼ How to Lie with Statistics, by Darrell Huff

◼ UCT Statistical Sciences offers consulting services

http://www.stats.uct.ac.za/stats/consulting

Melissa Densmore, UCT CSC2002S

25
+
Qualitative
Approaches to
Understanding
Users

CS2002S: Mobile
Development and Design

CSC2002S Melissa Densmore, UCT

26
 +
Qualitative Observations
Explaining Phenomena and Thick Description

◼ Identify Values, Goals and Actions

Interviews
◼ Understand the Context Contextual Inquiry
Cultural Probes
◼ Track:
Focus Groups
◼ Tools
Workshops
◼ Documents
◼ Emotional Setting

Melissa Densmore, UCT CSC2002S

In a lot of cases, the observational stage is what you’ve been doing already –
watching the world around you and figuring out what the next best app is based on
what’s missing. However, you can also be more deliberate about it. While you are
working on A3 and A4, think about the contexts in which you imagine your app will
be used, and go and observe to see how people interact with their phones in those
contexts

27
+
Inquiry

◼ Contextual Inquiry:
1. The designer watches the participant begin an activity
2. The designer interrupts to question the reasons behind the
activity
3. The participant provides commentary, explaining while doing

◼ Diary Studies
◼ Have participant record their activities at a regular interval
◼ Can be automated using software on phones, or interactive

◼ Interviews
◼ Structured → Semi-structured → Unstructured
◼ Prepare interview questions, a “guide”

Melissa Densmore, UCT CSC2002S

28
 +
Cultural Probes (Gaver, 1999)

Melissa Densmore, UCT CSC2002S

Bill Gaver left packets of postcards, maps, a camera, and other things with their
participants. Participants were invited to return postcard with questions answered, to
take photos and to keep a diary of specific activities.

The goal was inspiration rather than information, and through this project they were
able to bridge an age difference and geographic distance, and helped to capture
aspects of culture that wouldn’t have necessarily been revealed by traditional means.

29
+
Focus Groups and Workshops
◼ Choose your participants
◼ 3-7 for a focus group
◼ more for workshops

Wyche et al. Re-placing Faith. CHI’08

Melissa Densmore, UCT
◼ Prepare a guide
◼ Outline questions and
activities

◼ Choose your location: your

place or mine? Where will
they feel comfortable?

◼ 30m-4h session led by a

trained moderator

◼ Follow up with analysis and

written report

CSC2002S

30
+
The Focus Group Moderator
Some Tips
Characteristics of a good
1. Observe/watch other moderator
moderators in action first!
Prepared | Listens | Neutral
2. Facilitate a natural
conversation, make people Flexible | Patient | Observant
comfortable

3. Dig deeper, ask open-ended

questions

4. Direct participation, provide

opportunities for everyone to
have a voice

5. Moderate, don’t participate!

Melissa Densmore, UCT CSC2002S

31
 +
Making Sense of Observations

Melissa Densmore, UCT CSC2002S

There’s a lot of different ways to make sense of your observations. For A5, you’ll be
doing affinity mapping – effectively sorting the ideas out into categories and
subcategories. Then you’ll prioritize by taking those ideas and interpret them into an
empathy map.

Resources
NN Group Affinity Mapping
https://www.youtube.com/watch?v=C4nYxZxteJY

32
 +
Analysis
◼ Transcribing ◼ Reflecting and interpreting

◼ Categorizing and organizing ◼ Making consumable

(“Coding”) (“Implications for Design”)

Melissa Densmore, UCT CSC2002S

So this process of analysis has four main steps. First – transcribe your data. You can
use transcription services, but I also find that transcribing recordings yourself helps
you to familiarize with what people say.

Then you look through those transcripts noting, or coding, main ideas. This can be as
simple as highlighting and “tagging” notable quotes and responses. You can do this by
hand, producing stickies as you go, or use software such as Nvivo or Atlas.TI or even
Excel to capture the codes and quotes.

Next comes reflection and interpretation. This is where you start taking those codes,
or categories, and you sort and prioritize them. You’ve seen this already as “affinity
maps” If you don’t have lots of stickies and a blank wall, you can use virtual stickies
on a virtual wall. Google Jamboard is useful for this, or any number of similar services
such as Miro or Stormboard. You can even just start creating text boxes in a
powerpoint slide.

Analysis usually follows these steps. There’s variations on the theme. Sometimes
categories are emergent, based on the ideas and themes that occur frequently in the
data. Or perhaps you have a particular theory or set of questions you are interested
in, and you can define the categories and codes ahead of time. Noting frequency,
variations on a theme, and priority of ideas helps you to systematically understand
your data to identify phenomena in your research. As you uncover patterns, you’ll

33
find yourself better able to understand users and explain their approaches and
reactions to interfaces.

33
+

What’s next?

GDS Design Principles (flickr)

Melissa Densmore, UCT CSC2002S

34