CP 260

User Experience Theory

LEC#2
Lec 2

 Define usability and usability testing

 Differentiate formative testing and summative testing

 Discuss traditional testing practices

 Identify need for a small or large study

 Defining Usability

From ISO standard 924-11:

“The extent to which a product can be

used by specified users to achieve
specified goals with effectiveness,
efficiency, and satisfaction in a
specified context of use.”

 This definition encompasses specific

users, seeking to accomplish their
specific goals within their typical
environment
 This definition focuses on critical
measures of usability
Measures of Usability
Measures of Usability

What is a measure or a metric?

 A metric is a “system or standard of measurement” represented

in units that can be utilized to describe more than one attribute.
Metrics come in very handy when it comes to quantify usability
during the usability evaluation of hardware, software, websites
and applications.

Source: http://usabilitygeek.com/usability-metrics-a-guide-to-quantify-system-
usability/
Measures of Usability

 What is the aim of measuring usability?

Measures of Usability

Critical Measures of Usability

 Effectiveness – how completely and accurately the work or
experience is completed or goals reached
 Efficiency – how quickly the work can be completed
One of the dimensions of that is whether the product supports the user
in a way that is better than the current way in which the user works.
 i.e. Electronic Banking, Chip Cards, NFC, Airdrop….
 Satisfaction – the users desirability in the product
Measures of Usability

Critical Measures of Usability

 Effectiveness – how completely and accurately the work or
experience is completed or goals reached
 It is defined in 2 ways:
 Completion Rate

 Number of Errors when attempting to complete a task

Source:
http://usabilitygeek.com/usability-
metrics-a-guide-to-quantify-system-
usability/
 Measures of Usability

Critical Measures of Usability

 Efficiency – Effectiveness divided by the time spent by the user
 One of the dimensions is whether the product supports the
user in a way that is better than the current way in which
the user works.
 Again, we have 2 aspects to Efficiency
 Time Based Efficiency
 How quickly the work can be completed
 Overall Relative Efficiency
 The ratio of the time taken by the users who successfully
Source: completed the task in relation to the total time taken
http://usabilitygeek.com/usability- by all users.
metrics-a-guide-to-quantify-system-
usability/
Measures of Usability

Critical Measures of Usability

 Satisfaction – the users desirability in the product
Usability Testing 11

Models
 The field of usability testing has a number of usability models
which we can use to determine the usability of a product /
service / website / etc.

The 3 most prevalent usability models are:

 Whitney Quesenbery – “5E’s model”

 Peter Morville – “User Experience Honeycomb model”
 Jakob Nielsen’s – “10 Usability Heuristics for U.I. Design”
Usability Testing 12

Models
 Arnie Lund – “Expert Ratings of Usability Maxims”

 Bruce Tognazzini – “Principles of Interaction Design”

 Ben Shneiderman – “Eight Golden Rules of Interface Design”

 Andy Budd – “Web Application Heuristics”

 Dr. David Travis – “247 Usability Guidelines”

Measures of Usability

 Whitney Quesenbery also developed a

usability definition that includes easy to
remember dimensions of usability – the 5E’s
Effective

Efficient

Engaging

Error Tolerant
Easy to Learn
Quesenbery’s Def’n

The 5e’s
 Effective
 How completely and accurately the work or
experience is completed or the goals are reached
 Efficient
 How quickly this work can be completed
 Engaging
 How well the interface draws the user into the
interaction and how pleasant and satisfying it is to
use
Quesenbery’s Def’n

The 5e’s
 Error Tolerant
 How well the project prevents errors and can help
the user recover from mistakes that do occur
 Easy to Learn
 How well the product supports both the initial
orientation and continued learning throughout the
complete lifetime of use
Morville’s Definition

Peter Morville
 The User Experience Honeycomb
Morville’s Definition

Peter Morville
 The User Experience Honeycomb
 He notes that in order for there to be a meaningful and
valuable user experience, information must be:
 Useful: Your content should be original and fulfill a need
 Usable: Site must be easy to use
 Desirable: Image, identity, brand, and other design
elements are used to evoke emotion and appreciation
 Findable: Content needs to be navigable and locatable
onsite and offsite
 Accessible: Content needs to be accessible to people
with disabilities
 Credible: Users must trust and believe what you tell
them
Measures of Usability
Advantages
•It can provide some quick and
relatively inexpensive feedback to
designers.
•You can obtain feedback early in
the design process.
•Assigning the correct heuristic can
help suggest the best corrective
measures to designers.
•You can use it together with other
usability testing methodologies.
•You can conduct usability testing to
Disadvantages
•It requires knowledge and
experience to apply the heuristics
effectively.
•Trained usability experts are
sometimes hard to find and can be
expensive.
•You should use multiple experts
and aggregate their results.
•The evaluation may identify more
minor issues and fewer major
issues.
 Usability Testing

 The activity of observing users working

with a product, performing real-world or
relevant work-related tasks
 Generally, there are two types of testing:
 Formative testing
 Summative testing
 Formative testing – conducted during
product development intended to
diagnose and fix problems, and often
done in small groups throughout
iterations
 Summative testing – performed on a
completed product intended to establish
baselines and validate requirements,
done in larger groups for statistical
validity
Formative Testing

 It is an assessment for learning purposes – process focus

 Formative testing – conducted during product development
intended to diagnose and fix problems, and often done in
small groups throughout iterations
 These tests also reveal what users like, and these findings
should be implemented as the product moves through
development
 Great tool for ending arguments. Developers find out what
works best for the users, not what a manager or vocal team
member thinks works best
Summative Testing

 It is an assessment of learning – outcome focus

 Summative testing – performed on a completed product
intended to establish baselines and validate requirements,
done in larger groups for statistical validity
 The established baseline can also be used in future product
development
Risks of not Testing

 If you don't do Formative Usability Testing, you might end up

designing something that simply will never be a usable solution
 (e.g. a baby crib made from barb wire and broken
glass, vs. one of wood and fabric).

 If you don't do Summative Usability Testing then you won't find

out if your potentially usable solution has any problems or not
 (e.g. a baby crib made of wood and fabric, but with
awkward latches, lead paint, and nasty nails sticking
out everywhere).
How much testing…?

 Early 1990’s Nielsen and Landauer determined the maximum

cost-benefit ratio is achieved when you test with three to five
participants
 weigh the cost of testing and the benefits gained
 more than 5 participants did not yield any
significant benefits
Need a small study?

 Categorized as a formative usability tests, could be informal

 Tests are small in scope and repeated throughout the SDLC
 Main benefit is identifying issues for the development team to
fix, and performing a small secondary study to determine if the
fixes worked
Need a large study?

 Most large studies are summative in nature

 Conducted on completed product with a large number of
users because we are trying to produce metrics

• success/failure on tasks
• average time on task
• completion rates
• error rates
• optimal navigation
• search results
• other measures
Big Picture

 Time & money versus desired outcome of a usability study

 If outcome is to uncover bugs,
and the budget for testing is
small then perhaps a small study
is appropriate
 Whereas, complex systems with
a large user base require
multiple users testing to
represent multiple goals,
provided a sufficient budget
 27
References

1. https://techterms.com/definition/enduser
2. http://architectingusability.com/2012/06/15/user-requirements-understa
nding-your-users-characteristics/
3. http://semanticstudios.com/user_experience_design/
4. https://www.usability.gov/what-and-why/user-experience.html