3/21/2023

Defining Usability

The ISO defines usability as “the extent to which a

product can be used by specified users to achieve
USABILITY ENGINEERING specified goals with effectiveness, efficiency and
satisfaction in a specified context of use.”
Fenty Eka Muzayyana Agustin,MKom

Page 1 Page 2

Indikator Usability Atribut Usability

Effectiveness: accuracy and completeness with which users

achieve specified goals.
Efficiency: resources expended in relation to the accuracy and
completeness with which users achieve goals.
Satisfaction: freedom from discomfort, and positive attitudes
towards the use of the product.

A model of the attributes of system acceptability, based on Figure 1

Page 3 of Nielsen [1993b, page 25]. Page 4
 3/21/2023

Six Usability Attributes Measuring Usability Attributes

Combining the three ISO usability attributes with Nielsen’s five 1. Efectiveness: decide on definition of success. For example, number of
usability attributes, leads to the following six usability attributes: substitution words spotted in a text, or binary measure of success (order
completed or not)
1. Efectiveness: completeness with which users achieve their
2. Learnability: pick novice users of system, measure time to perform certain
goal.
tasks. Distinguish between no/some general computer experience..
2. Learnability: ease of learning for novice users.
3. Effeciency: decide definition of expertise, get sample expert users (difficult),
3. Effeciency: steady-state performance of expert users. measure time to perform typical tasks..
4. Memorability: ease of using system intermittently for casual 4. Memorability: get sample casual users (away from system for certain time),
users. measure time to perform typical tasks..
5. Errors: error rate for minor and catastrophic errors. 5. Errors: count minor and catastrophic errors made by users while performing

6. Satisfaction: how satisfying a system is to use, from user’s point some specified task. For example, number of deviations from optimal click path.

of view. 6. Satisfaction: ask users’ subjective opinion (questionnaire), after trying system
for real task.
Page 5 Page 6

Usability Evaluation Exploratory Evaluation

There are four types of evaluation, according to
the purpose of the evaluation:
• Explores current usage and the potential
• Exploratory - how is it (or will it be) used?
design space for new designs.
• Predictive - estimating how good it will be. – Done before interface development.
• Formative - how can it be made better? – Learn which software is used, how often, and
• Summative - how good is it? what for.
Definitions [Andrews, 2008] are adapted from those of several authors [Stone et
– Collect usage data – statistical summaries
al., 2005; Rubin, 1994; Lockee et al., 2002; Ellis and Dix, 2006]. and observations of usage.

Page 7 Page 8

Predictive Evaluation
• Estimates the overall quality of an
interface (like a summative evaluation, but
a prediction made in advance).
– Done once a design has been done, but
before implementation proceeds.
Page 9
Summative Evaluation
• Assesses the overall quality of an
interface.
– Done once an interface is (more or less)
finished.
– Either compare alternative designs, or test
definite performance requirements.
– Collect bottom-line data – quantitative
measurements of performance: how long
did users take, were they successful, how
many errors did they make.
Page 11
3/21/2023
Formative Evaluation
• Informs the design process and helps
improve an interface during design.
– Done during interface development.
– Learn why something went wrong, not just
that it went wrong.
– Collect process data – qualitative
observations of what happened and why.
Page 10
Modified Soup Analogy
• Extending Robert Stake’s soup analogy
[Stake, 1976; Lockee et al., 2002]:
“When the cook tastes other cooks’ soups, that’s exploratory.
When the cook assesses a certain recipe, that’s predictive.
When the cook tastes the soup while making it, that’s formative.
When the guests (or food critics) taste the soup, that’s
summative.”
Page 12
 3/21/2023

Usability Evaluation Methods Usability

Evaluation
• Usability Inspection Methods Methods
Inspection of interface design by usability
specialists using heuristics and judgement (no
test users).
• Usability Testing Methods
Empirical testing of interface design with real
users.

Page 13 Page 14

The The Usability Engineering Lifecycle

Usability
Engineering 1. Know the User
Life Cycle 2. Usability Benchmarking
3. Goal-Oriented Interaction Design
4. Iterative Design:
(a) Prototyping
(b) Formative Usability Evaluation (Inspection
and/or Testing)
5. Summative Usability Evaluation
6. Follow-up Studies Lanjuuuut…..

Page 15 Page 16
 3/21/2023

Know The User Usability Benchmarking

• Qualitative research: observation of users • Analyse competing products or

and interviews.
interfaces heuristically and empirically.
• Exploratory evaluation: which software is
used, how is it used, and what is it used for. • Set measurable usability targets for your
• Draw up a user profile for each (potential) own interface.
class of user, based on behavioural and
demographic variables.
• Identify user goals and attitudes.
• Analyse workflow and context of work.
• Draw up a set of typical user scenarios
Back Back

Page 17 Page 18

Interaction Design Iterative Design

• Goal-oriented initial design of interface. “Design, Test, Redesign.”

Build and evaluate prototype interface, then:
• Severity ratings of usability problems
discovered.
• Fix problems!new version of interface.
• Capture design rationale: record reasons why
changes were made.
• Evaluate new version of interface.
until time and/or money runs out.
A cycle of continuous improvement.
Back Back

Page 19 Page 20
 3/21/2023

Building Prototypes Formative and Summative

Usability Evaluation
• Verbal description. • The usability evaluation methods are
• Paper prototype. described according to who performs
• Working prototype. them:
– Usability inspection methods: Chapter 8.
• Implementation of final design.
– Usability testing methods: Chapter 9.

Back Back

Page 21 Page 22

Follow-Up Studies Planning Usability Activities

Important usability data can be gathered after the 1. Prioritise activities.

release of a product for the next version:
• Specific field studies (interviews, questionnaires,
observation).
• Standard marketing studies (what people are
saying in the newsgroups and mailing lists,
reviews and tests in magazines, etc.).
• Instrumented versions of software log data.
• Analyse user complaints to hotline, modification
requests, bug reports.
Back
2. Write down explicit plan for each
activity.
3. Subject plan to independent review
(e.g. colleague from different project).
4. Perform pilot activity with about 10% of
total resources, then revise plan for
remaining 90%.

Page 23 [Always perform a pilot study!] Page 24

 3/21/2023

TUGAS KELOMPOK Contoh Pengukuran Atribut

Usability
• Kelompok Terdiri dari 2 Orang • Aplikasi BNI SMS Banking
• Efectiveness:
• Pilih salah satu aplikasi E-government (contoh: – Tujuan aplikasi ini adalah memudahkan pengguna agar tidak perlu menghapal format SMS ketika
ingin bertransaksi.
imigrasi.go.id). • Learnability:

• Buatlah evaluasi terhadap atribut usability – Pengamatan dengan Mr. ABC yang baru pertama kali melihat aplikasi BNI SMS Banking, hasilnya
Mr. ABC bisa menggunakan BNI SMS Banking dalam waktu 10 menit
– Pengamatan dengan Mr. XYZ yang sudah terbiasa bertransaksi menggunakan SMS Banking.
Hasilnya Mr. XYZ bisa menggunakan seluruh menu dalam waktu 5 menit.
• Effeciency:
– Pengukuran hampir sama dengan learnability tapi lebih mendetail pada tiap fungsi aplikasi (menu)
• Memorability:
– get sample casual users (away from system for certain time), measure time to perform typical
tasks..
• Errors:
– Hitung tingkat kesalahan yang terjadi ketika menggunakan aplikasi, bedakan antara kesalahan
dari sistem dengan kesalahan diluar sistem (human error).
• Satisfaction:
– Pendapat pribadi dari pengguna
Page 25 Page 26