MODULE 6 and 7

Evaluation Techniques
and
Universal Design
Why Evaluate?
In HCI we evaluate interfaces and systems to:

● Determine how usable they are for different user groups

● Identify good and bad features to inform future design
● Compare design choices to assist us in making decisions
● Observe the effects of specific interfaces on users

There are two main types of evaluation:

● evaluation by system designers or experts - typically conducted
early in the development lifecycle, though may give late feedback
on system performance
● evaluation by end users - typically conducted late in the
development lifecycle, though may give early feedback on system
design
Goals of Evaluation
There are 3 main goals of user interface evaluation:

❖ System functionality

● does the system meet the user’s requirements?

● is the system clear to operate?
● does the system help make the user effective at her task?

❖ user’s experience is the interface usable?

● is the user satisfied?
● is the user’s experience using the interface pleasant?
● is the user happy/angry/frustrated when using the interface?
● take into account the user’s task, for example, if the interface is
for a game, then the user should have fun using the system
(playing the game).

❖ problem identification

● does the system produce errors?

● is the user confused when using the system?
 Evaluation by Experts
There are 4 types of evaluations conducted by experts or system
designers:

● cognitive walkthrough
● heuristic evaluation
● model-based
● based on prior studies
Experts: Cognitive Walkthrough

❖ Experts: Cognitive Walkthrough

● “walkthrough” means a sequence of steps

● e.g., “code walkthrough” is when a programmer shows her

source code to others and explains, line by line, how the code
works

● cognitive walkthrough is when a user explains how she is using an

interface, talking as she performs each action and explaining as
she goes then, experts analyze the user’s process
 for each step:

● is the effect of the user’s action the same as the user’s goal?

● will the user see that the necessary action is available (in the
interface)?

● will the user know which widget in the interface enables the
desired action?

● after the action is performed (e.g., clicking on a button), will the

user receive understandable feedback?
❖ Expert: Heuristic Evaluation
Heuristics evaluation is a methodical procedure to check user
interface for usability problems. Once a usability problem is
detected in design, they are attended as an integral part of
constant design processes. Heuristic evaluation method
includes some usability principles such as Nielsen’s ten
Usability principles.
❖ Experts: Model-Based
● high-level models (such as GOMS) are used to predict user’s
performance
● low-level models (such as Fitts law—keystroke, dialogue timing) are
used to assess limits of user’s performance
❖ Experts: Based on Prior Studies
● empirical models evolved from previous studies give indicators of
generic interface elements
● for example, menu types, command name recall, color schemes,
button labels, icon designs, etc.

❖ Evaluation by Users
● can be done at different stages of development
● system developers can simulate missing (undeveloped) pieces of
interface using techniques like “Wizard of Oz” where a human
takes the part of the pieces that will later be automated
● elements of user evaluation:
● styles of evaluation
● experimental evaluation design
● observational techniques o query techniques
● evaluation through physiological responses

❖ Users: Styles of Evaluation

● laboratory studies
○ conducted in controlled settings
○ advantages: no distractions
○ disadvantages: but no context
❖ Users: Experimental Evaluation Design
● for conducting “controlled experiments” where the goal is to
support a claim or a hypothesis

● participants:
○ choose carefully, to “simulate” actual end users
○ “sample size” (number of users): usability studies
recommend 3-5 users, also called “subjects”; controlled
experiments recommend at least twice this number; even
more when statistical significance is sought from results
● variables:

○ manipulated to create different conditions for experimental

comparison
○ independent versus dependent
■ independent: variable that changes; that is manipulated
(e.g., menu composition in an interface)
■ dependent: variable that doesn’t change; that is
measured (e.g., speed of selecting items on the menu)
● hypothesis:
○ prediction of outcome
○ framed in terms of independent and dependent variables
○ null hypothesis—the experimental results are random; not
caused by the value(s) of the independent variable
○ alternate hypothesis—the experimental results are not
random, but are caused by the changes to the independent
variable (i.e., what we are testing)

● experimental design:
○ choose the hypothesis
○ choose the experimental method
○ between subjects
○ within subjects
❖ Users: Observational Techniques
● protocols: – “think aloud”
● protocol analysis:

❖ Users: Query Techniques

● evaluator asks user directly about the interface methods:

● interview o questionnaire types of questions in a typical
questionnaire:
● general (e.g., user background information, demographics)
● open-ended o scalar (e.g., “rate this on a scale of 1 to 5...”)
● multiple choice o ranked (e.g., “pick the best of ...” )
❖ Users: Evaluation Through Physiological Responses
● use medical devices to record user’s physiological responses
while using interface

● types of bodily functions tracked:

○ `eye tracking ∗ record the number of fixations (times when
the user’s gaze is fixed on the same location for a minimum
period of time) ∗ record the length of each fixation ∗ record
the user’s scan path (the trajectory across the interface
where the user’s gaze moves)
○ heart rate, pulse – breathing – skin secretions (sweat) –
muscle activity – brain activity (e.g., EEG)
 Choosing Evaluation Techniques

● there are 8 factors to take into consideration when choosing

the appropriate evaluation technique:

1. stage in development cycle design (early) versus implementation

(late) stage early stage should be quick and cheap

2. style of evaluation i.e., laboratory versus field study

3. level of subjectivity or objectivity- how much will results rely on

the (subjective) interpretation of the evaluator? versus quantitative
measures, like time to complete a task
4. types of measures - quantitative versus qualitative versus mixed
methods
5. information provided - is high-level information about the
interface sought— e.g., is it usable? which would require interviews,
questionnaires, etc., is low-level information sought—e.g., which
font looks better?
6. immediacy of responses needed
● are evaluation results needed immediately, at the time of
the experiments? then use something like a “walkthrough”
or interview
● are results needed later? then use techniques that require
analysis, such as computer logs or user logs
 7. level of interference
● how intrusive is the evaluation process?
● the evaluation should NOT impact how the user uses the
interface...
8. resources required
● how much time, money, personnel is required to conduct
the evaluation?
 MODULE 7:
Universal Design
Topic Outline:
● Universal Design
○ Define Universal Design
○ Seven Principles of universal design of Interactive System

● Multi-Modal Technology
● Accessibility in user interfaces
● Emerging technologies in interface design
 UNIVERSAL DESIGN

➢ Universal design is about designing systems so that they can be used

by anyone in any circumstance
➢ Universal design is the process of designing products so that they
can be used by as many people as possible in as many situations as
possible
➢ In reality, we may not be able to design everything to be accessible
to everyone. We certainly cannot ensure that everyone has the same
experience of using a product. But we can work toward the aim of
universal design and try to provide an equivalent experience
➢ Universal design is primarily about trying to ensure that you do not
exclude anyone through the design choices you make but, by giving
thought to these issues, you will invariably make your design better
for everyone
 UNIVERSAL DESIGN

➢ Universal design means designing for diversity

● People with sensory, physical or cognitive impairment
● People of different ages
● People from different cultures or backgrounds

❖ Seven Principles of Universal Design of Interactive System

1. Equitable use
2. Flexibility in use
3. Simple and intuitive to use
4. Perceptible information
5. Tolerance for error
6. Low physical effort
7. Size and space for approach and use
Seven Principles of Universal Design
of Interactive System

● Equitable use

○ The design is useful to people with range of ability and

appealing to all
○ No user is excluded or stigmatized
○ Access should be the same for all
○ Safety, security and privacy should be available to all
Seven Principles of Universal Design
of Interactive System
❖ Flexibility in use

● The design allows for range of ability and preference, through

choice of methods of use and adaptively to the user’s pace,
precision and custom
● Simple and intuitive to use
● Simple and intuitive to use regardless of knowledge,
experience, language or level of concentration of the user.
● The design needs to support the user’s expectations and
accommodate different language and literacy skills.
● Provide prompting and feedback.
Seven Principles of Universal Design of
Interactive System
❖ Perceptible information
❖
● Design should provide effective communication of information
regardless of the environmental condition or the user’s abilities.
● Essential information should be emphasized and differentiated
clearly from the peripheral content.

❖ Tolerance for error

● Minimizing the impact and damage caused by mistakes or
unintended behavior.
● Potentially dangerous situations should be removed or made
hard to reach.
● Potential hazards should be shielded by warnings.
Seven Principles of Universal Design of
Interactive System
❖ Low physical effort

● System should be designed to be comfortable to use,

minimizing physical effort and fatigue.
● The physical design of the system should allow the user to
maintain a natural posture with reasonable operating effort.
❖ Size and space for approach and use
❖
● The placement of the system should be such that it can be reached
and used by any user regardless of body size, posture and mobility.
● Allow for variation in hand size
● Provide room for assistive devices
 Multi-Modal Technology

● Providing access to information through more than one mode

of interaction is an important principle of universal design.

● The design known as multi-modal interaction

● There are five senses:

○ Sight
○ Sound
○ Touch
○ Taste
○ Smell
❖ Alternative Mode of HCI

1. Sound in the interface

2. Touch in the interface
3. Handwriting recognition
4. Gesture recognition

1. Sound in the interface

● Sound – important contributor to usability
● Experimental evidences:
○ Addition of audio confirmation of modes, in form of changes in
key clicks, reduces errors
○ Video games: Experts tend to score less well when the sound is
turned off than when it is on
● Dual presentation of information through sound and vision
supports universal design, by enabling access for users with
visual and hearing impairments respectively
● Two general types:
○ Speech
○ Non-Speech

Examples:
● Speech-based word processors
● Telephone-based system
● Interactive systems that give feedback
 2. Touch in the interface
● Touch is the only sense that can be used to both send and receive
information
● Used of touch in the interface is known as haptic interaction
● Haptics is a generic term relating to touch, but it can be roughly
divided into two areas:
● Cutaneous perception
○ Concerned with tactile
○ sensations through skin
● Kinesthetic
○ Perception of movement and position
● Tactile devices:
○ Electronic braille display
○ Force feedback devices in VR equipment
3. Handwriting Recognition
● Handwriting – natural form of communication
● Using digitizing tablet, refined by incorporating a thin screen on top
to display the information, producing electronic paper

● Example:
○ Digitizing tablet Electronic paper
 4. Gesture Recognition
● Being able to control the computer with certain movement of the
hand would be advantageous in many situations where there is no
possibility of typing.
● Could also support communication for people who have hearing
loss
● Technology of capturing gestures is expensive
● Examples:
● Data glove Computer vision
 ACCESSIBILITY IN USER
INTERFACES
● The needs of the accessibility guidelines and standards
● Accessibility features in user interfaces:
○ Biometric Access Control (e.g: Fingerprint,Palm Scan,Hand
Geometry, Iris Scan, Signature Dynamics,Keyboard Dynamics,
Voice Print, Facial Scan, Hand Topography)
○ Narrator (e.g : Jenwave, Powertalk,Speechexpert)
❖ Biometric Access Control
● Verifies an identity by analyzing a unique person attribute or
behavior (e.g., what a person “is”).
● Most expensive way to prove identity, also has difficulties with user
acceptance.
● Access Controls: The security features that control how users and
systems communicate and interact with one another.
● Access: The flow of information between subject and object
● Subject: An active entity that requests access to an object or
the data in an object
● Object: A passive entity that contains information

❖ Narrator

● Narrator is a Text-to-Speech utility for users who are blind or

have impaired vision.
● Narrator reads what is displayed on your screen: the contents
of the active window, menu options, or the text that you type.
● Narrator is designed to work with the Notepad, WordPad,
Control Panel programs, Microsoft Internet Explorer, the
Windows desktop, and Windows Setup.
● Narrator may not read words aloud correctly in other programs.

● Narrator Software:
a. Jenwave
b. Powertalk
c. Speech expert
EMERGING TECHNOLOGY OVERVIEW

● Emerging technologies are those which are just beginning to be

adopted or are at the initial acceptance stage, and are expected
to make a significant contribution to the world of IT and the
businesses that implement them.
● A novel computer technology with promising properties, use
and significance, for which the implications for
human-computer interaction are still unclear
● Examples:
● The Burton Amp Jacket
● BMW Assist o Disney’s Pirates of the Caribbean
● Are different from desktop computers in physical appearance
use contexts
● All represent different directions of development
○ Mobility
○ Immersion
○ Context-awareness
○ Multi-modality…
❖ The Burton Amp Jacket

● A mobile music system built into a jacket

● User controls MP3 player by touching specific areas of his jacket
❖ BMW Assist

● A context-aware car navigation system

● User drives the car and receives route planning information
based on the car’s location through a mix of text, graphics and
speech.
❖ Disney’s Pirates of the Caribbean

● An immersive theme park ride

● Users steer a physical boat through a virtual environment by
collectively operating helm and cannons
 Input and Output Technologies

● Input technologies allow people and other technologies to put

data into a computer.
● The two main types of input devices are:
a. Human Data Entry devices include keyboards, mouse,
trackball, joystick, touch screen, stylus and voice recognition;

b. Source-data automation devices input data with minimal

human intervention (e.g. barcode reader)

● Speed up data collection

● Reduce errors
● Gather data at the source of a transaction or other event.
Introduction of Mobile Computing &
Wearable Computing
● Mobile Computing –paradigm in which users carry their
personal computers and retain connectivity to other machines.
Examples: Handheld computing (use PDAs, iPhones)
○ Systems running on small, lightweight computing devices
○ Technology so small it fits in one‟s hand or pocket •
Facilitates high level of mobility
○ Portability-are fundamentally different from stationary
machines such as desktop computers
○ Small Storage Capacity
○ Low Power
○ Small User Interface
○ Risks to Data-more prone to complete data loss
○ Allows user to access information while moving through
physical space
● Wearable Computing –paradigm where users carry wearable
computing devices on their person (clothes, watches, rings, ..)
● A wearable computer facilitates a new form of
human-computer interaction‟ that isn‟t possible with even
today’s most sophisticated devices.
● This means that the computer is always with the user and
always interacting, completely intertwined with the user.
● A wearable computer should be a continuation of its user,
working like a natural part of the body, not an external
apparatus.
● Example :
○ Augmented Reality, VR
 Impact of Emerging Technologies

● Some emerging technologies may result in the lengthening of

human life, the development of artificial intelligence and the
enhancement of human sensory perception.
● Such results can be controversial and can inspire debates about
ethical use, topics that organizations such as the Institute for
Ethics and Emerging Technologies seek explicitly to address.
● The effect that emerging technologies have on the world can
vary greatly depending on the type of technology.
● The National Science Foundation examines the effects of
nanotechnology and biotechnology on the environment with
programs such as sensor networks to measure and monitor the
environmental impact of nanoparticles.