Chapter 5

EMOTIONAL
INTERACTION
 Overview
• Emotions and the user experience
• Expressive and annoying interface
– how the ‘appearance’ of an interface can affect users
• Models of emotion
– Ortony et al (2005)

• Automatic emotion recognition and emotional technologies

• Persuasive technologies and behavioral change
– how technologies can be designed to change people’s
attitudes and behavior
• Anthropomorphism
– The pros and cons

www.id-book.com 2
 Emotions and the user
experience
• HCI has traditionally been about designing efficient and
effective systems

• Now more about how to design interactive systems that

make people respond in certain ways
– e.g. to be happy, to be trusting, to learn, to be motivated

• Emotional interaction is concerned with how we feel and

react when interacting with technologies

www.id-book.com 3
 Is this form fun to fill in?

“My goal was to design

Wufoo to feel like
something Fisher-Price
would make.”
- Kevin Hale, Wufoo director

www.id-book.com 4
 Emotional interaction
• What makes us happy, sad, annoyed, anxious,
frustrated, motivated, delirious and so on
– translating this into different aspects of the user experience

• Why people become emotionally attached to certain

products (e.g. virtual pets)

• Can social robots help reduce loneliness and improve

wellbeing?

• How to change human behavior through the use of

emotive feedback

www.id-book.com 5
 Activity
• Try to remember the emotions you went
through when buying a big ticket item
online (e.g. a fridge, a vacation, a
computer)

• How many different emotions did you go

through?

www.id-book.com 6
 Emotional design model
• Norman, Ortony and Revelle (2004) model of emotion

www.id-book.com 7
 Claims from model
• Our emotional state changes how we think
– when frightened or angry we focus narrowly and body
responds by tensing muscles and sweating
• more likely to be less tolerant

– when happy we are less focused and the body

relaxes
• more likely to overlook minor problems and be more creative

www.id-book.com 8
 Activity
• Do you feel more creative when you are in
a happy mood? Do you get less work done
when you are feeling stressed?

www.id-book.com 9
 Expressive interfaces
• Provide reassuring feedback that can be both
informative and fun
• But can also be intrusive, causing people to get annoyed
and even angry
• Color, icons, sounds, graphical elements and animations
are used to make the ‘look and feel’ of an interface
appealing
– conveys an emotional state
• In turn this can affect the usability of an interface
– people are prepared to put up with certain aspects of an interface (e.g.
slow download rate) if the end result is appealing and aesthetic

www.id-book.com 10
 Friendly interfaces
• Microsoft pioneered friendly interfaces for
technophobes - ‘At home with Bob’ software

• 3D metaphors based on familiar places (e.g.

living rooms)

• Agents in the guise of pets (e.g. bunny, dog)

were included to talk to the user
– Make users feel more at ease and comfortable

www.id-book.com 11
 Bob

www.id-book.com 12
 Clippy
• Why was Clippy
disliked
by so many?
• Was it annoying,
distracting,
patronising or
other?
• What sort of user
liked Clippy?

www.id-book.com 13
 Frustrating interfaces
• Many causes:
– When an application doesn’t work properly or crashes
– When a system doesn’t do what the user wants it to do
– When a user’s expectations are not met
– When a system does not provide sufficient information to
enable the user to know what to do
– When error messages pop up that are vague, obtuse or
condemning
– When the appearance of an interface is garish, noisy,
gimmicky or patronizing
– When a system requires users to carry out too many steps to
perform a task, only to discover a mistake was made earlier
and they need to start all over again

www.id-book.com 14
 Gimmicks
• Amusing to the designer but not the user,
e.g.
– Clicking on a link to a website only to discover
that it is still ‘under construction’

www.id-book.com 15
 Error messages
“The application Word Wonder has unexpectedly quit due to a
type 2 error.”
Why not instead:
“the application has expectedly quit due to poor coding in the
operating system”

• Shneiderman’s guidelines for error messages include:

• avoid using terms like FATAL, INVALID, BAD
• Audio warnings
• Avoid UPPERCASE and long code numbers
• Messages should be precise rather than vague
• Provide context-sensitive help

www.id-book.com 16
Website error messages

www.id-book.com 17
More helpful error message
“The requested page /helpme is not available on the web server.

If you followed a link or bookmark to get to this page, please let us

know, so that we can fix the problem. Please include the URL of the
referring page as well as the URL of the missing page.

Otherwise check that you have typed the address of the web page
correctly.

The Web site you seek

Cannot be located, but
Countless more exist.”

www.id-book.com 18
 Should computers say they’re sorry?
• Reeves and Naas (1996) argue that computers should be
made to apologize
• Should emulate human etiquette
• Would users be as forgiving of computers saying sorry as
people are of each other when saying sorry?
• How sincere would they think the computer was being? For
example, after a system crash:
– “I’m really sorry I crashed. I’ll try not to do it again”

• How else should computers communicate with users?

www.id-book.com 19
 Detecting emotions and emotional
technology
• Sensing technologies used to measure
GSR, facial expressions, gestures, body
movement
• Aim is to predict user’s emotions and
aspects of their behavior –
• E.g. what is someone most likely to buy
online when feeling sad, bored or happy

www.id-book.com 20
 Facial Coding
• Measures a user’s emotions as they interact
with a computer or tablet
• .

• Analyses images captured by a webcam of

their face
• Uses this to gauge how engaged the user is
when looking at movies, online shopping
sites and ads
• 6 core expressions - sadness, happiness,
disgust, fear, surprise and anger

www.id-book.com 21
www.id-book.com 22
How to use the emotional data?
• If user screws up their face when an ad
pops up -> feel disgust
• If start smiling -> they are feeling happy
• Website can adapt its ad, movie storyline
or content to match user’s emotional state
• Eye-tracking, finger pulse, speech and
words/phrases also analysed when
tweeting or posting to Facebook
www.id-book.com 23
 Indirect emotion detection
• Beginning to be used more to infer or
predict someone’s behavior
• For example, determining a person’s
suitability for a job, or how they will vote at
an election
• Do you think it is creepy that technology
can read your emotions from your facial
expressions or from your tweets?

www.id-book.com 24
 Persuasive technologies and
behavioral change
• Interactive computing systems deliberately
designed to change people’s attitudes and
behaviors (Fogg, 2003)

• A diversity of techniques now used to change

what they do or think
– Pop-up ads, warning messages, reminders, prompts,
personalized messages, recommendations, Amazon 1-click

– Commonly referred to as nudging

www.id-book.com 25
 Nintendo’s Pocket Pikachu
• Changing bad habits and improving well being
– Designed to motivate children to be more physically
active on a regular basis

– owner of the digital pet that ‘lives’ in the device is

required to walk, run, or jump

– If owner does not exercise the virtual pet becomes

angry and refuses to play anymore

www.id-book.com 26
 How effective?
• Is the use of novel forms of interactive
technologies (e.g., the combination of
sensors and dynamically updated
information) that monitor, nag, or send
personalized messages intermittently to a
person more effective at changing a person’s
behavior than non-interactive methods, such
as the placement of warning signs, labels, or
ads in prominent positions?

www.id-book.com 27
Which is most effective?

www.id-book.com 28
 Tracking devices
• Mobile apps designed to help people
monitor and change their behaviour (e.g.
fitness, sleeping, weight)
• Can compare with online leader boards
and charts, to show how they have done in
relation to their peers and friends
• Also apps that encourage reflection that in
turn increase well-being and happiness
www.id-book.com 29
Energy reduction

www.id-book.com 30
 The Tidy Street project
• large-scale visualization of the street’s
electricity usage
– stenciled display on the road surface using chalk

– provided realtime feedback that all could see change each day

– reduced electricity consumption by 15%

www.id-book.com 31
 Phishing and trust
• Web used to deceive people into parting with
personal details
– e.g. Paypal, eBay and won the lottery letters

• Allows Internet fraudsters to access their

bank accounts and draw money from them
• Many vulnerable people fall for it
• The art of deception is centuries old but
internet allows ever more ingenious ways to
trick people

www.id-book.com 32
 Anthropomorphism
• Attributing human-like qualities to inanimate objects
(e.g. cars, computers)

• Well known phenomenon in advertising

– Dancing butter, drinks, breakfast cereals

• Much exploited in human-computer interaction

– Make user experience more enjoyable, more
motivating, make people feel at ease, reduce
anxiety

www.id-book.com 33
 Which do you prefer?
1. As a welcome message

• “Hello Chris! Nice to see you again. Welcome

back. Now what were we doing last time? Oh
yes, exercise 5. Let’s start again.”

• “User 24, commence exercise 5.”

www.id-book.com 34
 Which do you prefer?

2. Feedback when get something wrong

1. “Now Chris, that’s not right. You can do better

than that. Try again.”
2. “Incorrect. Try again.”

Is there a difference as to what you prefer

depending on type of message? Why?

www.id-book.com 35
Evidence to support anthropomorphism

• Reeves and Naas (1996) found that

computers that flatter and praise users in
education software programs -> positive
impact on them
“Your question makes an important and useful
distinction. Great job!”
• Students were more willing to continue with
exercises with this kind of feedback

www.id-book.com 36
 Criticism of anthropomorphism
• Deceptive, make people feel anxious, inferior or stupid
• People tend not to like screen characters that wave their
fingers at the user and say:
– Now Chris, that’s not right. You can do better than
that.Try again.”

• Many prefer the more impersonal:

– “Incorrect. Try again.”
• Studies have shown that personalized feedback is
considered to be less honest and makes users feel less
responsible for their actions (e.g. Quintanar, 1982)

www.id-book.com 37
 Virtual characters
• Appearing on our screens in the form of:
– Sales agents, characters in videogames,
learning companions, wizards, pets,
newsreaders

• Provides a persona that is welcoming, has

personality and makes user feel involved
with them

www.id-book.com 38
 Disadvantages
• Can lead people into false sense of belief,
enticing them to confide personal secrets with
chatterbots

• Annoying and frustrating

– e.g. Clippy

• May not be trustworthy

– virtual shop assistants?

www.id-book.com 39
 Virtual agents
• What do the virtual agents do?
• Do they elicit an emotional response in
you?
• Do you trust them?
• What is the style of interaction?
• What facial expression do they have?
• Are they believable, pushy, helpful?
• Would it be different if they were male?
If so, how?
www.id-book.com 40
What makes a virtual agent believable?
• Believability refers to the extent to which
users come to believe an agent’s intentions
and personality
• Appearance is very important
– Are simple cartoon-like characters or more realistic characters, resembling
the human form more believable?

• Behaviour is very important

– How an agent moves, gestures and refers to objects on the screen

– Exaggeration of facial expressions and gestures to show underlying

emotions (c.f. animation industry)

www.id-book.com 41
 Robot-like or cuddly?
• Which do you prefer and why?

www.id-book.com 42
 Implications
• Should we create products that adapt according
to people’s different emotional states?
– When people are feeling angry should an interface
be more attentive and informative than when they
are happy?

• Is Norman right?
– designers “can get away with more” for products
intended to be used during leisure time than those
designed for serious tasks

www.id-book.com 43
 Summary
• Emotional aspects of interaction design concerned with how to
facilitate certain states (e.g. pleasure) or avoid reactions (e.g.
frustration)
• Well-designed interfaces can elicit good feelings in people
• Aesthetically pleasing interfaces can be a pleasure to use
• Expressive interfaces can provide reassuring feedback to users
• Badly designed interfaces make people frustrated, annoyed, or
angry
• Emotional technologies can be designed to persuade people to
change their behaviors or attitudes
• Anthropomorphism is the attribution of human qualities to
objects
• Virtual agents and robot pets have been developed to make
people feel motivated, reassured, and in a good mood

www.id-book.com 44
 Chapter 6
INTERFACES
 Overview
• Interface types
– highlight the main design and research issues
for each of the different interfaces

• Consider which interface is best for a

given application or activity

www.id-book.com 2
 1. Command-based
• Commands such as abbreviations (e.g. ls) typed
in at the prompt to which the system responds
(e.g. listing current files)
• Some are hard wired at keyboard, others can be
assigned to keys
• Efficient, precise, and fast
• Large overhead to learning set of commands

www.id-book.com 3
Second Life command-based interface
for visually impaired users

www.id-book.com 4
 Research and design issues
• Form, name types and structure are key
research questions

• Consistency is most important design principle

– e.g. always use first letter of command

• Command interfaces popular for web scripting

www.id-book.com 5
 2. WIMP and GUI
• Xerox Star first WIMP -> rise to GUIs
• Windows
– could be scrolled, stretched, overlapped, opened, closed, and
moved around the screen using the mouse

• Icons
– represented applications, objects, commands, and tools that
were opened when clicked on

• Menus
– offering lists of options that could be scrolled through and
selected

• Pointing device
– a mouse controlling the cursor as a point of entry to the
windows, menus, and icons on the screen
www.id-book.com 6
 GUIs
• Same basic building blocks as WIMPs but
more varied
– Color, 3D, sound, animation,
– Many types of menus, icons, windows
• New graphical elements, e.g.
– toolbars, docks, rollovers
• Challenge now is to design GUIs that are
best suited for tablet, smartphone and
smartwatch interfaces

www.id-book.com 7
 Windows
• Windows were invented to overcome physical
constraints of a computer display
– enable more information to be viewed and tasks to be
performed

• Scroll bars within windows also enable more

information to be viewed
• Multiple windows can make it difficult to find
desired one
– listing, iconising, shrinking are techniques that help
www.id-book.com 8
www.id-book.com 9
Apple’s shrinking windows

www.id-book.com 10
Safari panorama window view

www.id-book.com 11
Selecting a country from a
scrolling window

www.id-book.com 12
Is this method any better?

www.id-book.com 13
 Research and design issues
• Window management
– enables users to move fluidly between different
windows (and monitors)

• How to switch attention between windows

without getting distracted

• Design principles of spacing, grouping, and

simplicity should be used

www.id-book.com 14
 Menus
• A number of menu interface styles
– flat lists, drop-down, pop-up, contextual, and expanding ones,
e.g., scrolling and cascading
• Flat menus
– good at displaying a small number of options at the same time
and where the size of the display is small, e.g. iPods

– but have to nest the lists of options within each other, requiring
several steps to get to the list with the desired option

– moving through previous screens can be tedious

www.id-book.com 15
 Expanding menus
• Enables more options to be shown on a single
screen than is possible with a single flat menu
• More flexible navigation, allowing for selection of
options to be done in the same window
• Most popular are cascading ones
– primary, secondary and even tertiary menus
– downside is that they require precise mouse control
– can result in overshooting or selecting wrong options

www.id-book.com 16
Cascading menu

www.id-book.com 17
 Contextual menus
• Provide access to often-used commands that
make sense in the context of a current task
• Appear when the user presses the Control key
while clicking on an interface element
– e.g., clicking on a photo in a website together with holding down
the Control key results in options ‘open it in a new window,’
‘save it,’ or ‘copy it’

• Helps overcome some of the navigation

problems associated with cascading menus

www.id-book.com 18
Windows Jump List Menu

www.id-book.com 19
 Research and design issues
• What are best names/labels/phrases to use?
• Placement in list is critical
– Quit and save need to be far apart
• Choice of menu to use determined by
application and type of system
– flat menus are best for displaying a small number of
options at one time
– expanding menus are good for showing a large
number of options

www.id-book.com 20
 Icon design
• Icons are assumed to be easier to learn and
remember than commands
• Can be designed to be compact and variably
positioned on a screen
• Now pervasive in every interface
– e.g. represent desktop objects, tools (e.g. paintbrush),
applications (e.g. web browser), and operations (e.g.
cut, paste, next, accept, change)

www.id-book.com 21
 Icons
• Since the Xerox Star days icons have changed
in their look and feel:
– black and white -> color, shadowing, photorealistic
images, 3D rendering, and animation

• Many designed to be very detailed and animated

making them both visually attractive and
informative
• GUIs now highly inviting, emotionally appealing,
and feel alive

www.id-book.com 22
 Icon forms
• The mapping between the representation and
underlying referent can be:
– similar (e.g., a picture of a file to represent the object file)
– analogical (e.g., a picture of a pair of scissors to represent ‘cut)
– arbitrary (e.g., the use of an X to represent ‘delete’)
• Most effective icons are similar ones
• Many operations are actions making it more difficult
to represent them
– use a combination of objects and symbols that capture the salient
part of an action

www.id-book.com 23
Early icons

www.id-book.com 24
Newer icons

www.id-book.com 25
Simple flat 2D icons

www.id-book.com 26
 Activity
• Sketch simple icons to represent the following
operations to appear on a digital camera screen:
– Turn image 90 degrees sideways

– Auto-enhance the image

– Fix red-eye

– Crop the image

• Show them to someone else and see if they can

understand what each represents

www.id-book.com 27
 Basic edit icons on iPhone
• Which is which?

• Are they easy to understand

• Are they distinguishable?

• What representation forms

are used?

• How do yours compare?

www.id-book.com 28
 Research and design issues
• There is a wealth of resources now so do not
have to draw or invent new icons from scratch
– guidelines, style guides, icon builders, libraries

• Text labels can be used alongside icons to help

identification for small icon sets

• For large icon sets (e.g. photo editing or word

processing) use rollovers

www.id-book.com 29
 3. Multimedia
• Combines different media within a single
interface with various forms of interactivity
– graphics, text, video, sound, and animations

• Users click on links in an image or text

-> another part of the program
-> an animation or a video clip is played
->can return to where they were or move on to another
place

www.id-book.com 30
BioBlast Multimedia Learning Environment

www.id-book.com 31
 Pros and cons
• Facilitates rapid access to multiple representations of
information
• Can provide better ways of presenting information than
can any media alone
• Can enable easier learning, better understanding, more
engagement, and more pleasure
• Can encourage users to explore different parts of a
game or story
• Tendency to play video clips and animations, while
skimming through accompanying text or diagrams

www.id-book.com 32
 Research and design issues
• How to design multimedia to help users explore,
keep track of, and integrate the multiple
representations
– provide hands-on interactivities and simulations that the user has
to complete to solve a task
– Use ‘dynalinking,’ where information depicted in one window
explicitly changes in relation to what happens in another (Scaife
and Rogers, 1996).

• Several guidelines that recommend how to

combine multiple media for different kinds of
task

www.id-book.com 33
 4. Virtual reality
• Computer-generated graphical simulations
providing:
– “the illusion of participation in a synthetic environment
rather than external observation of such an
environment” (Gigante, 1993)

• Provide new kinds of experience, enabling users

to interact with objects and navigate in 3D space
• Create highly engaging user experiences

www.id-book.com 34
 Pros and cons
• Can have a higher level of fidelity with objects they
represent compared to multimedia
• Induces a sense of presence where someone is totally
engrossed by the experience
– “a state of consciousness, the (psychological) sense of being in
the virtual environment” (Slater and Wilbur, 1999)

• Provides different viewpoints: 1st and 3rd person

• Head-mounted displays are uncomfortable to wear, and
can cause motion sickness and disorientation

www.id-book.com 35
 Research and design issues
• Much research on how to design safe and realistic VRs to
facilitate training
– e.g. flying simulators
– help people overcome phobias (e.g. spiders, talking in public)

• Design issues
– how best to navigate through them (e.g. first versus third person)
– how to control interactions and movements (e.g. use of head and
body movements)
– how best to interact with information (e.g. use of keypads, pointing,
joystick buttons);
– level of realism to aim for to engender a sense of presence

www.id-book.com 36
Which is the most engaging game of
Snake?

www.id-book.com 37
 5. Information visualization and
dashboards
• Computer-generated interactive graphics of complex data

• Amplify human cognition, enabling users to see patterns,

trends, and anomalies in the visualization (Card et al, 1999)

• Aim is to enhance discovery, decision-making, and

explanation of phenomena

• Techniques include:
– 3D interactive maps that can be zoomed in and out of and which
present data via webs, trees, clusters, scatterplot diagrams, and
interconnected nodes

www.id-book.com 38
 Dashboards
• Show screenshots of data updated over
periods of time - to be read at a glance
• Usually not interactive - slices of data that
depict current state of a system or process
• Need to provide digestible and legible
information for users
– design its spatial layout so intuitive to read when
first looking at it
– should also direct a user’s attention to anomalies
or unexpected deviations

www.id-book.com 39
Which dashboard is best?

www.id-book.com 40
Which dashboard is best?

www.id-book.com 41
 Research and design issues
• Whether to use animation and/or interactivity
• What form of coding to use, e.g. color or text
labels
• Whether to use a 2D or 3D representational
format
• What forms of navigation, e.g. zooming or
panning,
• What kinds and how much additional information
to provide, e.g. rollovers or tables of text
• What navigational metaphor to use

www.id-book.com 42
 6. Web
• Early websites were largely text-based, providing
hyperlinks

• Concern was with how best to structure information

to enable users to navigate and access it easily and
quickly
• Nowadays, more emphasis on making pages
distinctive, striking, and pleasurable
• Need to think of how to design information for multi-
platforms - keyboard or touch?
– e.g. smartphones, tablets, PCs

www.id-book.com 43
 Usability versus attractive?
• Vanilla or multi-flavor design?
– Ease of finding something versus aesthetic and
enjoyable experience
• Web designers are:
– “thinking great literature”
• Users read the web like a:
– “billboard going by at 60 miles an hour” (Krug, 2000)
• Need to determine how to brand a web page to
catch and keep ‘eyeballs’

www.id-book.com 44
 In your face ads
• Web advertising is often intrusive and
pervasive
• Flashing, aggressive, persistent, annoying
• Often need to be ‘actioned’ to get rid of
• What is the alternative?

www.id-book.com 45
 Research and design issues
• Need to consider how best to design, present,
and structure information and system behavior
• But also content and navigation are central
• Veen’s (2001) design principles
(1)Where am I?

(2)Where can I go?

(3) What’s here?

www.id-book.com 46
 Activity
• Look at the Nike.com website
• What kind of website is it?
• How does it contravene the design principles
outlined by Veen?
• Does it matter?
• What kind of user experience is it providing for?
• What was your experience of engaging with it?

www.id-book.com 47
 7. Consumer electronics and
appliances
• Everyday devices in home, public place, or car
– e.g. washing machines, remotes, photocopiers, printers and
navigation systems)

• And personal devices

– e.g. MP3 player, digital clock and digital camera

• Used for short periods

– e.g. putting the washing on, watching a program, buying a ticket,
changing the time, taking a snapshot

• Need to be usable with minimal, if any, learning

www.id-book.com 48
A toaster

www.id-book.com 49
 Research and design issues
• Need to design as transient interfaces with
short interactions
• Simple interfaces
• Consider trade-off between soft and hard
controls
– e.g. buttons or keys, dials or scrolling

www.id-book.com 50
 8. Mobile
• Handheld devices intended to be used while on
the move
• Have become pervasive, increasingly used in all
aspects of everyday and working life
• Apps running on mobiles have greatly expanded,
e.g.
– used in restaurants to take orders
– car rentals to check in car returns
– supermarkets for checking stock
– in the streets for multi-user gaming
– in education to support life-long learning

www.id-book.com 51
 The advent of the iPhone app
• A whole new user experience that was
designed primarily for people to enjoy
– many apps not designed for any need, want or use
but purely for idle moments to have some fun

– e.g. iBeer developed by magician Steve Sheraton

– ingenious use of the accelerometer that is inside the

phone

www.id-book.com 52
iBeer app

www.id-book.com 53
QR codes and cell phones

www.id-book.com 54
 Mobile challenges
• Smaller screens, small number of physical keys and
restricted number of controls
• Innovative physical designs including:
– roller wheels, rocker dials, up/down ‘lips’ on the face of
phones, 2-way and 4-way directional keypads, softkeys,
silk-screened buttons

• Usability and preference varies

– depends on the dexterity and commitment of the user

• Smartphones overcome mobile physical

constraints through using multi-touch displays
www.id-book.com 55
 Research and design issues
• Mobile interfaces can be tricky and
cumbersome to use for those with poor
manual dexterity or ‘fat’ fingers
• Key concern is hit area
– area on the phone display that the user touches
to make something happen, such as a key, an
icon, a button or an app
– space needs to be big enough for fat fingers to
accurately press
– if too small the user may accidentally press the
wrong key

www.id-book.com 56
 9. Speech
• Where a person talks with a system that has a
spoken language application, e.g. timetable,
travel planner
• Used most for inquiring about very specific
information, e.g. flight times or to perform a
transaction, e.g. buy a ticket
• Also used by people with disabilities
– e.g. speech recognition word processors, page
scanners, web readers, home control systems

www.id-book.com 57
Have speech interfaces come of
age?

www.id-book.com 58
 Get me a human operator!
• Most popular use of speech interfaces currently
is for call routing
• Caller-led speech where users state their needs
in their own words
– e.g. “I’m having problems with my voice mail”

• Idea is they are automatically forwarded to the

appropriate service
• What is your experience of speech systems?

www.id-book.com 59
 Format
• Directed dialogs are where the system is in control of the
conversation
• Ask specific questions and require specific responses
• More flexible systems allow the user to take the initiative:
– e.g. “I’d like to go to Paris next Monday for two weeks.”
• More chance of error, since caller might assume that the
system is like a human
• Guided prompts can help callers back on track
– e.g. “Sorry I did not get all that. Did you say you wanted to fly next
Monday?”

www.id-book.com 60
 Research and design issues
• How to design systems that can keep conversation
on track
– help people navigate efficiently through a menu system
– enable them to easily recover from errors
– guide those who are vague or ambiguous in their requests
for information or services

• Type of voice actor (e.g. male, female, neutral, or

dialect)
– do people prefer to listen to and are more patient with a
female or male voice, a northern or southern accent?

www.id-book.com 61
 10. Pen
• Enable people to write, draw, select, and move objects at
an interface using lightpens or styluses

– capitalize on the well-honed drawing skills developed

from childhood

• Digital pens, e.g. Anoto, use a combination of

ordinary ink pen with digital camera that digitally
records everything written with the pen on
special paper

www.id-book.com 62
 Pros and cons
• Allows users to quickly and easily
annotate existing documents
• Can be difficult to see options on the
screen because a user’s hand can occlude
part of it when writing
• Can have lag and feel clunky

www.id-book.com 63
 11. Touch
• Touch screens, such as walk-up kiosks, detect
the presence and location of a person’s touch on
the display

• Multi-touch support a range of more dynamic

finger tip actions, e.g. swiping, flicking, pinching,
pushing and tapping

• Now used for many kinds of displays, such as

Smartphones, iPods, tablets and tabletops

www.id-book.com 64
 Research and design issues
• More fluid and direct styles of interaction involving
freehand and pen-based gestures
• Core design concerns include whether size, orientation,
and shape of touch displays effect collaboration
• Much faster to scroll through wheels, carousels and bars
of thumbnail images or lists of options by finger flicking
• More cumbersome, error-prone and slower to type using
a virtual keyboard on a touch display than using a
physical keyboard

www.id-book.com 65
 Research and design issues
• Will finger-
flicking, swiping,
stroking and
touching a
screen result in
new ways of
consuming,
reading,
creating and
searching digital
content?

www.id-book.com 66
 12. Air-based gestures
• Uses camera recognition, sensor and computer
vision techniques
– can recognize people’s body, arm and hand gestures in a
room
– systems include Kinect

• Movements are mapped onto a variety of gaming

motions, such as swinging, bowling, hitting and
punching
• Players represented on the screen as avatars doing
same actions
www.id-book.com 67
 Home entertainment

• Universal
appeal
– young
children,
grandparent
s,
professional
gamers,
technophobe
s

www.id-book.com 68
 Gestures in the operating theatre
• A touchless
system that
recognizes
gestures

• surgeons can
interact with and
manipulate MRI
or CT images
– e.g. two-handed
gestures for
zooming and
panning

www.id-book.com 69
 Research and design issues
• How does computer recognize and
delineate user’s gestures?
– Deictic and hand waving

• Does holding a control device feel more

intuitive than controller free gestures?
– For gaming, exercising, dancing

www.id-book.com 70
 13. Haptic
• Tactile feedback
– applying vibration and forces to a person’s body,
using actuators that are embedded in their clothing or
a device they are carrying, such as a smartphone

• Can enrich user experience or nudge them to

correct error
• Can also be used to simulate the sense of touch
between remote people who want to
communicate

www.id-book.com 71
 Realtime vibrotactile feedback
• Provides nudges when
playing incorrectly
• Uses motion capture
• Nudges are vibrations
on arms and hands

www.id-book.com 72
 Research and design issues
• Where best to place actuators on body
• Whether to use single or sequence of ‘touches’
• When to buzz and how intense
• How does the wearer feel it in different contexts?
• What kind of new smartphones/smart-watches
apps can use vibrotactile creatively?
– e.g. slow tapping to feel like water dropping that is
meant to indicate it is about to rain and heavy tapping
to indicate a thunderstorm is looming

www.id-book.com 73
 14. Multi-modal
• Meant to provide enriched and complex
user experiences
– multiplying how information is experienced and
detected using different modalities, i.e. touch, sight,
sound, speech

– support more flexible, efficient, and expressive means

of human–computer interaction

– Most common is speech and vision

www.id-book.com 74
 Research and design issues
• Need to recognize and analyse speech,
gesture, and eye gaze
• What is gained from combining different
input and outputs
• Is talking and gesturing, as humans do
with other humans, a natural way of
interacting with a computer?

www.id-book.com 75
 15. Shareable
• Shareable interfaces are designed for more than
one person to use
– provide multiple inputs and sometimes allow
simultaneous input by co-located groups
– large wall displays where people use their own pens
or gestures
– interactive tabletops where small groups interact with
information using their fingertips
– e.g. DiamondTouch, Smart Table and Surface

www.id-book.com 76
A smartboard

www.id-book.com 77
DiamondTouch Tabletop

www.id-book.com 78
 Advantages
• Provide a large interactional space that can
support flexible group working
• Can be used by multiple users
– Can point to and touch information being displayed
– Simultaneously view the interactions and have same
shared point of reference as others

• Can support more equitable participation

compared with groups using single PC

www.id-book.com 79
 Research and design issues
• More fluid and direct styles of interaction involving
freehand and pen-based gestures
• Core design concerns include whether size,
orientation, and shape of the display have an effect on
collaboration
• Horizontal surfaces compared with vertical ones
support more turn-taking and collaborative working in
co-located groups
• Providing larger-sized tabletops does not improve
group working but encourages more division of labor

www.id-book.com 80
 16. Tangible
• Type of sensor-based interaction, where
physical objects, e.g., bricks, are coupled with
digital representations
• When a person manipulates the physical
object/s it causes a digital effect to occur, e.g. an
animation
• Digital effects can take place in a number of
media and places or can be embedded in the
physical object

www.id-book.com 81
 Examples
• Chromarium cubes
– when turned over digital animations of color are mixed on an
adjacent wall
– faciliates creativity and collaborative exploration

• Flow Blocks
– depict changing numbers and lights embedded in the blocks
– vary depending on how they are connected together
• Urp
– physical models of buildings moved around on tabletop
– used in combination with tokens for wind and shadows -> digital
shadows surrounding them to change over time

www.id-book.com 82
 Benefits
• Can be held in both hands and combined and
manipulated in ways not possible using other interfaces
– allows for more than one person to explore the interface
together
– objects can be placed on top of each other, beside each other,
and inside each other
– encourages different ways of representing and exploring a
problem space
• People are able to see and understand situations
differently
– can lead to greater insight, learning, and problem-solving than
with other kinds of interfaces
– can facilitate creativity and reflection

www.id-book.com 83
 VoxBox
• A tangible system that gathers opinions at events through playful
and engaging interaction (Goldsteijn et al, 2015)

www.id-book.com 84
 Research and design issues
• Develop new conceptual frameworks that identify novel and
specific features
• The kind of coupling to use between the physical action and
digital effect
– If it is to support learning then an explicit mapping between action
and effect is critical
– If it is for entertainment then can be better to design it to be more
implicit and unexpected
• What kind of physical artifact to use
– Bricks, cubes, and other component sets are most commonly used
because of flexibility and simplicity
– Stickies and cardboard tokens can also be used for placing material
onto a surface

www.id-book.com 85
 17. Augmented and mixed reality
• Augmented reality - virtual representations are
superimposed on physical devices and objects

• Mixed reality - views of the real world are

combined with views of a virtual environment

• Many applications including medicine, games,

flying, and everyday exploring

www.id-book.com 86
 Examples
• In medicine
– virtual objects, e.g. X-rays and scans, are overlaid on part
of a patient’s body
– aid the physician’s understanding of what is being
examined or operated

• In air traffic control

– dynamic information about aircraft overlaid on a video
screen showing the real planes, etc. landing, taking off,
and taxiing
– Helps identify planes difficult to make out

www.id-book.com 87
An augmented map

www.id-book.com 88
Top Gear James May in AR
• Appears as a 3D character to act as personal
tour guide at Science Museum

www.id-book.com 89
 Research and design issues
• What kind of digital augmentation?
– When and where in physical environment?

– Needs to stand out but not distract from ongoing task

– Need to be able to align with real world objects

• What kind of device?

– Smartphone, head up display or other?

www.id-book.com 90
 18.Wearables
• First developments were head- and eyewear-mounted
cameras that enabled user to record what was seen and
to access digital information

• Since, jewellery, head-mounted caps, smart fabrics,

glasses, shoes, and jackets have all been used
– provide the user with a means of interacting with digital
information while on the move

• Applications include automatic diaries, tour guides, cycle

indicators and fashion clothing

www.id-book.com 91
 Google Glass: short-lived

• What were the pros and cons?

www.id-book.com 92
 Research and design issues
• Comfort
– needs to be light, small, not get in the way, fashionable, and
preferably hidden in the clothing

• Hygiene
– is it possible to wash or clean the clothing once worn?

• Ease of wear
– how easy is it to remove the electronic gadgetry and replace it?

• Usability
– how does the user control the devices that are embedded in the
clothing?

www.id-book.com 93
 19. Robots and drones
• Four types of robot
– remote robots used in hazardous settings
– domestic robots helping around the house
– pet robots as human companions
– sociable robots that work collaboratively with humans,
and communicate and socialize with them – as if they
were our peers

www.id-book.com 94
 Advantages
• Pet robots are assumed to have therapeutic qualities,
helping to reduce stress and loneliness
• Remote robots can be controlled to investigate bombs
and other dangerous materials

www.id-book.com 95
 Drones
• Unmanned aircraft that are controlled remotely and
used in a number of contexts
– e.g. entertainment, such as carrying drinks and food to
people at festivals and parties;
– agricultural applications, such as flying them over
vineyards and fields to collect data that is useful to farmers
– helping to track poachers in wildlife parks in Africa
• Can fly low and and stream photos to a ground station,
where images can be stitched together into maps
• Can be used to determine the health of a crop or when
it is the best time to harvest the crop
www.id-book.com 96
Drone in vineyard

www.id-book.com 97
 Research and design issues
• How do humans react to physical robots designed to
exhibit behaviors (e.g. making facial expressions)
compared with virtual ones?
• Should robots be designed to be human-like or look like
and behave like robots that serve a clearly defined
purpose?
• Should the interaction be designed to enable people to
interact with the robot as if it was another human being
or more human-computer-like (e.g. pressing buttons to
issue commands)?
• Is it acceptable to use unmanned drones to take a series
of images or videos of fields, towns, and private property
without permission or people knowing what is
happening?

www.id-book.com 98
 20. Brain-computer interfaces
• Brain–computer interfaces (BCI) provide a
communication pathway between a person’s brain
waves and an external device, such as a cursor on a
screen
• Person is trained to concentrate on the task, e.g. moving
the cursor
• BCIs work through detecting changes in the neural
functioning in the brain
• BCIs apps:
– Games
– enable people who are paralysed to control robots

www.id-book.com 99
Brainball game

www.id-book.com 100
 Which interface?
• Is multimedia better than tangible interfaces for learning?

• Is speech as effective as a command-based interface?

• Is a multimodal interface more effective than a monomodal

interface?

• Will wearable interfaces be better than mobile interfaces for helping

people find information in foreign cities?

• Are virtual environments the ultimate interface for playing games?

• Will shareable interfaces be better at supporting communication

and collaboration compared with using networked desktop PCs?

www.id-book.com 101
 Which interface?
• Will depend on task, users, context, cost,
robustness, etc.
• Mobile platforms taking over from PCs
• Speech interfaces also being used much
more for a variety of commercial services
• Appliance and vehicle interfaces becoming
more important
• Shareable and tangible interfaces entering
our homes, schools, public places, and
workplaces
www.id-book.com 102
 Summary
• Many innovative interfaces have emerged post
the WIMP/GUI era, including speech, wearable,
mobile, brain and tangible
• Raises many design and research questions to
decide which to use
– e.g. how best to represent information to the user so
they can carry out ongoing activity or task
• New interfaces that are context-aware or monitor
raise ethical issues concerned with what data is
being collected and what it is used for

www.id-book.com 103
 Chapter 7
GATHERING DATA
 Aims
• Discuss how to plan and run a successful data
gathering program.

• Enable you to plan and run an interview.

• Enable you to design a simple questionnaire.

• Enable you to plan and carry out an

observation.

www.id-book.com 2
 Five key issues
1. Setting goals
• Decide how to analyze data once collected
2. Identifying participants
• Decide who to gather data from
3. Relationship with participants
• Clear and professional
• Informed consent when appropriate
4. Triangulation
• Look at data from more than one perspective
• Collect more than one type of data, eg qualitative from
experiments and qualitative from interviews
5. Pilot studies
• Small trial of main study

www.id-book.com 3
 Data recording
• Notes, audio, video, photographs can be
used individually or in combination:
– Notes plus photographs
– Audio plus photographs
– Video

• Different challenges and advantages

with each combination

4
www.id-book.com
 Interviews
• Unstructured - are not directed by a script.
Rich but not replicable.
• Structured - are tightly scripted, often like a
questionnaire. Replicable but may lack
richness.
• Semi-structured - guided by a script but
interesting issues can be explored in more
depth. Can provide a good balance between
richness and replicability.
• Focus groups – a group interview
www.id-book.com 5
 Interview questions
• Two types:
− ‘closed questions’ have a predetermined answer format, e.g..
‘yes’ or ‘no’
− ‘open questions’ do not have a predetermined format

• Closed questions are easier to analyze

• Avoid:
− Long questions
− Compound sentences - split them into two
− Jargon and language that the interviewee may not understand
− Leading questions that make assumptions e.g.. why do you
like …?
− Unconscious biases e.g.. gender stereotypes
www.id-book.com 6
 Running the interview
• Introduction – introduce yourself, explain the goals of the
interview, reassure about the ethical issues, ask to record,
present the informed consent form.

• Warm-up – make first questions easy and non-threatening.

• Main body – present questions in a logical order

• A cool-off period – include a few easy questions to defuse

tension at the end

• Closure – thank interviewee, signal the end,

eg. switch recorder off.

www.id-book.com 7
 Enriching the interview process
• Props - devices for prompting interviewee, e.g. use a
prototype, scenario

www.id-book.com 8
 Questionnaires
• Questions can be closed or open

• Closed questions are easier to analyze, and

may be distributed and analyzed by computer

• Can be administered to large populations

• Disseminated by paper, email and the web

• Sampling can be a problem when the size of a

population is unknown as is common online
evaluation
www.id-book.com 9
 Questionnaire design
• The impact of a question can be influenced by question
order.
• You may need different versions of the questionnaire for
different populations.
• Provide clear instructions on how to complete the
questionnaire.
• Strike a balance between using white space and keeping
the questionnaire compact.
• Avoid very long questionnaires
• Decide on whether phrases will all be positive, all negative
or mixed.
www.id-book.com 10
 Question and response format
• ‘Yes’ and ‘No’ checkboxes
• Checkboxes that offer many options
• Rating scales
– Likert scales
– semantic scales
– 3, 5, 7 or more points

• Open-ended responses
11
www.id-book.com
 Encouraging a good response
• Make sure purpose of study is clear
• Promise anonymity
• Ensure questionnaire is well designed
• Offer a short version for those who do not have
time to complete a long questionnaire
• If mailed, include a stamped addressed envelope
• Follow-up with emails, phone calls, letters
• Provide an incentive
• 40% response rate is good, 20% is often
acceptable www.id-book.com 12
 Advantages of online
questionnaires
• Relatively easy and quick to distribute

• Responses are usually received quickly

• No copying and postage costs

• Data can be collected in database for analysis

• Time required for data analysis is reduced

• Errors can be corrected easily

13
www.id-book.com
Example of an online questionnaire

14
www.id-book.com
Problems with online questionnaires
• Sampling is problematic if population size
is unknown

• Preventing individuals from responding

more than once can be a problem

• Individuals have also been known to

change questions in email questionnaires

www.id-book.com 15
 Observation
• Direct observation in the field
– Structuring frameworks
– Degree of participation (insider or outsider)
– Ethnography

• Direct observation in controlled environments

• Indirect observation: tracking users’ activities

– Diaries
– Interaction logging
– Video and photographs collected
remotely by drones or other
equipment

16
www.id-book.com
Observation

17
www.id-book.com
Structuring frameworks to guide observation
• Three easy-to-remember parts:
– The person: Who?
– The place: Where?
– The thing: What?

• A more detailed framework (Robson, 2014):

– Space: What is the physical space like and how is it laid out?
– Actors: What are the names and relevant details of the
people involved?
– Activities: What are the actors doing and why?
– Objects: What physical objects are present, such as furniture
– Acts: What are specifi c individual actions?
– Events: Is what you observe part of a special event?
– Time: What is the sequence of events?
– Goals: What are the actors trying to accomplish?
– Feelings: What is the mood of the group and of individuals?
www.id-book.com 18
 Planning and conducting
observation in the field
• Decide on how involved you will be: passive
observer to active participant
• How to gain acceptance
• How to handle sensitive topics, eg. culture,
private spaces, etc.
• How to collect the data:
– What data to collect
– What equipment to use
– When to stop observing

www.id-book.com 19
 Ethnography (1)
• Ethnography is a philosophy with a set of techniques
that include participant observation and interviews
• Debate about differences between participant
observation and ethnography
• Ethnographers immerse themselves in the culture that
they study
• A researcher’s degree of participation can vary along a
scale from ‘outside’ to ‘inside’
• Analyzing video and data logs can be time-consuming
• Collections of comments, incidents, and artifacts are
made
www.id-book.com
 Ethnography (2)
• Co-operation of people being observed is required

• Informants are useful

• Data analysis is continuous

• Interpretivist technique

• Questions get refined as understanding grows

• Reports usually contain examples

21
www.id-book.com
Ethnography (2)

22
www.id-book.com
 Online Ethnography
• Virtual, Online, Netnography
• Online and offline activity
• Interaction online differs from face-to-face
• Virtual worlds have a persistence that
physical worlds do not have
• Ethical considerations and presentation of
results are different

23
www.id-book.com
 Observations and materials that
might be collected (Crabtree, 2007)
• Activity or job descriptions.
• Rules and procedures that govern particular activities.
• Descriptions of activities observed.
• Recordings of the talk taking place between parties.
• Informal interviews with participants explaining the detail of observed
activities.
• Diagrams of the physical layout, including the position of artifacts.
• Other information collected when observing activities:
– Photographs of artifacts (documents, diagrams, forms, computers, etc.)
– Videos of artifacts.
– Descriptions of artifacts.
– Workflow diagrams showing the sequential order of tasks.
– Process maps showing connections between activities.

24
www.id-book.com
Observation in a controlled environment
• Direct observation
– Think aloud techniques
• Indirect observation – tracking users’
activities
– Diaries
– Interaction logs
– Web analytics
• Video, audio, photos, notes are used
to capture data in both types of
observations

25
www.id-book.com
 Web analytics
• A system of tools and techniques for
optimizing web usage by:
– Measuring,
– Collecting,
– Analyzing, and
– Reporting web data

• Typically focus on the number of web

visitors and page views.

26
www.id-book.com
A section of Google analytics dashboard for
id-book.com

27
www.id-book.com
Choosing and combining techniques

• Depends on the:
– Focus of the study

– Participants involved

– Nature of the technique(s)

– Resources available

– Time available

28
www.id-book.com
 Summary
• Data gathering sessions should have clear goals.
• An informed consent may be needed.
• Five key issues of data gathering are: goals, choosing
participants, triangulation, participant relationship, pilot.
• Data may be recorded using handwritten notes, audio or
video recording, a camera, or any combination of these.
• Interviews may be structured, semi-structured or
unstructured
• Focus groups are group interviews
• Questionnaires may be on paper, online or telephone
• Observation may be direct or indirect, in the field or in
controlled settings.
• Techniques can be combined depending on the study
focus, participants, nature of technique, available
resources and time.
29
www.id-book.com
 Chapter 8
Data Analysis, Interpretation and Presentation
 Aims
• Discuss the difference between qualitative and
quantitative data and analysis.
• Enable you to analyze data gathered from:
– Questionnaires.
– Interviews.
– Observation studies.
• Make you aware of software packages that are
available to help your analysis.
• Identify common pitfalls in data analysis,
interpretation, and presentation.
• Enable you to interpret and present your findings in
appropriate ways.
www.id-book.com 2
 Quantitative and qualitative
• Quantitative data – expressed as numbers

• Qualitative data – difficult to measure sensibly as numbers, e.g.

count number of words to measure dissatisfaction

• Quantitative analysis – numerical methods to ascertain size,

magnitude, amount

• Qualitative analysis – expresses the nature of elements and is

represented as themes, patterns, stories

• Be careful how you manipulate data and numbers!

www.id-book.com 3
 Simple quantitative analysis
• Averages
– Mean: add up values and divide by number of data points
– Median: middle value of data when ranked
– Mode: figure that appears most often in the data
• Percentages
• Be careful not to mislead with numbers!
• Graphical representations give overview of data
Number of errors made Internet use Number of errors made

10
Number of errors made

4.5

Number of errors made

< once a day 4
8
3.5
6 once a day 3
2.5
4 2
once a week 1.5
2 1
0
2 or 3 times a week 0.5
0
0 5 10 15 20
once a month 1 3 5 7 9 11 13 15 17
User
User

www.id-book.com 4
 Visualizing log data
Interaction profiles of players in online game

www.id-book.com 5
Visualizing log data
Log of web page activity

www.id-book.com 6
Web analytics

www.id-book.com 7
 Simple qualitative analysis
• Recurring patterns or themes
– Emergent from data, dependent on observation framework if used
• Categorizing data
– Categorization scheme may be emergent or pre-specified
• Looking for critical incidents
– Helps to focus in on key events

www.id-book.com 8
 Tools to support data analysis
• Spreadsheet – simple to use, basic graphs

• Statistical packages, e.g. SPSS

• Qualitative data analysis tools

– Categorization and theme-based analysis
– Quantitative analysis of text-based data

• Nvivo and Atlas.ti support qualitative data analysis

• CAQDAS Networking Project, based at the University of
Surrey (http://caqdas.soc.surrey.ac.uk/)

www.id-book.com 9
 Theoretical frameworks for
qualitative analysis
• Basing data analysis around theoretical frameworks
provides further insight

• Three such frameworks are:

– Grounded Theory

– Distributed Cognition

– Activity Theory

www.id-book.com 10
 Grounded Theory
• Aims to derive theory from systematic analysis of data

• Based on categorization approach (called here ‘coding’)

• Three levels of ‘coding’

– Open: identify categories

– Axial: flesh out and link to subcategories

– Selective: form theoretical scheme

• Researchers are encouraged to draw on own theoretical

backgrounds to inform analysis

www.id-book.com 11
Code book used in grounded theory analysis

www.id-book.com 12
Excerpt showing axial coding

www.id-book.com 13
 Distributed Cognition
• The people, environment & artefacts
are regarded as one cognitive system
• Used for analyzing collaborative work
• Focuses on information propagation
& transformation

www.id-book.com 14
 Activity Theory
• Explains human behaviour in terms of our practical
activity in the world

• Provides a framework that focuses analysis on the

concept of an ‘activity’ and helps to identify tensions
between the different elements of the system

• Two key models: one outlines what constitutes an

‘activity’; one models the mediating role of artifacts

www.id-book.com 15
Individual model

www.id-book.com 16
Engeström’s (1999) activity
system model

17
www.id-book.com
 Presenting the findings
• Only make claims that your data can support
• The best way to present your findings depends on the
audience, the purpose, and the data gathering and
analysis undertaken
• Graphical representations (as discussed above) may
be appropriate for presentation
• Other techniques are:
– Rigorous notations, e.g. UML
– Using stories, e.g. to create scenarios
– Summarizing the findings

www.id-book.com 18
 Summary
• The data analysis that can be done depends on the
data gathering that was done
• Qualitative and quantitative data may be gathered from
any of the three main data gathering approaches
• Percentages and averages are commonly used in
Interaction Design
• Mean, median and mode are different kinds of
‘average’ and can have very different answers for the
same set of data
• Grounded Theory, Distributed Cognition and Activity
Theory are theoretical frameworks to support data
analysis
• Presentation of the findings should not overstate the
evidence
19
www.id-book.com
 Chapter 9
THE PROCESS OF INTERACTION DESIGN
 Overview
• What is involved in Interaction Design?
– Importance of involving users
– Degrees of user involvement
– What is a user-centered approach?
– Four basic activities

• Some practical issues

– Who are the users?
– What are ‘needs’?
– Where do alternatives come from?
– How to choose among alternatives?
– How to integrate interaction design activities in other lifecycle
models?

www.id-book.com 2
 What is involved in Interaction
Design?
• It is a process:
– a goal-directed problem solving activity informed by
intended use, target domain, materials, cost, and feasibility
– a creative activity
– a decision-making activity to balance trade-offs

• Generating alternatives and choosing between them

is key

• Four approaches: user-centered design, activity-

centered design, systems design, and genius design

www.id-book.com 3
 Importance of involving users
• Expectation management
– Realistic expectations
– No surprises, no disappointments
– Timely training
– Communication, but no hype
• Ownership
– Make the users active stakeholders
– More likely to forgive or accept problems
– Can make a big difference to acceptance and
success of product
www.id-book.com 4
 Degrees of user involvement
• Member of the design team
– Full time: constant input, but lose touch with users
– Part time: patchy input, and very stressful
– Short term: inconsistent across project life
– Long term: consistent, but lose touch with users

• Newsletters and other dissemination devices

– Reach wider selection of users
– Need communication both ways

• User involvement after product is released

• Combination of these approaches
www.id-book.com 5
What is a user-centered approach?
User-centered approach is based on:

– Early focus on users and tasks: directly studying cognitive,

behavioral, anthropomorphic & attitudinal characteristics

– Empirical measurement: users’ reactions and

performance to scenarios, manuals, simulations &
prototypes are observed, recorded and analysed

– Iterative design: when problems are found in user testing,

fix them and carry out more tests

www.id-book.com 6
 Four basic activities in
Interaction Design
1. Establishing requirements

2. Designing alternatives

3. Prototyping

4. Evaluating

www.id-book.com 7
A simple interaction design lifecycle
model
Exemplifies a user-centered design approach

www.id-book.com 8
 Some practical issues
• Who are the users?

• What do we mean by ‘needs’?

• How to generate alternatives

• How to choose among alternatives

• How to integrate interaction design

activities with other lifecycle models?
www.id-book.com 9
 Who are the users/stakeholders?
• Not as obvious as you think:
– those who interact directly with the product
– those who manage direct users
– those who receive output from the product
– those who make the purchasing decision
– those who use competitor’s products

• Three categories of user (Eason, 1987):

– primary: frequent hands-on
– secondary: occasional or via someone else
– tertiary: affected by its introduction, or will influence its purchase

www.id-book.com 10
 Who are the stakeholders?

Check-out operators

• Suppliers
• Local shop
owners

Customers
Managers and owners
www.id-book.com 11
 What do we mean by ‘needs’?
• Users rarely know what is possible
• Users can’t tell you what they ‘need’ to help them
achieve their goals
• Instead, look at existing tasks:
– their context
– what information do they require?
– who collaborates to achieve the task?
– why is the task achieved the way it is?

• Envisioned tasks:
– can be rooted in existing behaviour
– can be described as future scenarios
www.id-book.com 12
 How to generate alternatives
• Humans stick to what they know works

• But considering alternatives is important to ‘break out of

the box’

• Designers are trained to consider alternatives, software

people generally are not

• How do you generate alternatives?

— ‘Flair and creativity’: research and synthesis

— Seek inspiration: look at similar products or look at very
different products

www.id-book.com 13
 IDEO TechBox
• Library, database and website all-in-one

• Contains physical gizmos for inspiration

www.id-book.com 14
The TechBox

www.id-book.com 15
How to choose among alternatives
• Evaluation with users or with peers, e.g. prototypes
• Technical feasibility: some not possible
• Quality thresholds: Usability goals lead to usability
criteria set early on and check regularly
– safety: how safe?
– utility: which functions are superfluous?
– effectiveness: appropriate support? task coverage,
information available
– efficiency: performance measurements
– learnability: is the time taken to learn a function acceptable
to the users?
– memorability: can infrequent users remember how to
achieve their goal?
www.id-book.com 16
Testing prototypes to choose
among alternatives

www.id-book.com 17
How to integrate interaction design
in other models
• Integrating interaction design activities in lifecycle
models from other disciplines needs careful planning
• Several software engineering lifecycle models have
been considered
• Integrating with agile software development is
promising
– it stresses the importance of iteration
– it champions early and regular feedback
– it handles emergent requirements
– it aims to strike a balance between flexibility and structure

www.id-book.com 18
 Summary
Four basic activities in the design process
1. Establishing requirements
2. Designing alternatives
3. Prototyping
4. Evaluating

User-centered design rests on three principles

1. Early focus on users and tasks
2. Empirical measurement using quantifiable &
measurable usability criteria
3. Iterative design
www.id-book.com 19
 Chapter 10
ESTABLISHING REQUIREMENTS
 Overview
• The importance of requirements
• Different types of requirements
• Data gathering for requirements
• Data analysis and presentation
• Task description: Scenarios
Use Cases
Essential use cases
• Task analysis: HTA

www.id-book.com 2
 What, how and why?
What needs to be achieved?
1. Understand as much as possible about users, task, context

2. Produce a stable set of requirements

How can this be done?

• Data gathering activities

• Data analysis activities

• Expression as ‘requirements’

• All of this is iterative

www.id-book.com 3
 What, how and why?
• Why bother?
Requirements
definition is the
stage where
failure occurs
most
commonly

Getting requirements right is crucial

www.id-book.com 4
 Establishing requirements
• What do users want? What do users ‘need’?

Requirements need clarification, refinement, completion,

re-scoping

Input: Requirements document (maybe)

Output: stable requirements

• Why ‘establish’?

Requirements arise from understanding users’ needs

Requirements can be justified & related to data

www.id-book.com 5
Volere shell

www.id-book.com 6
Volere requirements template

www.id-book.com 7
 Different kinds of requirements
• Functional:
—What the system should do

• (Non-functional: security, response time...)

• Data:
—What kinds of data need to be stored?

—How will they be stored (e.g. database)?

www.id-book.com 8
 Different kinds of requirements
Environment or context of use:
• physical: dusty? noisy? vibration? light? heat?
humidity? …. (e.g. ATM)

• social: sharing of files, of displays, in paper, across

great distances, synchronous, privacy for clients

• organisational: hierarchy, IT department’s attitude

and remit, user support, communications structure
and infrastructure, availability of training

www.id-book.com 9
Underwater computing

www.id-book.com 10
Underwater computing

www.id-book.com 11
Different kinds of requirements
Users: Who are they?
— Characteristics: nationality, educational background,
attitude to computers

— System use: novice, expert, casual, frequent

— Novice: prompted, constrained, clear

— Expert: flexibility, access/power

— Frequent: short cuts

— Casual/infrequent: clear menu paths

www.id-book.com 12
 What are the users’ capabilities?
Humans vary in many dimensions:
— size of hands may affect the size and positioning of input buttons

— motor abilities may affect the suitability of certain input and output
devices

— height if designing a physical kiosk

— strength - a child’s toy requires little strength to operate, but

greater strength to change batteries

— disabilities (e.g. sight, hearing, dexterity)

www.id-book.com 13
 Personas
• Capture a set of user characteristics (user
profile)

• Not real people, but synthesised from real users

• Should not be idealised

• Bring them to life with a name, characteristics,

goals, personal background

• Develop a small set of personas with one primary

www.id-book.com 14
Example Persona

www.id-book.com 15
Data gathering for requirements
• Interviews:
— Props, e.g. sample scenarios of use,
prototypes, can be used in interviews
— Good for exploring issues
— Development team members can connect with stakeholders

• Focus groups:
— Group interviews

— Good at gaining a consensus view and/or highlighting areas of

conflict

— But can be dominated by individuals

www.id-book.com 16
Data gathering for requirements
• Questionnaires:
— Often used in conjunction with other
techniques
— Can give quantitative or qualitative data
— Good for answering specific questions from
a large, dispersed group of people
• Researching similar products:
— Good for prompting requirements

www.id-book.com 17
Data gathering for requirements
• Direct observation:
— Gain insights into stakeholders’ tasks
— Good for understanding the nature and
context of the tasks
— But, it requires time and commitment
from a member of the design team, and
it can result in a huge amount of data
• Indirect observation:
— Not often used in requirements activity
— Good for logging current tasks

www.id-book.com 18
Data gathering for requirements
Studying documentation:
— Procedures and rules are often written
down in manuals

— Good source of data about the steps

involved in an activity, and any
regulations governing a task

— Not to be used in isolation

— Good for understanding legislation, and

getting background information

— No stakeholder time, which is a limiting

factor on the other techniques
www.id-book.com 19
Some examples

Cultural probes

www.id-book.com 20
 Some examples
Ethnographic study, interviews, usability tests, and user
participation

www.id-book.com 21
 Contextual Inquiry
• An approach to ethnographic study where user is expert,
designer is apprentice
• A form of interview, but
— at users’ workplace (workstation)
— 2 to 3 hours long

• Four main principles:

— Context: see workplace & what happens
— Partnership: user and developer collaborate
— Interpretation: observations interpreted by user and developer
together
— Focus: project focus to understand what to look for
www.id-book.com 22
 Considerations for data gathering (1)
• Identifying and involving stakeholders:
users, managers, developers, customer reps?, union reps?,
shareholders?

• Involving stakeholders: workshops, interviews, workplace

studies, co-opt stakeholders onto the development team

• ‘Real’ users, not managers

• Political problems within the organisation

• Dominance of certain stakeholders

• Economic and business environment changes

• Balancing functional and usability demands

www.id-book.com 23
 Considerations for data gathering (2)
• Requirements management: version control, ownership

• Communication between parties:

—within development team
—with customer/user
—between users… different parts of an
organisation use different terminology
• Domain knowledge distributed and implicit:
—difficult to dig up and understand
—knowledge articulation: how do you walk?
• Availability of key people
www.id-book.com 24
 Data gathering guidelines
• Focus on identifying the stakeholders’ needs
• Involve all the stakeholder groups
• Involve more than one representative from
each stakeholder group
• Use a combination of data gathering
techniques
• Support the process with props such as
prototypes and task descriptions
www.id-book.com 25
Data interpretation and analysis
• Start soon after data gathering session

• Initial interpretation before deeper analysis

• Different approaches emphasize different

elements e.g. class diagrams for object-
oriented systems, entity-relationship
diagrams for data intensive systems

www.id-book.com 26
 Task descriptions
• Scenarios
― an informal narrative story, simple, ‘natural’,
personal, not generalisable
• Use cases
— assume interaction with a system
— assume detailed understanding of the interaction

• Essential use cases

— abstract away from the details
— does not have the same assumptions as use cases

www.id-book.com 27
 Scenario for travel organizer
“The Thomson family enjoy outdoor activities and want to try their hand at
sailing this year. There are four family members: Sky (10 years old),
Eamonn (15 years old), Claire (35), and Will (40). One evening after dinner
they decide to start exploring the possibilities. They all gather around the
travel organizer and enter their initial set of requirements – a sailing trip for
four novices in the Mediterranean. The console is designed so that all
members of the family can interact easily and comfortably with it. The
system’s initial suggestion is a flotilla, where several crews (with various
levels of experience) sail together on separate boats. Sky and Eamonn
aren’t very happy at the idea of going on vacation with a group of other
people, even though the Thomsons would have their own boat. The travel
organizer shows them descriptions of flotillas from other children their ages
and they are all very positive, so eventually, everyone agrees to explore
flotilla opportunities. Will confirms this recommendation and asks for detailed
options. As it’s getting late, he asks for the details to be saved so everyone
can consider them tomorrow. The travel organizer emails them a summary
of the different options available.”
www.id-book.com 28
Scenarios and Personas

www.id-book.com 29
 Use case for travel organizer
1. The system displays options for investigating visa and vaccination
requirements.
2. The user chooses the option to find out about visa requirements.
3. The system prompts user for the name of the destination country.
4. The user enters the country’s name.
5. The system checks that the country is valid.
6. The system prompts the user for her nationality.
7. The user enters her nationality.
8. The system checks the visa requirements of the entered country for a
passport holder of her nationality.
9. The system displays the visa requirements.
10. The system displays the option to print out the visa requirements.
11. The user chooses to print the requirements.
www.id-book.com 30
Alternative courses for travel organizer
Some alternative courses:

6. If the country name is invalid:

6.1 The system displays an error message.
6.2 The system returns to step 3.
8. If the nationality is invalid:
8.1 The system displays an error message.
8.2 The system returns to step 6.
9. If no information about visa requirements is found:
9.1 The system displays a suitable message.
9.2 The system returns to step 1.
www.id-book.com 31
Example use case diagram for travel organizer

www.id-book.com 32
 Example essential use case for travel organizer

retrieve Visa

USER INTENTION SYSTEM RESPONSIBILITY

find visa requirements request destination and

nationality

supply required information

obtain appropriate visa info
obtain copy of visa info
offer info in different formats
choose suitable format
provide info in chosen format

www.id-book.com 33
 Task analysis
• Task descriptions are often used to envision new systems or
devices

• Task analysis is used mainly to investigate an existing

situation

• It is important not to focus on superficial activities

– What are people trying to achieve?

– Why are they trying to achieve it?

– How are they going about it?

• Many techniques, the most popular is Hierarchical Task

Analysis (HTA)

www.id-book.com 34
 Hierarchical Task Analysis
• Involves breaking a task down into subtasks, then sub-
sub-tasks and so on. These are grouped as plans which
specify how the tasks might be performed in practice

• HTA focuses on physical and observable actions, and

includes looking at actions not related to software or an
interaction device

• Start with a user goal which is examined and the main

tasks for achieving it are identified

• Tasks are sub-divided into sub-tasks

www.id-book.com 35
 Example Hierarchical Task Analysis

0. In order to buy a DVD

1. locate DVD
2. add DVD to shopping basket
3. enter payment details
4. complete address
5. confirm order

plan 0: If regular user do 1-2-5.

If new user do 1-2-3-4-5.
www.id-book.com 36
Example Hierarchical Task Analysis
(graphical)

www.id-book.com 37
 Summary
• Getting requirements right is crucial

• There are different kinds of requirement, each is

significant for interaction design

• The most commonly-used techniques for data

gathering are: questionnaires, interviews, focus
groups, direct observation, studying documentation
and researching similar products

• Scenarios, use cases and essential use cases can be

used to articulate existing and envisioned work
practices.

• Task analysis techniques such as HTA help to

investigate existing systems and practices
www.id-book.com 38