The Human

Human Computer Interaction

-Dix, Finlay, Abowd, Beatle-

(Lecture Notes)

Ngo Thi Duyen

(duyennt@vnu.edu.vn)
Faculty of Information Technology
Vietnam National University, Hanoi
 Contents
1. Introduction
2. Input-output channels
3. Human Memory
4. Thinking: reasoning & problem solving
5. Emotion
6. Individual difference
7. Psychology and the design of interactive systems

03/22/23 The Human Page 2

 Introduction

 Information i/o …
 visual, auditory, haptic, movement

 Information stored in memory

 sensory, short-term, long-term

 Information processed and applied

 reasoning, problem solving, skill, error

 Emotion influences human capabilities

 Each person is different

03/22/23 The Human Page 3

 Input-output channels
 In interaction with a computer
 The human input is the data output by the computer and vice verse
 In humans, input mainly occurs through the senses and output through the
motor controls of the effectors

 The most important senses in HCI

 Vision, hearing, and touch

 The primary effectors

 Eyes, ears, fingers, head and body position
 Visual perception can be divided into 2 stages
 Physical reception of the stimulus from the outside world
 The processing and interpretation of that stimulus

03/22/23 The Human Page 4

Input-output channels

Vision
 Eye: is a mechanism for receiving light and transforming it into electrical energy
 Light is reflected from objects in the visual field and their image is focused on the
back of the eye, where it is transformed into an electrical signal and passed to the
brain
 The most important components are the cornea (giác mạc) and lens (thủy tinh
thể/đồn tử) and the retina (võng mạc) with the blind spot and photoreceptors(các tế
bào nhận kích thích ánh sáng): rods, cones located on the fovea (hố võng mạc)

 Rods are highly sensitive to light

and are usable under low
illumination
 Cones are less sensitive to light
and can distinguish color
03/22/23 The Human Page 5
Input-output channels

Vision: The signal interpretation

 Size and depth

 Visual angle indicates how much of view an object occupies
(relates to size and distance from eye)
 Visual acuity (thị lực) is ability to perceive detail
 Familiar objects perceived as constant size
(in spite of changes in visual angle when far away)

03/22/23 The Human Page 6

Input-output channels

Vision: The signal interpretation (cont’d)

 Perception of brightness
 is a subjective reaction to levels of light emitted by an object
 affected by luminance of object

 Colour
 The eye perceive color because the cones are sensitive to light of
different wavelengths
 3-4% of the fovea is sensitive to blue, making blue acuity lower
 In reality, about 8% males and 1% females are colour blind

03/22/23 The Human Page 7

Input-output channels

Vision: The signal interpretation (cont’d)

 The visual system compensates for

the human mind
 Movements
judges an object's
 Changes in luminance
size based on its
 The context in which an object appears background
the Ponzo
allows us to clearly disambiguate the illusion
interpretation of the object

 Optical illusions sometimes occur due

to over compensation
the Muller
Lyer illusion

03/22/23 The Human Page 8

Input-output channels

Vision: Reading

 Consists of several stages

 The visual patterns of the words are perceived
 Decoded with reference to an internal representation of
language
 The word is processed as part of the sentence or phrase using
syntactic and semantic analysis

 Reading involves saccades(cử động giật giật của mắt)

and fixations
 Perception occurs during fixation

03/22/23 The Human Page 9

Input-output channels

Hearing
 Provide information about environment: distances, directions, objects, etc.

 The ear receives vibrations in the air and transmits them through various
stages to the auditory nerves

 Physical sections
 Outer ear: protects inner ear and amplifies sound
 Middle ear: transmit sound waves like vibrations to inner ear
 Inner ear: chemical transmitters are released and cause impulses in the auditory

nerves (thần kinh thính giác )

Amplitude
 Features of sound
 Pitch(độ cao thấp): frequency of the sound
 Loudness(cường độ): amplitude of the sound
 Timbre(âm sắc): type/quality of the sound Frequency

03/22/23 The Human Page 10

Input-output channels

Hearing (cont.)
 Humans can hear sound at frequencies from 20Hz to 15kHz

 Can perceive changes at low frequency about 1.5Hz

 The higher the frequency, the more difficult to distinguish

 Human auditory system has a filtering system which filter out the
distracting noise to concentrate on important sounds
 “Cocktail party effect” – we still can concentrate on the conversation

03/22/23 The Human Page 11

Input-output channels

Hearing withHCI
 Currently sounds are still mainly used to inform some thing, e.g.,
 When pressing a wrong button
 Welcoming to Windows when booting
 Low battery status

 Sound research has been carried out

 Synthesize speech: Listening to an audio material instead of reading, which
is especially beneficial to blind people & those who have weak visual acuity
 Use sounds to create effect in content displaying

03/22/23 The Human Page 12

Input-output channels

Touch
 Provides important feedback about environment
 Stimuli are received via the receptors (cơ quan nhận cảm) in the
skin
 Thermoreceptors: hot and cold
 Nociceptors: pain
 Mechanoreceptors (cơ quan cảm thụ cơ học): pressure

 Kinesthesis (cảm giác về sự vận động của tứ chi & cơ thể)

 Second aspect of haptic perception
 Awareness of the position of the body and limbs due to receptors in the
joints

03/22/23 The Human Page 13

Input-output channels

Touch with HCI

 E.g., we can perceive when a button is being pressed

 New devices: haptic

03/22/23 The Human Page 14

Input-output channels

Movement
 When making movements, a stimulus is received through the receptors and
transmitted to the brain
 After processing, the brain tells the appropriate muscle to respond
 The movement time is dependent on the physical characteristics of the subjects
(age, fitness )
 The reaction time dependent on the stimulus type
 visual ~ 200ms
 auditory ~ 150 ms
 pain ~ 700ms

 A fast response doesn’t always mean a less accurate response

 Fitts' Law describes the time taken to hit a target
Movement time = a + b log2(Distance/Size + 1)
a and b are empirically determined constants

03/22/23 The Human Page 15

 Memory
There are three types of memory function:

Sensory memories

Short-term memory or working memory

Long-term memory

Selection of stimuli governed by level of arousal

03/22/23 The Human Page 16

Memory

Sensory memory

 Act as buffers for stimuli received via each of senses

 iconic memory: visual stimuli
 echoic memory: aural stimuli
 haptic memory: tactile stimuli

 These memories are constantly overwritten by new

information coming in on these channels

 Information is passed to sensory memory into short-term

memory by attention

03/22/23 The Human Page 17

Memory

Short-term memory
 Is used to store information which is only required
fleetingly
 Can be accessed rapidly: ~ 70ms
 Also decay rapidly: ~ 200ms
 Has a limited capacity
 Humans can store 7± 2 chunks of information
 E.g.,
212348278493202
0121 414 2626
HEC ATR ANU PTH ETR EET

03/22/23 The Human Page 18

Memory

Long-term memory
 Has an unlimited capacity, a slow access time and forgetting occurs
more slowly or not at all
 Information is stored here from the STM through rehearsal

 2 types of LTM
 Episodic (tình tiết) memory represents our memory of event & experiences
in a serial form
 Semantic memory is a structured record of facts, concepts, skills that we
have acquired, derived from the episodic memory

 Semantic memory is structured as a network

 The more general the information is, the higher is the level on which it is
stored
  allows us to generalize about specific cases

03/22/23 The Human Page 19

Memory

LTM: example of semantic network model

03/22/23 The Human Page 20

Memory

LTM: Frames

Other models about the organization of LTM…

 Frames
 Information is organized in data structures
 Have slots to add attribute values

DOG COLLIE

Fixed Fixed
legs: 4 breed of: DOG
type: sheepdog
Default
diet: carniverous Default
sound: bark size: 65 cm
Variable Variable
size: colour
colour

03/22/23 The Human Page 21

Memory

LTM: Scripts
 Comprise a number of elements (like slots) which can be filled with
appropriate information
Script for a visit to the vet

Entry conditions: dog ill Roles: vet examines

vet open diagnoses
owner has money treats
owner brings dog in
Result: dog better
pays
owner poorer
takes dog out
vet richer
Scenes: arriving at reception
Props: examination table
waiting in room
medicine
examination
instruments
paying
Tracks: dog needs medicine
dog needs operation

03/22/23 The Human Page 22

Memory

LTM: Production rules

 IF-THEN rules

 If information coming into the STM matches one of the

condition in the LTM, the appropriate action is executed

IF a dog is wagging tail

THEN pat the dog

IF a dog is growling
THEN run away

03/22/23 The Human Page 23

Memory

LTM: 3 main activities

 Storage of information
 Rehearsal of a piece of information from the STM (moved to LTM)

 Total time hypothesis (Ebbinghaus 1885)

 If the total learning time is increased, information is remembered better
(amount retained is proportional to rehearsal time)

 Distribution of practice effect (Baddeley & Longman 1978)

 Optimization: the learning time should be well spread over time

 Repetition alone is enough?

 To help learn efficiently, information should be meaningful & familiar
 So that, it can be related to existing structures
  more easily incorporated into memory

03/22/23 The Human Page 24

Memory

LTM: 3 main activities

 Forgetting: 2 main theories of forgetting

 Decay
 information held in the LTM is gradually lost but slowly

 Inference
 New information replaces old one (retroactive interference)
 The older information interferes with the newly acquired information
(proactive inhibition)

 Forgetting is affected by emotional factors too

03/22/23 The Human Page 25

Memory

LTM: 3 main activities

 Retrieval: 2 types of retrieval

 Recall
 information reproduced from memory can be assisted by cues (e.g.,
the category in which information can be placed)
 Recognition
 information gives knowledge that it has been seen before
 Less complex than Recall

03/22/23 The Human Page 26

 Thinking

 Require different amount of knowledge

 Some thinking are very directed and the knowledge required is
constrained
 Others require vast amounts of knowledge from different
domains

 Thinking can be divided in

 Reasoning : deductive(diễn dịch), inductive(quy nạp),
aductive(Truy kích)
 Problem solving

03/22/23 The Human Page 27

Thinking

Reasoning
 Is the process in which we use the knowledge to draw conclusion or infer
something new about the domain of interest
 Deductive reasoning
 Derive the logically necessary conclusion from the given premises
 E.g. If it is Friday then she will go to work

It is Friday
Therefore she will go to work
 Logical conclusion not necessarily true
 E.g. If it is raining then the ground is dry

It is raining
Therefore the ground is dry
 When the truth and logical validity clash
 E.g. Some people are babies
Some babies cry
Inference - Some people cry

03/22/23 The Human Page 28

Thinking

Reasoning

 Inductive reasoning

 generalizing from cases we have seen to infer information about

cases we have not seen
 E.g., all elephants we’ve been have trunks, therefore all elephants
have trunks

 Unbelievable
 Can only prove False not True

03/22/23 The Human Page 29

Thinking

Reasoning

 Adduction
 Reasons from event to cause

 E.g., Sam drives fast when drunk

 If I see him driving fast, assume drunk

 Unbelievable

 Can lead to false explanation

 But still useful

03/22/23 The Human Page 30

Thinking

Problem solving
 Is the process of finding a solution to unfamiliar task, using the
knowledge we have

 There are different views on problem solving

 Gestalt theory
 problem solving is both productive and reproductive
 productive draws on insight and restructuring of problem
 attractive but not enough evidence to explain `insight' etc.
 move away from behaviourism and lead towards information processing
theories

03/22/23 The Human Page 31

Thinking

Problem solving (cont.)

 Problem space theory
 problem space comprises problem states
 problem solving involves generating states using legal operators
 People use these operators to move from the initial state to the goal state
 heuristics may be employed to select operators

 Use of analogy
 Problems solved by mapping knowledge relating to a similar known
problem domain to the new problem
 Analogical mapping

03/22/23 The Human Page 32

Thinking

Skill acquisition

 Information structure is fine tuned at a deep level to

enable efficient and accurate retrieval

 Skills acquired via 3 levels

 The learners uses general-purpose rules to interpret facts about
a problem
 Develop rules specific to the task using proceduralization
 The rules are tuned to speed up performance, using
generalization

03/22/23 The Human Page 33

Thinking

Errors & mental models

Types of errors
 slips
 Right intention, but failed to do it right
 Causes: poor physical skill, inattention etc.
 Changes in context of skilled behaviour can cause error
 Mistakes
 wrong intention
 An incorrect understanding of a situation can cause errors because
humans tend to create mental models , based on experience, which
may differ from the actual situation

03/22/23 The Human Page 34

 Emotion

 Various theories of how emotion works

 James-Lange: emotion is our interpretation of a physiological response

to a stimuli

 Cannon: emotion is a psychological response to a stimuli

 Schacter-Singer: emotion is the result of our evaluation of our

physiological responses, in the light of the whole situation we are in

 Emotion clearly involves both cognitive and physical responses to

stimuli

03/22/23 The Human Page 35

 Emotion (cont.)

 The biological response to physical stimuli is called affect

 Affect influences how we respond to situations

 positive  creative problem solving
 negative  narrow thinking

“Negative affect can make it harder to do even easy tasks; positive

affect can make it easier to do difficult tasks”
(Donald Norman)

03/22/23 The Human Page 36

 Emotion (cont.)

 Implications for interface design

 stress will increase the difficulty of problem solving

 relaxed users will be more forgiving of shortcomings

in design

 aesthetically pleasing and rewarding interfaces will

increase positive affect

03/22/23 The Human Page 37

 Individual differences

 The principles and properties discussed apply to the

majority of people

 But humans are NOT the same at all

 Differences should be taken into account in the design

 E.g., Divide the users into the target groups

03/22/23 The Human Page 38

 Psychology & the design of interactive systems

 Some direct applications

 e.g. blue acuity is poor
 blue should not be used for important detail
 However, correct application generally requires understanding of
context in psychology, and an understanding of particular
experimental conditions

 A lot of knowledge has been distilled in

 guidelines (chap 7)
 cognitive models (chap 12)
 experimental and analytic evaluation techniques (chap 9)

03/22/23 The Human Page 39

 Summary
 Input-output channels
 Human Memory
 Thinking
 Emotion
 Individual difference
 Psychology and the design of interactive systems

03/22/23 The Human Page 40