Professional Documents
Culture Documents
Slide 2
Speaking and understanding spoken language is a natural human ability, but reading is not.
Over hundreds of thousands—perhaps millions—of years, the human brain evolved the neural
structures necessary to support the acquisition and the execution of spoken languages. As a
result, normal humans are born with an innate ability to learn as toddlers, with no systematic
training, whatever language they are exposed to.
Conversely, When it comes to reading we need to receive a systematic training
Slide 3
Speaking and understanding a language
To acquire the necessary linguistic competences to understand and speak a language there is a
critical period: which is usually up to 8 years old (to achieve full fluency). In this period we
need to be exposed and have the opportunity to practice regularly a certain language.
Up to this period of our development, we may acquire very easily new languages. After this
period our ability to learn a new language decreases constantly.
Slide 4
To learn how to write and read require specific training in terms of competences.
At no time during childhood, our brain shows any special innate ability to learn to read or
write. We learn these competencies by instruction and practice, like playing a violin or
reading music.
Slide 5
Reading and writing imply learning how to interact and manipulate with an
artefact: The alphabet.
If we do not know a script and we are not able to interpret the code (alphabet)
we talk about ILLITERACY. This means that we cannot understand (interact
with) the content of the message, like in the examples I am reporting here from
your handbook.
Slide 6
Recognize patterns and their combination
Learning to read involves training our visual system to recognize patterns—the
patterns of text (characters)
• We do not need to learn the basic visual features like lines, shapes etc.
• We need to learn the combinations of the features (rules) to create
characters — e.g., letters, numeric digits, and other standard symbols.
This means that we need to be able to recognize the patterns created by
the standard arrangement of certain features.
• Then we need to learn how these symbols can be combined to generate
words, sentences, periods etc.
Slide 7
To read we need to perform the following processes:
• To be able to recognize
• Phonemes (the smallest unit of a spoken language)
• Graphemes (the smallest unit of a written language)
• Morphemes (the smallest unit of meaning that cannot be further
divided)
• Acquire the syntax to use words and phrases (features of the
language system).
• to acquire experience about the logic context in which words are used and
their sense (context of words, phrases and sentences)
Slide 8
To handle all these processes concurrently we should learn how to read by
performing in parallel bottom-up and top-down activities:
• Bottom-up (FEATURE DRIVEN)
Once we learn how to recognize the feature (forms of letters) and their
combinations we can read the text
• Top-down (CONTEXT DRIVEN)
We use the context and the meaning of phrases to speed up the recognition
of the feature and the meaning, and to speed up the reading process
Slide 9
Here you have an example from your book of a broken linguistic code.
You are all good readers, therefore with a minimal effort you could make sense
of this text, inferring by the context of the text (the most probable association
between the words):
<<Mary had a little lamb. Its fleece was white as snow. And everywhere that
Mary went. Mary went, Mary went. Everywhere that Mary went. The lamb was
sure to go>>
The context (Top-Down) in reading is important especially for the poor readers
who have little skills / expertise in recognizing and combining the symbols,
therefore they strongly rely on context to read and understand the text. Because
of that reading for poor readers is complex, time consuming and demanding
from the cognitive point of view and their comprehension of the text is minimal.
Also experts readers use the context to extrapolate the overall meaning of the
contents of a text, but they can read and understand better because for them it is
easy to recognize the patterns (minimal workload) therefore they can better
reflect on the words and their meaning.
Slide 10
Design and Reading
Don’t disrupt reading; support it! This means:
1. Ensure that the text of your product enable feature-based processes
(visibility of text): avoid difficult or tiny fonts, patterned backgrounds,
centering, etc.
2. Use restricted, highly consistent vocabularies. Help poor readers with
plain language or simplified language.
3. Format text to create a visual hierarchy: facilitate easy scanning of the
text e.g. use headings, bulleted lists, tables, and visually emphasized
words. (Support information seeking)
Slide 11
Here I am reporting one of the many examples regarding the importance of
facilitating reading with products. In this specific case for medical products 1
The tips about reading are relevant to guide the design of interfaces, but also of
elements such as: product guidelines, packaging, instructions.
Remember that: The text you are using on an interface or associated with a
product is part of your system. Usability analysis should also include the usage,
for instance, of instructions and guidelines to check if issues due to
inappropriate words (or instructions, or label) affect the safety of the interaction
with a system.
This is true for high-risk products (e.g., medical products) but in general for any
type of interactives artefact (digital or physical)
Slide 12
As we already discussed we are information seekers
Slide 13
If you also recall, we discussed about the fact that we can follow the scent of
information
Slide 14
Moreover, we discussed that Information can be designed to grab our
attention and that when we are designing a product we want to know which
information is going to attract people attention (attentional shift).
Slide 15
The best way to answer the question regarding what people are looking at when
interacting with a product is to use an ET system.
1
https://www.ema.europa.eu/en/human-regulatory/marketing-
authorisation/product-information/product-information-reference-documents-
guidelines
Slide 16
Just because I want you to be aware of that. Usually, we train groups of 20
students with 4 eye trackers at the master level on the usage of ET. As you can
easily understand it is not possible to train all of you on the usage of the fully
functional ET.
An ET system costs between 20 to 35 thousand Euros and therefore there is
limited availability of such tools.
Nevertheless, this year for the first time I decided to introduce the topic of ET at
the bachelor level.
Nevertheless, I believe that we found a way to help you in learning the basic
principles behind the use of eye-tracking tools so that (one day) you could apply
this know-how in a controlled setting by using fully functional eye-tracking
(ET) systems.
Slides 17
I want to provide you with initial knowledge and insights about:
• How and when you can use an ET
• Which data you can obtain from this equipment
• You can also have some practical insights about the fully functional ET by
following the optional tutorial online (see CANVAS) regarding how to
assemble an ET and analyse data of such a tool. This is optional but if you are
interested in ET this is a good starting point to learn how to handle some
practical aspects.
At the end of today, you will have the basic knowledge about ET and the metrics
that you can obtain from such a system. This will help you in future courses or
for your bachelor project. In general, the contents you are receiving today could
be considered a good starting point to build upon, and are equivalent to the
initial knowledge that you could receive on the first day of a master-level course
on ET.
Slide 18
To give you some insights regarding ET we created an online tool together with
the BMS LAB called BMSwebET.
The current version of the BMSwebET is the initial prototype. This was not
tested for usability (you are the first end-users). It is not as accurate as a fully
functional ET.
In general, web-based ET systems cannot be as accurate as fully functional
equipment that use specific sensors to collect eye movements, but we can
certainly improve the accuracy of the current prototype.
https://apps.bmslab.utwente.nl/
In this example, I set up the test so that each participant will see the two
INTERFACES (prototype and original interface), ONE AFTER ANOTHER,
FOR ABOUT 10 SECONDS.
FOR EACH INTERFACE, I ask participants to LOCATE WITH their EYES
THE MENU.
e.g. <<In this test, you will see the two interfaces, one after another, for about 10
seconds. For each interface, please try to locate with your eyes the location of
the menu.>>
Slide 19
If you are interested in this topic and you want to know how to use a fully
functional ET you can access the online tutorial and resources on Canvas.
This is just for your knowledge and expertise.
https://canvas.utwente.nl/courses/5914
Slide 20
Vision and Eye-tracking
Eye-tracking and reading are strongly connected
Slide 21
Even though our vision appears extremely stable, the eye is constantly moving
around to construct a picture of what we are looking at.
Our vision can be divided into:
• Fixation. This is the pause of the eye movement on a specific area of the
visual field.
• Saccades. These are rapid movements of the eye from one fixation to
another
Slide 22
ET and vision
An ET registers the eye fixations.
• Fixations take place in our foveal area (F). This accounts for nearly half of
the visual information sent to our brain, including colours
• The Foveal area is highly detailed and provides complete clarity about what
we are looking at.
• only objects in the very centre of your vision are in clear focus. This is
called the Foveal Spotlight.
• However, the foveal area is only 8% of our total vision.
• The rest of our vision is composed of parafoveal (Pf) and peripheral
vision(P).
• These regions surround our foveal vision and help us to gain a sense of
what is happening around us.
Slide 23
THE EYE IN READING
Our eyes fix the fovea on a word, pause there for a fraction of a second to allow
basic patterns to be captured and transmitted to the brain for further analysis,
then jump to the next important word.
Eye fixations while reading always land on words! usually near the center, never
on word boundaries.
Very common small connector and function words like “and,” “or,” “is,” “but”
are usually skipped over, their presence either detected in perifoveal vision or
simply assumed.
Slide 24
According to reading researcher Kevin Larson (2004) 2, the reading area in and
around the fovea consists of three distinct zones (for European-language
scripts):
1. Closest to the fixation point is where word recognition takes place. This zone
is usually large enough to capture the word being fixated and often includes
smaller function words directly to the right of the fixated word.
2. The next zone extends a few letters past the word recognition zone. Here
readers gather preliminary information about the next letters in this zone.
2
Larson, K., 2004. The Science of word recognition, Microsoft.com. July 2004, http://www.mic
rosoft.com/typography/ctfonts/WordRecognition.aspx.
3. The final zone extends out to 15 letters past the fixation point. Information
gathered out this far is used to identify the length of upcoming words and to
identify the best location for the next fixation point.
From a historical point of view, both the knowledge about vision and research
about reading are strongly connected with the development of eye-tracking
systems.
Slide 25
Around 1879 Louis Émile Javal noticed for the first time that readers’ eyes do
not skim fluently through the text while reading, but make quick movements
(saccades) mixed with short pauses (fixations).
In 1908, Edmund Huey built a device that could track eye movement during the
reading process and published The Psychology and Pedagogy of Reading.
Slide 26
Classic experiment of Alfred Yarbus
Yarbus in 1967 asked participants to look at this painting seven times, each time
with a different instruction before starting to look at the image…
Slide 27
Motivation/Goals and fixations
The seven instructions produced very different behaviour of the participant in
terms of eye movement:
Slide 28
Break
Slide 29
Eye-tracking (How it works)
Eye-tracking is the process of recording:
• the point of gaze where one person is looking at
• the motion of eyes towards a specific point
Slide 30
Eye-tracking general considerations
In general, with eye tracking, you can understand
• in real-time, what the user is looking at
• PAY ATTENTION THIS IS NOT a measure of the focus of attention!
• It could be considered a measure of attentional shift.
Slide 31
Pupil Centre Corneal Reflection
The most commonly used technique to track the eyes is the pupil centre corneal
reflection (PCCR).
The basic concept is to use a light source to illuminate the eye causing highly
visible reflections. The camera may capture an image of the eye showing these
reflections.
A filtered Infra-Red camera is embedded with the LED (light-emitting diode)
when the infrared rays hit the pupil in the cornea region the reflection produces a
glint on the cornea and in the pupil.
The equipment can then calculate a vector formed by the angle between the
cornea and pupil reflections—the direction of this vector, combined with other
geometrical features of the reflections, is then used to calculate the gaze
direction.
Once the image processing software has identified the centre of the pupil and the
location of the corneal reflection, the vector between them is measured.
Slide 32
Calibration and differences
Two general types of eye-tracking techniques are used:
bright-pupil and dark-pupil (depends on the type of eye-trackers).
Slide 33
Bright pupil
The light of the tracker is direct (coaxial with the optical path) and the eye acts
as a retroreflector creating a bright reflection
(this is similar to the red eyes effect in pictures)
Advantages
Good contrast and robust eye tracking
Low interference
disadvantages
Not effective for tracking outdoors (interference of sunlight)
Slide 34
Dark pupil
The light of the tracker is not direct and the reflection of the eye is minimal i.e.,
the pupil appears dark
Advantages
Better for outdoor
Disadvantages
Less accurate than the bright pupil detection
Slide 35
Screen ET - Combination of Bright and Dark pupil tracking
Slide 36
Wearable ET – Dark pupil tracking
Slide 37
Pros and Cons of ET
Advantages:
Unobtrusiveness doesnt attract attention/distract
It provides objective measures and understandable metrics
We can track eyes movement which means that we can track the shift or
the diversion of attention of users during the interaction with an
interface
Disadvantages
Perceiving an object is not equal to fixate an object with the eyes
We can look at something but our focus of attention can be somewhere
else. Therefore, to track the Eyes Movements is not equal to track
attention
Slide 38
4 points on ET and eye movements from Andrew Duchowski 3
1. When we look at something: <<… we move our eyes to bring a particular
portion of the visible field of view into high resolution so that we may see in
fine detail whatever is at the central direction of gaze.>>
Slide 39
2. When we move our eyes from one point to another: <<… we divert our
attention to a point so that we can focus our concentration (if only for a very
brief moment) on the object or region of interest.>>
Slide 40
ET tracks the “diversion of attention” not the attention: <<… we may presume
that if we can track someone’s eye movements, we can follow along the path of
attention deployed by the observer.>>
3
Duchowski, A. T. (2007). Eye tracking methodology. Theory and practice. Springer
Slide 41
4. ETs give us an insight on what grabs the attention of people: << This may
give us some insight into what the observer found interesting…what drew their
attention… provide a clue as to how that person perceived whatever scene she or
he was viewing.>>
Slide 42
Data that you can obtain by an ET
Slide 43
Micro metrics
• Fixations (frequency and duration)
• Gaze movements: gaze points are what the eyes are looking at (a cluster
of gaze around a point constitute a fixation)
• Saccades movements: are quick simultaneous movements of both the
eyes between points of fixations
• Pupils dilatation and size: it is usually associated in the literature with the
cognitive workload and task difficulty
• Blink of the eyes: when we are blinking (shutting and opening the eyelids)
there is a signal loss which is accounted for in the ET systems. However, too
much signal loss could be an indicator of errors (hardware, software, light
etc.) during the detection. Exclusion criteria for the data.
Slide 44
Macro Metrics
These macro Metrics are based on a combination of the micro metrics
• Scanpath Maps
• Heat Maps
• Areas of Interests (AOI)
• Time to First Fixation
• Time spent (Dwell Time)
• Ratio
• Revist
Slide 45
Scan path (gaze plot)
The use of gaze fixation and saccades metrics (time, duration) can tell you the
path used by participants:
Scan path duration, Scan path direction, Spatial density
Are used to visualize: What people are looking at in the product?
Slide 46
Heat map
By using the gaze points we can create heat areas. They are typically displayed
as a colour gradient overlay on the presented image or stimulus. The red, yellow,
and green colours represent in descending order the number of gaze points that
were directed towards parts of the image.
Slide 47
Areas of Interest
Usually, in ETs software you are asked to set up AREAS OF INTEREST (AOI).
These AOI are the areas of the objects you want to compare in terms of micro
metrics.
You can determine which regions of the visual area of the product you want to
extract metrics from.
For instance, in this picture, we can estimate the differences in terms of fixations
for the adverts in different positions.
To define AOI is useful to aggregate metrics to answer: Are people looking at
what we want (or we expect)?
Slide 48
Time to the first fixation
The amount of time that it takes a respondent (or all respondents on average) to
look at a specific AOI from stimulus onset.
This answer the question: how certain aspects of a visual scene are prioritized?
Banovic et al (2014) 4 show (see the picture) that when people have to choose a
product (food) they need significantly less time to fixate on the product they
4
Banović, M., Rosa, P. J., & Gamito, P. (2014). Eye of the beholder: Visual search, attention and product
choice. I see me, you see me: inferring cognitive and emotional processes from gazing behavior, 40-61.
want to choose compared to the products that they want to discard (non –chosen
products) at the first attempt.
The more they learn (stage 2 – after 12 tasks) the less they need to fixate
between the products they want to choose and the products they are not
choosing –i.e., the difference in time to the first fixation disappeared = learning
process.
Slide 49
Time spent on AOI (Dwell Time)
A long duration of looking at a certain region (AOI) can indicate a high level of
interest, while a shorter duration of fixations can indicate that other areas on the
screen might be more interesting or relevant.
This metric can answer questions regarding, e.g., How much time people spent
looking at certain things?
This can inform, for instance, about the importance that participants give to
different actions or information and the priority within different tasks or
information.
Slide 50
Ratio
It provides information about how many of your respondents guided their gaze
towards a specific AOI.
It could be a ratio, for instance, of fixations count and duration or a ratio of time
spent on saccade movement (searching) and time of fixation.
Usually, a high ratio indicates that something (e.g., a logo, the login etc.) is easy
to find, while a lower ratio that something requires effort to be fixated. In the
picture, the sign of “Harrods” is more attractive than the “Bus”.
This answer the question: Are people looking at what I want to?
5
Drew, T., Williams, L. H., Aldred, B., Heilbrun, M. E., & Minoshima, S. (2018). Quantifying the costs of
interruption during diagnostic radiology interpretation using mobile eye-tracking glasses. Journal of Medical
Imaging, 5(3), 031406.
Slide 51
Revisit
Is the number of returned gaze to a particular AOI (how many times a
participant look at the area).
Which are the hottest areas (for positive and negative reasons) and which are the
COLD areas.
Of course, the reasons why an area is attractive should be investigated.
Slide 52
Recap
• Eye trackers enable the observation and recording of eye behaviour such
as pupil dilation, fixation and movements.
• Eye trackers may not replace other techniques (qualitative and
quantitative). Eye-tracking is an enhancement!
• Eye trackers are not mind-reading devices, and they can only tell
us what the person looked at but not why
• Just because a fixation was registered (by a device) does not necessarily
mean that the user really saw an object i.e., that they registered
cognitively in their brain the object.
Slide 53
Best practices at a glance
• Have optimal environment and lighting conditions
• Work with a dual-screen configuration
• Clean your computer
• Appropriately train all people involved
• Use a clear and robust procedure
• Simplify your (lab) setup
Slide 54
Do I need the ET
A fundamental question is: Do I need an eye tracker to answer my question?
Keep in mind that eye trackers are useful tools only when:
- Results may bring actionable insights
- There are no simplest methods to answer a research question or a buy-in
boost i.e., higher support and participation
End