Chapter 1

Eye Tracking as a Research Tool:

An Introduction
See https://pubs.acs.org/sharingguidelines for options on how to legitimately share published articles.

Steven Cullipher,*,1 Sarah J. R. Hansen,*,2 and Jessica R. VandenPlas*,3

1Science and Mathematics Department, Massachusetts Maritime Academy,
Downloaded via 37.239.78.63 on January 11, 2023 at 21:36:47 (UTC).

Buzzards Bay, Massachusetts 02532, United States

2Department of Chemistry, Columbia University,

New York, New York 10027, United States

3Department of Chemistry, Grand Valley State University,

Allendale, Michigan 49401, United States

*E-mails: scullipher@maritime.edu (S.C.);

sjh2115@columbia.edu (S.J.R.H.); vandenpj@gvsu.edu (J.R.VP.).

Eye tracking can be a robust and rich source of data for

chemistry education research but is not appropriate for all
research questions. There are many variables to consider when
deciding to conduct an eye-tracking study, some of which
may not be obvious to the novice user. By the end of this
chapter, the reader should be able to: (1) decide whether or not
the research question can be answered with eye tracking; (2)
design an appropriate participant task; and (3) determine which
quantitative measures are appropriate to collect and analyze.

© 2018 American Chemical Society

VandenPlas et al.; Eye Tracking for the Chemistry Education Researcher
ACS Symposium Series; American Chemical Society: Washington, DC, 2018.
 Introduction
In recent years, eye tracking has become increasingly popular in the chemistry
education research (CER) community. Many researchers are looking to add this
technique to their toolbox, but because of its complexity and distinct differences
from other CER tools, they have hesitated to make the leap. Outside the field of
CER, eye tracking has been employed in many areas, including usability studies,
reading research, and visual search tasks (1–12). While these types of studies
provide important information on the range and usefulness of eye tracking, the
research questions that CER investigates are often unique and distinct. Before
venturing deeper into the world of eye tracking, and the remainder of this book, it
is important to understand the benefits, capabilities, and limitations of eye-tracking
applications.

Eye-Tracking Technology: How It Works

Most modern eye-tracking systems use a technique called pupil center corneal
reflection (PCCR) to track movements of the eyes while viewing a visual stimulus.
This technology uses near-infrared illumination to create reflection patterns on the
cornea and pupil of the eyes of the user. Image sensors in the eye-tracking unit are
then used to capture images of the eyes and reflections patterns of the near-infrared
light. Using advanced algorithms for image processing, as well as a physiological
3D model of the eye, the software is able to estimate the position of the eye in
space and the point of gaze on the visual stimulus. Sampling rates vary based on
the manufacturer and model of eye tracker, and should be considered if purchasing
a new eye tracker. A minimum sampling rate of 60 Hz is typically recommended,
but a review of the literature will provide best practices for the type of study being
conducted.

Eye Tracking as a Useful, Independent Research Tool

Eye tracking is a quantitative method for recording a participant’s eye
movements as they observe a visual stimulus. Thus, eye tracking can directly
answer the questions:

• At what part of the stimulus is the participant looking?

• How much time does the participant spend looking at a particular part of
the stimulus? Does the participant look at a particular part of the stimulus
more than the others?
• In what order does the participant view the various components of the
stimulus?

This information alone can be quite useful if the investigator is looking to

design, for instance, curriculum materials, dynamic representations, or interactive
simulations. Details on designing suitable stimuli for eye-tracking investigations
are given in Chapter 3 of this book.
2
VandenPlas et al.; Eye Tracking for the Chemistry Education Researcher
ACS Symposium Series; American Chemical Society: Washington, DC, 2018.
 Beyond the quantitative data, eye tracking can provide insight into the
participants’ underlying cognitive processes as they interact with visual stimuli.
Hoffman and Subramaniam have shown that if an individual’s eyes are focused
on an object, the attention of the individual is also on that object (13). The
relationship between mental processing and eye-movement data has also been
studied extensively (14–17). However, to accept the relationship between
cognition and eye movements, the investigator must rely on two core working
assumptions (14):

• The immediacy assumption states that the viewer begins processing the
object upon which they fixate immediately and before moving on to
the next object. As they begin fixating on the new object, the viewer
immediately begins processing this new information.
• The eye-mind assumption states that a link exists between the eyes and the
mind, such that whatever the eye fixates on, the mind processes. Based
on this assumption, it can be inferred that commonly occurring eye gaze
patterns might represent similar ways of processing a visual stimulus.

Thus, in addition to the questions listed above, eye tracking alone can be useful
to answer questions such as:

• What part of the stimulus does the participant spend most of their time
processing or trying to interpret?
• How much time does the participant spend processing different parts of
the stimulus?
• In what order does the participant process information presented in the
stimulus?

While all of these questions are worthwhile investigations, they only begin to
scratch the surface of the types of questions chemistry education researchers are
interested in investigating. Eye tracking alone cannot answer questions about what
the participant is thinking about the stimulus they are processing, or why they have
chosen to process this information in the first place. Such interpretations require
the use of auxiliary data collection methods.

Mixed Methods Approaches to Eye Tracking:

Methodological Triangulation for Enhanced Confidence
As with any research study, the use of additional data channels can serve
to boost confidence in the interpretation of results from a single data collection
method. Remember that eye tracking alone can only provide information about
which parts of a stimulus an individual is processing. It provides no information
on the types of processing that occur. Methodological triangulation is the use
of more than one data collection method to improve confidence in results when
investigating a research question (18). Because of the limitations of eye tracking,

3
VandenPlas et al.; Eye Tracking for the Chemistry Education Researcher
ACS Symposium Series; American Chemical Society: Washington, DC, 2018.
most research investigations should rely on methodological triangulation to
cement understanding of the quantitative findings.
Holmqvist et al. suggest several auxiliary methods to consider, each with
their own benefits and shortcomings (19). Many of the alternate data channels
that work well with eye tracking are already common tools used by CER
investigators, including cognitive interviewing, questionnaires, problem-solving
tasks, and thinking aloud. Other auxiliary data collection methods rely on
biological responses to gauge neurological functions. Some examples are
galvanic skin response, functional magnetic resonance imaging (fMRI), and
electroencephalography (EEG). For more information on the use of multiple
biometric methods to triangulate eye-tracking results, the reader is referred to
Chapter 7. Additional information on other biological data metrics, including
those listed, can be found elsewhere (19). Two considerations that the researcher
must consider when selecting such auxiliary data collection methods include:

• Will the auxiliary data collection method impact the eye-tracking data?
Kirk and Ashcraft have shown that verbal interaction with a participant
during eye tracking may alter their eye movements (20). Some studies
have shown that the increased cognitive load of concurrent verbalizations
slows down eye movements and learning processes (21, 22).
• How will the auxiliary data be linked to eye-tracking data? Most
data collection software for eye tracking provides the capability for
recording audio and video data while monitoring eye movements. While
concurrent verbalization has some drawbacks, the ability to provide an
in-the-moment perspective from the participant may serve as valuable
data, which can then be directly linked to eye movements during the
participant task.

The reader can find more information on identifying appropriate research

questions and designing an eye-tracking study to address these questions in
Chapter 2.

Eye-Tracking Measures
When designing an eye-tracking study, it is important to consider the project
as a whole, from formulation of the research question through analysis of findings,
from the outset. Because of the nature of eye tracking, decision-making in study
design must consider multiple factors. Chief among these decisions will be to
determine which measures are best suited for the study.
Before discussing these measures, there are some commonly used terms that
appear frequently in eye-tracking literature and that you will see throughout this
book:

• Fixation: a pause in eye movement in which the retina is stabilized over

a stationary object. It is estimated that 90% of viewing time is comprised
of fixations, which can last between 150–600 ms (23).
4
VandenPlas et al.; Eye Tracking for the Chemistry Education Researcher
ACS Symposium Series; American Chemical Society: Washington, DC, 2018.
 • Saccade: a rapid movement of the eye that occurs between fixations.
During this transition, the viewer is essentially blind. Saccadic
movements can be both voluntary and reflexive, so careful consideration
is important in analyzing data related to saccades (24).
• Area of interest (AOI): a region of the visual stimulus in which
measurements, primarily fixations, can be aggregated as part of analysis.
AOIs can be defined in two ways. First, AOIs can be defined by
the researcher before data collection, a technique that is commonly
used when the researcher wants to know if a participant looks at a
particular region of the visual stimulus. In the second method, AOIs are
defined after data collection based on cluster analysis. Cluster analysis
aggregates fixation data from all participants to indicate regions of the
visual stimulus that have a high concentration of fixations.
• Scanpath (fixation sequence): a series of eye fixations and saccades, most
commonly among AOIs, that occurs when a viewer is exposed to the
visual stimulus.

Based on these concepts, there are innumerable measures available to eye-

tracking users, ranging from the relatively simple to the cumbersome and complex.
Below is a discussion of the most frequently utilized eye-tracking measures and
their typical applications. The adventurous researcher is referred to Holmqvist et
al. and Duchowski for a more extensive compilation of eye-tracking measures (19,
24).

Fixation Count and Fixation Duration

The most common types of measures used in eye tracking involve fixations.
In particular, fixation count and fixation duration are frequently used to interpret
participant processing of information. Fixation count refers to the distinct number
of fixations within a particular AOI. This is a common measure to indicate how
frequently a participant processes information within the AOI. Fixation duration
is the length of time that a participant’s gaze remains within a particular AOI.
Duration can indicate a participant’s level of understanding, or the complexity,
difficulty, or importance of the information, depending on the task and other
auxiliary measures.
Although research has shown a strong correlation between fixation count and
fixation duration, it is important to consider these two measures separately to get
a complete picture of how a participant processes the information in a stimulus
(25, 26). The researcher should keep in mind that a correlation between fixation
count and fixation duration may not exist for their particular study; so it is always
important to examine both measures. Some research shows that fixation count is
an appropriate indicator of the importance of the information contained in an AOI,
with higher fixation counts corresponding to greater importance (27, 28). Fixation
duration, on the other hand, is indicative of the complexity of information within an
AOI, with longer fixation duration corresponding to greater complexity (7, 29–34).
Both fixation count and fixation duration can be analyzed in various ways.
For example, fixation duration could be taken as the average duration of individual
5
VandenPlas et al.; Eye Tracking for the Chemistry Education Researcher
ACS Symposium Series; American Chemical Society: Washington, DC, 2018.
fixations or the total duration of all fixations within the AOI. An AOI with a
high fixation count but a low average duration per fixation can indicate search
behavior, showing that the participant has little understanding of the presented
information (35). Williamson et al. used fixation duration to examine how
students used ball-and-stick representations and electrostatic potential maps to
answer questions about electron density, positive charge, proton attack, and
hydroxide attack for various molecules (36). Their results showed a correlation
between the accuracy of the students’ response and the fixations of students
within the provided representation. Hansen used fixation data to examine how
students transition between different symbolic reaction representations when
solving chemistry problems (33). Results showed that participants shifting their
viewing pattern regardless of success and similar viewing patterns were employed
by participants who were both successful and unsuccessful in solving visual
stoichiometry problems.
For a more in-depth discussion of how fixations may be collected and
analyzed, see Chapter 4.

Fixation Sequence or Scanpath

Eye fixation sequences (also known as scanpaths) refer to the order in which
a participant’s eye movements shift between AOIs. Whereas fixation count and
duration can provide information on the importance and complexity of the various
AOIs, fixation sequences can reveal perceptual strategies that people develop for
interpreting the sum of a visual stimulus (25, 34, 35, 37–39). These strategies
reflect the individual’s cognition processes and could be used to group participants
or differentiate between demographically grouped individuals. Cullipher and
Sevian used scanpath analysis to compare students at various education levels and
problem-solving strategies (35). Their results showed that students with greater
understanding showed distinct fixation sequences that correlated to examining the
presented information in a direct and reasoned fashion. Additional applications
of sequence analysis are discussed in Chapter 5.

Pupil Diameter
The use of pupil diameter measures, referred to as pupillometry, is another
technique that can be used to indicate cognitive functions of viewers. A thorough
history and explanation of pupillometry can be found in Chapter 8. In the chapter,
Karch also provides evidence for the relationship between pupil dilation and
cognitive function.

Analyzing Eye-Tracking Data

Once the aforementioned measures have been collected by the researcher,
some data analysis, likely including statistical comparison, must take place.
The type of statistics that can be applied to eye-tracking data share many
commonalities with the type of statistics the CER community applies to other
quantitative data. However, some special considerations for running statistical
6
VandenPlas et al.; Eye Tracking for the Chemistry Education Researcher
ACS Symposium Series; American Chemical Society: Washington, DC, 2018.
analysis of eye tracking data are given in Chapter 6, along with suggestions for
using the statistical program R to complete these analyses.

Putting It All Together

Building an eye-tracking study takes careful planning and consideration.
From selecting appropriate research questions, to designing stimuli and concurrent
data collection methods to ensure these questions are properly addressed, to
selecting the right metrics to collect, to finding the right statistical analyses to
apply to the data, the researcher has many decisions to make. The final two
chapters of this book, Chapters 9 and 10, give detailed examples of how CER can
be applied to particular topics within CER, and show how other researchers have
tackled some of these questions. We hope this book will help the researcher get
started and feel more confident making some of these decisions.

References
1. Van Gog, T.; Paas, F.; Van Merriënboer, J. J. G. Uncovering Expertise-
Related Differences in Troubleshooting Performance: Combining Eye
Movement and Concurrent Verbal Protocol Data. Appl. Cogn. Psychol.
2005, 19, 205–221.
2. Van Gog, T.; Paas, F.; van Merriënboer, J. J. G.; Witte, P. Uncovering the
Problem-Solving Process: Cued Retrospective Reporting versus Concurrent
and Retrospective Reporting. J. Exp. Psychol. Appl. 2005, 11, 237.
3. Goldberg, J. H.; Wichansky, A. M. Eye Tracking in Usability Evaluation: A
Practitioner’s Guide. In The Mind’s Eye; Hyona, J., Radach, R., Deubel, H.,
Eds.; Elsevier, 2003, pp 493–516.
4. Jacob, R. J. K.; Karn, K. S. Eye Tracking in Human-Computer Interaction
and Usability Research: Ready to Deliver the Promises. Mind 2003, 2, 4.
5. Land, M. F. Eye Movements and the Control of Actions in Everyday Life.
Prog. Retin. Eye Res. 2006, 25, 296–324.
6. Reder, S. M. On-Line Monitoring of Eye-Position Signals in Contingent and
Noncontingent Paradigms. Behav. Res. Methods Instrum. 1973, 5, 218–228.
7. Rayner, K. Eye Movements in Reading and Information Processing: 20 Years
of Research. Psychol. Bull. 1998, 124, 372–422.
8. Rayner, K.; Pollatsek, A. The Psychology of Reading; Prentice Hall:
Englewood Cliffs, NJ, 1989.
9. Inhoff, A. W.; Radach, R. Definition and Computation of Oculomotor
Measures in the Study of Cognitive Processes. In Eye Guidance in Reading
and Scene Perception; Underwood, G. M., Ed.; Elsevier Science, Ltd.:
Oxford, 1998; pp 29–53.
10. Engbert, R.; Longtin, A.; Kliegl, R. A Dynamical Model of Saccade
Generation in Reading Based on Spatially Distributed Lexical Processing.
Vision Res. 2002, 42, 621–636.
11. Wolfe, J. M. What Can 1 Million Trials Tell Us about Visual Search? Psychol.
Sci. 1998, 9, 33–39.
7
VandenPlas et al.; Eye Tracking for the Chemistry Education Researcher
ACS Symposium Series; American Chemical Society: Washington, DC, 2018.
12. Wolfe, J. M. Visual Search: A Review. In Attention; Pashler, H., Ed.;
University College London Press: London, 1998.
13. Hoffman, J.; Subramaniam, B. The Role of Visual Attention in Saccadic Eye
Movements. Percept. Psychophys. 1995, 57 (6), 787–795.
14. Just, M. A.; Carpenter, P. A. A Theory of Reading: From Eye Fixations to
Comprehension. Psychol. Rev. 1980, 87, 329.
15. Rayner, K.; Raney, G. E.; Pollatsek, A. Eye Movements and Discourse
Processing. In Sources of Coherence in Reading; Lorch, R. F., O’Brien, J. E.
J., Eds.; Lawrence Erlbaum Associates, Inc: Hillsdale, NJ, 1995; pp 9–35.
16. Rayner, K. Eye Movements and Attention in Reading, Scene Perception, and
Visual Search. Q. J. Exp. Psychol. 2009, 62, 1457–1506.
17. Anderson, J. R.; Bothell, D.; Douglass, S. Eye Movements Do Not Reflect
Retrieval Processes: Limits of the Eye-Mind Hypothesis. Psychol. Sci.
2004, 15, 225–231.
18. Denzin, N. K. The Research Act in Sociology: A Theoretical Introduction to
Sociological Methods; Transaction Publishers: Piscataway, NJ, 1973.
19. Holmqvist, K.; Nyström, M.; Andersson, R.; Dewhurst, R.; Jarodzka, H.;
Van de Weijer, J. Eye Tracking: A Comprehensive Guide to Methods and
Measures; Oxford University Press, 2011.
20. Kirk, E. P.; Ashcraft, M. H. Telling Stories: The Perils and Promise of Using
Verbal Reports to Study Math Strategies. J. Exp. Psychol. Learn. Mem.
Cogn. 2001, 27, 157.
21. Nielsen, J.; Clemmensen, T.; Yssing, C. Getting Access to What Goes on in
People’s Heads?: Reflections on the Think-Aloud Technique. In Proceedings
of the Second Nordic Conference on Human-Computer Interaction; ACM:
New York, 2002; pp 101–110.
22. Van Someren, M. W.; Barnard, Y. F.; Sandberg, J. A. C. The Think Aloud
Method: A Practical Guide to Modelling Cognitive Processes; Academic
Press London: London, 1994; Vol. 2.
23. Irwin, D. E. Visual Memory Within and Across Fixations. In Eye Movements
and Visual Cognition: Scene Perception and Reading; Rayner, K., Ed.;
Springer: New York, 1992; pp 146–165.
24. Duchowski, A. T. Eye Tracking Methodology, 3rd ed.; Springer: London,
2017.
25. Tang, H.; Day, E.; Kendhammer, L.; Moore, J.; Brown, S.; Pienta, N. J. Eye
Movement Patterns in Solving Science Ordering Problems. J. Eye Mov. Res.
2016, 9, 1–13.
26. Stieff, M.; Hegarty, M.; Deslongchamps, G. Identifying Representational
Competence With Multi-Representational Displays. Cogn. Instr. 2011, 29,
123–145.
27. Hegarty, M.; Mayer, R. E.; Green, C. E. Comprehension of Arithmetic Word
Problems: Evidence from Students’ Eye Fixations. J. Educ. Psychol. 1992,
84, 76.
28. Green, H. J.; Lemaire, P.; Dufau, S. Eye Movement Correlates of Younger
and Older Adults’ Strategies for Complex Addition. Acta Psychol. (Amst).
2007, 125, 257–278.

8
VandenPlas et al.; Eye Tracking for the Chemistry Education Researcher
ACS Symposium Series; American Chemical Society: Washington, DC, 2018.
29. de Corte, E.; Verschaffel, L.; Pauwels, A. Influence of the Semantic Structure
of Word Problems on Second Graders’ Eye Movements. J. Educ. Psychol.
1990, 82, 359–365.
30. Tang, H.; Pienta, N. Eye-Tracking Study of Complexity in Gas Law
Problems. J. Chem. Educ. 2012, 89, 988–994.
31. Schuttlefield, J. D.; Kirk, J.; Pienta, N. J.; Tang, H. Investigating the Effect
of Complexity Factors in Gas Law Problems. J. Chem. Educ. 2012, 89,
586–591.
32. Topczewski, J.; Topczewski, A. M.; Tang, H.; Kendhammer, L.; Pienta, N. J.
NMR Spectra through the Eyes of a Student: Eye Tracking Applied to NMR
Items. J. Chem. Educ. 2017, 94, 29–37.
33. Hansen, S. J. R. Multimodal Study of Visual Problem Solving in Chemistry
with Multiple Representations. Ph.D. Thesis, Columbia University, 2014.
34. Havanki, K. L.; VandenPlas, J. R. Eye Tracking Methodology for Chemistry
Education Research. In Tools of Chemistry Education Research; Bunce, D.
M.; Cole, R. S., Eds.; ACS Symposium Series 1166; American Chemical
Society, 2014; pp 11–191.
35. Cullipher, S.; Sevian, H. Atoms versus Bonds: How Students Look at
Spectra. J. Chem. Educ. 2015, 92, 1996–2005.
36. Williamson, V. M.; Hegarty, M.; Deslongchamps, G.; Williamson, K. C.;
Shultz, M. J. Identifying Student Use of Ball-and-Stick Images versus
Electrostatic Potential Map Images via Eye Tracking. J. Chem. Educ. 2013,
90, 159–164.
37. Havanki, K. L. A Process Model for the Comprehension of Organic
Chemistry Notation, The Catholic University of America, 2012.
38. Augustyniak, P.; Tadeusiewicz, R. Assessment of Electrocardiogram Visual
Interpretation Strategy Based on Scanpath Analysis. Physiol. Meas. 2006,
27, 597.
39. Slykhuis, D. A.; Wiebe, E. N.; Annetta, L. A. Eye-Tracking Students’
Attention to PowerPoint Photographs in a Science Education Setting. J. Sci.
Educ. Technol. 2005, 14, 509–520.

9
VandenPlas et al.; Eye Tracking for the Chemistry Education Researcher
ACS Symposium Series; American Chemical Society: Washington, DC, 2018.