Corpus Pragmatics

https://doi.org/10.1007/s41701-020-00082-0

BOOK REVIEW

Geert Brône and Bert Oben (Eds.): Eye‑tracking

in interaction
(2018), ISBN 978 90 272 0152 2, Amsterdam/Philadelphia: John
Benjamins Publishing Company, 329 pp

Alessandro Ansani1,2

Received: 19 March 2020 / Accepted: 25 March 2020

© Springer Nature Switzerland AG 2020

Eye-tracking research methodology has grown ceaselessly since its origins in the
late 1960s with seminal works as those by Kendon (1967), Yarbus (1967), and the
first review by Rayner (1978). Inevitably, its domains of application have broadened
accordingly: starting with a conspicuous interest in strictly computational tasks
such as reading (Rayner 1977), perception of complex objects (Yarbus 1967) and
attentional processes (Gopher 1973), from the 1970s on researchers began to engage
in other—at the time very promising—fields, more pertaining to psychology and
psycholinguistics, soon revealing fascinating applications in language production,
lexical acquisition (Tomasello and Todd 1983), learning, and joint visual attention
(Scaife and Bruner 1975). The volume Eye-tracking in Interaction, edited by Geert
Brône and Bert Oben, now emerges as the first collection of articles on the contribu-
tion of eye-tracking technology in the domain of human interaction. Its articles pro-
vide a fascinating insight into several applications, including theoretical and meth-
odological aspects.
Since the 1970s technological advances have become more and more interesting
for this domain. The extraordinary progress of optical engineering and information
sciences allowed for the development of automated and less obstructive eye-tracking
recording devices in several shapes, the most important of which being remote eye-
tracking systems, eye-tracking glasses and table-top eye-trackers (p. 7). This whole
process led to the possibility of exploring one of the most complex phenomena in
nature: human interaction, and, in particular, dyadic interactions, here convincingly
considered as a “complex dynamical system” (p. 71).

* Alessandro Ansani
alessandro.ansani@uniroma1.it
1
Department of Psychology, Sapienza University of Rome, via dei Marsi, 76, Rome 00185, Italy
2
Cosmic Lab, Department of Philosophy, Communication and Performing Arts, Roma Tre
University, via Ostiense, 234, Rome 00146, Italy

13
Vol.:(0123456789)
 A. Ansani

Since tools have become more affordable, an increasing number of researchers

from different disciplines have started to approach eye-tracking; they are mainly
attracted by its apparently simple use, but soon come to face the great non-spec-
ificity of gaze against which many of the authors of this book warn the reader. In
the first place there is the “Midas Touch” problem (p. 36), viz. the problem of dis-
tinguishing, in the context of human-computer interactions, the basic functions of
the eye (i.e. to look and perceive visual information) from deliberate interactions
(Meena et al. 2017; Wu and Wang 2016; Velichkovsky et al. 2014). Then there is
also the more general risk of overinterpreting casual or exogenous attentional gaze
paths by assigning them intentionality and fallacious meanings. All of these issues
merit great attention, and they are well addressed in the text.
Human interaction viewed through the lenses of eye-tracking (pun intended) is
what this book is all about. The volume is divided into three parts whose respec-
tive focuses are: theoretical considerations, methodological considerations, and case
studies. Such a division appears to be very useful, especially for the less-informed
readers: one of the critical risks when dealing with eye tracking studies is that the
theoretical foundations underlying experimental paradigms are left unexplained or
are insufficiently explained; the fact that in this volume a whole section us dedicated
to the theoretical foundations is remarkable. On the other hand, this three-fold divi-
sion has not over-compartmentalized the book: very often methodological issues are
discussed together with theoretical ones, while the case studies establish the link
between theory and application more directly.
In the first part of the book (Theoretical foundations), three chapters deal with
the delicate theme of the relationship between gaze and language: Chapters 3 and 5
share a focus on how the speaker’s gaze affects listeners’ language processing and
comprehension, while Chapter 4 tackles coordination in a multimodal perspective.
Gaze as a cue for intentionality is the central theme of Chapter 2, which is probably
the least linguistic one. The same theme is addressed in Chapter 5, where elegant
experiments are described by Staudte and Crocker in support of the “intentional
account” of gaze, the theory according to which “speaker gaze additionally signals
some sort of intention by the speaker to now refer to that particular object” (p. 95).
This article launches a first bridge to the part on applications, where the same debate
on gaze for visual perception only vs. gaze for signaling meaning (and/or intention-
ality) emerges in the final remarks of Chapter 10.
In the second part (Methodological considerations), the reader is provided with a
broad spectrum of practical solutions for eye-tracking: Chapter 6 presents 3D simu-
lation modeling of deictic gazes, gestures, and their interaction, Chapter 7 tackles
the learning and generation of social gaze patterns in avatars, and, most importantly,
the whole of Chapter 8 is dedicated to the problem of how to avoid the “painstaking
task of manually coding large amounts of data, which is extremely time-consuming
without losing the full potential of mobile eye-tracking systems” (p. 170). Especially
this latter chapter has interesting potential, since the authors’ methodological pro-
posal appears to be most valid, especially in the light of the high agreement ratings
between their automatic analysis and a manual one (p. 190).
It is essential to underline—as indeed some contributors do—that not all of
the proposed methodologies (experimental setting, devices, tools, data analysis

13
Geert Brône and Bert Oben (Eds.): Eye‑tracking in interaction

techniques) are readily affordable for any university or research center, particularly
for departments whose primary research methodologies are not massively engineer-
ing-based. Very appropriately, Pfeiffer and Renner (p. 133) provide a short but use-
ful outlook on some cheaper eye-tracking solutions.
The third part of the book (Case studies) offers a selection of four case studies on
several topics. Chapter 9 is devoted to the underexplored multimodal approach to
turn-taking. One of its conclusions is that “the co-participant last gazed in a Turn-
Constructional Unit by the speaker has a privileged status with respect to turn-tak-
ing” (p. 218), which is an interesting insight even if further research still has to shed
light on the cases in which this privileged status seems to vanish and another co-
participant self-selects. Chapter 10 tackles the differences in gaze behavior between
lexical-gaze and gestural-gaze alignment, showing that the former is somewhat
stronger than the latter, that is “if a speaker is looking at an addressee’s face while
uttering a target word, this significantly increases the probability that the addressee
will use that same word later in the conversation. If a speaker is looking at an
addressee’s face while performing a target gesture, there is no correlation with sub-
sequent gesture production by that addressee” (p. 233; results are broadly discussed
on p. 256). Chapter 11 deepens our knowledge of deictic behavior using a mobile
dual eye-tracking paradigm in face-to-face naturally occurring interactions. Its con-
tribution is quite innovative, primarily since previous studies on deixis have mainly
been based on video-recordings, and therefore lack an entirely satisfying ecological
validity. Chapter 12 deserves great attention given its highly specific theme and its
novel approach: gaze analysis of “multimodal display of recipiency in one naturally
occurring interpreter-mediated therapeutic encounter” (p. 302), i.e. the encounter of
a Russian-speaking asylum seeker and his Dutch psychotherapist in a mental health
institution in the Netherlands. The authors manage to shed light on some interest-
ing micro-phenomena of collaboration between the interlocutors expressed by gaze
paths.
The use of eye-tracking techniques has some great potential in several research
fields, but the domain of human interaction had nevertheless not been thematized to
this extent so far. This publication fills a gap in such research by offering the reader
theoretical, methodological, and applicative perspectives in an organic fashion.
The whole book is solid and its topics are discussed at a satisfactory level of anal-
ysis. Each chapter is convincingly referenced so that the reader aiming at an even
better grasp of any of the theme-related literature can go more in depth into the tack-
led topics.
The book can by no means be regarded as an eye-tracking guide for beginners
(nor does it aim to be one). As a matter of fact, the text would chiefly be recom-
mended to intermediate users of eye-tracking techniques who already have some
basic skills of experimental methodologies and a high competence in data analysis.
Nevertheless, less-skilled users can find the reading useful as well, notably the first
part in which several theoretical frameworks are presented.
In spite of the broad variation of paradigms represented in the volume, one
tool appears to remain absent: pupillometry, i.e. the measurement of pupil size
and reactivity. This has been considered a promising research tool since the 1960s
(Hess and Polt 1964; Hess wt al. 1965) and its effects have been shown in several

13
 A. Ansani

domains such as perception, language processing (as a dynamic marker of pro-

cessing), memory, decision-making, emotion, cognition and cognitive develop-
ment (for reviews, see Laeng and Alnaes 2019; Mathôt 2018; Sirois and Brisson
2014). In reading the book, one could wonder whether pupillometry could also be
a beneficial tool in interaction studies, not merely as an attentional marker. To be
honest, one can imagine that no relevant result has emerged so far, since continu-
ous pupil diameter measurement is a common feature in all eye-tracking software.
Nevertheless, a brief elaboration on this matter could have been profitable, even
from a negative findings perspective: as researchers, we all would often like to
know not only what has been tested and proved, but also what others have tried to
find without a meaningful result.
Another commonly encountered problem in multimodal approaches to human
interaction is the synchronization of data from several sensors. A significant
number of articles in the book address this issue more or less straightforwardly,
proposing various synchronization methodologies, but also in this respect some
novel and more affordable solutions could have been mentioned. Some software is
currently available, such as i­Motions®, which has been designed to integrate and
synchronize behavioral outcomes from different biosensors such as eye-tracking
(screen-based, glasses, VR), automated facial expression analysis (FEA through
Affectiva’s AFFDEX algorithms), electroencephalography (EEG), electrodermal
activity (Galvanic Skin Response), electrocardiography (ECG), and electromyo-
graphy (EMG). Such tools enrich the potential of eye-tracking techniques to make
sense of everyday human interaction.
Given the massive number of hypotheses, techniques and results provided
across the book, more schematization in stating research questions, used method-
ologies and main obtained results would have been useful in a concluding chap-
ter. The reader would have taken great advantage of a final chart with all this
information schematized in one place.
To conclude, this volume is overall a good tool for those eye tracking users
whose expertise is already above average; especially those who are interested in
creating innovative experimental paradigms, settings and tools. Both the theoreti-
cal contents and the practical methodological solutions are in most cases highly
specific, which is both a blessing and a curse, as it can encourage skilled users
but discourage less skilled ones. However, the latter have the opportunity to reach
a better understanding of the eye-tracking techniques by consulting the references
in each chapter, which are very informative, detailed and thorough.

Compliance with ethical standards

Conflict of interest The author declares that this book review was written in the absence of any commercial
or financial relationships that could be construed as a potential conflict of interest.

13
Geert Brône and Bert Oben (Eds.): Eye‑tracking in interaction

References
Gopher, D. (1973). Eye-movement patterns in selective listening tasks of focused attention. Perception
and Psychophysics, 14(2), 259–264.
Hess, E. H., & Polt, J. M. (1964). Pupil size in relation to mental activity during simple problem-solving.
Science, 143(3611), 1190–1192.
Hess, E. H., Seltzer, A. L., & Shlien, J. M. (1965). Pupil response of hetero- and homosexual males to
pictures of men and women: A pilot study. Journal of Abnormal Psychology, 70(3), 165–168.
Kendon, A. (1967). Some functions of gaze-direction in social interaction. Acta Psychologica, 26, 22–63.
Laeng, B., & Alnaes, D. (2019). Pupillometry. In C. Klein & U. Ettinger (Eds.), Eye movement research
studies in neuroscience, psychology and behavioral economics (pp. 449–502). Cham: Springer.
Mathôt, S. (2018). Pupillometry: Psychology, physiology, and function. Journal of Cognition, 1(1), 1–23.
Meena, Y. K., Cecotti, H., Wong-Lin, K., & Prasad, G. (2017). A multimodal interface to resolve the
Midas-Touch problem in gaze-controlled wheelchair. In Proceedings of the annual international
conference of the IEEE engineering in medicine and biology society, EMBS (pp. 905–908).
Rayner, K. (1977). Visual attention in reading: Eye movements reflect cognitive processes. Memory &
Cognition, 5(4), 443–448.
Rayner, K. (1978). Eye movements in reading and information processing. Psychological Bulletin, 85(3),
618–660.
Scaife, M., & Bruner, J. S. (1975). The capacity for joint visual attention in the infant. Nature, 253,
265–266.
Sirois, S., & Brisson, J. (2014). Pupillometry. Wiley interdisciplinary reviews. Cognitive Science, 5(6),
679–692.
Tomasello, M., & Todd, J. (1983). Joint attention and lexical acquisition style. First Language, 4,
197–211.
Velichkovsky, B. B., Rumyantsev, M. A., & Morozov, M. A. (2014). New solution to the Midas touch
problem: Identification of visual commands via extraction of focal fixations. In Paper presented at
the procedia computer science. Vol. 39, No. C (pp. 75–82).
Wu, H., & Wang, J. (2016). A visual attention-based method to address the Midas touch problem existing
in gesture-based interaction. The Visual Computer, 32(1), 123–136.
Yarbus, A. L. (1967). Eye movements and vision. Boston, MA: Springer.

Publisher’s Note Springer Nature remains neutral with regard to jurisdictional claims in published
maps and institutional affiliations.

13