Brussels, 27 May 2022

COST 045/22

DECISION

Subject: Memorandum of Understanding for the implementation of the COST Action “Enabling
multilingual eye-tracking data collection for human and machine language processing
research” (MultiplEYE) CA21131

The COST Member Countries will find attached the Memorandum of Understanding for the COST Action
Enabling multilingual eye-tracking data collection for human and machine language processing research
approved by the Committee of Senior Officials through written procedure on 27 May 2022.
 MEMORANDUM OF UNDERSTANDING

For the implementation of a COST Action designated as

COST Action CA21131

ENABLING MULTILINGUAL EYE-TRACKING DATA COLLECTION FOR HUMAN AND MACHINE
LANGUAGE PROCESSING RESEARCH (MultiplEYE)

The COST Members through the present Memorandum of Understanding (MoU) wish to undertake joint
activities of mutual interest and declare their common intention to participate in the COST Action, referred
to above and described in the Technical Annex of this MoU.

The Action will be carried out in accordance with the set of COST Implementation Rules approved by the
Committee of Senior Officials (CSO), or any document amending or replacing them.

The main aim and objective of the Action is to foster an interdisciplinary network of research groups
working on collecting eye tracking data from reading in many languages.. This will be achieved through the
specific objectives detailed in the Technical Annex.

The present MoU enters into force on the date of the approval of the COST Action by the CSO.

2
 TECHNICAL ANNEX
OVERVIEW

Summary
The MultiplEYE COST Action aims to foster an interdisciplinary network of research groups working on
collecting eye tracking data from reading in many languages. The goal is to support the development of a
large multilingual eye tracking corpus and enable researchers to collect data by sharing infrastructure and
their knowledge between various fields, including linguistics, psychology, and computer science. This data
collection can then be used to study human language processing from a psycholinguistic perspective as
well as to improve and evaluate computational language processing from a machine learning perspective.

The MultiplEYE COST Action has three core goals: (1) To provide a platform for discussing the desiderata
and reaching a common ground between psycholinguists and computational linguists for a multilingual eye-
tracking and self-paced reading data collection. This includes developing and reaching a consensus
concerning experiment design, stimulus selection, stimulus layout, experimental procedure, and data
preprocessing. (2) To enable discussions on the psycholinguistic research questions that can be addressed
with multilingual eye movement data and providing a broad network to initiate collaborations focusing on
cross-linguistic and multilingual projects. (3) To advance the natural language processing and machine
learning applications that leverage eye-tracking data and improve their cross-linguistic generalization
abilities by bringing researchers from psycholinguistics and computational linguistics closer together.

Areas of Expertise Relevant for the Action Keywords

● Languages and literature: Databases, data mining, data ● eye tracking
curation, computational modelling ● natural language processing
● Psychology: Psycholinguistics: acquisition and knowledge of ● multilingual
language, language pathologies ● psycholinguistics
● Computer and Information Sciences: Machine learning ● low resource languages
algorithms

Specific Objectives
To achieve the main objective described in this MoU, the following specific objectives shall be
accomplished:

Research Coordination
● Create a common understanding between researchers from psycholinguistics and computational
linguistics concerning the research questions that shall be addressed with the multilingual eye-tracking
corpus.
● Develop a joint standard methodology for eye-tracking-while-reading experiments considering all aspects
of the experimental design.
● Investigate the possibilities of using low-cost eye-tracking and self-paced reading for collecting accurate
reading data.
● Define the specific desiderata for a multilingual eye-tracking-while-reading corpus.
● Coordinate data collection efforts across countries.
● Provide support for inclusiveness target countries with lower-resource languages without eye-
tracking equipment by sharing expertise and infrastructure.
● Coordinate potential collaborations between the subfields of linguistics.
● Cultivate a diverse and dynamic environment of experienced and young researchers enjoying
and fostering interdisciplinary work.

3
 ● Disseminate the insights and findings generated within this COST Action with the larger
research community and the general public.

Capacity Building
● Promote interdisciplinary research by providing a growing network of experts.
● Support and connect exceptional young researchers and students in various fields of linguistics, as well
as psychology, engineering, and graphic design.
● Develop Training Schools and Workshops that will empower researchers to ...
- design, implement and conduct eye-tracking experiments.
- statistically model eye-tracking data to address psycholinguistic research questions.
- apply and develop machine learning methods to process eye-tracking-data for language technological
applications.

4
Powered by TCPDF (www.tcpdf.org)
TECHNICAL ANNEX

1 S&T EXCELLENCE
1.1 SOUNDNESS OF THE CHALLENGE

1.1.1 DESCRIPTION OF THE STATE-OF-THE-ART

1. Relevance of eye-tracking and self-paced reading data in linguistics

Eye-tracking technology is widely used in various scientific fields including cognitive psychology,
linguistics, and computational sciences. It is considered a gold-standard method in reading research
(Rayner, 1998; Rayner & Carroll, 2018). Due to its high temporal resolution (compared to, e.g., fMRI),
eye-tracking is suitable for real-time reading processing research. It is non-invasive and unobtrusive,
and therefore allows for naturalistic experiment settings.

When humans read a text, they do not focus on every single word. The number of fixations and the
fixation duration on a word depend on several linguistic factors (Clifton et al., 2007; Demberg & Keller,
2008). Different eye-tracking features allow us to study early and late cognitive processing separately.
First, word length, frequency and predictability from context affect fixation duration and fixation count.
The frequency effect was first noted by Rayner (1977) and has been consistently reported in various
studies since (e.g., Just & Carpenter, 1980; Rayner & Duffy, 1986; Cop et al., 2017). Second, readers
are more likely to fixate on content words (Carpenter & Just, 1983). It even appears that eye movements
are reliable indicators of syntactic categories (Barrett & Søgaard, 2015). Word familiarity, too, influences
how long readers look at a word. Although two words may have the same frequency, they may differ in
familiarity and predictability from context. Effects of word familiarity on fixation duration have also been
demonstrated in a number of studies (Juhasz & Rayner, 2003; Williams & Morris, 2004) as have word
predictability effects, e.g., McDonald & Shillcock (2003).

As a low-cost alternative to eye-tracking, self-paced reading (SPR) is often used in reading research
(Jegerski, 2013). In self-paced reading experiments, the participants read a passage word-by-word by
pressing a button to trigger the display of the next word and thus processing times can be recorded.
Low-cost eye-tracking functionalities have the advantage of providing access to a wider population of
participants.

2. State-of-the-art eye-tracking data collection methods

Current eye-tracking technology provides gaze position with high accuracy and precision. Eye-tracking
software solutions are very flexible and allow many options for experiment configuration (Feit et al.,
2017). This richness of possibilities demands a consensus for experiment design. Even though eye-
tracking research for reading has been conducted for decades, there is no common standard for
stimulus presentation (e.g., the number of characters displayed within the fovea, mono- vs. non-
monospaced fonts, spacing between lines, etc.). Moreover, there is no common standard for data
preprocessing to answer open questions such as the minimum duration of fixations and saccades,
velocity thresholds for saccades, the size of interest areas in reading (i.e., how many pixels above and
below a character). Open issues are also found in post-hoc calibration correction, data cleaning, and
the handling of missing values. Finally, the interpretation of different reading time measures must be
studied in more detail.

Self-paced reading experiments are inexpensive and straight-forward to implement. Besides a standard
personal computer, no additional equipment is required. While the naturalness and richness of eye-
tracking data is well established, the correspondence between eye-tracking measures and self-paced
reading times is not clear. Various reading measures extracted from eye-tracking data have been shown
to be similar to SPR (Witzel et al., 2012; Frank et al., 2013; Boyce et al., 2020), but in order to understand
the link between the two methods, more research is required that compares both signals in a principled
way at a large scale across languages and scripts.

3. State-of-the-art psycholinguistic research on human reading

Psycholinguistics investigates the psychological processes involved in the use of language, including
language comprehension, language production, and first and second language acquisition. The benefit
of eye-tracking in human language processing is supported by intensive study in psycholinguistics. An
immense number of studies investigating language processing using eye-tracking or self-paced reading
have been conducted (Rayner, 1998; Vasishth et al., 2013; Jäger et al., 2017; Godfroid, 2019; Marsden,
et al., 2018; Lau, E. & Tanaka, 2021). The overwhelming majority of these studies are planned or
controlled experiments. There are certain limitations to a large part of the existing research which limit
the generalizability of the results: controlled experiment conditions, focus on English language, neglect
of individual differences between readers, and disregard of different demographic groups.

Typically, psycholinguistic theories are developed and tested based on controlled experiments with
minimal pairs. However, as highlighted by Demberg and Keller (2019), there is a need for evaluating
psycholinguistic theories on natural reading corpora rather than on highly infrequent sentence
constructions specifically designed for the purpose of one experiment. Recent research has shown the
advantages of naturalistic reading studies not only when recording eye movements, but also for other
types of cognitive signals. High-quality eye-tracking devices support very small areas of interest,
meaning that even when reading text from multiple lines on a screen, accurate reading times can be
extracted for individual characters (e.g., Demberg & Keller, 2008). Therefore, eye-tracking recordings
of naturalistic reading allows for the extraction of features on various levels of linguistic processing, i.e.,
document level, sentence level, word level, and fixation level (Hamilton & Huth, 2018). An ecologically
valid experiment setup with real-world stimuli is crucial for studying language processing (Nastase et
al., 2020).

State-of-the-art research of psycholinguistic research is biased towards the English language (Evans
& Levinson, 2009). Psycholinguistic theories are typically developed based on English data (Anand et
al., 2011), but many of these theories either explicitly or implicitly aim to explain human language
processing in general. However, it has been shown that many of the existing findings from English do
not generalize to other languages. Indeed, within the last decades, several classic psycholinguistic
theories stumbled in light of cross-linguistic evidence. For instance, Frazier’s (1987) influential Garden
Path Theory turned out to be inconsistent with data from Spanish (Cuetos & Mitchell, 1988) or Gibson’s
(2000) Dependency Locality Theory was found to be inconsistent with anti-locality effects first observed
in German (Konieczny, 2000). Many languages, even within Europe where lab infrastructures do exist,
are extremely understudied in psycholinguistics (e.g., Sekerina (2017) on the situation of Slavic
psycholinguistics). To develop psycholinguistic theories that generalize, these need to be evaluated on
a large set of languages with different properties.

It is well-known that the population of participants in psychological and behavioral research is mainly
restricted to well-educated adults in their early 20s (Henrich et al., 2010). There are a few eye-tracking
studies on second language speakers (Parshina et al., 2021), but mainly involving English as one of the
languages (e.g., Cop et al., 2017). Additionally, some research has been conducted on children (e.g.,
Bingel et al., 2018), but very little research on reading processing of elderly participants is available.
Moreover, there is a lack of systematic investigation on the impact of socio-economic factors on written
language processing. It has been shown that native language can be predicted from gaze fixations when
reading English text (Berzak et al., 2017), but there is no large-scale investigation comparing intra- vs.
inter-group differences and/or comparing speakers from different languages.

As of yet, researchers have mainly focused on population effects and largely neglected the individual
level. Individual differences between readers are often treated as a source of variance to be controlled
using a hierarchical model, but only few papers look further into the individual variability (Kuperman &
Van Dyke, 2011; Nicenboim et al., 2015; Kidd et al., 2018; Gordon et al., 2020; Cunnings & Fujita, 2021).
In addition to the differences between individual readers, there are also differences in the data between
two experiment sessions of the same subject. In psychological vision research, a large-scale study has
investigated the stability of certain individual characteristics of the eye movements across recording

oc-2021-1 2
sessions (Bagary et al., 2017). However, in reading research, there is no work yet that investigates
sentence processing within and between individuals across multiple recording sessions.

4. State-of-the-art of cognitively-enhanced natural language processing

Natural language processing (NLP) is a subfield of linguistics, computer science, and artificial
intelligence concerned with the interactions between computers and human language, in particular, how
to program computers to process and analyze large amounts of natural language data. In recent years,
there has been an increase in research trying to bridge the gap between human and machine language
processing by leveraging cognitive signals, such as eye movement and brain activity data, to analyze
and enhance computational models of language understanding (Hollenstein et al., 2020; Hale et al.,
2021).

Researchers have proposed a range of work that uses eye-tracking signals to improve various NLP
tasks, such as part-of-speech tagging (Barrett et al., 2016), sentiment analysis (Mishra et al., 2017),
named entity recognition (Hollenstein & Zhang, 2019), prediction of multiword expressions (Rohanian
et al., 2017), sentence compression (Klerke et al., 2016), predicting text readability (González-Garduño
& Søgaard), generating image captions (Takmaz et al., 2020), among other tasks (Barrett & Hollenstein,
2020). Furthermore, gaze data has been used to regularize attention in neural architectures on NLP
classification tasks (Barrett et al., 2018).

Cognitive language processing signals can also be leveraged to analyze and evaluate computational
language models. Eye movement data is useful to investigate the cognitive plausibility of computational
language models (Keller, 2010; Sood et al., 2020a). Approaches to directly predict cognitive signals
(e.g., brain activity) indicate that computational representations implicitly encode similar information as
humans (Wehbe et al., 2014; Abnar et al., 2019; Sood et al., 2020b; Schrimpf et al., 2020). Eye
movement data are well suited to analyze these questions in more detail (Merkx & Frank, 2021;
Hollenstein et al., 2021). A large, unified dataset of eye-tracking data from many languages would allow
researchers to investigate the inner workings of state-of-the-art multilingual language models as well as
monolingual models of smaller languages.

Enriching NLP models with eye-tracking features can make their inductive bias more human-like. Most
approaches achieve a modest but consistent improvement across many NLP tasks, even over strong
baselines such as word frequency. While the potential of leveraging eye-tracking data for natural
language processing has been explored, there is a need for richer feature extraction and more
sophisticated and efficient models. Moreover, gaze patterns can be used to analyze the cognitive
plausibility of computational language models. However, due to the limited size of available datasets of
cognitive signals, this research is still exploratory and requires collaborative efforts between various
fields to investigate the similarities and differences between human and machine language processing.
Finally, advances in natural language processing can also benefit our understanding of human language
processing (Hahn & Keller, 2016; Schwartz & Mitchell, 2019; Zuidema et al., 2020).

The bias towards conducting research on English language data is also present in NLP (Bender, 2018).
Moreover, when leveraging eye movement data for NLP, there are still many open questions that have
not been addressed as of yet due to the lack of a large multilingual data source. This includes the types
of features to include in the models, the type of machine learning models to implement, and the
interpretation of results. Collaborative efforts in collecting multilingual eye-tracking data are crucial to
further reduce the gap between linguistics and NLP.

5. Limited availability and limitations of existing eye-tracking datasets

Monolingual eye-tracking datasets are currently available for English (Dundee Corpus (Kennedy et al.,
2003), UCL Corpus (Frank et al., 2013), Provo Corpus (Luke et al., 2018), GECO (Cop et al., 2017),
ZuCo (Hollenstein et al., 2018), GazeBase (Griffith et al., 2021), and a few other languages including
Dutch (Cop et al., 2017), German (Jäger et al., 2021), Russian (Laurinavichyute et al., 2019). However,
these existing datasets have a series of shortcomings, which will be discussed in the following.

Existing eye-tracking and self-paced reading datasets are normally efforts from single research groups.
Hence, the number of sentences and participants is often low (Barrett & Hollenstein, 2020). Additionally,
the included participants mostly cover a restricted population of undergraduate students, which leads to
ethical considerations regarding participant demographics that need to be considered (Henrich et al.,

oc-2021-1 3
2010). Moreover, most existing data sets include controlled experiments with highly unnatural and
manually constructed stimulus sentences; typically, minimal pairs testing one particular syntactic or
semantic manipulation (Hale, 2017).

Many of the existing eye-tracking corpora (e.g., Potsdam Sentence Corpus (Kliegel et al., 2004),
Russian Sentence Corpus (Laurinavichyute et al., 2019)) are single sentence datasets. This means that
individual sentences are presented to the participants out of context. Therefore, any aspects of language
processing beyond the sentence level such as discourse and pragmatic processing, coreference
resolution, context effects, and textual coherence cannot be analyzed. The same applies to available
self-paced reading datasets (Boyce et al., 2020; Futtrell et al., 2021). Currently, the GECO corpus (Cop
et al., 2017) is the only dataset that covers two languages (Dutch and English) and includes reading a
full novel instead of single sentence stimuli.

The available datasets do not contain any data about working memory capacity or cognitive control,
although in psycholinguistics it is well known that these affect sentence processing; indeed several
theories (Cunnings & Felser, 2013; Nicenboim et al., 2016; Engelmann et al., 2019) make claims about
different processing strategies as a function of working memory capacity. Such claims can only be tested
when scores from cognitive tests are available.

Finally, most datasets (except for ZuCo and GECO) include merely a single recording session from
each participant. This is a critical limitation for the choice of evaluation protocols. For instance, it is
impossible to evaluate a machine learning model’s generalization ability across sessions when only data
from a single session is available. Nevertheless, this is one of the most realistic use case scenarios for
many NLP applications. From a psycholinguistic perspective, multiple sessions would allow for teasing
apart individual characteristics from session-specific features. Multiple sessions would allow testing
intra- vs. interindividual differences.

Currently, the MECO project (Siegelman et al., forthcoming) is the most similar dataset to the data
collection envisioned by this Action. The first release will include single-session data from native readers
of 12 languages (Dutch, English, Finnish, German, Greek, Hebrew, Italian, Korean, Norwegian,
Russian, Spanish, and Turkish). The project also plans to release MECO-L2, which will include data on
the same participants reading in English (Kuperman et al., forthcoming).

1.1.2 DESCRIPTION OF THE CHALLENGE (MAIN AIM)

Linguistic diversity and multilingualism is an identity-creating feature of the European Union and its
neighboring countries. From a linguist's point of view, this linguistic diversity is extremely valuable. On
the one hand, it allows the study of linguistic processing in a wide variety of languages, including to-
date psycholinguistically understudied languages such as the Slavic languages (Sekerina, 2017) or
even less studied languages such as Albanian or Maltese. The rich European linguistic diversity also
helps to overcome the traditional research focus on a few (mainly Germanic) languages in countries
where renowned psycholinguistic labs happen to be located, and to advance our understanding of
human language processing in general.

From the perspective of computational linguistics, on the other hand, the recent technological advances
that have been reached for a handful of high-resource languages need to be extended to low-resource
languages. Thereby smaller linguistic communities can equally benefit from the technological advances
which ultimately contribute to Europe’s ideal of being “unified in diversity”. From a machine learning
perspective, the combination of Europe’s high- and low-resource languages within a large multilingual
dataset offers optimal conditions for transfer learning from high- to low-resource languages.

However, collecting a parallel corpus of language processing data from a wide range of languages
across Europe is a major challenge, mainly in terms of networking and communication between the
research groups. The involved researchers need to reach a common understanding of the goals and
ultimately design the stimulus materials and the experiment methodology in a coordinated manner. This
challenge is a great opportunity to bring together experts from different fields and different languages.
Such an effort will advance eye-tracking-while-reading methodology and the psycholinguistic and
computational applications of eye movement data.

oc-2021-1 4
The MultiplEYE COST Action aims to foster an interdisciplinary network of research groups working on
collecting eye-tracking data from reading for NLP and for the cross-linguistic evaluation of
psycholinguistic theories. Many of the currently open research questions can only be answered with
very large samples across many languages, which do not exist yet. Therefore, the MultiplEYE COST
Action wants to enable researchers to contribute to a joint data collection and to share their knowledge
between various fields.

The MultiplEYE COST Action will focus on three core challenges:

1. Providing a platform to discuss the desiderata and reaching a common ground between
psycholinguists and computational linguists for a multilingual eye-tracking and self-paced
reading data collection. This includes developing and reaching a consensus concerning
experiment design, stimulus selection, stimulus layout, experimental procedure, data
preprocessing, etc.
2. Enabling discussions on the psycholinguistic research questions that can be addressed with
multilingual eye movement data and providing a broad network to initiate collaborations focusing
on cross-linguistic and multilingual projects.
3. Advancing the natural language processing and machine learning applications that leverage
eye-tracking data and improve their cross-linguistic generalization abilities by bringing
researchers from psycholinguistics and computational linguistics closer together.

1.2 PROGRESS BEYOND THE STATE-OF-THE-ART

1.2.1 APPROACH TO THE CHALLENGE AND PROGRESS BEYOND THE STATE-OF-THE-ART

The various limitations of existing eye-tracking-while-reading research cannot be addressed by a single

research group or within a single language. The Action will drive scientific progress and achieve results
beyond the state-of-the-art by bringing together multiple disciplines and languages, and by sharing
and propagating knowledge and expertise in three main aspects: eye-tracking methodology,
psycholinguistics, and natural language processing. In the following, the Action’s approaches to
each of these aspects will be described.

First, the Action will approach the challenges posed by bringing together researchers across Europe
and beyond from different fields to encourage interdisciplinary solutions. To collect eye-tracking data
and define a common ground for methodological decisions and experimental design, eye-tracking
hardware and engineering expertise needs to be combined with experimental psycholinguistic expertise
and even aspects concerning graphical design. First, to create and select appropriate reading materials
and decide the experimental procedures, psycholinguists, language teaching and education experts as
well as computational linguists are needed. Moreover, the Action will address general questions
concerning the impact of stimulus presentation mode on eye movements (layout, typeface, line spacing,
etc.). Finally, the applications of eye-tracking-while-reading data are manifold, and computer science,
machine learning, natural language processing, as well as computational cognitive modeling skills will
improve the processing of such data. The Action’s network of proposers fulfills these criteria, often also
combining experts from various fields within and across countries. This selection of interdisciplinary
researchers will bridge the gap between various subfields of linguistics and enable the team to address
currently unresolved methodological issues that go beyond language research.

Second, this broad network of researchers will also provide access to low-resource and typologically
diverse languages. The Action aims to counterbalance the strong focus on the English language in
psycholinguistics and computational linguistics by supporting and including as many languages as
possible in this COST Action. The MultiplEYE Action includes researchers from countries with high
resource and low resource languages. The network of proposers supports access to native speakers
(and language learners) of 27 languages (Albanian, Arabic, Basque, Croatian, Czech, Danish, Dutch,
English, Estonian, French, German, Greek, Hebrew, Latvian, Lithuanian, Macedonian, Maltese, Polish,
Portuguese, Romanian, Romansh, Russian, Serbian, Slovenian, Spanish, Swedish, Turkish), and more
will be added as participants during the lifetime of MultiplEYE. This not only supports the cultural heritage
and technological advances for smaller languages, but it allows the Action to study specific language
combinations, such as high-resource and low-resource languages from the same language family (for
example, Spanish and French vs. Romansh), similar languages with diverging scripts (Croatian and
Serbian), comparison between more or less phonemic orthographies (Danish versus Croatian),
comparison of eye movements between fluent bilingual speakers of Slavic languages (e.g., Czech and

oc-2021-1 5
Slovak bilinguals); as well as individual representatives of a language family such as Maltese as the
only European Semitic language and language isolates such as Basque.

This “breadth and depth” approach of including several similar languages from the same family (e.g.,
Slavic languages, Germanic languages, Romance languages) and at the same time a wide variety of
typologically very different languages (e.g., Basque, Albanian, Hebrew, Turkish, Arabic) will provide
access points to an immense number of research questions. By moving away from the traditional
anglocentrism, potential findings can be corroborated across language boundaries. Hence, the
conclusions will be more generalizable and scalable than what is currently possible. The psycholinguistic
study of European languages and concurrent advancements of NLP technologies for these languages
can enhance their place in a modern multilingual society. The MultiplEYE action contributes to Europe’s
status as a world-leading hub for diverse linguistic research.

The Action will advance eye-tracking methodology for reading experiments by jointly developing best
practices for the design of naturalistic reading in ecological experiments. This will take into account the
multilingual approach of the Action’s endeavor, when selecting reading materials or computationally
processing the texts. The Action will discuss neglected aspects in experiment design such as text
presentation, design of comprehension questions, linguistic annotations, number of experimental
sessions and the assessment of the participants’ cognitive and linguistic skills (e.g., a series of
psychometric tests assessing working memory capacity, cognitive control, vocabulary size, print
exposure). This will result in a common understanding and shared foundation, and thereby influence
and encourage future data collections to drive scientific discoveries. The Action will furthermore explore
to what extent low-cost eye-tracking solutions can be used for linguistic reading research. Low-cost eye-
tracking is advantageous in that it provides a method to reach a wider population of participants from
understudied languages. Finally, the Action aims to provide the first principled comparison between eye-
tracking-while-reading data and self-paced-reading data. The Action will explore to what degree the two
methods are interchangeable and, more specifically, the Action will, for the first time, analyze the linking
function between the different reading measures computed from eye-tracking data and self-paced
reading reaction times. This will provide the empirical and theoretical foundation of replacing high-
precision eye-tracking with low-cost solutions and hence open the possibility to conduct large-scale
web-based experiments or experimental field work in remote places where no lab infrastructure is
available.

Enabling researchers to collect eye-tracking data from reading will provide resources to approach the
psycholinguistic research aims of this COST Action: It will provide a reference data set to evaluate
existing and new psycholinguistic theories cross-linguistically. As a result, reading and sentence
processing research can be investigated in understudied languages and the results can be compared
directly to the patterns observed in well-studied languages such as English, German, or French. The
assessment of a range of psychometric, linguistic, and demographic variables allows the Action to study
individual differences as well as group-level differences in reading in a principled way across languages.

Finally, natural language processing technologies will also greatly benefit from multilingual eye-
tracking data. It will permit the development of generalizable methods across languages. The Action
can explore the potential of eye-tracking data as an inductive bias for models of languages where
training data is scarce. Eye-tracking data will enable computer scientists to better understand machine
learning methods in terms of cognitive plausibility, explainability and interpretability. Current state-of-
the-art machine learning algorithms for language understanding still lack certain skills that humans can
perform automatically and effortlessly. Therefore, the Action can leverage the insights gained from the
psycholinguistic study of human language processing and the newly available eye-tracking data to
improve and evaluate monolingual and multilingual NLP models. As eye-tracking patterns have been
shown to be consistent across languages for certain phenomena (Liversedge et al., 2016), they will
allow the Action to investigate whether multilingual models provide cognitively more plausible
representations than monolingual models. Insights from eye movement data will facilitate transfer
learning from larger language models to low-resource language models through the implementation of
more human-like algorithms. Lastly, advances on the computational side can also enrich
psycholinguistic research: More accurate computational models and our understanding thereof will be
beneficial for generating insights into human language processing models.

oc-2021-1 6
1.2.2 OBJECTIVES

1.2.2.1 Research Coordination Objectives

The research coordination objectives of this COST Action can be described as follows:

• Create a common understanding between researchers from psycholinguistics and

computational linguistics concerning the research questions that shall be addressed with the
multilingual eye-tracking corpus.
• Develop a joint standard methodology for eye-tracking-while-reading experiments considering
all aspects of the experimental design.
• Investigate the possibilities of using low-cost eye-tracking and self-paced reading for collecting
accurate reading data.
• Define the specific desiderata for a multilingual eye-tracking-while-reading corpus.
• Coordinate data collection efforts across countries.
• Provide support for inclusiveness target countries with lower-resource languages without eye-
tracking equipment by sharing expertise and infrastructure.
• Coordinate potential collaborations between the subfields of linguistics.
• Cultivate a diverse and dynamic environment of experienced and young researchers enjoying
and fostering interdisciplinary work.
• Disseminate the insights and findings generated within this COST Action with the larger
research community and the general public.

1.2.2.2 Capacity-building Objectives

The capacity-building objectives of this COST Action will ensure the critical mass and drive scientific
progress in European research on various levels of participation. The Action’s objectives are three-fold:

• Promote interdisciplinary research by providing a growing network of experts.

• Support and connect exceptional young researchers and students in various fields of linguistics,
as well as psychology, engineering, and graphic design.
• Develop Training Schools and Workshops that will empower researchers to …
o design, implement and conduct eye-tracking experiments.
o statistically model eye-tracking data to address psycholinguistic research questions.
o apply and develop machine learning methods to process eye-tracking-data for
language technological applications.

2 NETWORKING EXCELLENCE
2.1 ADDED VALUE OF NETWORKING IN S&T EXCELLENCE

2.1.1 ADDED VALUE IN RELATION TO EXISTING EFFORTS AT EUROPEAN AND/OR

INTERNATIONAL LEVEL

The only comparable research project is MECO, a recently started North American-based project, which
plans to collect multilingual eye-tracking data. However, the MECO project differs from MultiplEYE in
that it mainly includes data from larger languages and L2 data only in English. MultiplEYE will tie in with
the MECO data collection and benefit from their findings. The Action will ensure regular communication
to explore possible collaborations and joint networking activities.

MultiplEYE will build upon the outcomes of previous COST Actions such as the European Literacy
Network (ELN; IS1401), which analyzed some of the challenges in literacy of a multilingual Europe
(Papadopoulos et al., 2021). Current related COST Actions are Multi3Generation (CA18231), which
focuses on multilingual, multi-modal, and multi-task language generation, and Nexus Linguarum
(CA18209), which brings together data scientists and linguists. The joint interest in multilingual language
research and NLP applications will favor fruitful exchanges with both actions. Moreover, MultiplEYE will
interact with resource building programs such as CLARIN (FP7 212230) for natural language resources.
The Action will investigate possible collaborative efforts for hosting eye-tracking data. In addition,

oc-2021-1 7
MultiplEYE can incorporate the findings of initiatives such as the European Language Equality initiative
by META-NET.

Where possible, MultiplEYE will liaise with these other actions and research projects. First, the Action
will ensure regular contact and inform the respective coordinators on the action’s progress. Second, the
Action will strive to co-organize events in appropriate domains. And lastly, the Action will promote the
funding activities of the COST Action, such as Short-Term Scientific Missions and Training Schools.

2.2 ADDED VALUE OF NETWORKING IN IMPACT

2.2.1 SECURING THE CRITICAL MASS AND EXPERTISE

Europe counts many leading experts in research fields such as linguistics, psycholinguistics, and NLP.
MultiplEYE leverages this advantage and brings together complementary skills wherever possible. This
COST Action unites psycholinguists and computational scientists, graphic designers, and education
specialists, neurolinguists and NLP researchers, bilingualism and corpus creation experts, translation
and machine learning experts. The Action’s network is comprised of a balanced panel of researchers in
multiple fields. The varying backgrounds and skills will be key to foster the cross-disciplinary mindset
envisioned by MultiplEYE. Not only do the partners contribute with their specialized expertise profiles,
but also by guaranteeing access to native speakers and learners of their local languages. During the
course of the COST Action, MultiplEYE aims to engage with participants from additional countries to
expand the selection of languages even further.

Most researchers of the Action’s network either have their own eye-tracking lab or have access to a
shared eye-tracking lab where eye-tracking devices suitable for reading research are available. To
ensure lab access and continuity of the intended data collection, those researchers who will be directly
responsible for conducting the data collection are mostly Tenure-Track Assistant Professors or tenured
Professors. Tremendous effort has gone into building the initial network of proposers, including 40
scientists from 26 countries. MultiplEYE reinforces its cross-disciplinary goals by having multiple
proposers from the same country with different backgrounds where complementary expertise is sensible
(in a few cases even the same institution, but from different departments) to strengthen collaboration
not only across countries but also at the source, i.e., within and across institutions of the same country.
This horizontal and vertical selection of countries and institutions in the network of proposers will be
helpful in counterbalancing the research communities’ unequal access to knowledge, infrastructures,
funding, and resources. Moreover, the selection of researchers ensures that, for the majority of
languages, expertise in both psycholinguistics and computational linguistics is available. This is key for
assembling the stimulus materials for an eye-tracking-while-reading corpus intended to serve both
communities as a long-lasting resource.

2.2.2 INVOLVEMENT OF STAKEHOLDERS

There are two main stakeholders of this COST Action: the research scientists involved in MultiplEYE
(professors, assistant professors, and postdoctoral researchers) and the next generation of cross-
disciplinary PhD students. First, the most relevant stakeholders in the MultiplEYE COST Action are the
researchers involved. The main aim of MultiplEYE is to enhance knowledge transfer between various
related subfields of linguistics. All participants will be involved by enabling collaborations and sharing
resources and infrastructures between researchers. Second, MultiplEYE seeks a longer-term impact on
interdisciplinary linguistic research. Hence, junior researchers (PhD students and postdocs) are also
important stakeholders. The Action wishes to inspire and promote junior researchers with the Action’s
network such that their careers may benefit. The Action’s network of interdisciplinary researchers will
be an invaluable resource for junior researchers by providing opportunities to work with experts in many
fields and to get in-depth training from experts in a range of highly relevant theoretical and practical
skills, such as machine learning, natural language processing, statistical and computational modeling
of psycholinguistic data, and experimental methodology.

The Action will achieve a high level of involvement from all stakeholders through:

• Regular meetings and Workshops, to enhance the participants’ networking capabilities and
collaboration opportunities.

oc-2021-1 8
 • Short-Term Scientific Missions (STSMs) and Training Schools, to acquire new skills and transfer
existing knowledge.
• Dissemination and outreach activities by exchanging resources and guidelines and connecting
with the research community at large.

Due to its flexible membership structure, this COST Action will be open to all interested parties. As
described, the Action will encourage participants from further countries to join. While the initial network
focuses on academia, the Action will actively seek participation of industrial partners. This could include
SMEs working on eye-tracking hardware or commercial applications of multilingual technology, such as
personalized document summarization, adaptive e-learning, and machine translation, where eye-
tracking data will be beneficial.

2.2.3 MUTUAL BENEFITS OF THE INVOLVEMENT OF SECONDARY PROPOSERS FROM NEAR

NEIGHBOUR OR INTERNATIONAL PARTNER COUNTRIES OR INTERNATIONAL
ORGANISATIONS

In the Action’s initial network of proposers, the Action includes a Near Neighbor Country. The COST
Action will benefit from their involvement by covering further languages and including the expertise of
non-European experts. Moreover, their interest in hosting and sending guest researchers through
STSMs will facilitate multinational knowledge sharing as well as collaboration on research project
proposals. Specifically, the collaboration with the Russian Federation will provide access to linguistic
expertise about multiple languages spoken in the Russian Federation. The Action also includes two
International Partner Countries, namely Canada and USA. This will facilitate the involvement of the
MECO project as described in Section 1.1.1.

3 IMPACT
3.1 IMPACT TO SCIENCE, SOCIETY AND COMPETITIVENESS, AND
POTENTIAL FOR INNOVATION/BREAK-THROUGHS

3.1.1 SCIENTIFIC, TECHNOLOGICAL, AND/OR SOCIOECONOMIC IMPACTS (INCLUDING

POTENTIAL INNOVATIONS AND/OR BREAKTHROUGHS)

Scientific & technological impact

This COST Action will focus heavily on European languages. The prevailing anglocentrism in linguistic
research (both in psycholinguistics and computational linguistics) is dramatic and hinders advancement
in research. Theories of human language processing cannot be validated appropriately in a cross-
linguistic way and NLP technologies do not generalize from English to other languages. MultiplEYE aims
to counteract these tendencies.

For psycholinguistics, the high diversity of European languages in combination with the good research
infrastructure and training that becomes available through MultiplEYE, is a real treasure: it enables
researchers of all countries to collect and process human language processing data in their own
language. By studying and comparing a wide range of closely related and unrelated European
languages, most of which are extremely understudied, the topics addressed in MultiplEYE will shed new
light on both classical theories and provide a valuable resource to address innovative psycholinguistic
research questions from a cross-linguistic perspective.

By giving more weight to other European languages, MultiplEYE will also advance NLP research beyond
state-of-the-art. NLP is a chance to keep Europe as multilingual as it is (e.g., through machine
translation), but only if these technologies support as many European languages as possible. A large
corpus of multilingual eye-tracking data will provide a valuable resource to study the linguistic
capabilities of computational multilingual language models. In short, MultiplEYE will empower
researchers to make technological breakthroughs in explainable artificial intelligence through cognitive
data, making machine learning models more “human-like”, and extending advances in language
technology to small languages.

oc-2021-1 9
Socio-economic impact

In unison with the European Union’s language policy, based on the motto ‘unified in diversity’, the
multinational structure of COST Actions benefits MultiplEYE not only from a demographic perspective
but also with a direct impact on the research possibilities provided by the large number of represented
languages. Moreover, the Action recognizes the asset that data is for the economy and society and
therefore, the potential impact of a large multilingual eye-tracking resource. In parallel with the European
Commission’s efforts to generate value at the different stages of the data chain, the Action believes that
such a dataset will increase productivity of researchers in various disciplines, address research
questions more efficiently, and speed up innovation and research progress. Additionally, fruitful
interdisciplinary collaboration will reduce costs through shared efforts and infrastructure. Finally, a cross-
national and cross-disciplinary initiative such as MultiplEYE will promote Europe as an attractive location
for science and research by exchanging ideas and advancing methodologies.

3.2 MEASURES TO MAXIMISE IMPACT

3.2.1 KNOWLEDGE CREATION, TRANSFER OF KNOWLEDGE AND CAREER DEVELOPMENT

Knowledge creation & knowledge transfer

The initial proposers include researchers with deep expertise in their respective areas and experience
in collaborative projects. This COST Action will ensure knowledge creation & transfer through the
following practices:

• Workshops and Training Schools will build skills across various disciplines: eye-tracking
methodology, NLP, and machine learning, data science and statistics for linguists. The goal of
these activities is to enable researchers to enhance their skills outside of their current field of
expertise and to advance multiple linguistic subfields in parallel. MultiplEYE will help to bridge
the gap between experimental psycholinguistics and computational linguistics by providing
technical and methodological training.
• The initial team of proposers are participating in many national and European projects related
to the themes of the action. The MultiplEYE participants therefore have the necessary skills and
experience as well as funds and facilities to achieve the objectives listed in Section 1.2. In their
institutions, they are currently supervising a considerable number of graduate students and
postdocs. Therefore, active participation in the COST activities is expected.
• Most of the initial proposers work at institutions or labs where eye-tracking equipment is already
available. The COST’s Short-Term Scientific Missions will provide a medium to share data
collection expertise, equipment, and infrastructure within the network.
• Regular digital communication and physical meetings will be cultivated to ensure a continuous
flow of knowledge transfer. The time schedule in Section 4 provides further details.

Career development

The MultiplEYE action will foster the next generation of researchers concerned with multilingualism. The
network of proposers involves many early career investigators that will build their research groups
focusing on the objectives of this action. This long-term investment by YRIs will also provide continuity
beyond the scope and timeframe of MultiplEYE. Future students and young researchers will benefit from
this dynamic environment and from the COST activities such as Training Schools and Short-Term
Scientific Missions.

3.2.2 PLAN FOR DISSEMINATION AND/OR EXPLOITATION AND DIALOGUE WITH THE
GENERAL PUBLIC OR POLICY

Dissemination within and beyond the research community is a process that will be encouraged,
maintained, and enhanced throughout the whole project. The activities and outcomes of the MultiplEYE
action will be disseminated through the project’s website, and through an active online community
environment including social media channels. The website will contain the information of all current
participants as well as a collection of materials and guidelines created for the Workshops and Training
Schools. Additionally, it will present a compilation of all publications, data repositories, preprocessing
packages, and software tools. All software and code will be made publicly available. The COST Action

oc-2021-1 10
will harness social media to reach a broader European and international audience and inform the
general public of how the investments are leveraged to promote human-centered artificial intelligence.

The public plays an important role as participants in eye-tracking experiments. Since collecting
multilingual eye-tracking data will require a large body of participants, advertisement through both social
media as well as traditional channels is of vital importance. MultiplEYE will ensure a continued
information channel to share the findings with the participants.

Dissemination of the project’s scientific contributions will occur through national and international
conferences and journals. The Action will choose publication venues that provide full open access.
MultiplEYE will target international and European NLP conferences (e.g., ACL, NAACL, EMNLP,
COLING, KONVENS, NoDaLiDa), and psycholinguistics conferences (e.g., CUNY Conference on
Human Sentence Processing; AMLaP Conference); psycholinguistic journals (e.g., Journal of Memory
and Language; Cognition; Cognitive Science; Language, Cognition and Neuroscience; Journal of
Experimental Psychology: Learning, Memory and Cognition), and methodological journals (e.g.,
Behavior Research Methods). Moreover, code and software packages for data preprocessing, stimulus
preparation and experiment presentation will be made publicly available on open-source repositories.

At the end of the COST Action’s lifetime, the Action will organize and advertise a final conference for
compiling and sharing the main findings, targeting interested researchers in all fields of linguistics. The
conference will include a discussion of concrete applications and possible follow-up projects.

4 IMPLEMENTATION
4.1 COHERENCE AND EFFECTIVENESS OF THE WORK PLAN

MultiplEYE will be coordinated by a Management Committee (MC). The MC will oversee the internal
organization of the Working Groups (WGs; Section 4.1.1) and the efficient use of the available
networking tools (Section 4.1.3). The MC will be responsible for ensuring that the COST Action runs
according to plan, reacting quickly to take immediate contingency measures (Section 4.1.2). A MC Chair
and Vice-Chair will be elected at the beginning of the Action. Chair and Vice-Chair will preside over the
MC and oversee the work of the five WGs. Each WG is managed by a WG Leader and Vice-Leader to
ensure the timely production of deliverables and to facilitate and coordinate between the participants.

4.1.1 DESCRIPTION OF WORKING GROUPS, TASKS AND ACTIVITIES

The research topics will be structured in five main Working Groups (WGs) within the MultiplEYE COST
Action:

WG1: Enabling eye-tracking data collection

This Working Group will focus on the multilingual data collection, compiling a large resource of eye-
tracking data from natural reading in many languages. These data collection efforts will include larger
European languages as well as smaller languages. At the moment of submission, all proposers are
interested in contributing to this data collection with their native languages. Researchers will be enabled
to conduct eye-tracking experiments by sharing expertise and infrastructure through Short-Term
Scientific Missions.
One of the main aims is to provide a resource of reading from naturally occurring texts including
comprehension questions. Eye-tracking data will be collected from both native readers (L1) and second
language learners (L2). The goal is to cover a wider population in terms of age and socio-economic
background than in the existing corpora. Moreover, the data collection also encompasses low-cost eye-
tracking and self-paced reading experiments. In addition to the reading data, the Action will assess
participants’ cognitive and linguistic skills via psychometric tests assessing, for instance, working
memory capacity, cognitive control, vocabulary size, and print exposure, as well as some information
concerning their demographic and educational background.
WG1 will select, edit and potentially translate the reading materials, design the reading comprehension
questions, and select the additional assessments. It will further design and implement the experimental
procedure of the eye-tracking and self-paced reading experiments. WG1 is also responsible for
formulating the guidelines and data statement as well as for the final editing and documentation of the

oc-2021-1 11
data collection. The Action will harness the joint efforts of the action to secure additional funds to carry
out the data collection, such as participant payments and salaries for lab assistants.

WG2: Experiment design & methodology

Eye-tracking methodology and experimental decisions play a crucial role in this endeavor. There is a
range of methodological choices that needs to be carefully addressed. These include font choices in
reading experiments, paragraph-level and document-level linguistic and layout characteristics and other
formatting aspects. The objective of this Working Group is to converge to a standard experiment design
for natural reading experiments. This Working Group will also investigate current state-of-the-art
possibilities of using low-cost eye-tracking technology such as webcam-based eye-tracking for reading
studies and analyze to what extent and in what way self-paced reading data corresponds to certain
reading measures extracted from eye-tracking data. There is a lack of research in that area and the
quality of these methods has not yet been thoroughly evaluated for natural reading.

WG3: Eye-tracking for psycholinguistic research questions

A large multilingual eye-tracking corpus will provide a long-awaited resource for tackling psycholinguistic
research questions from a cross-linguistic perspective and hence allow for more generalizable results.
WG 3 will make use of this resource and, by means of statistical hypothesis testing as well as
computational cognitive modeling, will address questions such as (i) the variability in reading patterns
across languages (both L1 and L2) and the study of bilingualism (e.g., Basque-Spanish or Romansh-
German); (ii) individual differences and differences between subpopulations; (iii) a large-scale cross-
linguistic evaluation of psycholinguistic theories on natural reading and, more generally, on sentence
processing; (iv) studying text readability and reading comprehension. WG 3 is tightly coupled with WG
1 and will be in close contact to corroborate the linguistic characteristics of the reading corpus and the
additional assessments to be conducted.

WG4: Natural language processing applications leveraging eye-tracking data

Eye-tracking signals can be leveraged to provide an inductive bias to the computational language
models, which can improve their performance. Additionally, they can be used to evaluate the inner
workings of deep neural language models to increase their interpretability and explainability. For
instance, the human attention patterns in eye-tracking data can be compared to machine attention
mechanisms to gain a better understanding of the similarities and differences between human and
computational language processing. WG4 will investigate the potential applications of eye-tracking data
in natural language processing. The WG will study the cognitive plausibility of multilingual language
models by providing a method for intrinsic evaluation as well as opportunities to analyze the similarities
and differences in human and machine attention. The WG will further study how eye-tracking data can
be used to improve transfer learning between more or less similar languages as well as from a high-
resource to a low-resource language. The large eye tracking data collection will enable researchers to
study machine translation acceptability, and to develop personalized text summarization. WG4 will study
to what extent eye tracking data can be used as human rationales to evaluate the decision-making
process of NLP systems and to provide ground truth labels for machine learning models. More human-
centered NLP applications will improve the usability and acceptability of modern language technologies.

WG5: Dissemination
Working Group 5 will coordinate with the other four groups to ensure timely dissemination of the results
through various channels. Firstly, it will promote and coordinate the publication of the results on scientific
venues (conferences and journals) and advertise these via the COST Action’s website and social media
channels. Working Group 5 will further coordinate the publication of software and data on public
repositories and will be responsible for creating and maintaining an informative website presenting the
COST network, its activities and research outputs. This Working Group will further be in charge of
handling conference grants for junior researchers and open access publication costs and, lastly, of
writing the final project report.

4.1.2 DESCRIPTION OF DELIVERABLES AND TIMEFRAME

The following table describes the deliverables of each Working Group and their corresponding
timeframes (year - quarter) of this COST Action.

oc-2021-1 12
Table 1: Overview of deliverables.

Working Groups Tangible deliverables

• Data statement: format, documentation, and metadata

WG1: Enabling data requirements & open access repository to collect all sub-datasets
collection as well as tools and software useful for multilingual eye-tracking
data collection (Y1-Q3)
• Stimulus corpus: an initial collection of texts to be used in the
experiments for all involved languages (Y1-Q4)
• Training School and Workshop on data collection (Y1-Q4; Y2-Q2)

• Guidelines defining general desiderata for experiment design used

WG2: Eye-tracking for natural reading (Y2-Q1)
methodology • Publication(s) addressing methodological research questions (Y3 -
Y4) including a paper on lessons learned and best practices for
eye-tracking methodology for naturalistic reading experiments
• Training School and Workshop on eye-tracking methodology (Y1-
Q4; Y2-Q2)

• Publications in (psycho-)linguistics journals based on collected data

WG3: addressing the research questions of this Working Group (Y3 - Y4)
Psycholinguistics • Training School in statistics and computational modeling for
psycholinguistics (Y3-Q3)
• Workshop to address cross-linguistic research questions (Y4-Q1)

• Publications in NLP conference proceedings and journals including

WG4: Natural a review paper summarizing the applications of eye-tracking data in
language NLP (Y3 - Y4)
processing • Training School and Workshop in machine learning and NLP for
linguists (Y3-Q3; Y4-Q1)

• Project website (Y1-Q1)

WG5: • Social media channels (Y1-Q1)
Dissemination • Open access publications (Y1 - Y4)
• Final report on dissemination activities (Y4-Q2)

4.1.3 RISK ANALYSIS AND CONTINGENCY PLANS

Timely awareness of and reaction to potential problems will be crucial for effective risk management.
The following are the most important operational risks the project faces.

Table 2: Overview of risks and corresponding contingency plans.

Risk (Level) Contingency Plan

The Action will increase the awareness of the COST Action’s

Low participation in Training activities through the extended network and via all dissemination
Schools & STSMs (unlikely) channels.

Shortage of linguistic
expertise or text materials
for some languages and
disbalance between
languages (possible)
The Action will ensure the participation of researchers and native
speakers of low-resource languages to ensure linguistic expertise
and materials for all involved languages. If necessary, The Action will
connect with language councils at the national level to identify
support mechanisms and data sources.

oc-2021-1 13
 Delays in deliverables & The Action will invite further researchers with the required expertise
milestones (possible) and strongly encourage collaborations in the network.

Bottlenecks in data
collection (possible)
Researchers hosted for STSMs might be able to bring their own
portable eye-tracking equipment from universities where multiple
devices are available. The research will also address possibilities for
low-cost gaze tracking and self-paced reading, which will be an
option for research groups without high-quality equipment or in
situations where access to the labs are restricted. Researchers from
participating countries will be encouraged to apply for additional
national and international funds for data collection.

4.1.4 GANTT DIAGRAM

The COST Action duration is four years. In the following, the milestones will be described, and an
overview of the Action’s activities as well as a detailed time schedule of all intended activities will be
provided.

The MultiplEYE Action sets the following milestones around the individual WGs and their targeted
deliverables:
Milestone 1: By the end of the first quarter of the first year, the COST Action will be operative. The
website and social media channels will be available, the kick-off event will have taken place and
recruitment for the WGs has been initiated.
Milestone 2: By the end of the first year, the deliverables of Year 1 have been completed. The first
Workshop and Training School (WG1) have taken place, the dissemination process is well-established
and publication goals are met.
Milestone 3: By the end of the second year, the deliverables of Year 2 have been achieved. The
Workshop and Training School of WG2 have taken place and the dissemination goals are met.
Milestone 4: By the end of the third year, the deliverables of Year 3 have been achieved. The Workshop
and Training School of WG3 have taken place and the dissemination goals are met.
Milestone 5: By the end of the fourth year, the deliverables of Year 4 have been achieved and all final
events have taken place. The Workshop and Training School of WG4 have taken place and the
dissemination goals are met. The final conference will have been organized. The final report will have
been written. The final set of deliverables, including publications, datasets and tools are available.

These milestones are control points in the COST Action that help to map progress. They will be used in
MC meetings to guide and ensure the progress of the activities. If problems and risks arise, they will
allow corrective measures to be taken in time.

The following activities are included in the COST Action:

• MC meetings: The initial MC meeting is the kick-off meeting and will officially start the COST
Action. 2 MC meetings will be organized annually, whenever possible these will be co-located
with other activities. If necessary, the meetings will be held virtually.
• WG meetings: Each WG will organize 2 annual WG meetings. WG meetings at the end of
Years 2 and 4 will be organized jointly. Year 1 and 3 meetings will be co-located with
international conferences or Workshops.
• Workshops and Training Schools: WGs 1-4 will organize Workshops for the action
participants. Each of these Working Groups will also organize Training Schools for participants
and externals.
• STSMs: Throughout the Action, STSMs will be set up within and between WGs. At least five
STSMs per year will take place to stimulate collaboration between members of different WGs.
• Other dissemination activities: Dissemination via the public website and social media will
start immediately.

The table below provides a detailed time plan of COST Action activities and deliverables split by Working
Groups (green cells mark official COST meetings, blue stands for general COST events, yellow for
deliverables and red for Training Schools and Workshops).

oc-2021-1 14
Table 3: MultiplEYE COST Action time schedule.

Meetings & activities Year 1 Year 2 Year 3 Year 4

Q1 Q2 Q3 Q4 Q1 Q2 Q3 Q4 Q1 Q2 Q3 Q4 Q1 Q2 Q3 Q4
Kick-off event
MC meetings
Mid-term conference
Final conference

WG1: Enabling data collection

Guidelines: Data statement
Data repository
Stimulus corpus
Training school
Workshop
WG meetings

WG2: Eye-tracking methodology

Guidelines: Eye tracking methodology
Publications: Research results
Training school
Workshop
WG meetings

WG3: Psycholinguistics
Publications: Research results
Training school
Workshop
WG meetings

WG4: Natural language processing

Publications: Research results
Review paper: Eye tracking in NLP
Training school
Workshop
WG meetings

WG5: Dissemination
Project website
Social media channels
Report: Dissemination activities
WG meetings

oc-2021-1 15