Computers & Education 125 (2018) 376–388

Contents lists available at ScienceDirect

Computers & Education

journal homepage: www.elsevier.com/locate/compedu

When two computer-supported learning strategies are better than

T
one: An eye-tracking study
Héctor R. Poncea,∗, Richard E. Mayerb, María Soledad Loyolac, Mario J. Lópezd,
Ester E. Méndezc
a
Facultad de Administración y Economía, Universidad de Santiago de Chile (USACH), Av. L. B. O'Higgins, 3363, Santiago, Chile
b
Department of Psychological and Brain Sciences, University of California, Santa Barbara, CA, 93106-9660, USA
c
Laboratorio de Tecnologias Interactivas (VirtuaLab), Universidad de Santiago de Chile (USACH), Av. L. B. O'Higgins, 3363, Santiago, Chile
d
Departamento de Ingeniería Industrial, Universidad de Santiago de Chile (USACH), Av. L. B. O'Higgins, 3363, Santiago, Chile

A R T IC LE I N F O ABS TRA CT

Keywords: The aim of this study is to examine the effects of using one or two computer-supported learning
Learning strategies strategies on learning processes (as measured by integrative eye movements through eye-tracking
Eye-tracking methodology) and learning outcome (as measured by memory and comprehension tests). Sixth-
Notetaking grade students selected from four schools located in Santiago, Chile read a two-paragraph text on
Highlighting
a computer screen and were asked either to read it (read-only group), highlight text as they read
Graphic organizers
(highlighting group), type notes from the text into a textbox as they read (notetaking group), fill
in an interactive graphic organizer (graphic organizer group), highlight first and then take notes
from the text (highlighting + notetaking group) or highlight first and then fill in an interactive
graphic organizer (highlighting + graphic organizer group). The graphic organizer group,
highlighting + graphic organizer group, and highlighting + notetaking group each made sig-
nificantly more integrative eye movements between the two paragraphs during learning (in-
dicating better learning processes) and produced higher comprehension test scores (indicating
better learning outcomes) as compared to the read-only group, whereas the highlighting group
and notetaking group did not. Results are consistent with the idea that filling in graphic orga-
nizers is a generative learning strategy, whereas highlighting and typing notes into a textbox are
not generative learning strategies. Similarly using two learning strategies together (notetaking
and highlighting) leads to generative learning even though using either one of those strategies
alone does not, so in this case two learning strategies are better than one.

1. Introduction

1.1. Objective and rationale

Consider a learning situation in which a sixth-grade student reads an expository text such as shown in Fig. 1 and then takes a test
on the material. Previous research shows that students often take a linear approach to reading expository text, in which they read the
words in order without much effort to mentally organize the material and integrate it with their prior knowledge (Fiorella & Mayer,
2015). When students take a linear reading approach their eye movements do not show many transitions between paragraphs and

∗
Corresponding author.
E-mail addresses: hector.ponce@usach.cl (H.R. Ponce), rich.mayer@psych.ucsb.edu (R.E. Mayer), maria.loyola.f@usach.cl (M.S. Loyola),
mario.lopez@usach.cl (M.J. López), ester.mendez@usach.cl (E.E. Méndez).

https://doi.org/10.1016/j.compedu.2018.06.024
Received 20 September 2017; Received in revised form 20 June 2018; Accepted 24 June 2018
Available online 27 June 2018
0360-1315/ © 2018 Elsevier Ltd. All rights reserved.
H.R. Ponce et al. Computers & Education 125 (2018) 376–388

Fig. 1. The Changos and Rapa-nui passage in the read-only condition.

their posttest scores of reading comprehension are not as high as they could be (Ponce & Mayer, 2014a, 2014b). Given the renewed
emphasis on helping students learn to make sense of expository text (Porter, McMaken, Hwang, & Yang, 2011), how can we prime
students to take a generative approach to reading expository text as reflected in more eye movement transitions between paragraphs
and higher posttest scores?
In the present study, we explore the value-added of asking students to engage in various learning strategies during reading an
onscreen text—electronically highlighting parts of an onscreen text with a yellow background (highlighting), typing notes into a text
box (notetaking), filling in an interactive graphic organizer (graphic organizer), or using two of these learning strategies (highlighting
and notetaking or highlighting and graphic organizers). Previous studies using an eye-tracking methodology have explored the effects
on learning outcomes and observed eye movement patterns fostered by the use of single learning strategies such as highlighting text,
notetaking, and filling in graphic organizers while reading texts (Ponce & Mayer, 2014a, 2014b). Our primary goals in this study are
to determine whether adding one or two learning strategies improves learning processes and outcomes as compared to reading only,
and to determine whether using two learning strategies can be more effective than using one.

1.2. Research on highlighting, notetaking and graphic organizers

A learning strategy can be defined as an intentional activity carried out by a learner during learning with the intention of
improving the learning outcome. Some well-known learning strategies for learning from text include taking notes, highlighting
important ideas in the text, or filling in a graphic organizer during reading a text (Dole, Nokes, & Drits, 2009; Dunlosky, Rawson,
Marsh, Nathan, & Willingham, 2013; Fiorella & Mayer, 2015; Miyatsu, Nguyen, & McDaniel, 2018; Ponce & Mayer, 2014b). These
learning strategies were selected for the present study because they are commonly used and vary in the degree to which they are
intended to foster generative processing, with graphic organizers seen as most likely to promote generative processing (Fiorella &
Mayer, 2015).
Highlighting is a basic learning strategy in which readers mark those elements in the text (e.g., words, phrases, or sentences) that
they consider important (Ponce, Mayer, Figueroa, & López, 2018). It helps to direct attention to important information and facilitates
storing the highlighted information in working memory for further processing (Weinstein & Mayer, 1986). The most consistent
finding with highlighting is that it mainly improves memory or recall of information, which leads to superficial learning (Dunlosky
et al., 2013). In a study conducted by Hartley, Bartlett, & Branthwaite, 1980, a group of sixth graders was asked to study a text that
was already highlighted and another group was asked to study the same text but without highlighting (i.e., plain text). They found
that the group that read the highlighted text outperformed the plain text group on immediate and delayed cloze tests, indicating that
highlighting improves rote memory for the highlighted material. Similarly, a study conducted by Ponce and Mayer (2014b) found
that when college students were asked to highlight online text or were provided with highlighted text on the screen they scored
significantly higher on a memory test on the highlighted material but not on a comprehension test compared with a group that only
read the text.
Notetaking is a learning technique extensively used by students either during lectures or when studying texts. It is intended to
direct the reader's attention to relevant information, resulting in encoding it in working memory and storing it in long term memory
(Kiewra, 1985; Reimer, Brimhall, Cao, & O’Reilly, 2009; Weinstein & Mayer, 1986). The drawback with notetaking is that learners
may pay attention to irrelevant aspects of the learning material, take incomplete notes, or create verbatim records from lectures or
textbooks. The most consistent finding is that notetaking tends to improve memory or recall of information, and that only in special
circumstances (e.g., when training is provided) notetaking will improve comprehension of the learning material (Kiewra, 1985; Peper
& Mayer, 1978). A study conducted by Ponce and Mayer (2014a) found that when college students were asked, without previous
training, to take notes on the computer from an onscreen text they scored significantly higher on a memory test but not on a

377
H.R. Ponce et al. Computers & Education 125 (2018) 376–388

comprehension test compared with a group that only read the text.
Graphic organizers involve creating a spatial arrangement of verbal material (such as a matrix), and thereby represent a type of
generative learning strategy that Fiorella and Mayer (2015) refer to as mapping. Creating graphic organizers involves not only
selecting relevant verbal material but also organizing it into a coherent spatial structure (Beyer, 1997). Graphic organizers can be
associated with the development of cognitive skills, such as being able to categorize elements, make comparisons, identify sequences
or define concepts (Kauffman & Kiewra, 2010; Ponce, López, & Mayer, 2012). If such schemas are understood as a general me-
chanism, then they will be stored in long-term memory. For example, if learners are confronted with a text that compares two or more
elements (as shown in Fig. 1) then they can mentally use a matrix for selecting information (e.g., the elements being compared),
organizing the information (e.g., using attributes of comparison), and integrating it with previous knowledge (e.g., assess whether the
values of the attributes are similar or different). Studies have shown that graphic organizers tied to specific rhetorical structures (e.g.,
compare-and-contrast, sequence, cause-and-effect) improve memory and comprehension of the learning material (Kauffman &
Kiewra, 2010; Ponce, Mayer, & Lopez, 2013; Robinson & Kiewra, 1995).

1.3. Research on eye movement during reading

In research on eye movement during reading, it is useful to distinguish between fixations, which are defined as the eye coming to
rest on part of the text (e.g., a word or part of it), and saccades, which correspond to rapid movements of the eye from one fixation
point to another. Fixation duration and direction of saccades are influenced by word and sentence difficulty. Readers usually make
more regressions (i.e., backward saccades) and show longer fixations as texts become more difficult (Rayner, Chace, Slattery, &
Ashby, 2006).
An important advance involves examining the role of integrative saccades or transitions between areas of interest (also called
inter-scanning count) as an index of meaningful learning (Holmqvist et al., 2011; Lai et al., 2013). For example, Mason, Tornatora, &
Pluchino, 2013 used integrative saccades (i.e., readers' transitions between text and corresponding illustrations) to measure the
degree of cognitive integration of text and pictures, which is thought to be a fundamental cognitive process in multimedia learning
(Mayer, 2009, 2014). In another eye-tracking study, Ho, Tsai, Wang, & Tsai, 2014 found that high prior knowledge learners were
more likely than low prior knowledge learners to make inner-scanning transitions (also called integrative saccades) between words
and corresponding graphics in viewing a web-based science report. Johnson and Mayer (2012) found that students made more
integrative saccades and scored higher on posttests from multimedia lessons that adhered to the spatial contiguity principle (by
placing words next to corresponding graphs rather than presents words as a caption at the bottom of the graphic). In addition, Tsai,
Huang, Hou, Hsu, & Chiou, 2016 identified qualitatively and quantitatively different patterns of visual attention transitions based on
sequences of eye fixations to distinguish between high and low achieving players in game-based learning. Overall, researchers have
shown the utility of examining integrative saccades between areas of interest during learning in addition to fixations on key areas of
interest as an index of the depth of cognitive processing during learning.
Although, there is a long research tradition on eye movement during reading (Lai et al., 2013; Liu, 2014; Rayner, 1998), few
studies have investigated eye movement patterns while learners read a text and process it with a learning strategy such as high-
lighting or graphic organizer (Lou, Peteranetz, Kiewra, & Flanigan, 2017). For example, a study conducted by Ponce and Mayer
(2014a) used an eye-tracking methodology to infer cognitive processes primed by notetaking and filling in a graphic organizer.
College students read a passage on the screen that compared eastern steamboats (top section of the paragraph) and western
steamboats (bottom section of the paragraph) and were asked to either read it twice (control group), type notes into a textbook next
to the text (notetaking group), or fill in a graphic organizer that was presented next to the text (graphic organizer group). Fixations
and transitions (i.e., saccades between the top and bottom sections of the text) were recorded while learners executed the tasks, and
memory and comprehension tests were administered after they completed the task. Eye-tracking measures indicated that different
learning strategies produced different patterns of saccades and fixations. Learners in the graphic organizer group made more tran-
sitions from one part of the paragraph to another than learners in the notetaking and read-only groups, indicating that different
cognitive processes were involved. The graphic organizer groups scored significantly higher than the read-only group on both
memory and comprehension tests. The notetaking group improved only on memory but not on comprehension compared with the
read-only group.
In a follow-up study, Ponce and Mayer (2014b) examined how highlighting and graphic organizer strategies affected cognitive
processing. College students were randomly assigned to five experimental conditions: one group read a plain text (control group), a
second group read the same text with key words colored, a third group read the same text along with a filled-in graphic organizer; a
fourth group was asked to highlight key words in the text; and a fifth group was asked to fill in a compare-and-contrast graphic
organizer. The first three conditions were regarded as behaviorally passive and the last two as behaviorally active. The experimental
material was the same as in the previous study. Eye-tracking measures showed that participants in the graphic organizer conditions
made more transitions between the top and bottom sections of the text than the highlighting conditions in both the behaviorally
passive and active conditions. Furthermore, learning outcomes showed that the groups that either filled in the graphic organizer or
were presented with the filled-in graphic organizer outscored the read-only group on both the memory and comprehension tests;
whereas the groups that either highlighted text or read the text with highlighted words outscored the read-only group on the memory
test only. These results suggest that learning strategies involving graphic organizers are more effective in priming integrative (or
generative) learning processes than are learning strategies involving highlighting.

378
H.R. Ponce et al. Computers & Education 125 (2018) 376–388

1.4. Theory and predictions

According to the cognitive theory of multimedia learning (Mayer, 2009, 2014), learners engage in some amount of each of three
kinds of cognitive processing when confronted with a lesson: extraneous processing, which is cognitive processing that does not serve
the instructional objective and is caused by poor instructional design; essential processing, which is cognitive processing needed to
mentally represent the essential material and is caused by the complexity of the material; and generative processing, which is cognitive
processing aimed at making sense of the material and is caused by the learner's motivation to exert effort to understand the material.
These three processes correspond to extraneous cognitive load, intrinsic cognitive load, and germane cognitive load, respectively, in
cognitive load theory (Sweller, Ayres, & Kalyuga, 2011). In particular, generative learning theory focuses on techniques for fostering
generative processing (or germane cognitive load).
It should be noted that some theorists (Kalyuga & Singh, 2016; Kalyuga, 2011) have argued for combining intrinsic and germane
load, thereby yielding just two types of cognitive load: cognitive load that is irrelevant to accomplishing the learning objective (i.e.,
extraneous load) and cognitive load that is relevant to accomplishing the learning objective (i.e., intrinsic and germane load).
However, others (Ayres & Paas, 2012; DeLeeuw & Mayer, 2008; Korbach, Brunken, & Park, 2017) have argued for retaining the
triarchic conceptualization of three types of cognitive load, and shown how eye-tracking measures can be useful in contributing to the
debate.
Generative learning theory states that the learner's cognitive processing during learning plays a vital role in explaining the quality
of his/her learning process and outcome (Fiorella & Mayer, 2015; Mayer, 2009; Wittrock, 1989). Consistent with the generative
processing in the cognitive theory of multimedia learning (Mayer, 2009, 2014), this theory indicates that meaningful learning occurs
when a learner engages in three cognitive processes that operate in working memory—selecting, organizing and integrating. Selecting
involves learners paying attention to relevant information in the learning material, such as distinguishing important ideas. Organizing
involves learners building a coherent structure of the information selected, such as using a comparison schema to connect different
pieces of information. Integrating involves learners relating the incoming information with previous knowledge activated from long-
term memory, such as spatially relating the incoming information with known concepts and ideas.
Generative learning implies that learners engage in generative processing as opposed to linear processing. Generative processing is
a process that arises when learners engage in selecting, organizing, and integrating information during learning. Generative pro-
cessing leads to deep learning which is characterized by making sense of the learning material. Linear processing, on the other hand, is
a process that arises when learners mentally represent the material as presented during learning without further elaboration.
Engaging in linear processing leads to superficial learning, which is mainly characterized by memorizing information. This may occur
when the learner focuses principally on selecting information but fails to organize it through an appropriate schema or model.
Learners may also lack knowledge of appropriate schemas or strategies to process information leading to difficulties in engaging in
generative processing.
If learners engage in generative processing while reading the material in Fig. 1, then it will be reflected in more eye movements
between paragraphs 1 and 2 (i.e., up-down transitions) because it reflects that learners are selecting information from both para-
graphs and organizing it into a coherence structure such as a compare-and-contract schema. On the contrary, fewer eye movements
between paragraphs 1 and 2 reflect linear processing characterized mainly by selecting information in the same way as presented in
the text and storing it in long-term memory without further elaboration. Based on the up-down transition measure, we predict that
learners in the graphic organizer group will make more up-down transitions than the learners in the read-only group, indicating
generative processing; however, the highlighting group and notetaking group will not differ from the read-only group in up-down
transitions showing linear processing (hypothesis 1).
A new issue addressed in this study concerns whether asking students to process the text with two strategies can lead to generative
learning. Our hypothesis is that learners in the highlighting + notetaking will engage in generative processing and consequently
outperform the read-only group in up-down transitions. Although highlighting and notetaking strategies working separately prime
linear processing, by working together they can prime generative processing. In other words, asking learners to highlight the text first
helps to free up working memory capacity that can be used more effectively in the organizing and integrating processes during the
notetaking strategy. Likewise, we predict that the highlighting + graphic organizer group will outperform the read-only group on up-
down transition indicating generative processing (hypothesis 2). Highlighting, in this case, should have a similar effect of freeing up
memory capacity that can be used during filling in the graphic organizer. Additionally, we predict that adding a second strategy will
result in more up-down transitions than the use of a single strategy. Thus, the highlighting + notetaking group will produce more up-
down transitions than the highlighting group or the notetaking group, and the highlighting + graphic organizer group will produce
more up-down transitions than the highlighting group or the graphic organizer group (hypothesis 3).
Engaging in generative processing leads to meaningful learning whereas engaging in linear processing leads to superficial
learning. Meaningful learning is reflected by better test performance on retention (e.g., a memory test) and transfer (e.g., a com-
prehension test) whereas superficial learning is reflected by better performance on retention tests but not on transfer tests (Mayer,
2009, 2011). Consequently, we predict that the read-only group will be outperformed on memory and comprehension tests by the
graphic organizer group, the highlighting + notetaking group, and the highlighting + graphic organizer group (hypothesis 4). On
the contrary, we predict that the read-only group will be outperformed only on the memory test but not on the comprehension test by
the highlighting group and the notetaking group (hypothesis 5). Finally, we predict that using two learning strategies will generate
better learning outcomes than using one strategy when they involve only the least effective strategies (i.e., highlighting and note-
taking), but not when they include the effective one (i.e., graphic organizer) (hypothesis 6).
Finally, asking students to use one or two leaning strategies to process a text takes naturally more time. One may expect that more

379
H.R. Ponce et al. Computers & Education 125 (2018) 376–388

time studying a text could lead to better learning outcomes; however, previous research on single learning strategies demonstrates
that more time spent reading the text, taking notes, or highlighting main ideas on the text does not necessarily lead to an increase on
learning (Ponce & Mayer, 2014b). As Mayer (2009) indicates “meaningful learning depends on the learner's cognitive activity during
learning rather than on the learner's behavioral activity during learning” (p. 22). Therefore, we do not expect to observe that more
time spent performing the tasks involved for each strategy will necessarily lead to better performance on the learning measures
(hypothesis 7).

2. Method

2.1. Participants and design

The participants were 152 sixth grade students selected from 4 elementary schools located in Santiago, Chile. All students were
Spanish-speaking. The schools were selected based on previous contacts with the research team. The schools were privately owned,
received subsidies from the state under a voucher system based on the number of students enrolled, and served families with a social
and economic background classified by the Ministry of Education as middle-low. As a pretest, all students were asked to take a
standardized reading comprehension and writing test, known in Chile as CL-PT (Medina, Gajardo, & Fundación-Educacional-Arauco,
2009).
Students in each school were allocated to the control group or one of the experimental group using a between-subject design, in
which we can observe the effects of adding one feature (such as a specific learning strategy or a pair of learning strategies) as
compared to the identical group without any added features. Students were assigned to one of these conditions using a randomization
process that was sensitive to students' gender and the scores they achieved on the reading comprehension section of the CL-PT test. To
assign students to one of the groups, we listed all students in each school in a spreadsheet, including their pretest scores and gender.
An additional column used a function to generate a random number for each student that was used to sort out the list. Means and
standard deviations (SDs) on the pretest scores and gender (coded as 0 and 1) for six different groups were computed several times
until the means and SDs were very similar among the groups (statistically tested with an ANOVA test for pretest score and a chi-
square test for gender). Lastly, the subgroups were randomly assigned to the control or experimental groups. Consequently, the
number of participants (N), the mean score (M) and standard deviation (SD) on the reading comprehension pretest, and the pro-
portion of girls (PG) for each group were the following: read-only (N = 27, M = 28.31, SD = 8.43, PG = .67), highlighting (N = 24,
M = 27.85, SD = 8.09, PG = .71), notetaking (N = 25, M = 27.64, SD = 9.13, PG = .60), graphic organizer (N = 26, M = 26.29,
SD = 8.74, PG = .62), highlighting + notetaking (N = 24, M = 27.17, SD = 5.86, PG = .67), and highlighting + graphic organizer
(N = 26, M = 26.50, SD = 7.75, PG = .69). An analysis of variance conducted on the reading comprehension pretest found no
significant differences among the groups, F (5, 146) = 0.250, p = .939. A chi-square test found no significant differences among the
groups in the proportion of girls and boys, X2 (5) = 1.00, p = .963.

2.2. Materials

We developed two sets of materials—for training and experimental purposes. The training materials consisted of a PowerPoint
presentation for teaching and handouts for students to practice each strategy. The PowerPoint presentation included an explanation
of the process involved in making a comparison and how to use each strategy to analyze this type of text. The presentation began with
a slide that included the question, “What does it mean to compare?”, a picture of a red apple and a green apple, and a spider map
graphic organizer. Slide 2 gave a definition of the compare-and-contrast strategy and a third slide identified the steps involved in
making a comparison. Slide 4 introduced the graphic organizer, notetaking, and highlighting strategies. Slide 5 presented a one-
paragraph passage titled, “The Brothers”, which compared two brothers based on characteristics such as age, personality, and in-
terests. Slide 6 presented a compare-and-contrast graphic organizer filled in with information from “The Brothers” passage that was
used to explain the use of the organizer. Similarly, slides 7 and 8 used the same passage to explain the use of notetaking and
highlighting. Slide 9 showed how to write a summary of the “The Brothers” passage following a comparison structure that was pre-
filled with information from the text. Slide 10 included an additional one-paragraph text titled, “House and Apartment”, which was
used to practice each strategy in a whole classroom activity. Slides 11, 12, and 13 showed an empty graphic organizer, an editor to
take notes, and another editor with the “House and Apartment” text to highlight, respectively. Slide 14 included the structure of a
summary to be completed. A final slide gave instructions to students on an individual activity involving the handouts focusing on
practicing each strategy.
The handouts included two pages. Page 1 contained a short two-paragraph text that showed similarities and differences between
the cities of Acapulco and Cancun and below the text an empty compare-and-contrast graphic organizer. The instructions were: “Read
the text, highlight relevant information in the text with a red pen and fill in the graphic organizer.” Page 2 included a short two-
paragraph text about how to dress in summer and winter and below the text an empty box to take notes. The instructions were similar
to the previous activity, but in this case, we asked students to read the text, highlight relevant information, and take notes based on
the highlighted text.
The experimental materials included two sets of PowerPoint slides. The first set was developed to practice with the respective
strategy that each student would use during the eye-tracking experiment and make sure they did not experience difficulties in using
any of the embedded software components. It consisted of one slide that displayed a very short text on similarities and differences
between oranges and apples and a software component that implemented either a compare-and-contrast graphic organizer or an

380
H.R. Ponce et al. Computers & Education 125 (2018) 376–388

Fig. 2. The Changos and Rapa-nui passage and the interactive graphic organizer.

editor to take notes. The highlighting technique was implemented through the highlighter functionality available in PowerPoint
presentation mode.
For the eye-tracking experiment, the materials consisted of two slides. The first slide provided a series of simple instructions to
follow during the eye-tracking study. The second slide contained a 200-word text titled, “Chile's Indigenous Peoples: Changos and
Rapa-nui” (as shown in Fig. 1). Next to the text, we embedded a software component that implemented either an interactive graphic
organizer (as shown in Fig. 2) or a text editor to take notes (as shown in Fig. 3). The highlighting technique was implemented with
PowerPoint highlighter functionality that is active during the presentation mode. The graphic organizer was implemented in Adobe
Flash and was embedded as an object into PowerPoint. The editor for notetaking was implemented in PowerPoint by integrating a
textbox object that worked as a basic text editor.
The eye tracker used in this study was the Tobii Pro X2-60 mounted on a personal computer with a 23-inch monitor and Tobii
Studio software installed on the computer. The eye-tracker run at 60 Hz.

Fig. 3. The Changos and Rapa-nui passage and the editor for notetaking.

381
H.R. Ponce et al. Computers & Education 125 (2018) 376–388

2.3. Measures

2.3.1. Eye movement measures

Two eye movement measures were employed for each participant in this study: up-down transitions and fixation duration. Up-
down transitions (or integrative saccades) are the number of saccades from the first paragraph to the second paragraph and from the
second paragraph to the first paragraph. Total fixation duration is the total time spent on all eye fixations on the first paragraph and
on the second paragraph on the Chango and Rapa-nui text. Measures of fixations and saccades are fundamental eye-tracking measures
of cognitive processing from texts (Cook, Wei, & Preziosi, 2018; Holmqvist et al., 2011).
For the Chango and Rapa-nui passage used in the eye-tracking experiment, we defined paragraph 1 and paragraph 2 as two
separate areas of interest (AOIs), and recorded fixation duration in each paragraph and counted the number of up-down transitions
between them. To make the comparison between the Chango and Rapa-nui people, information from the first paragraph must be
associated with information in the second paragraph and vice versa. The more up-down transitions are observed the most likely is the
possibility that learners are attempting to integrate the information from both paragraphs. In contrast, total fixation duration is
intended as a measure of the cognitive process of attending to the information, which is a prerequisite for the deeper cognitive
processes of organizing and integrating. In all conditions, the size and location on the screen of the text occupied by paragraphs 1 and
2 were the same.

2.3.2. Learning outcomes measures

We considered two learning outcome measures for each participant in this study: rote memory and reading comprehension. We
asked participants to complete a cloze test to measure memory and to write a summary of the experimental text to measure com-
prehension.
The cloze test was implemented on paper and consisted of the original Chango and Rapa-nui text with 14 words deleted and
replaced with blank spaces for students to fill in. During scoring, students received one point for each correct word they filled in. The
total possible score is 14 points.
The summary test was implemented on paper and consisted of an instruction given to students to write a summary of the Chango
and Rapa-nui text. To score the summaries, we developed a four criteria rubric that measured the level of understanding of the
comparison being made in the original text. The criteria were: (a) summary text structure, (b) elements that are compared in the text,
(c) attributes of comparison, and (d) degree of similarity or difference mentioned. For each criterion, between one and four points
were given during the scoring process, according to the guidelines indicated in the rubric. The maximum score for this test was 16
points. Two members of the research team (both teachers) blindly marked each summary. Another member of the research team filled
in a spreadsheet with the scores from both markers, if a difference was found between the total scores for each summary of more than
5 points, then the scorers got together to score the summaries again. Eleven cases were in this condition. The reliability of the
marking process was highly consistent between the two evaluators, with r (152) = .75. Given these results, we decided to compute an
average of the two scores as a final score in the summary test.

2.3.3. Pretest
Furthermore, a standardized test called in Spanish CL-PT (Prueba de Comprensión Lectora y Producción de Textos) was ad-
ministered (Medina et al., 2009) to all sixth graders in each participant school. The scores in the reading comprehension section of the
CL-PT test was used to control for the random assignment of students to each experimental condition (as described in the participants
and design section).

2.3.4. Time on task measure

Time on task corresponds to the amount of time spent by each student studying and processing the text with the respective
learning strategy in each group. It was recorded directly with the eye tracker.

2.4. Procedure

We ran the study during the first school semester. The first step was to administer the paper-based version of CL-PT test for sixth
graders four weeks after the semester began. The administration of the CL-PT test lasted 60 min and it was administered as a whole
classroom activity. Next, we graded the test and sent a report to each school (as agreed in advance with each deputy principal). Three
weeks later, an instructor from the research team visited the four schools to teach each sixth-grade classroom how to use the
highlighting, notetaking, and graphic organizer strategies during a 90-min session. The instructor employed the PowerPoint pre-
sentation, which was presented on screen for the entire class to see, and handouts as described in the materials section. The emphasis
of this session was on how to analyze texts with a comparison structure by filling in a graphic organizer, taking notes, and high-
lighting main ideas in the text. The first part of the session was dedicated to making students aware of the process involved in making
a comparison (which lasted 10–15 min). The instructor employed a brainstorming strategy to obtain responses from the students to
the question, “What does it mean to compare?” She wrote students' answers directly into the PowerPoint slide that had an embedded
application implementing a diagram like a “spider” (known as spider map), in which the question was written in the body and the
legs were editable for writing text. Subsequently, the instructor taught students the process involved in making a comparison: first,
identify the elements being compared; second, recognize the attributes of comparison; third, identify the values of each attribute; and
fourth, assess whether such values are similar or different.

382
H.R. Ponce et al. Computers & Education 125 (2018) 376–388

To provide practice (which lasted 15–20 min), the instructor presented the slide containing “The Brother” text and asked students
what type of text was. Next, the instructor explained the use of the graphic organizer, notetaking, and highlighting strategies based on
the “The Brother” text; each contained in an individual slide and with information already processed from the text (e.g., the graphic
organizer was already filled in). The analysis of this text was completed by having the instructor explain how to make a summary by
following a specific structure (i.e., first, indicate the elements being compared; second, write about the similarities; and third, write
about the differences between the elements).
Afterward, to reinforce the concepts and the use of the strategies (which lasted approximately 20 min), the instructor presented
the “House and Apartment” text on a slide and asked students to read it and indicate what the text was about. Later, she presented a
slide with an empty graphic organizer and students were asked to indicate what information from the text to type in the graphic
organizer. Next, the same activity was realized with notetaking and highlighting techniques, respectively, on separate slides. As a
final activity with the “House and Apartment” text, the instructor wrote a summary of the text by following the structure indicated in
the previous paragraph. Students also participated by indicating what to write in the summary structure.
During the last 15 min, the instructor gave each student a handout to work on in class as described in the material section. The
instructor supervised the students during this activity and responded to individual questions that students asked. At the end of the
session, the instructor collected the handouts with the activities completed.
Two weeks after the training session, we proceeded to conduct the eye-tracking experiment. We visited each school as previously
agreed with each deputy principal. Deputy principals arranged for us to have access to an empty classroom or meeting room prepared
for the study. Based on the randomized list (see participants and design section), one of the researchers went to the respective
classroom and asked students to go one by one with her to the meeting room. We asked each student to sit in front of the computer
that had the eye tracker and experimental PowerPoint presentation installed. The first step was to remind students about the training
session they had participated in. We run a 5-min pre-training session on how to work with the application. For example, if the student
was selected to read the text and fill in the graphic organizer, we presented a PowerPoint slide that contained a short text and next to
it an empty graphic organizer. The student was reminded about the use of the graphic organizer and asked to read the text and
complete the organizer. A similar procedure was followed with students who had to take notes or highlight texts. For the high-
lighting + notetaking and highlighting + graphic organizer groups, the training session took 3–5 min longer.
Subsequently, we gave each student a short explanation of the eye tracker and proceeded to calibrate it. Once the eye tracker was
calibrated, we opened the PowerPoint presentation that contained the “Changos and Rapa-nui” text with the respective strategy. In
the case of the read-only group, students were given a maximum of 5 min to read the text as many times as possible. We encouraged
students to read it more than once and they were told that they would have to take a test after they finished. The notetaking and
highlighting groups were given 10 min to finish the task. In the notetaking conditions, 5 students asked for additional time to
complete the task since they were slower in writing with the keyboard, which added an additional 3.6 min on average. Regarding the
multiple strategy groups, the instructions were first to read the text, second highlight the text, and third to take note using the editor
or fill in the graphic organizer. All students followed these instructions and we gave 15 min to complete the task.
After we run the experiment, the actual average time on task (in minutes), as recorded with the eye-tracker, taken by each group
was the following: read-only (M = 3.67, SD = 0.76), highlighting (M = 5.27, SD = 1.83), notetaking (M = 10.27, SD = 2.18), gra-
phic organizer (M = 9.71, SD = 2.78), highlighting + notetaking (M = 14.16, SD = 2.01) and highlighting + graphic organizer
(M = 12.10, SD = 3.20). Although, the differences in time on task among the groups was statistically significant (F (5,
142) = 79.194, p < .001), it did not have a significant effect on learning outcomes (memory and reading comprehension) as shown
in the results section.
Once each student completed the task on the eye tracker, the student was asked to move to another desk and we handed out a
sheet of paper with instruction to write a summary of the text they read on the computer. We repeated the instruction verbally and
told the students that they had 15 min to write the summary. Next, we collected the summary and handed out a sheet of paper
containing the cloze test. The student was given 10 min to complete this task but did not have access to the original text, nor the
information they created using their respective strategy.

3. Results

3.1. Do the groups display different cognitive processes as indicated by different eye movement patterns during learning?

The major issue addressed in this study is whether the groups display different cognitive processing during learning, as indicated
by patterns of eye movements. We are interested in whether students who engage in generative learning strategies—such as creating
graphic organizers—show more up-down transitions or integrative saccades as compared to the read-only group, and we are in-
terested in whether students who engage in two strategies show more up-down transitions than those who engage in only one
strategy. As noted in the introduction, we posit that up-down transitions indicate attempts to integrate information from the first
paragraph with corresponding information in the second paragraphs and vice versa, and thus we use up-down transitions as our
major measure of generative cognitive processing during learning. The analysis of up-down transitions was divided into two phases
during learning: the up-down transitions observed during performance of the first strategy and the up-down transitions observed
during performance of the second strategy.
Table 1 shows the mean number of up-down transitions during the first strategy and the second strategy (for the two groups that
used two strategies) as well as the standard deviations for each group. Data from the eye-tracker for 4 students were not of sufficient
quality so they were excluded from the eye movement analysis. A one-way ANOVA shows significant differences among the groups on

383
H.R. Ponce et al. Computers & Education 125 (2018) 376–388

Table 1
Means and standard deviations on up-down transitions during first strategy and second strategy.
Group N First Strategy Second Strategy Total

M SD M SD M SD d

Read-only 27 7.11 3.39 – – 7.11 3.39 –

Highlighting 24 9.88 12.43 – – 9.88 12.43 0.31
Notetaking 25 11.64 6.43 – – 11.64 6.43 0.89
Graphic Organizer 25 18.56 9.46 – – 18.56 9.46 1.64
Highlighting + Notetaking 23 12.35 12.98 5.91 3.91 18.26 14.78 1.08
Highlighting + Graphic Organizer 24 12.79 11.31 16.96 8.47 29.75 15.56 2.07

Note: Effect sizes (d) computed in relation to the read-only group.

up-down transitions during the performance of the first strategy, F (5, 142) = 3.85, p = .003. A follow-up post-hoc analysis using a
Dunnett test (with p < .05) indicates that only the graphic organizer group made significantly more up-down transitions than the
read-only group (p < .001), with a large effect size of d = 1.64. This finding supports hypothesis 1. A t-test showed that the
highlighting + graphic organizer group made significantly more up-down transitions during the second strategy than did the
highlight + notetaking group, t (45) = 5.70, p < .001, d = 1.66.
A one-way ANOVA involving all six groups shows significant differences among the groups on all up-down transitions made
during the entire learning period, F (5, 142) = 13.61, p < .001. A follow-up Dunnett test (with p < .05) indicates that compared to
the read-only group, more up-down transitions were made by the highlighting + notetaking group (p = 002, d = 1.08) and by the
highlighting + graphic organizer group (p < .001, d = 2.07). These findings support hypothesis 2. Asking students to use a second
strategy to analyze the text had an impact on how they processed it. First, the highlighting + notetaking group made a similar
number of up-down transitions as the graphic organizer group that used it as a single strategy; and second, the highlighting + graphic
organizer group ended up making a higher number of transitions.
We focus on two important findings: First, if only one strategy is being used, people show deeper learning (as indicated by
significantly more up-down transitions compared to the read-only group) when they use a graphic organizer learning strategy but not
when they use highlighting or notetaking. This points to the power of generative learning strategies to cause deeper cognitive
processing during learning. Second, adding a second strategy resulted in deeper cognitive processing—as indicated by up-down
transitions—in which the highlighting + graphic organizer group made more up-down transitions than the highlighting group
(p < .001, d = 1.41) or the graphic organizer group (p = .006, d = 0.87), based on an ANOVA, F (2, 70) = 14.83, p < .001, and
post hoc Dunnett test (with p < .05). Similarly, based on an ANOVA and post hoc Dunnett test (with p < .05), the high-
lighting + notetaking group made more up-down transitions than the highlighting group (p = .030, d = 0.61) but the difference
failed to reach significance for the notetaking group (p = .095, d = 0.59), although the effect size shows the practical relevance of
this difference. These results are consistent with hypothesis 3. Overall, we conclude that (1) using a generative strategy such as in the
graphic organizer group leads to deeper cognitive processing during learning than using none, and (2) using two learning strategies
under certain circumstances leads to deeper cognitive processing during learning than using one. This points to the power of gen-
erative learning strategies and to the power of combining two learning strategies rather than using only one. This represents the
major new empirical contributions of this study.

3.2. Do the groups differ on the time they attend to the text?

A secondary issue concerns whether the groups differ in the amount of time they spend looking at the text (paragraphs 1 and 2
combined) in the lesson. The fixation time analysis was divided in two: fixation time during the performance of the first strategy and
fixation time during the performance of the second strategy. Table 2 shows the mean for total fixation time (and standard deviation)
during the first strategy and the second strategy (for the two groups that used two strategies). A one-way ANOVA shows significant
differences among the groups in average total fixation duration during the first strategy, F (5, 142) = 6.51, p < .001. A follow-up
post-hoc analysis using a Dunnett test (with p < .05) indicates that the groups that spent significantly more time than the read-only

Table 2
Means and standard deviations on total fixation duration on the text during first and second strategy.
Group N First Strategy Second Strategy

M SD M SD

Read-only 27 96.36 42.76 – –

Highlighting 24 157.28 63.58 – –
Notetaking 25 100.29 30.06 – –
Graphic Organizer 25 123.67 57.24 – –
Highlighting + Notetaking 23 172.26 81.51 44.54 22.00
Highlighting + Graphic Organizer 24 147.66 76.57 63.02 29.64

384
H.R. Ponce et al. Computers & Education 125 (2018) 376–388

Table 3
Means, standard deviations and effect sizes on memory and comprehension tests for each group.
Group N Memory Reading Comprehension

M SD d M SD d

Read-only 27 6.11 3.07 – 7.10 2.94 –

Highlighting 24 6.46 2.45 0.13 7.54 2.86 0.15
Notetaking 25 7.29 2.21 0.44 6.78 2.84 −0.11
Graphic Organizer 26 7.92 2.99 0.60 9.86 3.50 0.86
Highlighting + Notetaking 24 8.46 2.55 0.83 9.25 2.77 0.75
Highlighting + Graphic Organizer 26 8.62 2.28 0.93 10.73 2.39 1.35

Note: Effect sizes (d) computed in relation to the read-only group.

group were those that highlighted the text as single strategy and as the first strategy during the two-strategy approach; that is, the
highlighting group (p = .002, d = 1.14), the highlighting + notetaking group (p < .001, d = 1.19), and the highlighting + graphic
organizer group (p = .014, d = 0.84). Interestingly, asking students to perform the second strategy after they highlighted the text,
required less fixation time than working on a single strategy, presumably because students had already spent a lot of time with the
text. We conclude that highlighting drew students' attention to the text whereas notetaking and graphic organizers caused learners to
share some of their attention with the portion of screen containing their notes or graphic organizer.
As an additional analysis that takes total time into account, we divided total fixation duration on text by total time on task to
produce an index of the proportion of time that learners were looking at the text. Time on saccades in not included in this analysis
since it is assumed that information is encoded only during fixations (Rayner, 1998). The resulting means and standard deviations
were: read-only (M = 0.44, SD = 0.19), highlighting (M = 0.51, SD = 0.15), notetaking (M = 0.17, SD = 0.05), graphic organizer
(M = 0.22, SD = 0.10), highlighting + notetaking (M = 0.26, SD = 0.10), and highlighting + graphic organizer (M = 0.29,
SD = 0.08). An analysis of variance found a significant difference among the groups in the proportion of time looking at the text, F (5,
142) = 28.51, p < .001. Pairwise Dunnett tests (with p < .05) indicated that the read-only group and the highlighting group spent a
significantly greater proportion of time looking at the text compared to the other groups. Thus, even though students in the read-only
group spent the least amount of time on the text overall, they spent a greater proportion of their time looking at the text.

3.3. Do the groups differ on learning outcomes (as indicated by the memory and reading comprehension tests)?

In parallel to our analysis of learning processes comparing the groups on up-down transitions during learning, the next major issue
addressed in this study was whether the groups differed in learning outcomes (as measured by scores on the memory test and
comprehension test) as shown in Table 3, in which the means (M), standard deviations (SD) and Cohen's effect sizes (d) are presented
for each measure. We are particularly interested in whether learning with a generative learning strategy such as in the graphic
organizer group leads to better scores on the tests than the read-alone group, and whether learning with two strategies leads to better
test scores than learning with one.
Does using a learning strategy improve memory over simply reading the text? A one-way ANOVA on the memory test score
indicates significant differences among the six groups, F (5, 146) = 3.02, p = .002. A Dunnett post-hoc analysis (with p < .05)
showed that the highlighting + notetaking and highlighting + graphic organizer groups outperformed the read-only group, with
p = .008, and p = .003 (with large effect sizes as shown in Table 3) respectively, indicating that using two strategies is better than
none; and the difference between the graphic organizer group and the read-only group reach marginal significance, with p = .054,
and an effect size of practical significance (d = 0.60 .95% [0.05–1.15]). These findings are consistent with hypothesis 4. The no-
tetaking group and the highlighting group did not score significantly higher than the read-only group, partially rejecting hypothesis
5. This pattern of results shows that using one strategy is better than none in improving memory for the material when that strategy
requires a generative processing (i.e., graphic organizer) but not when it just requires lower level processing (i.e., highlighting and
notetaking).
Does using two learning strategies improve comprehension more than using one strategy? In a one-way ANOVA involving the
three groups of highlighting + notetaking, highlighting, and notetaking, the groups differed significantly on memory test score, F (2,
70) = 4.19, p = .019, with the highlighting + notetaking group scoring significantly higher on the memory test than the highlighting
group (p = .010) but not the notetaking group (p = 0.165), based on a Dunnett test. In a one-way ANOVA involving the three groups
of highlighting + graphic organizer, highlighting, and graphic organizer, the groups differed significantly on memory test score, F (2,
73) = 4.46, p = .015, with the highlighting + graphic organizer group scoring significantly higher than the highlighting group
(p = .009), but not the graphic organizer group (p = .532), based on a Dunnett test. This pattern shows that two learning strategies
are better than one when compared against the least cognitively demanding strategy (i.e., notetaking) but not when compared against
the most cognitively demanding strategy (i.e., graphic organizer). These results are consistent with hypothesis 6.
Does using a learning strategy improve comprehension over simply reading the text? A one-way ANOVA on the comprehension
test indicates significant differences among the six groups, F (5, 146) = 8.04, p < .001. A Dunnett post-hoc analysis showed that the
groups that outperformed the read-only group were the graphic organizer group (p = .003), the highlighting + notetaking group
(p = .039), and the highlighting + graphic organizer group (p < .001), with large effect sizes, as shown in Table 3. This shows that

385
H.R. Ponce et al. Computers & Education 125 (2018) 376–388

using one strategy (i.e., graphic organizer) is better than no strategy when the strategy is a learning strategy that requires generative
processing (i.e., graphic organizer) but not when it is a less cognitively demanding strategy (i.e., highlighting or notetaking). This also
shows that two strategies are better than no strategy (i.e., highlighting + notetaking and highlighting + graphic organizer). These
results support hypothesis 4 and partially support hypothesis 5.
Does using two learning strategies improve comprehension over using one strategy? In a one-way ANOVA involving the three
groups of highlighting + notetaking, highlighting, and notetaking, there was a significant difference among the groups in com-
prehension test scores, F (2, 70) = 4.88, p = .010, with the highlighting + notetaking group scoring significantly higher on com-
prehension than the notetaking group (p = .006), but not significantly higher than the highlighting group (p = .073), based on a
Dunnett test (with p < .05). Similarly, in a one-way ANOVA involving the three groups of highlighting + graphic organizer,
highlighting, and graphic organizer, there was a significant difference among the groups on the comprehension test, F (2, 73) = 7.70,
p = .001, with the highlighting + graphic organizer group scoring significantly higher than the highlighting group (p = .001), but
not the graphic organizer group (p = .467), based on a Dunnett test (with p < .05). These results support hypothesis 6.
Overall, we conclude that (1) using a generative strategy such as in the graphic organizer group leads to better learning than using
none, and (2) using two learning strategies under certain circumstances leads to better learning than using one (and always better
than using none). This pattern is consistent with the idea that creating graphic organizers or using two strategies can be considered
generative learning strategies whereas highlighting and typing linear notes in a textbox are not generative learning strategies.

3.4. Does the amount of time spent on each learning strategy predict learning outcomes?

Finally, as indicated in the procedure section, the differences in average time on the task were statistically significant among the
groups. We examined whether these differences influenced learning outcome measures. For this, we run two multiple regression
analyses; in which the first regression used the scores on the memory test as the dependent variable, and the second regression, the
scores on the reading comprehension test as the dependent variable. In both regressions, the predictors were learning strategy type
and time on task. The results of the first regression predicting memory performance indicated the two independent variables ex-
plained 11.4% of the variance (R2 = .114, F (2, 145) = 10.420, p < .001). We found that learning strategy type significantly pre-
dicted memory performance (β = .476, p = .018) but not time on task (β = .045, p = .572). Similarly, the results of the second
regression predicting reading comprehension performance indicated the two predictors explained 15.5% of the variance (R2 = .155,
F (2, 145) = 14.468, p < .001). We found that learning strategy type significantly predicted memory performance (β = .807,
p = .001) but not time on task (β = −.021, p = .821). Therefore, the evidence suggests that only the type of learning strategy used by
students predict their performance on the memory and reading comprehension measures but not the time spent on the task. These
results support hypothesis 7.

4. Discussion

4.1. Empirical contributions

First, concerning the three basic learning strategies, filling in a graphic organizer resulted in more integrative eye movements and
higher comprehension test scores than simply reading a text, but highlighting and notetaking did not. These results show that using
graphic organizers was an effective learning strategy whereas highlighting and notetaking were not. This pattern of findings is
consistent with previous work, confirms the added value of eye-tracking methodology, and adds replicated support to the effec-
tiveness of using graphic organizers as an effective learning strategy (Lou et al., 2017; Ponce & Mayer, 2014a, 2014b).
Second, concerning the two combined learning strategies (i.e., highlighting + notetaking and highlighting + graphic organizer),
there is evidence that using two learning strategies can result in more integrative eye movements and better comprehension test
scores than using one of the constituent learning strategies. In addition, although highlighting alone and notetaking alone were not
effective as compared to the read-only group, there is evidence that the combination of both of them was effective in increasing
integrative eye movements and comprehension test scores. These findings extend previous work by showing that two learning
strategies can be more effective than one.

4.2. Theoretical contributions

This study was intended to examine the distinction between linear reading strategies, in which learners simply process the
material in the order presented, and generative reading strategies, in which learners mentally reorganize the material into a coherent
structure. We start with the idea that the students who are asked to read without a learning strategy are more likely to use a linear
reading approach, as reflected in fewer up-down transitions and lower reading comprehension scores. In the present study, asking
students to highlight or type notes into a box, also resulted in a linear reading approach, yielding up-down transitions and reading
comprehension scores that were indistinguishable from the read-only group. We conclude that highlighting and notetaking did not
encourage learners to mentally reorganize the material. In contrast, asking students to complete a graphic organizer, or to engage in
two learning strategies, appears to have primed a generative reading approach in learners, as is reflected in more up-down transitions
and higher reading comprehension scores than the read-only group. An intriguing finding that warrants further study is that although
highlighting and notetaking were not effective learning strategies when used individually, they were effective in promoting an
integrative approach when combined (in the highlighting + notetaking group). Consistent, with Fiorella and Mayer's (2015) call for

386
H.R. Ponce et al. Computers & Education 125 (2018) 376–388

research on generative learning strategies—that is, learning strategies that promote understanding through priming a generative
approach—the present study distinguishes between strategies that are generative—graphic organizer, highlighting-and-graphic-or-
ganizer, and highlighting-and-notetaking—from those that are not—highlighting alone and notetaking alone.

4.3. Practical contributions

The present study calls into question the use of highlighting as an effective learning strategy when used without any other
learning strategies. This assessment is consistent with Dunlosky et al.'s (2013, p. 21) conclusion: “On the basis of the available
evidence, we rate highlighting and underlining as having low utility.” An important new practical contribution is that highlighting
appears to be useful when used in conjunction with other basic learning strategies such as notetaking. Thus, the combination of
highlighting and notetaking can be an effective generative learning strategy.
The present study reaffirms the use of graphic organizers as an effective learning strategy even when it is used alone (Lou et al.,
2017; Ponce & Mayer, 2014a, 2014b). A new contribution is that graphic organizers are also effective when used in combination with
highlighting. Overall, if students use only one learning strategy, then graphic organizers would be preferable to highlighting or
unguided notetaking. Another worthwhile approach would be to use a combination of learning strategies (Glover, Xu, & Hardaker,
2007; Ponce et al., 2013; Reimer et al., 2009).

4.4. Methodological contributions

The use of eye-tracking measures—particularly, up-down transitions—allowed us to examine cognitive processing during learning
in a more direct way than using self-report surveys. Like most intervention studies, we included measures of learning out-
come—memory and comprehension tests—which are essential for testing instructional effectiveness. However, adding a direct
measure of cognitive processing during learning (i.e., eye-tracking measures) allowed for a more complete test of the idea that the
higher scoring groups were engaging in more generative processing during learning.

4.5. Limitations and future directions

Although this study was conducted in schools and used academic texts, it is limited because it is a single, short-term study, with
immediate tests. It could be improved by including other forms of measuring learning outcomes such as asking students to construct
conceptual maps and introducing delayed testing. Also, it would be useful to explore the most effective way to employ learning
strategies over a longer instructional period, such as an entire term using multiple tests.
Another issue warranting further study concerns whether the effectiveness of learning strategies could be improved through
training and guidance. In addition, it would be useful to examine the effectiveness of other combinations of learning strategies
beyond those tested in this study.

Acknowledgment

This study was partially funded by CONICYT of Chile through grant FONDECYT #1151092 and Proyectos Basales at the
University of Santiago of Chile.

References

Ayres, P., & Paas, F. (2012). Cognitive load theory: New directions and challenges. Applied Cognitive Psychology, 26. 877–832 https://doi.org/10.1002/acp.2882.
Beyer, B. K. (1997). Improving student thinking: A comprehensive approach. Boston, MA: Allyn and Bacon.
Cook, A. E., Wei, W., & Preziosi, M. A. (2018). The use of ocular-motor measures in a convergent approach to studying cognitive load. In R. Z. Zheng (Ed.). Cognitive
load measurement and application: A theoretical framework for meaningful research and practice (pp. 112–128). New York: Routledge.
DeLeeuw, K. E., & Mayer, R. E. (2008). A comparison of three measures of cognitive load: Evidence for separable measures of intrinsic, extraneous, and germane load.
Journal of Educational Psychology, 100(1), 223–234.
Dole, J. A., Nokes, J. D., & Drits, D. (2009). Cognitive strategy instruction. In S. E. Israel, & G. G. Duffy (Eds.). Handbook of research on reading comprehension (pp. 347–
372). New York: Routledge.
Dunlosky, J., Rawson, K. A., Marsh, E. J., Nathan, M. J., & Willingham, D. T. (2013). Improving students' learning with effective learning techniques: Promising
directions from cognitive and educational psychology. Psychological Science in the Public Interest, 14(1), 4–58. https://doi.org/10.1177/1529100612453266.
Fiorella, L., & Mayer, R. E. (2015). Learning as a generative activity. New York: Cambridge University Press.
Glover, I., Xu, Z., & Hardaker, G. (2007). Online annotation – research and practices. Computers & Education, 49(4), 1308–1320. https://doi.org/. https://doi.org/10.
1016/j.compedu.2006.02.006.
Hartley, J., Bartlett, S., & Branthwaite, A. (1980). Underlining can make a difference: Sometimes. The Journal of Educational Research, 73(4), 218–224. https://doi.org/
10.2307/27539753.
Holmqvist, K., Nystrom, M., Anderson, R., Dewhurst, R., Jarodzka, H., & van de Weijer, J. (2011). Eye tracking: A comprehensive guide to methods and measures. Oxford:
Oxford University Press.
Ho, H. N. J., Tsai, M.-J., Wang, C.-Y., & Tsai, C.-C. (2014). Prior knowledge and online inquiry-based science reading: Evidence from eye tracking. International Journal
of Science and Mathematics Education, 12(3), 525–554. https://doi.org/10.1007/s10763-013-9489-6.
Johnson, C. I., & Mayer, R. E. (2012). An eye movement analysis of the spatial contiguity effect in multimedia learning. Journal of Experimental Psychology: Applied,
18(2), 178–191. https://doi.org/10.1037/0033-2909.124.3.37210.1037/a0026923.
Kalyuga, S. (2011). Cognitive load theory: How many types of load does it really need. Educational Psychology Review, 23, 1–19. https://doi.org/10.1007/s10648-010-
9150-7.
Kalyuga, S., & Singh, A. (2016). Rethinking the boundaries of cognitive load theory in complex learning. Educational Psychology Review, 28, 831–852. https://doi.org/

387
H.R. Ponce et al. Computers & Education 125 (2018) 376–388

10.1007/s10648-015-9352-0.
Kauffman, D. F., & Kiewra, K. A. (2010). What makes a matrix so effective? An empirical test of the relative benefits of signaling, extraction, and localization.
Instructional Science, 38(6), 679–705. https://doi.org/10.1007/s11251-009-9095-8.
Kiewra, K. A. (1985). Investigating notetaking and review: A depth of processing alternative. Educational Psychologist, 20(1), 23–32.
Korbach, A., Brunken, R., & Park, B. (2017). Measurement of cognitive load in multimedia learning: A comparison of different objective measures. Instructional Science,
45, 515–536.
Lai, M.-L., Tsai, M.-J., Yang, F.-Y., Hsu, C.-Y., Liu, T.-C., Lee, S. W.-Y., et al. (2013). A review of using eye-tracking technology in exploring learning from 2000 to 2012.
Educational Research Review, 10, 90–115. https://doi.org/10.1016/j.edurev.2013.10.001.
Liu, P.-L. (2014). Using eye tracking to understand learners' reading process through the concept-mapping learning strategy. Computers & Education, 78, 237–249.
https://doi.org/. https://doi.org/10.1016/j.compedu.2014.05.011.
Lou, L., Peteranetz, M. S., Kiewra, K. A., & Flanigan, A. E. (2017). Using eye-tracking technology to understand how graphic organizers aid student learning. In C. Was,
F. Sansosti, & B. Morris (Eds.). Eye-tracking technology applications in educational research (pp. 220–238). Hershey, PA: IGI Global. https://doi.org/10.4018/978-1-
5225-1005-5.ch011.
Mason, L., Tornatora, M. C., & Pluchino, P. (2013). Do fourth graders integrate text and picture in processing and learning from an illustrated science text? Evidence
from eye-movement patterns. Computers & Education, 60(1), 95–109. https://doi.org/https://doi.org/10.1016/j.compedu.2012.07.011.
Mayer, R. E. (2009). Multimedia learning (2nd ed.). New York: Cambridge University Press.
Mayer, R. E. (2011). Applying the science of learning. Boston, MA: Pearson/Allyn & Bacon.
Mayer, R. E. (2014). Cognitive theory of multimedia learning. In R. E. Mayer (Ed.). The Cambridge handbook of multimedia learning (pp. 43–71). (2nd ed.). New York:
Cambridge University Press.
Medina, A., Gajardo, A. M., & Fundación-Educacional-Arauco (2009). Prueba de comprensión lectora y producción de textos (CL-PT): Kinder a 4to año básico. Santiago de
Chile: Ediciones Universidad Católica de Chile.
Miyatsu, T., Nguyen, K., & McDaniel, M. A. (2018). Five popular study strategies: Their pitfalls and optimal implementations. Perspectives on Psychological Science,
13(3), 390–407. https://doi.org/10.1177/1745691617710510.
Peper, R. J., & Mayer, R. E. (1978). Note taking as a generative activity. Journal of Educational Psychology, 70(4), 514–522.
Ponce, H. R., López, M. J., & Mayer, R. E. (2012). Instructional effectiveness of a computer-supported program for teaching reading comprehension strategies.
Computers & Education, 59(4), 1170–1183. https://doi.org/10.1016/j.compedu.2012.05.013.
Ponce, H. R., & Mayer, R. E. (2014a). Qualitatively different cognitive processing during online reading primed by different study activities. Computers in Human
Behavior, 30, 121–130. https://doi.org/10.1016/j.chb.2013.07.054.
Ponce, H. R., & Mayer, R. E. (2014b). An eye movement analysis of highlighting and graphic organizer study aids for learning from expository text. Computers in Human
Behavior, 41, 21–32. https://doi.org/10.1016/j.chb.2014.09.010.
Ponce, H. R., Mayer, R. E., Figueroa, V. A., & López, M. J. (2018). Interactive highlighting for just-in-time formative assessment during whole-class instruction: Effects
on vocabulary learning and reading comprehension. Interactive Learning Environments, 26(1)https://doi.org/10.1080/10494820.2017.1282878.
Ponce, H. R., Mayer, R. E., & Lopez, M. J. (2013). A computer-based spatial learning strategy approach that improves reading comprehension and writing. Educational
Technology Research & Development, 61(5), 819–840. https://doi.org/10.1007/s11423-013-9310-9.
Porter, A., McMaken, J., Hwang, J., & Yang, R. (2011). Common Core Standards: The new U.S. intended curriculum. Educational Researcher, 40(3), 103–116. https://
doi.org/10.3102/0013189x11405038.
Rayner, K. (1998). Eye movements in reading and information processing: 20 years of research. Psychological Bulletin, 124(3), 372–422. https://doi.org/10.1037/
0033-2909.124.3.372.
Rayner, K., Chace, K. H., Slattery, T. J., & Ashby, J. (2006). Eye movements as reflections of comprehension processes in reading. Scientific Studies of Reading, 10(3),
241–255. https://doi.org/10.1207/s1532799xssr1003_3.
Reimer, Y. J., Brimhall, E., Cao, C., & O'Reilly, K. (2009). Empirical user studies inform the design of an e-notetaking and information assimilation system for students
in higher education. Computers & Education, 52(4), 893–913. https://doi.org/. https://doi.org/10.1016/j.compedu.2008.12.013.
Robinson, D. H., & Kiewra, K. A. (1995). Visual argument: Graphic organizers are superior to outlines in improving learning from text. Journal of Educational
Psychology, 87(3), 455–467. https://doi.org/10.1037/0022-0663.87.3.455.
Sweller, J., Ayres, P. L., & Kalyuga, S. (2011). Cognitive load theory. Explorations in the learning sciences, instructional systems and performance technologies. New York:
Springer.
Tsai, M.-J., Huang, L.-J., Hou, H.-T., Hsu, C.-Y., & Chiou, G.-L. (2016). Visual behavior, flow and achievement in game-based learning. Computers & Education, 98,
115–129. https://doi.org/https://doi.org/10.1016/j.compedu.2016.03.011.
Weinstein, C. E., & Mayer, R. E. (1986). The teaching of learning strategies. In M. C. Wittrock (Ed.). Handbook of research on teaching (pp. 315–327). (3rd ed.). New
York: Macmillan.
Wittrock, M. C. (1989). Generative processes of comprehension. Educational Psychologist, 24(4), 345–376. https://doi.org/10.1207/s15326985ep2404_2.

388