Educational Research Review: Sascha Schneider, Maik Beege, Steve Nebel, Günter Daniel Rey

Educational Research Review 23 (2018) 1–24
Contents lists available at ScienceDirect
Educational Research Review

journal homepage: www.elsevier.com/locate/edurev
A meta-analysis of how signaling affects learning with media

T
∗,1 1
Sascha Schneider , Maik Beege , Steve Nebel, Günter Daniel Rey
Psychology of Learning with Digital Media, Faculty of Humanities, Chemnitz University of Technology, Germany
AR TI CLE I NF O AB S T R A CT
Keywords: The signaling effect states that learners profit from cues that highlight the organization of specific
Cueing relevant information within materials. This meta-analysis includes 103 studies and N = 12,201
Signaling participants. 139 retention and 70 transfer performance measures were used to determine se-
Meta-Analysis parate mean effect sizes. Cognitive load, motivation/affect, learning time, and eye-tracking data
Learning with media
were included as dependent variables to explain possible effect mechanisms. Additionally, nine
Cognitive load
possible moderators (e.g., type of signaling) were identified. The retention (g+ = 0.53, 95% CI
[0.42, 0.64]) and transfer (g+ = 0.33, 95% CI [0.22, 0.43]) sizes support the positive effect of
signaling on motivation/affect, learning time, and learning-relevant fixations. Cognitive load was
significantly reduced. In contrast to the expertise reversal effect hypothesis, prior knowledge was
not identified as a moderator of the signaling effect. The results were interpreted using media
learning theories. Recommendations for future studies are included herein.
1. Introduction
Almost every learning situation involves media that include verbal and visual information, such as texts and pictures that must be
continuously analyzed and processed by learners. Moreover, in order to reach a learning goal, this information needs to be integrated
into a coherent mental model and stored in long-term memory (Mayer, 2014a). However, prior to the storage of this information, its
relevance must be determined (Awh & Jonides, 2001). This is a challenging task, as instructional environments increase in fidelity
and visual richness (e.g., Chen, Liu, & Hwang, 2015). Therefore, instructional designers use different methods to attract the learners’
attention and highlight important information (e.g., underlining) in order to improve learning outcomes. This principle is generally
referred to as signaling – the attempt to highlight organizational structures (e.g., main aspects or learning-relevant information) to
foster goal-oriented learning (Van Gog, 2014). This study summarized experimental studies on the advantages and disadvantages of
signaling features through a theoretical and meta-analytical review.
2. Theoretical background
2.1. The signaling principle and media design theories
The signaling principle, also known as the cueing principle, is based on the finding that people learn better when instructional
materials include cues that highlight relevant elements or the organization of the material (for reviews, see Corkill, 1992; De Koning,
Tabbers, Rikers, & Paas, 2007; Dodd & Antonenko, 2012; Richter, Scheiter, & Eitel, 2016; Spyridakis, 1989; and Van Gog, 2014). This
∗
Corresponding author. Psychology of Learning with Digital Media, Faculty of Humanities, Chemnitz University of Technology, Straße der Nationen 12, 09111
Chemnitz, Germany.
E-mail address: sascha.schneider@phil.tu-chemnitz.de (S. Schneider).
1
Contributed equally.
https://doi.org/10.1016/j.edurev.2017.11.001
Received 14 August 2017; Received in revised form 21 November 2017; Accepted 21 November 2017
Available online 23 November 2017
1747-938X/ © 2017 Elsevier Ltd. All rights reserved.
S. Schneider et al. Educational Research Review 23 (2018) 1–24
Fig. 1. Overview of possible signaling types.
principle is also referred to as the attention-guiding principle because most cues highlight information to attract attention. According
to Van Gog (2014), two main signaling modes can be distinguished: signals within texts and signals within pictures (including other
forms of graphics, like diagrams, videos, or animations). Textual signaling can be distinguished into five subtypes: organizational
signals (e.g., headings or summaries), colors (e.g., font colors), text-picture references (e.g., “see the picture”), intonation (e.g., in
auditory texts), or a mixture of types (e.g., coloring and text-picture referencing). Graphical signals can be subdivided into pointing
gestures (e.g., arrows or gestures of pedagogical agents), colors (e.g., parts of a picture), labels (e.g., naming parts of an animation),
flashing, spotlights (also called anti-cueing; e.g., greying out parts of an illustration), graphic organizers, or, again, a mixture of two
or more types (e.g., pointing gestures and labels). A graphical overview of signaling types is shown in Fig. 1.
Research on the signaling principle is based on a long tradition of experimental studies. First studies occurred in the early 1970s
(e.g., Cashen & Leicht, 1970), but the debate about new forms of signaling is ongoing (e.g., Gordon, Tindall-Ford, Agostinho, & Paas,
2016). Earlier studies focused on cues within texts (e.g., outlines; Glynn & Di Vesta, 1977), while the latest studies generally focus on
new signaling methods in, for example, video-based materials (e.g., Ouwehand, van Gog, & Paas, 2015a), in connection with ped-
agogical agents (e.g., Johnson, Ozogul, & Reisslein, 2015), or in virtual reality environments (e.g., Dodd & Antonenko, 2012).
The signaling principle can be based on two media design theories. According to the Cognitive Theory of Multimedia Learning
(CTML; Mayer, 2014a), learning is based on three assumptions about meaningful cognitive processes: (1) humans process information
via two channels (visual/pictorial & auditory/verbal), (2) each channel has a limited information processing capacity, and (3)
learning only takes place if cognitive processes are coordinated (active processing). Moreover, learning-relevant information needs to
be selected, organized and integrated into long-term memory (SOI-assumption of CTML). Since not all learning situations involve
teachers who monitor the learning progress, the selection of relevant information must be coordinated using attention-guiding
features (signals) within the learning material. However, not every signaling attempt is helpful; some guiding features actually
distract learners or fail to effectively convey meaning. In this case, signaling leads to additional cognitive processes that do not
contribute to goal-oriented learning. Within CTML, this is referred to as extraneous processing. In contrast, essential processing
defines the cognitive processes needed to represent a mental model of the instructional material, while generative processing sums up
all processes that try to make sense of and store information. In conclusion, CTML is not limited to multimedia instructional materials,
since even simple instructional materials like a learning text might evoke verbal and nonverbal mental models and can be transferred
via mental conversions (Mayer, 2014a).
The separation of cognitive processes postulated in the CTML, however, was based on the Cognitive Load Theory (CLT; e.g., Paas &
2
Sweller, 2014), which states that the overall cognitive capacity is limited and each additional cognitive process represents a separate
cognitive load. This theory includes three categories of cognitive load concerned with the acquisition, storage, and retrieval of
information. Intrinsic Cognitive Load (ICL) is based on the complexity of the information presented, while Extraneous Cognitive Load
(ECL) emerges through the design of instructional materials and is not directly connected with an increase in learning-relevant
processes. In contrast, Germane Cognitive Load (GCL) focuses on all learning-relevant processes which are needed transfer and store
information into the long-term memory system. ICL is material-dependent, determined by the material's element interactivity and is
commonly understood as information complexity. This complexity depends on a learner's domain-specific prior knowledge (Sweller,
1998). For example, learning single words of a foreign language requires a lower understanding of interacting elements than learning
phases of cell division. If working memory capacity is not exceeded by the total amount of ICL and ECL, learning (i.e., GCL processes)
can take place. In this connection it should be noted that more recent approaches of the CLT subsume all GCL processes within the
concept of ICL to combine both learning-relevant load types (Kalyuga & Singh, 2016). If cognitive capacity is exceeded, a cognitive
overload occurs which leads to a decreased learning performance.
As readers might not intuitively recognize which information in a new instructional material is learning-relevant and which might
be irrelevant for a learning goal, signaling can be used to shift the focus from learning-irrelevant, design-based information to goal-
based information. In conclusion, signaling reduces ECL and increases GCL processes, as shown in several signaling studies based on
the CLT (e.g., Amadieu, Mariné, & Laimay, 2011). However, signaling can also lead to additional cognitive processes because of the
additional costs of understanding the implemented cues’ meanings (e.g., Huk, Steinke, & Floto, 2010; Seufert & Brünken, 2006).
2.2. The impact of signaling on learning processes
Studies showed that signaling can enhance retention (e.g., Boucheix, Lowe, Putri, & Groff, 2013; Seufert & Brünken, 2006), and
transfer performance during problem-based tasks (e.g., Liu, Lin, & Paas, 2013; Lorch, Lemarié, & Grant, 2011). According to Mayer
(2014b), retention can be defined as “remembering” – recognizing or reproducing learning content. In contrast, the transfer
knowledge is defined as “understanding.” Learners need a coherent mental model to help them solve novel problems not explicitly
presented in the instructional material (Mayer, 2014b). In conclusion, this meta-analysis examined the effect of signaling on both the
retention and transfer learning performance.
However, the signaling techniques might be dependent from the setting of the learning material. In a situation where no time limit
is given, signaling should increase both the overall cognitive load and learning time, since learners may need extra time to process the
additional information, identify the structures and aims of signaling, and understand the meanings of combined elements (Richter
et al., 2016). Since cueing represents an additional information that needs to be processed, associations for the to-be-learned material
might increase because of highlighting nature of signaling (Kahana, 1996). For example, if the material incorporates picture-text
references (e.g., connecting lines between words and parts of the picture), readers need more time to analyze what is combined and
why. Additional gazes and transitions and increased learning-relevant eye fixations should, therefore, be further examined (Richter
et al., 2016). However, fixations should focus on relevant parts of the instructional material due to the definition of signaling as a
highlighter learning-relevant information.
This additional information (i.e., how to cope with signaling) might also affect motivational or affective states. For example,
studies showed that learners’ stress levels decreased (e.g., Skuballa, Schwonke, & Renkl, 2012), learners enjoyed their material to a
greater extent (e.g., Johnson et al., 2015), the attractiveness of the material increased (e.g., Huk, Steinke, & Floto, 2003), and the
intrinsic motivation of learners was enhanced (Lin, 2011). Even if signaling was shown to influence learning, cognitive load, learning
time, fixation times and also motivation/affect in most of the studies, several boundary conditions might affect the signaling effect.
2.3. Moderators of the signaling effect
2.3.1. The influence of prior knowledge

CTML, CLT as well as other theories of learning with media (e.g., Chi, De Leeuw, Chiu, & LaVancher, 1994; Graesser, Singer, &
Trabasso, 1994) consider learners’ knowledge bases to be the most important moderating influence. Some signals that are needed by
low prior knowledge learners in text-picture scenarios (for an overview, see Richter et al., 2016), might be redundant for high prior
knowledge learners due to their existing schema in long-term memory (Kalyuga, 2009). This expertise reversal effect (Kalyuga, 2007)
suggests that learners with a low prior knowledge profit more from highlighting cues than high prior knowledge learners since
learners with a low prior knowledge are hardly able to disregard irrelevant details and do not automatically focus on relevant
information (Johnson et al., 2015). In the case of a high prior knowledge, signaling techniques may contradict learners existing
mental representations of new material (Mayer, 2014b). This represents a conflict that must be solved and results in additional ECL
(Schnotz & Kürschner, 2007). McNamara, Kintsch, Songer, and Kintsch (1996) suggest that in contrast to a learner with a low prior
knowledge, learners with high prior knowledge need to process relationships between their existing knowledge bases and new
information. As the signaling effect is particularly sensitive to knowledge base differences (e.g., Arslan-Ari, 2013; Johnson et al.,
2015; Kriz & Hegarty, 2007), a similar pattern was expected for this meta-analysis.
2.3.2. Signaling mode

One of the more visible moderators of the signaling effect is the signaling mode, which is defined by the outer appearance of a
signaling feature. In line with Van Gog (2014), this meta-analysis divided the research field of signaling into two categories: signaling
in texts and signaling in graphics.
3
According to the literature, results across experiments with both textual and pictorial signaling techniques were shown to dif-
ferently affect learning. In a series of experiments by Glover et al. (1988) examining the effects of preview sentences (i.e. signaling in
texts) on learning texts in four experiments, results showed rather high effect sizes for comprehension and recall tests. In contrast, one
experiment by Loman and Mayer (1983) that used organizational signals (previews, summaries, and headings) revealed negative
impacts on learning. These contradictive results can also be seen in studies on graphical signals. A study by Jeung, Chandler, and
Sweller (1997) that used flashing appropriate diagram segments as a signaling mode (i.e. signaling in graphics) revealed detrimental
learning effect sizes in three experiments. In contrast, Ferrara and Butcher (2011) found that color-coding within a mind map was
highly effective. Therefore, it remains unclear whether signals are more or less effective when used within texts or graphics. Fur-
thermore, groups of graphical or textual cues might significantly vary in their effectiveness.
Regarding different types of textual cueing in detail (i.e., “organization highlighting,” “picture referencing,” “color coding,” and
“intonation; ” Van Gog, 2014), again, a multitude of results direct to the impression that one or another type is more effective. In the
study by Folker, Ritter, and Sichelschmidt (2005), picture referencing through color-grouping seemed ineffective as a signaling cue,
whereas in the experiment by Seufert and Brünken (2006), connecting lines as a phenotype of picture referencing resulted in a high
effect size. An even higher range of effect sizes can be seen within organizational highlighting cues; while advanced organizers in a
study by Titsworth and Kiewra (2004) resulted in high effect sizes, signaling words used in a study by Rickards, Fajen, Sullivan, and
Gillespie (1997) showed no enhancing effects. Regarding color, Fowler and Barker (1974) showed medium to high effect sizes with
colored underlining. In a study by Sung and Mayer (2012), five different signaling aids, an example of mixed signaling, generated a
high effect size. Overall, some text signaling types might be better than others. These signaling types were examined separately in the
meta-analysis. Since signaling might also occur in auditory texts (e.g. intonation of words), this type was also included. However,
only one study (to the best of our knowledge) investigated how intonation alone affects learning results (Mautone & Mayer, 2001); it
revealed a medium effect size for this signaling technique.
A similar pattern can be drawn from the data of all pictorial signaling types (i.e., “pointing gestures,” “color coding,” “picture
labeling,” “flashing,” “spotlight,” and “graphic organizers; ” Van Gog, 2014). Pointing gestures in a study by Ouwehand, van Gog, and
Paas (2015b) ranged from zero to medium effect sizes. Effect sizes for color signaling studies ranged from small negative (Arslan-Ari,
2013) to large (Ferrara & Butcher, 2011). Similar ranges were found for spotlights (e.g., De Koning, Tabbers, Rikers, & Paas, 2010b;
Jarodzka, van Gog, Dorr, Scheiter, & Gerjets, 2013), flashing (e.g., Jeung et al., 1997), and labeling (e.g., Huk et al., 2010; Johnson,
Butcher, Ozogul, & Reisslein, 2014).
2.3.3. Differences in instructional materials

Different aspects within instructional materials may also influence the signaling effect. Prior meta-analyses have shown that one
major influence might be the pacing of the materials (either system-paced or learner-paced; Ginns Martin, & Marsh, 2013). While a
learner-paced environment requires learners to time-independently discover their materials, system-paced materials provide a pre-
defined time. Therefore, learner-paced materials increase the demands of instructional control decisions and exploration of the
instructional material. This should add additional ECL to a learner's overall load (Carolan, Hutchins, Wickens, & Cumming, 2014),
which could lead to cognitive overload. With regards to signaling, learners in a self-paced scenario could, therefore, profit more from
additional instructional support. Signaling in a system-paced instructional material requires controlled attention in order to dis-
criminate between relevant and irrelevant information since too much time on irrelevant details might be even more harmful in a
limited time. Since this constant load is reflected as additional cognitive load, an attention-guiding support like signaling might also
help reduce the overall cognitive load (Karich, Burns, & Maki, 2014).
A second significant feature of instructional materials is their modality, defined as the number of human senses that is needed to
perceive all information of an instructional material. According to a review by Turk (2014), multimodal (more than one sense
needed) learning technologies offer more advantages in terms of natural and efficient interactions. One explanation is based on CLT's
presumptions of Baddeley's Model of Working Memory (1992), which claims that auditory and visual information is processed in two
distinct subsystems. Thus, multimodal instructions with signaling could be effective because resources from two subsystems can be
used to process the instructional material simultaneously (Low, Jin, & Sweller, 2011) and each subsystem can additionally be relieved
due to signaling techniques. However, multiple modes of presentation might also result in an additional overall cognitive load
(through further demands upon attention) and thus impair cognitive processes (Guichon & McLornan, 2008). Nonetheless, a reduced
ECL due to the inclusion of signaling cues should further promote the superiority of multimodality.
Following the suggestion of Leahy and Sweller (2005), the permanence of information in instructional materials is an important
moderator of design effects. If information is permanently accessible, for example in written texts, learners can easily switch between
different sources of information without any need of memorizing single information chunks. In the case of a transient information
presentation, for example in auditory texts or animations, the complexity increases since learners need to memorize each information
chunk in order to compare this information with all following information chunks. This adds additional cognitive load and might be
harmful to learning (Castro-Alonso, Ayres, & Paas, 2014). In both cases (permanent and transient materials), signaling can be seen as
a learning-enhancing feature.
On the one hand, signaling might be more effective when used in transient materials, because the cues help learners detect
important information, organize stimuli, and build coherent models (Kühl, Scheiter, & Gerjets, 2012), as important information might
be more difficult to find in continuous presentations of information (e.g., in animations).
On the other hand, signaling information in transient materials are rather brief in nature and can easily be missed. In detail,
signals might disappear within transient stimuli due to the overwhelming input of moving elements (De Koning et al., 2010b).
Although both categories of permanence have advantages and disadvantages, the extensive information of transient stimuli should
4
enhance the effectiveness of signaling as this effect is primarily defined by highlighting important information. Most reviews and
meta-analyses within the learning sciences underline the importance of the instructional material domain (e.g., biology, mathematics)
as a moderator of instructional design principles (e.g., D'Mello, 2013; Van der Kleij, Feskens, & Eggen, 2015). In addition, some
researchers consider the age differences between learners when examining the effects of instructional designs as an important
moderator (e.g., Van Gerven, Paas, Van Merriënboer, & Schmidt, 2006). Since age might not accurately reflect educational maturity,
the educational level (e.g., primary school, university) might provide more information on the effectiveness of signaling in different
stages of life. All these moderators were included in the present meta-analysis.
2.4. Research questions and hypotheses
Although some meta-analyses already exist in the field of signaling research, each of these analyses focused on either a different
media (i.e., animations vs. pictures; Höffler & Leutner, 2007) or combination of media codes (i.e., text-picture combinations; Richter
et al., 2016). To the best of our knowledge, the only review of overall signaling effect was by Spyridakis in 1989, meaning that there
are 27 years of missing data. This study aimed to collect all experimental data on the signaling effect by examining learning effects,
cognitive load, motivation/affect, learning time, and learning-relevant eye fixations. Furthermore, the literature-based moderators
were investigated to identify differences in signaling effect conditions.
Based on the literature review, the following hypothesis for the overall signaling effect was postulated:
H1. Overall effect: Learners with signaled instructional materials will achieve higher retention and transfer scores than learners
without signaled materials.
In addition, moderators were assumed according to the literature. With regards to prior knowledge, signaling should be more
effective for a learner with low prior knowledge:
H2. Expertise reversal effect: Learner with low prior knowledge profit more from signaled instructional materials than learners with
high prior knowledge.
The range of signaling modes was shown to be very diverse with no clear effect direction among all modes. In conclusion, the
hypothesis for signaling mode was kept omnidirectional:
H3. Signaling mode: There are differences in the effectiveness of signaling between all signaling modes.
Since no clear effect direction was found for the moderator pacing and two effect direction were shown, two contrasting hy-
potheses were postulated:
H4a. Pacing: The effectiveness of signaling is increased by system-paced learning materials in contrast to learner-paced materials.
H4b. Pacing: The effectiveness of signaling is increased by learner-paced learning materials in contrast to system-paced materials.
Since signaling might be more enhancing for transient materials, the following moderator hypothesis was postulated:
H5. The effectiveness of signaling is enhanced for transient materials in contrast to permanent materials.
Since instructional material domains might vary in their complexity of information presentation, signaling was postulate to differ
in its effectiveness.
H6. Instructional material domain: The effectiveness of signaling in instructional materials varies across the instructional material
domains.
Since the educational level is strongly correlated with domain-specific prior knowledge, younger samples should profit more from
signaling than older samples:
H7. Educational level: Younger samples benefit more from signaling in terms of retention and transfer scores than older samples.
3. Method
3.1. Data collection and inclusion criteria
The following keywords and combinations were used to search for relevant literature via Google Scholar and other relevant
databases (ERIC, PsychINFO, PsychARTICLES, and ScienceDirect): “cueing,” “cueing effect,” highlighting,” “attention cueing,”
“signal,” “signal(l)ing,” “organizational signals,” “graphic effects,” “(typo)graphical cues,” “visual cues,” “navigational aids,” “signal
(l)ing aids,” “encoding cues,” “across-chapter signals,” “summary signals,” “importance signals,” “color coding,” “visual indicators,”
and “verbal guidance.” Authors were contacted personally if relevant studies were unavailable online. Furthermore, grey literature
was searched to avoid publication bias. Academic theses, dissertations, conference papers, and conference posters were searched
systematically. Published literature was searched for additional studies or inspiration for additional research. Finally, authors were
contacted to obtain additional data. The study search ended on November 23, 2016.
After excluding studies that were not (a) signaling themed or (b) in English or German (due to a lack of language skills necessary
5
to read and interpret literature in other languages) and those (c) published more than once or (d) unavailable after contacting the
authors, 150 studies were identified and reviewed using the following inclusion criteria:
(1) Each study had to include at least one experiment. Studies consistently had between-subject designs; within-subject designs were
not excluded explicitly, but none were identified. Theoretical works (e.g., Mayer, 2010) were excluded from the meta-analysis.
(2) Each study had to investigate media learning. Investigations that did not examine learning or compute learning outcomes (e.g.,
Gozli, Aslam, & Pratt, 2015) were not included.
(3) Each study had to compare a control condition to at least one other condition using signaled instructional materials. The control
condition had to be exactly equivalent to the experimental condition and only the implementation of signaling varied. Studies
that compared signaling conditions without a control group (e.g., Fischer & Schwan, 2010) were excluded from the analysis.
(4) Statistical data had to be reported to calculate effect size. Several investigations that did not report enough statistical information
(e.g., means, test value, effect sizes; e.g., Coyan, Busch, & Reynolds, 2010) were excluded.
(5) The signaling techniques of the studies had to fit the definition of signaling (based on Van Gog, 2014) and the results had to be
traceable to the experimental manipulation of signaling. For example, studies that defined hyperlinks as a signaling technique
(Cuddihy & Spyridakis, 2012) or manipulated signaling and interactivity simultaneously (Sung & Mayer, 2013) were not included
in the meta-analysis.
One-hundred-and-three investigations fulfilled the outlined criteria and remained in the meta-analysis. All excluded studies are
summarized in Appendix A, which can be found in the supplemental material.
3.2. Study features
The characteristics of the included studies are shown in Table 1. Variables were coded by two scientific researchers who were pre-
trained. First, the educational level was described in detail. Then, to classify moderator variables, each sample was coded as “primary
students,” “secondary students,” “university students,” or “adult education.”
In most cases, the comparison sample size included all experimental group participants. In a few studies, experimental conditions and
therefore participants had to be excluded. Exclusions were made when there was no manipulation in terms of signaling within the
experimental group (e.g., McTigue, 2009) or a signaling condition did not match the definition of signaling (e.g., Skuballa et al., 2012).
Domain-specific prior knowledge was coded as “high prior knowledge” or “low prior knowledge. Information provided by the
authors of each study was used to classify prior knowledge. In the majority of the cases, no explicit information was specified;
therefore, statistical data from these studies were used to code prior knowledge. The mean percentage of correct answers in relation
to the maximum value reached in the prior knowledge test was computed for each study. The 50% mark was used to separate low and
high prior knowledge. If participants scored below 50% of a prior knowledge maximum score, they were designated as “low prior
knowledge” learners. If participants scored above 50%, they were classified as “high prior knowledge” learners. In some studies,
neither explicit information nor statistical data regarding prior knowledge was provided. These studies were assigned to a “not
available” category. Nevertheless, it could be problematic to use the pretest to calculate and code prior knowledge since the results of
the prior knowledge test strongly depend on the (arbitrary) difficulty of the questions. Since specific tests or questions were not
available for the current meta-analysis, results from these test might not be comparable to several studies. Therefore, an additional
analysis was conducted in which statistical data was not used for coding prior knowledge. For this second calculation, only explicit
information provided by the authors were used to code prior knowledge and the other studies were assigned to the “not available”
category.
Learner guidance was categorized in terms of how the instructional material was paced (“system-paced” vs. “learner-paced”). If
participants had no control over the instructional material or were unable to set the pace, the “system-paced” category was used. If
learners could modify the pace of the presentation (start, pause, stop, repeat sections, etc.), the material was categorized as “learner-
paced.”
An adapted version of the categorization used by Richter et al. (2016) was used to code the instructional material domains. The
domains were separated into “biology and ecology”; “chemistry”; “math and logic”; “physics and mechanics”; “geography and
geology”; “psychology”; “education” and “other.” The term “instructional material domain” was preferred over the term “subject
areas” because the contents of various instructional materials which were used in the included investigations did not reflect classical
subjects (e.g., logic, education, or mechanics).
The categorization used by Van Gog (2014) was used to code the signaling mode. Signaling could be implemented within the text
or via permanent or transient visualizations. Textual signals were divided into the following cues: “highlights the organization” of the
text, “picture referencing,” “color,” “intonation,” and “mixed.” Signals within a picture or animation were divided into “pointing
gestures” (arrows or pointing pedagogical agents), “color,” “labeling,” “flashing,” “spotlight,” “graphic organizers,” and “mixed.”
Furthermore, studies were categorized according to information permanence. If the presentation of the instructional material did
not change dynamically during learning, the information was coded “permanent” (i.e., permanent texts or graphics). When a pre-
sentation was subject to transient changes the information was categorized as “transient” (e.g., videos, animations, audio tracks, etc.).
Finally, studies were categorized in terms of publication date. It was distinguished whether the article was published before the
year 2000 or from the year 2000 onwards. The categorization of the moderator variables was conducted by two raters (Cohens κ
ranging from 0.63 to 0.81; Cohen, 1960). Non-congruent decisions between the two raters were discussed until a clear decision and
mutual agreement was reached.
6
Table 1
Overview of all meta-analysis-included studies.
Study Sample Comparison Mean Proportion of Prior Learner Instructional Signaling Signaling mode Signaling Type of Permanence
sample size age (in females knowledge guidance material domain mode (Text) mode Performance
S. Schneider et al.
years) based on pre- (Graphic) Measure

tests
Amadieu et al. (2011) university 36 22.6 0.83 high system-paced biology graphic spotlight retention, transient
students ecology transfer
Arslan-Ari (2013) university 105 20 0.53 low system-paced biology graphic color retention, transient
university 95 20 0.53 high system-paced biology graphic color retention, transient
Barley (2016) university 154 20.94 0.46 n. a. system-paced physics text organization retention permanent
students mechanics highlighting
Berthold and Renkl secondary 85 16.21 0.51 low learner-paced math logic text picture retention, permanent
(2009) students referencing transfer
Boucheix and Guignard mixed 117 n. a. n. a. n. a. mixed physics graphic mixed retention, permanent
(2005) mechanics transfer
Boucheix and Lowe university 57 20.7 0.91 low learner-paced physics graphic mixed retention transient
(2010) students mechanics
Boucheix et al. (2013) university 84 19.1 0.94 low learner-paced physics graphic mixed retention transient
students mechanics
Brooks, Dansereau, university 132 n. a. n. a. low learner-paced other text organization retention permanent
Spurlin, and Holley students highlighting
(1983)
7
Cashen and Leicht university 40 n. a. n. a. n. a. learner-paced psychology text organization retention permanent
(1970) students highlighting
Cheon and Grant university 80 n. a. 0.69 high learner-paced other graphic mixed retention transient
(2012) students
Craig, Twyford, adult 77 n. a. n. a. n. a. system-paced physics graphic pointing retention, transient
Irigoyen, and Zipp education mechanics gestures transfer
(2015)
Crooks, Cheon, Inan, university 135 19.5 0.50 low learner-paced physics graphic pointing retention transient
Ari, and Flores students mechanics gestures
(2012)
Crouse and Idstein university 36a n. a. n. a. n. a. system-paced other text organization retention permanent
university 66 n. a. n. a. n. a. system-paced education text organization retention permanent
students highlighting
Coles and Foster university 20 n. a. n. a. n. a. system-paced biology text organization retention permanent
(1975) students ecology highlighting
university 38 n. a. n. a. n. a. system-paced biology text organization retention permanent
students ecology highlighting
university 41 n. a. n. a. n. a. system-paced biology text organization retention permanent
(continued on next page)
Table 1 (continued)
S. Schneider et al.
tests
Cook, Friedman, primary 38 9.00 0.52 low system-paced math logic graphic pointing retention, transient
Duggan, Cui, and students gestures transfer
Popescu (2016)
Corkill, Bruning, and university 44 n. a. n. a. low system-paced physics text organization retention permanent
Glover (1988) students mechanics highlighting
university 47 n. a. n. a. low system-paced other text organization retention permanent
De Jong and Van Der university 46 n. a. n. a. low learner-paced physics graphic graphic retention permanent
Hulst (2002) students mechanics organizers
De Koning et al. (2007) university 40 26 0.75 n. a. system-paced biology graphic spotlight retention, transient
De Koning, Tabbers, university 40 21.43 0.68 high system-paced biology graphic spotlight retention, transient
Rikers, & Paas students ecology transfer
(2010a)
De Koning, Tabbers, university 76 20.38 0.74 high system-paced biology graphic labeling retention, transient
Rikers, & Paas students ecology transfer
(2010b)
De Koning, Tabbers, university 84 19.98 0.89 low system-paced biology graphic spotlight retention, transient
Rikers, and Paas students ecology transfer
(2011a)
8
De Koning, Tabbers, secondary 90 14.5 0.30 low system-paced biology graphic spotlight retention, transient
Rikers, and Paas students ecology transfer
(2011b)
Doolittle and university 105 19.4 0.3 n. a. system-paced physics graphic spotlight retention, permanent
Altstaedter (2009) students mechanics transfer
Ferrara and Butcher university 44 n. a. n. a. high learner-paced geography graphic color retention, permanent
(2011) students geology transfer
Fiorella and Mayer university 53b 19.1 0.71 low system-paced physics graphic pointing retention, transient
(2015) students mechanics gestures transfer
university 52c 19.1 0.71 low system-paced physics graphic pointing retention, transient
students mechanics gestures transfer
d
Florax and Ploetzner university 132 21.9 0.57 low system-paced biology graphic color retention, permanent
(2010) students ecology transfer
Folker et al. (2005) university 20 24.3 0.7 n. a. learner-paced biology text picture retention permanent
students ecology referencing
Fowler and Barker university 38 n. a. n. a. n. a. learner-paced psychology text color retention permanent
(1974) students
Table 1 (continued)
S. Schneider et al.
tests
Glover et al. (1988) university 26 n. a. n. a. n. a. learner-paced other text organization retention permanent
university 16 n. a. n. a. n. a. learner-paced other text organization retention permanent
Glynn & Di Vesta university 120 n. a. n. a. n. a. learner-paced geography text organization retention, permanent
(1977) students geology highlighting transfer
Golding and Fowler university 108 n. a. n. a. n. a. learner-paced geography text organization retention permanent
(1992) students geology highlighting
university 80 n. a. n. a. n. a. learner-paced geography text organization retention permanent
students geology highlighting
e
Gordon et al. (2016) primary 53 n. a. n. a. low system-paced physics graphic color retention, permanent
students mechanics transfer
Hartley and Trueman secondary 116 14.5 n. a. low learner-paced psychology text organization retention permanent
9
secondary 116 14.5 n. a. low learner-paced psychology text organization retention permanent
secondary 116 14.5 n. a. low learner-paced psychology text organization retention permanent
Harp and Mayer (1998) university 96 n. a. n. a. n. a. learner-paced physics text organization retention, permanent
students mechanics highlighting transfer
Hayes and Reinking secondary 277 n. a. n. a. n. a. learner-paced mixed text organization retention permanent
Holley et al. (1981) university 90 n. a. n. a. n. a. learner-paced other text organization retention permanent
Huk et al. (2003) secondary 129 n. a. n. a. n. a. system-paced biology graphic mixed retention transient
students ecology
Huk et al. (2010) university 83 22 0.65 n. a. system-paced chemistry graphic labeling retention, permanent
students transfer
university 164 22 0.65 n. a. system-paced chemistry graphic labeling retention, permanent
students transfer
Hyönä & Lorch (2004) university 59 n. a. n. a. n. a. learner-paced physics text organization retention permanent
Jamet (2014) university 32 22.4 0.69 low system-paced education graphic color retention, transient
students transfer
Jamet and Fernandez university 51 20.94 0.46 low learner-paced other text organization retention, transient
(2016) students highlighting transfer
Jamet, Gavota, and university 102 21.9 0.80 low system-paced biology graphic color retention, permanent
Quaireau (2008) students ecology transfer
Table 1 (continued)
S. Schneider et al.
tests
Jarodzka, van Gog, university 50f 22.83 0,67 low system-paced biology graphic spotlight transfer transient
Dorr, Scheiter, & students ecology
Gerjets (2013)
Jeung et al. (1997) primary 60 n. a. n. a. n. a. learner-paced math logic graphic flashing retention, permanent
students transfer
primary 30 n. a. n. a. n. a. learner-paced math logic graphic flashing retention, permanent
students transfer
primary 30 n. a. n. a. n. a. learner-pace math logic graphic flashing retention, permanent
students transfer
Johnson, Butcher, university 98 26.2 0.89 low learner-paced physics graphic color retention, transient
Ozogul, and students mechanics transfer
Reisslein (2013)
Johnson et al. (2014) university 126 23.36 0.75 high learner-paced physics graphic labeling retention, permanent
Johnson, Ozogul, secondary 297 12.8 0.57 low mixed physics graphic pointing retention transient
Moreno, and students mechanics gestures
Reisslein (2013)
Johnson et al. (2015) secondary 127g 12.5 0.50 low system-paced physics graphic pointing retention, transient
secondary 123h 12.5 0.50 high system-paced physics graphic pointing retention, transient
10
Kalyuga et al. (1999) adult 16 n. a. n. a. low learner-paced physics graphic color retention permanent
education mechanics
Kardash and Noel university 92 n. a. 0.81 n. a. learner-paced physics text organization retention permanent
(2000) students mechanics highlighting
Kornalijnslijper (2012) university 60 21 0.67 n. a. learner-paced mixed text picture retention permanent
students referencing
Kriz and Hegarty university 40i n. a. n. a. low learner-paced physics graphic pointing retention transient
(2007) students mechanics gestures
university 20 n. a. n. a. low learner-paced physics graphic pointing retention transient
students mechanics gestures
Krug, George, Hannon, university 62 n. a. n. a. n. a. learner-paced other text organization retention permanent
and Glover (1989) students highlighting
university 56 n. a. n. a. n. a. learner-paced psychology text organization retention permanent
university 60 n. a. n. a. n. a. learner-paced psychology text organization retention permanent
Kühl (2011) university 150 22.47 0.81 high system-paced physics graphic mixed retention, transient
Kühl et al. (2012) university 50 22.47 0.81 high system-paced physics graphic mixed retention, transient
university 50 22.47 0.81 high system-paced physics graphic mixed retention, transient
university 50 22.47 0.81 high system-paced physics graphic mixed retention, transient
Table 1 (continued)
S. Schneider et al.
tests
Lin (2011) university 126 21.69 0.58 low learner-paced biology graphic pointing retention transient
students ecology gestures
Lin and Atkinson university 112 25.57 0.49 high learner-paced geography graphic pointing retention transient
(2011) students geology gestures
Liu et al. (2013) primary 74 11 0.47 high learner-paced biology graphic pointing retention, permanent
students ecology gestures transfer
Loman and Mayer secondary 58 16 n. a. n. a. learner-paced biology text organization retention, transient
(1983) students ecology highlighting transfer
secondary 44 16 n. a. n. a. learner-paced geography text organization retention, transient
students geology highlighting transfer
Lorch et al. (2011) university 39 n. a. n. a. n. a. learner-paced physics text organization transfer permanent
university 35 n. a. n. a. n. a. learner-paced physics text organization transfer permanent
Lorch and Chen, 1986 university 120 n. a. n. a. n. a. learner-paced other text organization retention permanent
Lorch and Lorch (1986) university 75 n. a. n. a. n. a. learner-paced geography text organization retention permanent
11
university 80j n. a. n. a. n. a. learner-paced geography text organization retention permanent
Lorch, Lorch, and university 202 n. a. n. a. n. a. learner-paced other text organization retention permanent
Inman (1993) students highlighting
Lorch, Lorch, Ritchey, university 73 n. a. n. a. n. a. system-paced geography text organization retention permanent
McGovern, and students geology highlighting
Coleman (2001) university 104 n. a. n. a. n. a. system-paced geography text organization retention permanent
university 295 n. a. n. a. n. a. system-paced geography text organization retention permanent
Lorch, Lorch, and university 124 n. a. n. a. n. a. learner-paced geography text organization retention permanent
Klusewitz (1995) students geology highlighting
Table 1 (continued)
S. Schneider et al.
tests
Luo, Koszalka, and Zuo university 8 30.13 0.38 n. a. system-paced education mixed mixed mixed retention transient
(2016) students
k
Mason, Pluchino, and secondary 36 11.10 0.57 low system-paced physics graphic color retention, permanent
Tornatora (2013) students mechanics transfer
Mautone and Mayer university 48 18.66 0.62 n. a. learner-paced physics text organization retention, permanent
(2001) students mechanics highlighting transfer
university 48 18.55 0.73 n. a. system-paced physics text intonation retention, transient
university 86 18.54 0.87 n. a. system-paced physics mixed intonation mixed retention, transient
Mautone and Mayer university 102 n. a. 0.44 low learner-paced geography graphic mixed transfer permanent
(2007) students geology
Mayer, Dyck, and Cook university 30 n. a. n. a. low learner-paced chemistry text organization retention, permanent
(1984) students highlighting transfer
Meyer, Brandt, and secondary 122 n. a. n. a. n. a. system-paced other text organization retention permanent
Bluth (1980) students highlighting
Meyer and Poon (2001) adult 111 n. a. 0.62 low system-paced other text organization retention permanent
education highlighting
l
McTigue (2009) secondary 155 12 0.44 high learner-paced biology graphic labeling retention permanent
students ecology
12
Moreno (2007) university 121m 25.29 0.74 n. a. system-paced education graphic color retention, transient
students transfer
university 114n 25.03 0.73 n. a. system-paced education graphic color retention, transient
students transfer
Moreno et al. (2010) secondary 104 12.97 0.58 low learner-paced physics graphic pointing transfer transient
students mechanics gestures
Murray and McGlone university 62 n. a. n. a. n. a. system-paced other text organization retention permanent
university 62 n. a. n. a. n. a. system-paced other text organization retention permanent
Nevid and Lampmann university 80 n. a. 0.76 n. a. learner-paced psychology text organization retention permanent
Ouwehand et al. university 23o 22.7 0.59 n. a. system-paced math logic graphic pointing retention, transient
(2015a) students gestures transfer
Ouwehand, van Gog, & primary 92 11.34 0.55 n. a. mixed math logic graphic pointing retention, transient
Paas (2015b) students gestures transfer
university 59 20.73 0.81 n. a. mixed math logic graphic pointing retention, transient
students gestures transfer
adult 83 67.35 0.66 n. a. mixed math logic graphic pointing retention, transient
education gestures transfer
Table 1 (continued)
S. Schneider et al.
tests
Ozcelik, Arslan-Ari, university 37 21.63 0.58 low system-paced physics graphic color retention, transient
and Cagiltay students mechanics transfer
(2010)
Ozcelik, Karakus, university 52 19.45 0.29 low learner-paced biology graphic color retention, permanent
Kursun, and students ecology transfer
Cagiltay (2009)
Reinking, Hayes, and secondary 167 n. a. n. a. n. a. learner-paced other text organization retention permanent
McEneaney (1988) students highlighting
Reisslein, Johnson, and university 113 22.04 0.81 low learner-paced biology graphic mixed retention, transient
Reisslein (2015) students ecology transfer
Rey (2010) secondary 74 14.6 0.47 n. a. system-paced physics graphic color retention, transient
Rickards et al. (1997) university 40 n. a. n. a. n. a. learner-paced biology text organization retention transient
university 166 n. a. n. a. mixed biology text organization retention permanent
Roberts (2008) university 142 n. a. 0.15 n. a. system-paced physics graphic color retention transient
students mechanics
Scheiter and Eitel university 35 24.94 0.54 n. a. learner-paced biology mixed mixed mixed retention, permanent
13
Scheiter and Eitel university 55 n. a. 0.71 high system-paced biology graphic mixed retention, permanent
university 78 n. a. 0.71 high system-paced biology graphic mixed retention, permanent
Schrader and Rapp secondary 98 n. a. 0.51 n. a. learner-paced physics graphic labeling retention transient
(2016) students mechanics
Seufert and Brünken university 88 25.6 0.72 high learner-paced chemistry text picture retention permanent
(2006) students referencing
Skuballa et al. (2012) university 44p 22.42 0.73 low system-paced physics graphic spotlight retention transient
students mechanics
Spyridakis and Standal mixed 371 n. a. n. a. high learner-paced chemistry text organization transfer permanent
(1987) highlighting
q
Stull and Mayer (2007) university 105 19.4 0.71 n. a. learner-paced biology graphic graphic retention, permanent
students ecology organizers transfer
university 78r 18.8 0.60 n. a. learner-paced biology graphic graphic retention, permanent
university 65s 19.3 0.66 n. a. learner-paced biology graphic graphic retention, permanent
Sung and Mayer (2012) university 122 23.36 0.72 high learner-paced education text mixed retention, permanent
students transfer
Surber and Schroeder university 27t n. a. n. a. high learner-paced biology text organization retention permanent
(2007) students ecology highlighting
Table 1 (continued)
S. Schneider et al.
tests
Tabbers, Martens, and university 151 n. a. n. a. n. a. learner-paced education text picture retention, experimen-
van Merriënboer students referencing transfer tally varied
(2000)
Tabbers, Martens, van university 111 n. a. 0.86 low learner-paced education graphic color retention, transient
Merriënboer students transfer
(2004)
Titsworth and Kiewra university 60 22.58 0.53 n. a. system-paced psychology text organization retention transient
Van Oostendorp, university 30u n. a. 0.16 n. a. learner-paced other graphic color retention, transient
Beijersbergen, and students transfer
Solaimani (2008)
Yang (2016) university 169 19.30 0.82 n. a. system-paced biology graphic mixed retention, n. a.
Yung and Paas (2015) secondary 133 12 0.50 n. a. system-paced biology graphic pointing retention transient
students ecology gestures
a
“Generate Questions” and “Read Questions” conditions were not included.
b
Only LPLs were included. The “draw” condition was not included.
c
Only HPLs were included. The “draw” condition was not included.
d
“Integrated” condition was not included.
14
e
“Self-managed” condition was not included.
f
“Dot” condition was not included.
g
Only LPLs were included.
h
Only HPLs were included.
i
“Diagram” condition was not included.
j
“Half signals” condition was not included.
k
“Text only” condition was not included.
l
“Control” condition was not included.
m
n
o
„Gesture + Gaze cue” condition was not included.
p
“Instructed group” was not included.
q
“Learner Generated” condition was not included.
r
s
t
Only HPLs were included.
u
“Static condition” was not included.
3.3. Outcome measures
To obtain detailed insight into the effects of signaling on instructional materials, several outcome measures were investigated.
Outcome measures and study-specific effect sizes are shown in Appendix B (found in the supplemental material).
Performance measures were separated into “retention” and “transfer” performance. Inter-coder reliability of the assessment of
retention (κ = 0.77) and transfer (κ = 0.81) was high. Furthermore, CL, a key predictor of learning processes examined in numerous
studies, was investigated (Kalyuga & Singh, 2016; Sweller, 1988; Sweller, Ayres, & Kalyuga, 2011). Most of the included studies did
not examine cognitive load as a global construct. For example, studies measured perceived difficulty (Amadieu et al., 2011), GCL
(Cheon & Grant, 2012), or “subjective cognitive load” (Lin & Atkinson, 2011). Therefore, different sub-constructs were combined to
examine the general influence of signaling on CL. If learning-obstructive sub-constructs (e.g., perceived difficulty, subjective cog-
nitive load, etc.) were reduced by signaling, the effect sizes conducted were received positively in the main cognitive load effect size.
If learning-conducive sub-constructs (e.g., GCL) were reduced by signaling, the effect sizes conducted were received negatively in the
main cognitive load effect size. Inter-coder reliability of the assessment (two raters) was high (κ = 0.79).
Many of the collected studies investigated the affective and motivational effects of signaling in media environments (e.g., Huk
et al., 2010; Kardash & Noel, 2000; Kalyuga, Chandler, & Sweller, 1999; Kühl, 2011). However, diverse emotional and motivational
constructs were examined. For example, studies computed intrinsic motivation (Lin & Atkinson, 2011), affective rating (Kalyuga
et al., 1999), and attitude (Huk et al., 2010). Therefore, motivation/affect was investigated as a global construct in which all effect
sizes were aggregated. Otherwise, too many sub-constructs with too few effect sizes would have been integrated, whereby gen-
eralization of the results would be restricted. Inter-coder reliability of the assessment (two raters) was high (κ = 0.69).
Time measures were included as dependent variables. In order to examine the time, it took learners to process signaled in-
structional material versus non-signaled instructional material, effects for overall learning times were investigated. Furthermore,
effects for fixations on AOIs were included to differentiate how long learners dealt with specific information. Only the fixation areas
that were part of the signaling manipulation were compared. Thus, the areas relevant for learning and effect sizes reflect the mean
difference between the fixation times for the signaled information and corresponding non-signaled information. The time reported in
the original studies and measured in seconds, was used to calculate effect sizes for both time measures.
3.4. Analysis methods
The implementation and statistical evaluation of the meta-analysis were based on Field and Gillett's approach (2010). A small
sample-adjusted standardized mean difference (Hedges' g for effect sizes from single investigations, Hedges g+ for aggregated effect
sizes; e.g., Hedges & Olkin, 1985) was chosen as the standard effect size. The interpretation of the effect size was based on Hattie's
study (2008), which investigated over 800 meta-analyses. For educational achievements, values of d = 0.20, 0.40, and 0.60 were
used to describe small, medium, and large effects, respectively. These conventions were adapted to interpret values of Hedges' g+.
Effect sizes were positive when the signaling technique had a positive effect (higher learning score, reduced CL, higher motivation/
affective score, shorter learning time, longer fixations on AOIs). The effect sizes of all pair-wise comparisons were computed using the
means and standard deviations reported in the studies. For each outcome measure, only one mean effect size was computed per
experiment. Thus, the aggregated effect sizes from all the studies were independent even if the effect sizes within the studies were
dependent (Hedges, Tipton, & Johnson, 2010). When standard deviations were not reported and only means were displayed, test
scores (t or F values) were used to compute the average standard deviation. If only t or F values were reported and means and
standard deviations were not presented, these t and F values (or the corresponding p-values) and sample sizes were used to calculate
the effect sizes and the standard errors, using the Practical Meta-Analysis Effect Size Calculator (Wilson, 2001). If only the in-
formation “there was no significant difference between the experimental and control conditions” was given (F < 1; p < 0.05), a
null effect was adopted (g = 0) and a conservative high standard error was assumed (SE = 0.60). The computed effect sizes and
standard errors were compiled into SPSS 23.0 (IBM Corp, 2015).
Signaling techniques, signaling conditions, and the media under which signaling was operationalized varied significantly across
the studies. Therefore, a random-effects model was preferred to a fixed-effect model (Hedges & Vevea, 1998). This approach is based
on Field and Gillett (2010), who recommended a random-effects model be used in social sciences. Each computed effect size was
standardized by the inversed squared standard error to increase the weighting of studies with larger sample sizes (e.g., Cooper,
Hedges, & Valentine, 2009). The SPSS scripts “MetaES” and “MetaF” (Lipsey & Wilson, 2001; Wilson, 2010) were used to aggregate
effect sizes.
The publication bias analysis was carried out using two methods. First, funnel plots were conducted and observed (cf. Sterne,
Becker, & Egger, 2005). Additionally, the rank correlation was computed (Begg & Mazumdar, 1994).
4. Results
4.1. Sample analysis
One-hundred-and-three investigations and k = 145 pair-wise comparisons were included in the meta-analysis. The overall sample
size amounted to N = 12,201 (N = 6809 for the signaling condition). The sample sizes varied from N = 8 to N = 300. The mean
sample size was N = 84.14 (SD = 55.83). Pairwise comparisons are outlined in Appendix B, separated by outcome measure. The
studies were published between 1970 and 2016 and include 131 journal articles, six doctor theses, six conference papers, one book
15
chapter, and one online publication. The participants were comprised of primary students (4.9%), secondary students (13.9%),
university students (78.5%), and adults (2.8%). The language of the instructional material was English (59.7%) in most of the cases.
Studies were also conducted in German (11.8%), Dutch (9.0%), French (4.9%), and other languages (14.6%).
4.2. Publication bias analysis
Since most of the published studies were included in the analysis, a publication bias analysis was conducted. Therefore, a possible
publication bias distortion should be examined for all outcome measures. The funnel plot indicates a small publication bias for
retention performance. According to Sterne et al. (2005), negative effects are underrepresented (all funnel plots are displayed in
Appendix C, which can be found in the supplemental material). However, the rank correlation was non-significant
(τ(N = 139) = 0.04; p = 0.67), which indicates that a publication bias was not present for retention performance. An additional
Grubbs' test (Grubbs, 1969) found no outliers. With respect to transfer performance, the funnel plot showed an outlier. Following the
approach postulated by Hoaglin, Mosteller, and Tukey (1983), the effect size of g = 2.28 from Moreno, Reislein, and Ozogul (2010)
was excluded from publication bias and further analyses. An additional conducted Grubbs’ test supported this decision. The rank-
correlation revealed that there is no publication bias for data concerning transfer performance (τ(N = 70) = 0.15; p = 0.22).
Less empirical data was available concerning cognitive load measures. Nevertheless, the funnel plot showed an approximately
equal distribution of low and high effect sizes. No outliers were detected through Grubbs' tests. The rank correlation pointed out that
there was no significant publication bias in the cognitive load data (τ(N = 27) = 0.02; p = 0.92). Regarding motivation/affect, an
extreme value of g = 1.49 from Sung and Mayer (2012) was removed from the analysis which was supported by a conducted Grubbs'
test. The funnel plot showed a homogeneous cloud of data despite the fact that the motivation/affect category was rather unspecific.
The rank-correlation was non-significant, τ(N = 13) = 0.34; p = 0.26, which indicates that there was no publication bias for
motivation/affect data. In terms of learning time, the funnel plot revealed an outlier. The effect size of g = −4.15 from the in-
vestigation of Glover et al. (1988) was excluded based on the Grubbs' test. According to the rank correlation, τ(N = 27) = 0.25;
p = 0.20, there was no significant publication bias for learning time data. For fixation times on AOIs, an extreme value of g = 4.76
from Scheiter and Eitel (2010) was excluded because of a conducted Grubbs’ test. Furthermore, the funnel plot showed heterogeneous
data (ranging from g = −1.64 to g = 0.97). According to the rank-correlation, there was no significant publication bias in AOI data
(τ(N = 14) = −0.40; p = 0.15).
4.3. Overall signaling effect
An overview of the overall signaling effect on all outcome measures is provided in Table 2. Regarding retention performance, 117
out of 139 effect sizes were positive, meaning the signaled instructional materials appeared to impact retention performance posi-
tively. The aggregation of the effect sizes revealed a significant medium effect, g+ = 0.53, SE = 0.06, z = 9.35, p < 0.01. However,
the effect sizes were heterogeneous (Q = 12462.13, df = 138, p < 0.01), indicating the possible influence of moderator variables.
Concerning transfer performance, 55 out of 70 effect sizes were positive. Again, it can be suggested that signaled instructional
materials more effectively foster learning versus non-signaled materials. The computed effect size was significant: g+ = 0.33,
SE = 0.05, z = 6.10, p < 0.01. The homogeneity test showed that transfer effect sizes were heterogeneous (Q = 685.34, df = 69,
p < 0.01).
Nineteen out of 27 cognitive load effect sizes were positive, indicating that signaling techniques reduce CL. The aggregated effect
size was significant: g+ = 0.25, SE = 0.10, z = 2.37, p = 0.02. As mentioned previously, diverse measures were combined into a
global cognitive load construct. Therefore, effect sizes were rather heterogeneous (Q = 1272.78, df = 26, p < 0.01).
In terms of motivation/affect, 11 out of 13 effect sizes were positive. It could be assumed that learners were more motivated after
working with signaled instructional materials and thus gave higher ratings. The aggregation of the effect sizes revealed a significant
but small positive effect: g+ = 0.13, SE = 0.05, z = 2.79, p < 0.01. Again, the effect sizes were heterogeneous (Q = 27.50,
df = 12, p < 0.01).
Twenty out of 27 effect sizes were negative for learning time, indicating that it took more time to learn using signaled materials
versus non-signaled materials. The computed significant effect size was low to medium (g+ = −0.30, SE = 0.13, z = −2.38,
p = 0.02). The homogeneity test revealed that learning time effect sizes were heterogeneous (Q = 282.45, df = 26, p < 0.01).
Finally, 12 out of 14 effect sizes for fixation times on AOIs were positive. This indicates that learners look longer on signaled AOIs.
Table 2
Aggregated effect sizes and confidence intervals for outcome measures.
Outcome measure Number of comparisons k Number of participants n Effect size g+ 95% CI for g+
Retention 139 11571 0.53*** [0.42, 0.64]

Transfer 70 5499 0.33*** [0.22, 0.43]
Cognitive load 27 2994 0.25* [0.04, 0.45]
Motivation/affect 13 1616 0.13** [0.04, 0.22]
Learning Time 27 2220 −0.30* [-0.55, −0.05]
Fixation on AOIs 14 628 0.39** [0.09, 0.68]
Note. *p < 0.05. **p < 0.01. ***p < 0.001.
16
Table 3
Moderator analyses for retention performance.
Moderator Number of comparisons k Effect size g+ 95% CI for g+
Educational level
Primary students 7 0.19 [-0.22, 0.61]
Secondary students 19 0.34* [0.08, 0.60]
University students 108 0.60*** [0.49, 0.71]
Adults 4 0.50(*) [-0.05, 1.04]
Prior knowledge
Low 37 0.42*** [0.18, 0.65]
High 20 0.67*** [0.35, 1.00]
Prior knowledge (without pretest scores)
Low 26 0.46* [0.09, 0.82]
High 10 0.39 [-0.23, 1.01]
Learner guidance
System paced 57 0.57*** [0.39, 0.75]
Learner paced 76 0.53*** [0.38, 0.68]
Mixed 6 0.24 [-0.29, 0.77]
Instructional domain
Biology/ecology 32 0.35** [0.11, 0.59]
Chemistry 4 0.80* [0.15, 1.45]
Math/logic 9 0.08 [-0.32, 0.49]
Physics/mechanics 36 0.43*** [0.21, 0.65]
Geology/Geography 17 0.61*** [0.31, 0.92]
Psychology 9 0.84* [0.40, 1.28]
Education 8 0.53* [0.07, 0.99]
Other 24 0.86*** [0.59, 1.12]
Signaling mode
Text 66 0.68*** [0.52, 0.85]
Graphic 70 0.38*** [0.22, 0.53]
Mixed 3 0.87* [0.11, 1.62]
Text signaling
Organization 57 0.71*** [0.54, 0.88]
highlighting
Picture 5 0.38 [-0.21, 0.97]
referencing
Color 2 0.70 [-0.21, 1.60]
Intonation 2 0.43 [-0.46, 1.33]
Mixed 3 1.01** [0.26, 1.76]
Graphic signaling
Pointing gestures 19 0.33* [0.004, 0.66]
Color 18 0.44* [0.08, 0.79]
Labeling 6 0.32 [-0.29, 0.94]
Flashing 3 −0.56 [-1.38, −0.25]
Spotlight 7 0.17 [-0.40, 0.75]
Graphic organizers 4 0.32 [-0.41, 1.05]
Mixed 16 0.77*** [0.40, 1.14]
Permanence
Permanent 84 0.58*** [0.45, 0.72]
Transient 53 0.46*** [0.29, 0.63]
Publication date
Before 2000 52 0.65*** [0.42, 0.84]
Since 2000 87 0.46*** [0.31, 0.60]
Note. (*) p < 0.10. *p < 0.05. **p < 0.01. ***p < 0.001.
The aggregated significant effect size was low to medium (g+ = 0.39, SE = 0.15, z = 2.59, p < 0.01). The set of effect sizes was
homogeneous (Q = 10.79, df = 13, p = 0.63). In order to test specific moderator hypotheses and due to the heterogeneous effect
sizes of the outcome measures, moderator analyses were conducted in the following step.
4.4. Impacts of moderator variables
According to Ginns, Martin, and Marsh (2013), separate moderator analyses were computed for retention and transfer perfor-
mance, respectively. Statistical data for moderator retention performance are outlined in Table 3. Differences between the individual
moderator categories were tested using the 95% CIs for significance.
Retention performance was significantly moderated by the instructional domain (Q = 16.16; df = 7; p = 0.02). Thus, the benefits
of signaling regarding retention performance are dependent from the learning domain. CIs showed that signaling was most effective
in geology/geography/ecology and psychology/education. Biology and physics/mechanics had smaller significant effect sizes. Math/
statistics was the only category in which the effect was non-significant.
17
Table 4
Moderator analyses for transfer performance.
Moderator Number of comparisons k Effect size g+ 95% CI for g+
Educational level
Primary students 7 0.21 [-0.12, 0.54]
Secondary students 8 0.32* [0.02, 0.63]
University students 52 0.31*** [0.23, 0.50]
Adults 2 −0.04 [-0.70, 0.61]
Prior knowledge
Low 22 0.39*** [0.19, 0.60]
High 16 0.55*** [0.31, 0.78]
Prior knowledge (without pretest scores)
Low 14 0.43* [0.13, 0.74]
High 9 0.44* [0.07, 0.81]
Learner guidance
System paced 37 0.38*** [0.23, 0.53]
Learner paced 29 0.30*** [0.13, 0.47]
Mixed 4 0.05 [-0.40, 0.50]
Instructional domain
Biology/ecology 22 0.38*** [0.16, 0.60]
Chemistry 4 0.41(*) [-0.08, 0.90]
Math/logic 9 −0.03 [-0.34, 0.28]
Physics/mechanics 23 0.44*** [0.24, 0.64]
Geology/Geography 4 0.44(*) [-0.01, 0.90]
Psychology – – –
Education 6 0.24 [-0.16, 0.65]
Other 2 0.12 [-0.73, 0.97]
Signaling mode
Text 13 0.48*** [0.22, 0.75]
Graphic 55 0.30*** [0.17, 0.42]
Mixed 2 0.39 [-0.41, 1.20]
Text signaling
Organization 9 0.35*** [0.21, 0.49]
highlighting
Picture 2 0.27* [0.01, 0.54]
referencing
Color – – –
Intonation 2 0.70*** [0.31, 1.10]
Mixed 2 0.29 [-0.10, 0.67]
Graphic signaling
Pointing gestures 12 0.21 [-0.07, 0.48]
Color 16 0.39** [0.14, 0.64]
Labeling 4 0.38 [-0.08, 0.84]
Flashing 3 −0.47(*) [-0.99, 0.04]
Spotlight 7 0.29 [-0.09, 0.66]
Graphic organizers 3 0.32 [-0.23, 0.87]
Mixed 12 0.48*** [0.20, 0.76]
Permanence
Permanent 27 0.31*** [0.15, 0.46]
Transient 38 0.35*** [0.21, 0.49]
Publication date
Before 2000 9 0.06 [-0.22,0.34]
Since 2000 61 0.37*** [0.26, 0.49]
Note. (*) p < 0.10. *p < 0.05. **p < 0.01. ***p < 0.001.
Retention was significantly moderated by signaling mode (Q = 7.79; df = 2; p = 0.02). When the CIs were considered, signaling
appeared to be more beneficial when implemented within a text versus a graphic.
The publication date was a marginal significant moderator of retention performance (Q = 2,83; df = 1; p = 0.09). Investigations
which were published before the year 2000 reported higher effect sizes than studies published from the year 2000 onwards.
The other postulated moderators did not influence retention performance systematically. There were no differences in terms of
prior knowledge (Q = 1.58; df = 1; p = 0.21), adjusted prior knowledge (Q = 0.04; df = 1; p = 0.85), educational level (Q = 6.06;
df = 3; p = 0.11), learner guidance (Q = 1.36; df = 2; p = 0.51), type of text signaling (Q = 2.14; df = 4; p = 0.71), type of graphic
signaling (Q = 10.05; df = 6; p = 0.12), or permanence (Q = 1.26; df = 1; p = 0.26). It should be noted that many non-significant
results were based on limited effect sizes and unevenly distributed cells.
Statistical data for moderator transfer performance are outlined in Table 4. Again, differences between the individual categories
of moderators were tested using the 95% CIs for significance. Transfer was marginally significant moderated by type of graphic
signaling (Q = 11.29; df = 6; p = 0.08). Coloring had a positive influence, while pointing gestures, labeling, implementing spotlights
and graphic organizers had no influence and flashing had a negative impact on transfer performance.
18
Furthermore, the publication date was a significant moderator of transfer performance (Q = 4.14; df = 1; p = 0.04).
Investigations which were published since 2000 reported higher effect sizes than studies published before the year 2000.
There were no significant differences in terms of prior knowledge (Q = 0.92; df = 1; p = 0.34), adjusted prior knowledge
(Q < 0.01; df = 1; p = 0.97), educational level (Q = 2.04; df = 3; p = 0.56), learner guidance (Q = 2.11; df = 2; p = 0.35),
instructional domain (Q = 7.36; df = 6; p = 0.29), signaling mode (Q = 1.58; df = 2; p = 0.45), type or text signaling (Q = 3.50;
df = 3; p = 0.32), type of graphic signaling (Q = 7.94; df = 6; p = 0.24), or permanence (Q = 0.15; df = 1; p = 0.70). Compared to
retention performance, fewer effect sizes were available. Again, many non-significant results were based on limited effect sizes and
unevenly distributed cells.
5. Discussion
The benefit of incorporating signaling into instructional materials is based on a large number of experimental studies. According
to Van Gog (2014) and based on CTML, the incorporation of attention-guiding cues can improve instructional materials' effectiveness.
In fact, the meta-analysis revealed that retention and transfer performance significantly increased with interventions in all signaling
studies. Retention score enhancements reached a significantly medium to high mean effect size (g+ = 0.53), while transfer scores
were significantly increased by a low to medium effect size (g+ = 0.33). Based on the confidence intervals, both effect sizes differ
significantly from each other. One explanation for these different effect sizes might be that signaling highlights defined chunks of
information. Normally, retention performance focuses on inquiries about these chunks, whereas transfer performance can only be
partially supported through signaling. One of the most interesting results was reflected in the cognitive load measures. As shown in
the literature, signaling is usually explained by a reduced ECL (Mayer & Fiorella, 2014). The results of this meta-analysis showed a
significant decline (g+ = 0.25). One explanation for this rather small effect size could be that either GCL is enhanced and thus
cognitive load does not decline much, or other variables mediate this effect. Even if signaling has little influence on motivation and
emotion, this might provide some explanation for the signaling effect. Moreover, emotion and motivation might be correlated with
the cognitive measure. For example, if signaling reduces the complexity and difficulty of instructional materials, this might offer
space for elaboration strategies and positive emotions (Pekrun, Goetz, Titz, & Perry, 2002). If students are more motivated to learn
using signaled materials versus materials without cues, they will likely spend more time analyzing the signaled material. This is
reflected by the low to medium effect size for learning time. The more time students spend with their instructional material, the more
they process and store information – another explanation for the signaling effect. A medium effect size for AOI supports the eye-mind
hypothesis (Just & Carpenter, 1980), which states that eye fixation reflects cognitive processing of instructional material. Signaling,
as a director of attention, thus concentrates the learner's focus on the relevant material. Moreover, eye-tracking measures can serve as
manipulation checks for signaling research.
5.1. Moderating influences on signaling
The meta-analysis revealed the signaling effect's numerous moderating effects according to retention and transfer performance.
One of the most interesting results was the descriptively moderating influence of domain-specific prior knowledge based on the pre-
test scores. In contrast to the expertise reversal hypothesis, learners' domain-specific knowledge does positively influence the ef-
fectiveness of signaling techniques. This moderating influence contrasts with the results of the meta-analysis by Richter et al. (2016).
Once again, signaling seems to have multiple mechanisms of effect, as high prior knowledge learners are descriptively better able to
cope with the additional information of signaling than learner with low prior knowledge. This advantage has also been shown for
other effects (e.g., the imagination effect; Leahy & Sweller, 2005). It might be possible that higher domain-specific prior knowledge
helps learners connect the meaning of signaling techniques with new information. Similarly, the moderation effect of the sample
shows that primary education learners do not profit from signaling techniques, while signaling helps secondary students with
medium effect sizes. University and adult education students profit the most, with highest effect sizes. One interesting exception
occurs for adults and transfer performances – the high effect size decreases to a zero effect. This might be the result of fixed schemata
– older learners tend to rely on their longer learning history and schemata of how texts should be read; thus, signaling might help
them remember information (retention), but the information is not better integrated into their existing schemata and thus cannot be
better used for transfer tasks. Additionally, it was shown, that signaling was comparably learning-domain independent regarding
retention performance in every domain except mathematics. Learners in the mathematics domain did not profit from signaling;
perhaps abstract contents hinder effective signaling. For example, learning how proportions are related using additional pointing
gestures (Ouwehand et al., 2015a) does not automatically increase a learner's understanding of the logic behind the concept. In
contrast, all other domains rely on information that can be shown textually. However, analysis for the transfer performance revealed
smaller effects, in the case of education even a nonsignificant result. This implies that the moderating effects of learning domain
might decrease as the elaborateness of the learning task increases.
Based on signaling mode differentiation, textual cueing is shown to enhance learning more than pictorial cueing. One reason for
this may be the more complex variety of information within pictures. In contrast to texts, signaling in animations or videos might not
be salient enough or might be too brief, so learners will not be equally affected. Moreover, textual cueing consists of multiple types,
like advanced organizers and summaries, which might reduce the organization of the instructional material more effectively than
graphical cues. Furthermore, learners are more familiar with signaling in texts because of their daily confrontation with signaling in
school and study books. According to the differentiation of textual and pictorial cues, flashing negatively impacts learning. This might
be the result of the high amount of ECL needed to permanently process or ignore a flashing cue and needs to be examined in future
19
studies since the number of studies is relatively low by now. In contrast, color improves learning in both pictorial and textual cue
modes and might enhance the aesthetics of an environment and foster emotional and motivational variables that mediate higher
learning outcomes (emotional design hypothesis; Um, Plass, Hayward, & Homer, 2012). It might also be possible that colors require
very small amounts of additional cognitive processing – a reduction in ECL cost. Additionally, intonation and organization within text
signaling revealed strong effects, indicating that improved orientation and highlighting specific textual parts might have increased
learning relevant cognition and subsequently learning outcomes.
Interestingly, the signaling effect was descriptively more effective for retention tasks within permanent environments versus
transient instructional materials. This effect might have appeared as transient visualizations might reduce the effect of signaling by
showing signals less often or for shorter periods of time. Permanent materials like texts or pictures might be processed several times,
so the signaling effect increases. However, this advantage of signaling within permanent media disappears for transfer knowledge.
Again, as the elaborateness of the learning task increases the impact of signaling decreases. Interestingly, signaling effects for re-
tention decreased significantly in studies after 2000. Although this date might be arbitrary, a finer distinction of signaling methods in
later studies might be one reason for this results. Since transfer performance measures increased in latest studies, a non-significant
significant effect for studies before 2000 can be explained. Altogether, the field of signaling techniques is widespread and needs to be
regarded in the light of numerous moderators.
5.2. Implications
Several theoretical and practical implications can be drawn from the analysis. First, the calculations revealed a much more
complex view of the signaling effect than suggested by the relevant literature. Theories like CTML and CLT, with their cognition-
centered approaches, serve as the theoretical foundation of the signaling effect. Although some factors, such as cognitive load or
learning time, indicate the assumed relationship, it might be insufficient to understand the effect of signaling on learning. Therefore,
other aspects, like moderating factors or additional explanations from other fields like meta-cognition, should be included.
Additionally, motivational and affective factors play an important role in the signaling effect. The results of this meta-analysis support
the finding that differences in eye-tracking measures as well as motivational and affective factors are influenced by signaling and play
an important role in learning. This becomes more significant when taking into account this study's results, which are sometimes
counterintuitive. In contrast to the expertise reversal hypothesis, prior knowledge was not a significant moderator of signaling.
Interestingly, signaling had a greater impact on learners with high prior knowledge than on learner with low prior knowledge on a
descriptive level. In conclusion, moderating factors must be considered in experimental design in order to set boundary conditions for
the signaling effect. Additionally, this effect might be confounded by other factors that are not covered by the current meta-analysis,
such as experience with signaling methods.
The moderator analysis provided the most fruitful results regarding practical implications and could be used to determine which
types of signaling provide the most potential in each medium. For example, the analysis revealed a stronger signaling effect within
texts than within graphical presentations. Additionally, data regarding different types of learning outcomes (retention or transfer)
was provided. For example, graphical signaling with pointing gestures reached significance as a moderator of retention knowledge,
but not as a moderator for transfer knowledge. In line with the proposed connection between affective/motivational aspects and
cognitive processes, the analysis showed that factors like color elicit comparably strong effects. Finally, the empirical evidence
showed that signaling methods that harm usability (e.g., flashing) should be avoided.
5.3. Limitations
A general criticism of the meta-analysis is the varied inclusion criteria of the different studies. The results of the present analysis
can only be interpreted by considering instructional materials with textual or graphical signaling. Furthermore, the broad range of
included studies investigated many different constructs and had varied definitions for specific outcome measures. When conducting
the present meta-analysis, these slightly different constructs were bundled into an overall construct. Signaling was seen as a high-
lighting of relevant information, however, the relevance of information can vary across individual experiences. In more detail,
transfer performance, in particular, is often interpreted differently (e.g., Wittwer & Renkl, 2010), and aggregated cognitive load and
motivation effect size consist of many subgroups and constructs. In particular, the summation of eye-tracking data can be proble-
matic, since data of peripheral perception and implicit learning might be neglected (Van Gog & Scheiter, 2010). Therefore, the
aggregated effect sizes of the present meta-analysis should be interpreted with caution.
The categorization of signaling modes postulated by Van Gog (2014) proved to be problematic as the “organization highlighting”
category for text signalization was too broad. This category included underlining, headings, and every other signaling mode that
emphasizes the structure of a text. These subcategories should be considered separately because nearly all types of text signaling were
included in this broad category.
Each referenced study included at least one experiment. Thus, effect sizes are based on studies focused on internal validity. In
order to gain detailed information regarding the multiple influences of signaling techniques, laboratory experiments took place in
highly controlled environments. It might be difficult to apply and generalize these results to “natural” learning situations because
external validity was not necessarily the focus of these studies. Therefore, it would be negligent to transfer the overall effects, which
were based on different samples, media, and materials, to semantically different instructional settings.
Despite the fact that 95 investigations were included in the analysis, some studies only examined CL, learning time, AOIs, or
motivation/affect and were not included in the meta-analysis. In addition, some aggregated effect sizes were based on less than 30
20
pair-wise comparisons. There were only a few studies available for adult learning, specific learning domains (e.g., chemistry), and
different types of text and graphical signaling (e.g., coloring and intonation). Since some aggregated effect sizes were based on just
two comparisons, far-reaching interpretations were difficult. Consequently, cells in the moderator analyses were unequally dis-
tributed. This limitation clearly reveals the need for further investigations of terms of specific signaling techniques, instructional
materials, and participants. University student samples, natural science learning topics, and organizational signals were over-
represented and limit a generalization. Additionally, some results differed because of the publication date. Change in the research
focus over time provided more data, for example regarding transfer knowledge.
6. Conclusion
The fact that 95 studies were identified and included shows that signaling effects during learning that incorporates media have
been thoroughly investigated. The findings of this meta-analysis support the signaling effect under multiple conditions and cir-
cumstances. The diversity of the included studies (e.g., educational topics, samples, operationalization of signaling) provides a
comprehensive understanding of the positive and negative impacts of signaling. Elements of instructional materials, like texts,
graphics, and animations, should be adapted to the intended audience. Furthermore, learning domain is crucial to learning success,
especially regarding transfer knowledge. Designers should be aware of the potential of signaling, but they must also acknowledge that
instructional material should be adapted to the sample, domain, and other mentioned variables. In this case, instructional materials
could be optimized with relatively moderate changes.
Acknowledgements
We want to thank Simon Pradel, Matthias Niggehoff, Tobias Schmitt and Nico Münch for making this meta-analysis possible.
Appendix A. Supplementary data
Supplementary data related to this article can be found at http://dx.doi.org/10.1016/j.edurev.2017.11.001.
References
Note. Studies with * are included in the presented meta-analysis.

(∗)Amadieu, F., Mariné, C., & Laimay, C. (2011). The attention-guiding effect and cognitive load in the comprehension of animations. Computers in Human Behavior, 27,
36–40.
(∗). Arslan-Ari, I. (2013). Examining the effects of cueing and prior knowledge on learning, mental effort, and study time in a complex animationDoctoral dissertation. Texas
Tech University.
Awh, E., & Jonides, J. (2001). Overlapping mechanisms of attention and spatial working memory. Trends in Cognitive Sciences, 5, 119–126.
Baddeley, A. (1992). Working memory. Science, 255, 556–559.
Barley, N. (2016). Using visually disfluent fonts for cueing and increased readingDissertationhttp://digitalcommons.odu.edu/stemps_etds/7/, Accessed date: 25 November
2016.
Begg, C. B., & Mazumdar, M. (1994). Operating characteristics of a rank correlation test for publication bias. Biometrics, 50, 1088–1101.
(∗)Berthold, K., & Renkl, A. (2009). Instructional aids to support a conceptual understanding of multiple representations. Journal of Educational Psychology, 101, 70–87.
(∗)Boucheix, J. M., & Guignard, H. (2005). What animated illustrations conditions can improve technical document comprehension in young students? Format,
signaling and control of the presentation. European Journal of Psychology of Education, 20, 369–388.
(∗)Boucheix, J. M., & Lowe, R. K. (2010). An eye tracking comparison of external pointing cues and internal continuous cues in learning with complex animations.
Learning and Instruction, 20, 123–135.
(∗)Boucheix, J. M., Lowe, R. K., Putri, D. K., & Groff, J. (2013). Cueing animations: Dynamic signaling aids information extraction and comprehension. Learning and
Instruction, 25, 71–84.
(∗)Brooks, L. W., Dansereau, D. F., Spurlin, J. E., & Holley, C. D. (1983). Effects of headings on text processing. Journal of Educational Psychology, 75, 292–302.
Carolan, T. F., Hutchins, S. D., Wickens, C. D., & Cumming, J. M. (2014). Costs and benefits of more learner freedom meta-analyses of exploratory and learner control
training methods. Human Factors: The Journal of the Human Factors and Ergonomics Society, 56, 999–1014.
(∗)Cashen, V. M., & Leicht, K. L. (1970). Role of the isolation effect in a formal educational setting. Journal of Educational Psychology, 61, 484–486.
Castro-Alonso, J. C., Ayres, P., & Paas, F. (2014). Learning from observing hands in static and animated versions of non-manipulative tasks. Learning and Instruction, 34,
11–21.
Chen, C. H., Liu, G. Z., & Hwang, G. J. (2015). Interaction between gaming and multistage guiding strategies on students' field trip mobile learning performance and
motivation. British Journal of Educational Technology. http://dx.doi.org/10.1111/bjet.12270.
(∗)Cheon, J., & Grant, M. M. (2012). The effects of metaphorical interface on germane cognitive load in web-based instruction. Educational Technology Research and
Development, 60, 399–420.
Chi, M. T., De Leeuw, N., Chiu, M. H., & LaVancher, C. (1994). Eliciting self-explanations improves understanding. Cognitive Science, 18, 439–477.
Cohen, J. (1960). A coefficient of agreement for nominal scales. Educational and Psychological Measurement, 20, 37–46.
(∗)Coles, P., & Foster, J. (1975). Typographic cueing as an aid to learning from typewritten text. Programmed Learning and Educational Technology, 12, 102–108.
(∗)Cook, S. W., Friedman, H. S., Duggan, K. A., Cui, J., & Popescu, V. (2016). Hand gesture and mathematics learning: Lessons from an avatar. Cognitive Science, 1–18.
Cooper, H., Hedges, L. V., & Valentine, J. C. (2009). The handbook of research synthesis and meta-analysis. Russell Sage Foundation.
Corkill, A. J. (1992). Advance organizers: Facilitators of recall. Educational Psychology Review, 4, 33–67.
(∗)Corkill, A. J., Bruning, R. H., & Glover, J. A. (1988). Advance organizers: Concrete versus abstract. The Journal of Educational Research, 82, 76–81.
Coyan, J., Busch, M., & Reynolds, S. (2010, August). Using eye tracking to evaluate the effectiveness of signaling to promote disembedding of geologic features in
photographs. Spatial cognition 2010: Doctoral colloquium (pp. 15). .
(∗)Craig, S. D., Twyford, J., Irigoyen, N., & Zipp, S. A. (2015). A Test of spatial contiguity for virtual human's gestures in multimedia learning Environments. Journal of
Educational Computing Research, 53, 3–14.
(∗)Crooks, S. M., Cheon, J., Inan, F., Ari, F., & Flores, R. (2012). Modality and cueing in multimedia learning: Examining cognitive and perceptual explanations for the
modality effect. Computers in Human Behavior, 28, 1063–1071.
(∗)Crouse, J. H., & Idstein, P. (1972). Effects of encoding cues on prose learning. Journal of Educational Psychology, 63, 309–313.
21
Cuddihy, E., & Spyridakis, J. H. (2012). The effect of visual design and placement of intra-article navigation schemes on reading comprehension and website user
perceptions. Computers in Human Behavior, 28, 1399–1409.
(∗)De Jong, T., & Van Der Hulst, A. (2002). The effects of graphical overviews on knowledge acquisition in hypertext. Journal of Computer Assisted Learning, 18,
219–231.
(∗)De Koning, B. B., Tabbers, H. K., Rikers, R. M., & Paas, F. (2007). Attention cueing as a means to enhance learning from an animation. Applied Cognitive Psychology,
21, 731–746.
(∗)De Koning, B. B., Tabbers, H. K., Rikers, R. M., & Paas, F. (2011a). Attention cueing in an instructional animation: The role of presentation speed. Computers in
Human Behavior, 27, 41–45.
(∗)De Koning, B. B., Tabbers, H. K., Rikers, R. M., & Paas, F. (2011b). Improved effectiveness of cueing by self-explanations when learning from a complex animation.
Applied Cognitive Psychology, 25, 183–194.
(∗)De Koning, B. B., Tabbers, H. K., Rikers, R. M., & Paas, F. (2010a). Attention guidance in learning from a complex animation: Seeing is understanding? Learning and
(∗)De Koning, B. B., Tabbers, H. K., Rikers, R. M., & Paas, F. (2010b). Learning by generating vs. receiving instructional explanations: Two approaches to enhance
attention cueing in animations. Computers & Education, 55(2), 681–691.
Dodd, B. J., & Antonenko, P. D. (2012). Use of signaling to integrate desktop virtual reality and online learning management systems. Computers & Education, 59,
1099–1108.
(∗)Doolittle, P. E., & Altstaedter, L. L. (2009). The effect of working memory capacity on multimedia learning: Does attentional control result in improved performance.
Journal of Research in Innovative Teaching, 2, 7–25.
D'Mello, S. (2013). A selective meta-analysis on the relative incidence of discrete affective states during learning with technology. Journal of Educational Psychology,
105, 1082–1099.
(∗)Ferrara, L., & Butcher, K. R. (2011). Visualizing feedback: Using graphical cues to promote self-regulated learning. Proceedings of the thirty-third annual conference of
the cognitive science society. Boston: Cognitive Science Society.
Field, A. P., & Gillett, R. (2010). How to do a meta-analysis. British Journal of Mathematical and Statistical Psychology, 63, 665–694.
(∗) Fiorella, L., & Mayer, R. E. (2015). Effects of observing the instructor draw diagrams on learning from multimedia messages. Journal of Educational Psychology, 108,
528–546. Retrieved from http://psycnet.apa.org/psycinfo/2015-37517-001/.
Fischer, S., & Schwan, S. (2010). Comprehending animations: Effects of spatial cueing versus temporal scaling. Learning and Instruction, 20, 465–475.
(∗)Florax, M., & Ploetzner, R. (2010). What contributes to the split-attention effect? The role of text segmentation, picture labelling, and spatial proximity. Learning and
(∗)Folker, S., Ritter, H., & Sichelschmidt, L. (2005). Processing and integrating multimodal material: The influence of color-coding. In B. G. Bara, L. Barsalou, & M.
Bucciarelli (Eds.). Proceedings of the 27th annual conference of the cognitive science society (pp. 690–695). Mahwah, New Jersey: Lawrence Erlbaum Associates, Inc.
Fowler, R. L., & Barker, A. S. (1974). Effectiveness of highlighting for retention of text material. Journal of Applied Psychology, 59, 358–364.
Ginns, P., Martin, A. J., & Marsh, H. W. (2013). Designing instructional text in a conversational style: A meta-analysis. Educational Psychology Review, 25, 445–472.
(∗)Glover, J. A., Dinnel, D. L., Halpain, D. R., McKee, T. K., Corkill, A. J., & Wise, S. L. (1988). Effects of across-chapter signals on recall of text. Journal of Educational
Psychology, 80, 3–15.
(∗)Glynn, S. M., & Di Vesta, F. J. (1977). Outline and hierarchical organization as aids for study and retrieval. Journal of Educational Psychology, 69, 89–95.
(∗)Golding, J. M., & Fowler, S. B. (1992). The limited facilitative effect of typographical signals. Contemporary Educational Psychology, 17, 99–113.
(∗)Gordon, C., Tindall-Ford, S., Agostinho, S., & Paas, F. (2016). Learning from instructor-managed and self-managed split-attention materials. Applied Cognitive
Gozli, D. G., Aslam, H., & Pratt, J. (2015). Visuospatial cueing by self-caused features: Orienting of attention and action–outcome associative learning. Psychonomic
Bulletin & Review, 1–9.
Graesser, A. C., Singer, M., & Trabasso, T. (1994). Constructing inferences during narrative text comprehension. Psychological Review, 101, 371–395.
Grubbs, F. E. (1969). Procedures for detecting outlying observations in samples. Technometrics, 11, 1–21.
Guichon, N., & McLornan, S. (2008). The effects of multimodality on L2 learners: Implications for CALL resource design. System, 36, 85–93.
(∗)Harp, S. F., & Mayer, R. E. (1998). How seductive details do their damage: A theory of cognitive interest in science learning. Journal of Educational Psychology, 90,
414–434.
(∗)Hartley, J., & Trueman, M. (1983). The effects of headings in text on recall, search and retrieval. British Journal of Educational Psychology, 53, 205–214.
Hattie, J. (2008). Visible learning: A synthesis of over 800 meta-analyses relating to achievement. Routledge.
(∗)Hayes, D. A., & Reinking, D. (1991). Good and poor readers' use of graphic aids cued in texts and in adjunct study materials. Contemporary Educational Psychology,
16, 391–398.
Hedges, L. V., & Olkin, I. (1985). Statistical methods for meta-analysis. Orlando, FL: Academic Press.
Hedges, L. V., Tipton, E., & Johnson, M. C. (2010). Robust variance estimation in meta‐regression with dependent effect size estimates. Research Synthesis Methods, 1,
39–65.
Hedges, L. V., & Vevea, J. L. (1998). Fixed-and random-effects models in meta-analysis. Psychological Methods, 3, 486.
Hoaglin, D. C., Mosteller, F., & Tukey, J. W. (1983). Understanding robust and exploratory data analysis, Vol. 3. New York: Wiley.
Höffler, T. N., & Leutner, D. (2007). Instructional animation versus static pictures: A meta-analysis. Learning and Instruction, 17, 722–738.
(∗)Holley, C. D., Dansereau, D. F., Evans, S. H., Collins, K. W., Brooks, L., & Larson, D. (1981). Utilizing intact and embedded headings as processing aids with
nonnarrative text. Contemporary Educational Psychology, 6, 227–236.
(∗)Huk, T., Steinke, M., & Floto, C. (2003). Learning with educational hypermedia software: The impact of signals in animations. In P. Isaías, & N. Karmakar (Eds.).
Proceedings of IADIS international conference WWW/internet (ICWI) (pp. 689–695). (Algarve, Portugal).
(∗)Huk, T., Steinke, M., & Floto, C. (2010). The educational value of visual cues and 3D-representational format in a computer animation under restricted and realistic
conditions. Instructional Science, 38, 455–469.
(∗)Hyönä, J., & Lorch, R. F. (2004). Effects of topic headings on text processing: Evidence from adult readers' eye fixation patterns. Learning and Instruction, 14,
131–152.
IBM Corp (2015). IBM SPSS statistics for windows (version 23.0). Armonk, NY: IBM Corp.
(∗)Jamet, E. (2014). An eye-tracking study of cueing effects in multimedia learning. Computers in Human Behavior, 32, 47–53.
(∗)Jamet, E., & Fernandez, J. (2016). Enhancing interactive tutorial effectiveness through visual cueing. Educational Technology Research and Development, 64, 1–11.
(∗)Jamet, E., Gavota, M., & Quaireau, C. (2008). Attention guiding in multimedia learning. Learning and Instruction, 18, 135–145.
(∗)Jarodzka, H., van Gog, T., Dorr, M., Scheiter, K., & Gerjets, P. (2013). Learning to see: Guiding students' attention via a model's eye movements fosters learning.
Learning and Instruction, 25, 62–70.
(∗)Jeung, H. J., Chandler, P., & Sweller, J. (1997). The role of visual indicators in dual sensory mode instruction. Educational Psychology, 17, 329–345.
(∗)Johnson, A. M., Butcher, K. R., Ozogul, G., & Reisslein, M. (2013). Learning from abstract and contextualized representations: The effect of verbal guidance.
Computers in Human Behavior, 29, 2239–2247.
(∗)Johnson, A. M., Butcher, K. R., Ozogul, G., & Reisslein, M. (2014). Introductory circuit analysis learning from abstract and contextualized circuit representations:
Effects of diagram labels. IEEE Transactions on Education, 57, 160–168.
(∗)Johnson, A. M., Ozogul, G., Moreno, R., & Reisslein, M. (2013). Pedagogical agent signaling of multiple visual engineering representations: The case of the young
female agent. Journal of Engineering Education, 102, 319–337.
(∗)Johnson, A. M., Ozogul, G., & Reisslein, M. (2015). Supporting multimedia learning with visual signalling and animated pedagogical agent: Moderating effects of
prior knowledge. Journal of Computer Assisted Learning, 31, 97–115.
Just, M. A., & Carpenter, P. A. (1980). A theory of reading: From eye fixations to comprehension. Psychological Review, 87, 329–354.
22
Kahana, M. J. (1996). Associative retrieval processes in free recall. Memory & Cognition, 24, 103–109.
Kalyuga, S. (2007). Expertise reversal effect and its implications for learner-tailored instruction. Educational Psychology Review, 19, 509–539.
Kalyuga, S. (2009). Instructional designs for the development of transferable knowledge and skills: A cognitive load perspective. Computers in Human Behavior, 25,
332–338.
(∗)Kalyuga, S., Chandler, P., & Sweller, J. (1999). Managing split-attention and redundancy in multimedia instruction. Applied Cognitive Psychology, 13, 351–371.
Kalyuga, S., & Singh, A. M. (2016). Rethinking the boundaries of cognitive load theory in complex learning. Educational Psychology Review, 28, 831–852.
(∗)Kardash, C. M., & Noel, L. K. (2000). How organizational signals, need for cognition, and verbal ability affect text recall and recognition. Contemporary Educational
Karich, A. C., Burns, M. K., & Maki, K. E. (2014). Updated meta-analysis of learner control within educational technology. Review of Educational Research, 84, 392–410.
(∗). Kornalijnslijper, D. S. (2012). The user effects of using textual cues to increase image viewing attentionMaster thesis. University of Twente.
(∗)Kriz, S., & Hegarty, M. (2007). Top-down and bottom-up influences on learning from animations. International Journal of Human-computer Studies, 65, 911–930.
(∗)Krug, D., George, B., Hannon, S. A., & Glover, J. A. (1989). The effect of outlines and headings on readers' recall of text. Contemporary Educational Psychology, 14,
111–123.
(∗). Kühl, T. (2011). Optimizing learning with dynamic and static visualizations to foster understanding in the natural sciencesDoctoral dissertation. Universität Tübingen.
(∗)Kühl, T., Scheiter, K., & Gerjets, P. (2012). Enhancing learning from dynamic and static visualizations by means of cueing. Journal of Educational Multimedia and
Hypermedia, 21, 71–88.
Leahy, W., & Sweller, J. (2005). Interactions among the imagination, expertise reversal, and element interactivity effects. Journal of Experimental Psychology: Applied,
11, 266–276.
(∗). Lin, L. (2011). Learning with multimedia: Are visual cues and self-explanation prompts effective?Doctoral dissertation. Arizona State University.
(∗)Lin, L., & Atkinson, R. K. (2011). Using animations and visual cueing to support learning of scientific concepts and processes. Computers & Education, 56, 650–658.
Lipsey, M. W., & Wilson, D. B. (2001). Practical meta-analysis. Sage Publications, Inc.
Liu, T. C., Lin, Y. C., & Paas, F. (2013). Effects of cues and real objects on learning in a mobile device supported environment. British Journal of Educational Technology,
44, 386–399.
(∗)Loman, N. L., & Mayer, R. E. (1983). Signaling techniques that increase the understandability of expository prose. Journal of Educational Psychology, 75, 402–412.
(∗)Lorch, R. F., & Chen, A. H. (1986). Effects of number signals on reading and recall. Journal of Educational Psychology, 78, 263–270.
(∗)Lorch, R. F., Jr., & Lorch, E. P. (1995). Effects of organizational signals on text-processing strategies. Journal of Educational Psychology, 87, 537–544.
(∗)Lorch, R., Lemarié, J., & Grant, R. (2011). Signaling hierarchical and sequential organization in expository text. Scientific Studies of Reading, 15, 267–284.
(∗)Lorch, R. F., & Lorch, E. P. (1986). On-line processing of summary and importance signals in reading. Discourse Processes, 9, 489–496.
(∗)Lorch, R. F., & Lorch, E. P. (1996). Effects of organizational signals on free recall of expository text. Journal of Educational Psychology, 88, 38–48.
(∗)Lorch, R. F., Lorch, E. P., & Inman, W. E. (1993). Effects of signaling topic structure on text recall. Journal of Educational Psychology, 85, 281–290.
(∗)Lorch, R. F., Lorch, E. P., & Klusewitz, M. A. (1995). Effects of typographical cues on reading and recall of text. Contemporary Educational Psychology, 20, 51–64.
(∗)Lorch, R. F., Lorch, E. P., Ritchey, K., McGovern, L., & Coleman, D. (2001). Effects of headings on text summarization. Contemporary Educational Psychology, 26,
171–191.
Low, R., Jin, P., & Sweller, J. (2011). Cognitive load theory, attentional processes and optimized learning outcomes in a digital environment. In C. Roda (Ed.). Human
attention in digital environments (pp. 63–72). Springer International Publishing.
(∗)Luo, H., Koszalka, T., & Zuo, M. (2016). Investigating the effects of visual cues in multimedia instruction using eye tracking. In S. K. S. Cheung, L. Kwok, J. Shang, A.
Wang, & R. Kwan (Eds.). Blended Learning: Aligning theory with practices (pp. 63–72). Springer International Publishing.
(∗)Mason, L., Pluchino, P., & Tornatora, M. C. (2013). Effects of picture labeling on science text processing and learning: Evidence from eye movements. Reading
Research Quarterly, 48, 199–214.
(∗)Mautone, P. D., & Mayer, R. E. (2001). Signaling as a cognitive guide in multimedia learning. Journal of Educational Psychology, 93, 377–389.
(∗)Mautone, P. D., & Mayer, R. E. (2007). Cognitive aids for guiding graph comprehension. Journal of Educational Psychology, 99, 640–652.
Mayer, R. E. (2010). Unique contributions of eye-tracking research to the study of learning with graphics. Learning and Instruction, 20, 167–171.
Mayer, R. E. (2014a). Cognitive theory of multimedia learning. In R. E. Mayer (Ed.). The Cambridge handbook of multimedia learning (pp. 43–71). (2nd ed.). Cambridge:
Cambridge University Press.
Mayer, R. E. (2014b). The Cambridge handbook of multimedia learning (2nd ed.). Cambridge: Cambridge University Press.
(∗)Mayer, R. E., Dyck, J. L., & Cook, L. K. (1984). Techniques that help readers build mental models from scientific text: Definitions pretraining and signaling. Journal
of Educational Psychology, 76, 1089–1105.
Mayer, R. E., & Fiorella, L. (2014). Principles for reducing extraneous processing in multimedia learning: Coherence, signaling, redundancy, spatial contiguity, and
temporal contiguity principles. In R. E. Mayer (Ed.). The Cambridge handbook of multimedia learning (pp. 43–71). (2nd ed.). Cambridge: Cambridge University Press.
McNamara, D. S., Kintsch, E., Songer, N. B., & Kintsch, W. (1996). Are good texts always better? Interactions of text coherence, background knowledge, and levels of
understanding in learning from text. Cognition and Instruction, 14, 1–43.
McTigue, E. M. (2009). Does multimedia learning theory extend to middle-school students? Contemporary Educational Psychology, 34, 143–153.
(∗)Meyer, B. J., Brandt, D. M., & Bluth, G. J. (1980). Use of top-level structure in text: Key for reading comprehension of ninth-grade students. Reading Research
Quarterly, 16, 72–103.
(∗)Meyer, B. J., & Poon, L. W. (2001). Effects of structure strategy training and signaling on recall of text. Journal of Educational Psychology, 93, 141–159.
(∗)Moreno, R. (2007). Optimising learning from animations by minimising cognitive load: Cognitive and affective consequences of signalling and segmentation
methods. Applied Cognitive Psychology, 21, 765–781.
(∗)Moreno, R., Reislein, M., & Ozogul, G. (2010). Using virtual peers to guide visual attention during learning. Journal of Media Psychology, 22, 55–60.
(∗)Murray, J. D., & McGlone, C. (1997). Topic overviews and processing of topic structure. Journal of Educational Psychology, 89, 251–261.
(∗)Nevid, J. S., & Lampmann, J. L. (2003). Effects on content acquisition of signaling key concepts in text material. Teaching of Psychology, 30, 227–230.
(∗)Ouwehand, K., Gog, T., & Paas, F. (2015b). Effects of gestures on older adults' learning from video-based models. Applied Cognitive Psychology, 29, 115–128.
(∗)Ouwehand, K., van Gog, T., & Paas, F. (2015a). Designing effective video-based modeling examples using gaze and gesture cues. Journal of Educational Technology &
Society, 18, 78–88.
(∗)Ozcelik, E., Arslan-Ari, I., & Cagiltay, K. (2010). Why does signaling enhance multimedia learning? Evidence from eye movements. Computers in Human Behavior, 26,
110–117.
(∗)Ozcelik, E., Karakus, T., Kursun, E., & Cagiltay, K. (2009). An eye-tracking study of how color coding affects multimedia learning. Computers & Education, 53,
445–453.
Paas, F., & Sweller, J. (2014). Implications of cognitive load theory for multimedia learning. In R. E. Mayer (Ed.). The Cambridge handbook of multimedia learning (pp.
27–42). (2nd ed.). Cambridge: Cambridge University Press.
Pekrun, R., Goetz, T., Titz, W., & Perry, R. P. (2002). Academic emotions in students' self-regulated learning and achievement: A program of qualitative and quan-
titative research. Educational Psychologist, 37, 91–105.
(∗)Reinking, D., Hayes, D. A., & McEneaney, J. E. (1988). Good and poor readers' use of explicitly cued graphic aids. Journal of Literacy Research, 20, 229–247.
(∗)Reisslein, J., Johnson, A. M., & Reisslein, M. (2015). Color coding of circuit quantities in introductory circuit analysis instruction. IEEE Transactions on Education, 58,
7–14.
(∗)Rey, G. D. (2010). Reading direction and signaling in a simple computer simulation. Computers in Human Behavior, 26, 1176–1182.
Richter, J., Scheiter, K., & Eitel, A. (2016). Signaling text-picture relations in multimedia learning: A comprehensive meta-analysis. Educational Research Review, 17,
19–36.
(∗)Rickards, J. P., Fajen, B. R., Sullivan, J. F., & Gillespie, G. (1997). Signaling, notetaking, and field independence–dependence in text comprehension and recall.
Journal of Educational Psychology, 89, 508–517.
23
(∗)Roberts, W. E. (2008). The use of cues in multimedia instructions in technology as a way to reduce cognitive load. Ann Arbor: ProQuest LLC.
(∗)Scheiter, K., & Eitel, A. (2010). The effects of signals on learning from text and diagrams: How looking at diagrams earlier and more frequently improves un-
derstanding. In A. E. Goel, M. Jamnik, & N. H. Narayanan (Eds.). Diagrammatic representation and inference (pp. 264–270). Berlin, Heidelberg: Springer.
(∗)Scheiter, K., & Eitel, A. (2015). Signals foster multimedia learning by supporting integration of highlighted text and diagram elements. Learning and Instruction, 36,
11–26.
Schnotz, W., & Kürschner, C. (2007). A reconsideration of cognitive load theory. Educational Psychology Review, 19, 469–508.
(∗)Schrader, P. G., & Rapp, E. E. (2016). Does multimedia theory apply to all students? The impact of multimedia presentations on science learning. Journal of Learning
and Teaching in Digital Age, 1, 32–46.
(∗)Seufert, T., & Brünken, R. (2006). Cognitive load and the format of instructional aids for coherence formation. Applied Cognitive Psychology, 20, 321–331.
(∗)Skuballa, I. T., Schwonke, R., & Renkl, A. (2012). Learning from narrated animations with different support procedures: Working memory capacity matters. Applied
Cognitive Psychology, 26, 840–847.
Spyridakis, J. H. (1989). Signaling effects: A review of the research – Part I. Journal of Technical Writing and Communication, 19, 227–240.
(∗)Spyridakis, J. H., & Standal, T. C. (1987). Signals in expository prose: Effects on reading comprehension. Reading Research Quarterly, 22, 285–298.
Sterne, J. A., Becker, B. J., & Egger, M. (2005). The funnel plot. Publication bias in meta-analysis: Prevention, assessment and adjustments. 75–98.
(∗)Stull, A. T., & Mayer, R. E. (2007). Learning by doing versus learning by viewing: Three experimental comparisons of learner-generated versus author-provided
graphic organizers. Journal of Educational Psychology, 99, 808–820.
(∗)Sung, E., & Mayer, R. E. (2012). Affective impact of navigational and signaling aids to e-learning. Computers in Human Behavior, 28, 473–483.
Sung, E., & Mayer, R. E. (2013). Online multimedia learning with mobile devices and desktop computers: An experimental test of Clark's methods-not-media hy-
pothesis. Computers in Human Behavior, 29, 639–647.
(∗)Surber, J. R., & Schroeder, M. (2007). Effect of prior domain knowledge and headings on processing of informative text. Contemporary Educational Psychology, 32,
485–498.
Sweller, J. (1988). Cognitive load during problem solving: Effects on learning. Cognitive Science, 12, 257–285.
Sweller, J., Ayres, P., & Kalyuga, S. (2011). Cognitive load theory. New York: Springer Science + Business Media.
(∗)Tabbers, H. K., Martens, R. L., & Merriënboer, J. J. G. (2004). Multimedia instructions and cognitive load theory: Effects of modality and cueing. British Journal of
Educational Psychology, 74, 71–81.
(∗)Tabbers, H. K., Martens, R. L., & van Merriënboer, J. J. G. (2000). Multimedia instructions and cognitive load theory: Split-attention and modality effects. Paper
presented at the National convention of the association for educational communications and technology (Long Beach, CA).
(∗)Titsworth, B. S., & Kiewra, K. A. (2004). Spoken organizational lecture cues and student notetaking as facilitators of student learning. Contemporary Educational
Turk, M. (2014). Multimodal interaction: A review. Pattern Recognition Letters, 36, 189–195.
Um, E., Plass, J. L., Hayward, E. O., & Homer, B. D. (2012). Emotional design in multimedia learning. Journal of Educational Psychology, 104, 485–498.
Van Gerven, P. W., Paas, F., Van Merriënboer, J. J., & Schmidt, H. G. (2006). Modality and variability as factors in training the elderly. Applied Cognitive Psychology, 20,
311–320.
Van Gog, T. (2014). The signaling (or cueing) principle in multimedia learning. In R. E. Mayer (Ed.). The cambridge handbook of multimedia learning (pp. 263–278). (2nd
ed). Cambridge, MA: Cambridge University Press.
Van Gog, T., & Scheiter, K. (2010). Eye tracking as a tool to study and enhance multimedia learning. Learning and Instruction, 20, 95–99.
(∗)Van Oostendorp, H., Beijersbergen, M. J., & Solaimani, S. (2008). Conditions for learning from animations. Proceedings of the 8th international conference for the
learning sciences (pp. 438–445). International Society of the Learning Sciences.
Van der Kleij, F. M., Feskens, R. C., & Eggen, T. J. (2015). Effects of feedback in a computer-based learning environment on students' learning outcomes: A meta-
analysis. Review of Educational Research, 85, 475–511.
Wilson, D. B. (2001). Practical meta-analysis effect size calculator. http://www.campbellcollaboration.org/escalc/html/EffectSizeCalculator-SMD1.php, Accessed date:
25 November 2016.
Wilson, D. B. (2010). Meta-analysis macros for SAS, SPSS, and stata. Retrieved, February, 18, 2016, from http://mason.gmu.edu/∼dwilsonb/ma.html.
Wittwer, J., & Renkl, A. (2010). How effective are instructional explanations in example-based learning? A meta-analytic review. Educational Psychology Review, 22,
393–409.
(∗)Yang, H.-Y. (2016). The effects of attention cueing on visualizers' multimedia learning. Educational Technology & Society, 19, 249–262.
(∗)Yung, H. I., & Paas, F. (2015). Effects of cueing by a pedagogical agent in an instructional animation: A cognitive load approach. Journal of Educational Technology &
Society, 18, 153–160.
24

Educational Research Review: Sascha Schneider, Maik Beege, Steve Nebel, Günter Daniel Rey

Uploaded by

Document Information

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Educational Research Review: Sascha Schneider, Maik Beege, Steve Nebel, Günter Daniel Rey

Uploaded by

Copyright:

Available Formats

Educational Research Review 23 (2018) 1–24

Contents lists available at ScienceDirect

Educational Research Review

A meta-analysis of how signaling aﬀects learning with media

2.1. The signaling principle and media design theories

Fig. 1. Overview of possible signaling types.

2.2. The impact of signaling on learning processes

2.3. Moderators of the signaling eﬀect

2.3.1. The inﬂuence of prior knowledge

2.3.2. Signaling mode

2.3.3. Diﬀerences in instructional materials

2.4. Research questions and hypotheses

3.1. Data collection and inclusion criteria

3.2. Study features

years) based on pre- (Graphic) Measure

3.3. Outcome measures

3.4. Analysis methods

4.1. Sample analysis

4.2. Publication bias analysis

4.3. Overall signaling eﬀect

Retention 139 11571 0.53*** [0.42, 0.64]

Note. *p < 0.05. **p < 0.01. ***p < 0.001.

Moderator Number of comparisons k Eﬀect size g+ 95% CI for g+

4.4. Impacts of moderator variables

Moderator Number of comparisons k Eﬀect size g+ 95% CI for g+

5.1. Moderating inﬂuences on signaling

Appendix A. Supplementary data

Supplementary data related to this article can be found at http://dx.doi.org/10.1016/j.edurev.2017.11.001.

Note. Studies with * are included in the presented meta-analysis.

You might also like

Note. p < 0.05. p < 0.01. p < 0.001.