Learning and Instruction 29 (2014) 125–127

Contents lists available at SciVerse ScienceDirect

Learning and Instruction

journal homepage: www.elsevier.com/locate/learninstruc

Editorial

Cognitive and affective processes in multimedia learning

a b s t r a c t

Keywords: This special section focuses on cognitive and affective processes in multimedia learning in a range of
Multimedia learning learning domains. Expanding previous research that has taken a predominantly cognitive perspective of
Interest multimedia learning, recent studies have begun to consider affective aspects of multimedia learning with
Emotion
the aim of integrating emotion, motivation, and other affective variables into cognitive processing
Motivation
models. The articles included in this special section are examples of the various ways in which the
Affect
cognitive perspective can be enhanced by taking affective aspects of learning into account. Investigations
range from the study of confusion as an affective state that can be beneficial to learning, and the
consideration of the potential distracting or motivating function of decorative illustrations, to an inquiry
into how visual design can induce positive emotions in learners. The results of the studies included in
this section are in line with Moreno’s Cognitive-Affective Theory of Learning with Media (CATLM;
Moreno, 2006) and show how emotion and interest facilitate cognitive processing and improve cognitive
and affective outcomes.
Ó 2013 Elsevier Ltd. All rights reserved.

1. Introduction suggests that motivational factors mediate learning by increasing or

This special section is based on the invited SIG 6 Symposium on
Instructional Design, which took place during the 14th Biennial
Conference of the European Association of Research on Learning
and Instruction (EARLI) in summer 2011 in Exeter, U.K. This sympo-
sium was organized and chaired by Babette Park, Jan L. Plass, and
Roland Brünken, and was dedicated to Dr. Roxana Moreno,
who had unexpectedly passed away the previous summer. With
Dr. Moreno’s untimely death, our community lost one of our
most prolific researchers and highly productive scholars, a wonder-
ful human being who will be missed by many (see also Robinson,
2010). With this special section we would like to commemorate
Dr. Moreno’s most recent work on a Cognitive-Affective Theory of
Learning with Media (CATLM; Moreno, 2005, 2006, 2007, 2009)
by presenting papers that have provided empirical evidence for
the CATLM and that may inspire more research related to this
theory.
CATLM focuses on cognitive and affective processes in multimedia
learning. The theory is based on recent theoretical frameworks
of multimedia learning and enhances the cognitive perspective by
taking motivational and affective aspects into account. The model
includes four cognitive assumptions: (1) The existence of verbal
and non-verbal information processing channels that are relatively
independent of one another, (2) the limited capacity of working
memory, as well as a virtually unlimited capacity of long-term mem-
ory, (3) enhanced learning through dual coding, and (4) the need for
learners to actively process information to construct meaning. These
well-known assumptions, which are based on Mayer’s Cognitive
Theory of Multimedia Learning (Mayer, 2005), are augmented by
three new assumptions: (5) the affective mediation assumption, which
decreasing cognitive engagement (Gottfried, 1990; Moreno, Mayer,
Spires, & Lester, 2001; Park, Moreno, Seufert, & Brünken, 2011),
(6) the metacognitive mediation assumption, which suggests that
metacognitive factors mediate learning by regulating cognitive and
affective processes (McGuinness, 1990; Morris, 1990), and (7) the
individual differences assumption, which suggests that differences
in learners’ prior knowledge (Kalyuga, Ayres, Chandler, & Sweller,
2003; Moreno, 2004) and traits such as cognitive styles and abilities
(Moreno & Durán 2004; Plass, Chun, Mayer, & Leutner, 1998; Seufert,
Schütze & Brünken, 2009) affect the efficiency of learning with
methods and media. The resulting CATLM is shown in Fig. 1.
To consider these affective aspects in research on instructional
design, three contributions are presented in this special section
that focus on cognitive and affective processes in multimedia
learning. D’Mello, Lehman, Pekrun, and Graesser (2013) studied
whether confusion is an affective state that can be beneficial to
learning. Magner, Schwonke, Aleven, Popescu, and Renkl (2013)
investigated the effects of decorative illustrations and their dis-
tracting or motivating function. Plass, Heidig, Hayward, Homer,
and Um (2013) reported a study on how the design of multimedia
learning materials can induce positive emotions in learners, and
how these positive emotions facilitate cognitive processing and
improve cognitive and affective outcomes.
2. Overview of the studies
Each of the three studies presented in this special section
provides empirical support for different aspects of CATML. D’Mello
et al. (2013) conducted a study on the impact of a particular emotion,
confusion, on learning. Confusion can be a result of contradictions,

0959-4752/$ – see front matter Ó 2013 Elsevier Ltd. All rights reserved.
http://dx.doi.org/10.1016/j.learninstruc.2013.05.005
126 Editorial / Learning and Instruction 29 (2014) 125–127
Fig. 1. Cognitive-affective theory of learning with media. (Moreno, 2006).
conflicts, anomalies, erroneous information, and other discrepant
events in the learning materials, and can result in a state of cognitive
disequilibrium. Under specific conditions, however, confusion can be
beneficial to learning. These conditions include that confusion must
be induced, regulated, and resolved appropriately. The authors
examined the hypothesis that confusion could be beneficial to
learning in two experiments in which learners engaged in conversa-
tions with a tutor and a student peer, both played by animated
agents. These “trialogues on scientific reasoning concepts in a simu-
lated collaborative learning session” (p. 2) were hypothesized to sup-
port learning.
D’Mello et al. (2013) induced confusion in to the environment
via manipulation of the tutor, having the animated agents express
opinions that were at times incorrect and/or contradictory.
Learners were asked to decide the scientific merit of the different
opinions. The study found that these manipulations did not affect
learners’ self-reported confusion. However, learner responses that
immediately followed contradictions proved to be a more effective
indicator of confusion. Performance on post tests (multiple choice
comprehension tests and open-ended transfer tests) showed that
students learned more when the contradictions introduced by the
agent were successful in confusing them. These findings provide
evidence in support of the affective mediation assumption
(Moreno, 2006), which states that affective factors mediate
learning by impacting cognitive engagement, and show that confu-
sion is such an affective state that can be beneficial to learning.
Magner et al. (2013) focused on another affective aspect of the
CATML by investigating the impact of decorative illustrations on in-
terest in and learning from computer-based learning environments.
The researchers conducted two studies to investigate these ques-
tions. The pre-study, involving 87 students in 8th grade, explored
the effects of decorative illustrations on situational interest. Based
on the results of this study, illustrations that had been rated by stu-
dents as highly interesting were selected for use in the main study. In
the main study, which included 52 participants, Magner et al. inves-
tigated the effect of interesting decorative illustrations on immediate
and delayed test performance. The researchers found that decorative
illustrations resulted in lower near transfer for those students who
had low levels of prior knowledge, yet supported students who
had very high levels of prior knowledge. Results also showed that
the higher performance of students receiving decorative illustrations
on far transfer test was due to the enhanced situational interest
induced by these illustrations. However, these findings did not
persist in the delayed post test.
The findings of this study are in line with other research, such as
a study by Park et al. (2011), which found that seductive details
either hindered or fostered learning, depending on the level of
cognitive load they induced (Moreno & Park, 2010; Plass, Moreno,
& Brünken, 2010). Taken together, these findings suggest that the
cognitive processes of selecting relevant information and orga-
nizing this information into a coherent mental model can be
affected not only in a negative way through seductive details or
decorative illustrations, but also in a positive way if learners have
sufficient available resources to process non-redundant and inter-
esting, but irrelevant, learning material. Cognitive resources may
be available as a result of optimized design of the learning environ-
ment, which applied established multimedia learning principles
(Mayer, 2005), or due to learner differences in prior knowledge,
as assumed by the individual differences assumption of CATLM
(Moreno, 2006). The results of this study therefore support the af-
fective mediation assumption of CATLM (Moreno, 2006), suggest-
ing that a deep understanding of multimedia learning effects
requires the consideration of the interplay of cognitive and motiva-
tional factors, and that the motivational or arousing role of seduc-
tive details or decorative illustrations should be taken into account.
Another type of affect was investigated by Plass et al. (2013).
This paper focused on methods to induce positive emotions in
learners, and thereby to improve learning, through the design of
the learning materials. This study had two goals. The first goal
was to replicate findings from a previous study, which had shown
that the emotional design of multimedia learning environments
could induce positive emotions in learners, and that these positive
emotions facilitated learning both on a comprehension and a trans-
fer level (Um, Plass, Hayward, & Homer, 2012). The Plass et al.
(2013) study aimed to replicate these results with a different proce-
dure for mood induction, and with participants from a different
population. It also aimed to examine particular emotions that
learners may experience from different mood induction proce-
dures. Study 1 found indeed that the application of emotional
design principles, using round face-like shapes and warm colors,
was able to induce positive emotions and facilitate comprehension.
However, even though Um et al. (2012) had found an effect of
emotional design on transfer of learning, the Plass et al. (2013)
study was not able to replicate this effect.
 Editorial / Learning and Instruction 29 (2014) 125–127
Plass et al. (2013) next aimed to decompose the effects of two
emotional design elements, color and shape, which had been
used in compounded form in study 1. Results from study 2 showed
that round, face-like shapes induced positive emotions both alone
and in conjunction with warm colors. However, warm colors alone
did not affect learners’ emotions. Comprehension test results were
best for students receiving warm colors, round face-like shapes, or
combinations of warm colors and round face-like shapes. Transfer
test results were highest for round face-like shapes in combination
with neutral colors.
These findings also provide support for the affective mediation
assumption of CATLM (Moreno, 2006), suggesting that emotional
factors mediate learning by increasing the cognitive engagement
of the learner, with resulting gains in learning.
The present special section is discussed by two experts in the field
of multimedia learning, Detlev Leutner (2013) and Richard E. Mayer
(2013). Both commentaries offer critical reflections on the main is-
sues raised by the included papers, and discuss open questions for
future research. These two discussants also provide valuable sugges-
tions on future directions of research and theory development of
cognitive and affective processes in multimedia learning.
3. Conclusion and future directions
The papers in this special section demonstrate that the integra-
tion of cognitive and affective processes in multimedia learning is a
promising area of inquiry that, although still in its infancy, has the
potential to significantly broaden our understanding of multimedia
learning. Findings suggest that including emotion and other affec-
tive variables is not only essential in the process of designing multi-
media learning environments, but that these variables are also
critical in understanding and investigating learning, opening new
perspectives on the integrative nature of learning processes. Within
the frame of the resulting studies, which can be more complex than
studies focusing only on cognitive aspects, it may be necessary to
use more complex data analyses methods that show, for example,
indirect effects of mediation (Magner et al., 2013).
In summary, this special section aimed to bridge the gap be-
tween research on cognitive processes and research on affective
processes in learning and to contribute to the further development
of Roxana Moreno’s CATLM as an integrative framework for
research on learning and instruction. With the present special sec-
tion, we would like to provide momentum to research and theory
development on how to integrate cognitive and affective processes
in research on multimedia learning. We hope that this integrative
research field will grow within the next decades, continuing the
highly productive work initiated by Roxana Moreno.
Acknowledgments
The present work was in part supported by the German Federal
Ministry of Education and Research, as well as by a grant from
Microsoft Research.
References
D’Mello, S., Lehman, B., Pekrun, R., & Graesser, A. (2013). Confusion can be beneficial
for learning. Learning and Instruction, 29, 153–170.
Gottfried, A. (1990). Academic intrinsic motivation in young elementary school chil-
dren. Journal of Educational Psychology, 82, 525–538. http://dx.doi.org/10.1037/
0022-0663.82.3.525.
Kalyuga, S., Ayres, P., Chandler, P., & Sweller, J. (2003). The expertise reversal effect.
Educational Psychologist, 38, 23–32. http://dx.doi.org/10.1207/S15326985EP3801_4.
Leutner, D. (2013). Cognitive and affective processes in learning with multimedia:
commentary on papers by Magner et al., Plass et al., and D’Mello, et al. Learning
and Instruction. in press.
127
Magner, U. I. E., Schwonke, R., Aleven, V., Popescu, O., & Renkl, A. (2013). Triggering
situational interest by decorative illustrations both fosters and hinders learning
in computer-based learning environments. Learning and Instruction, 29, 141–
152.
Mayer, R. E. (Ed.). (2005). The cambridge handbook of multimedia learning. New York:
Cambridge University Press.
Mayer, R. E. (2013). Incorporating motivation into multimedia learning. Learning
and Instruction, 29, 171–173.
McGuinness, C. (1990). Talking about thinking: The role of metacognition in teach-
ing thinking. In K. Gilhooly, M. Deane, & G. Erdos (Eds.), Lines of thinking (pp.
310–312). San Diego: Academic Press.
Moreno, R. (2004). Decreasing cognitive load for novice students: effects of explan-
atory versus corrective feedback on discovery-based multimedia. Instructional
Science, 32, 99–113. http://dx.doi.org/10.1023/B: TRUC.0000021811.66966.1d.
Moreno, R. (2005). Instructional technology: promise and pitfalls. In L. Pytlik Zillig,
M. Bodvarsson, & R. Bruning (Eds.), Technology-based education: Bringing re-
searchers and practitioners together (pp. 1–19). Greenwich, CT: Information Age
Publishing.
Moreno, R. (2006). Does the modality principle hold for different media? A test of
the method-affects-learning hypothesis. Journal of Computer Assisted Learning,
22, 149–158. http://dx.doi.org/10.1111/j.1365-2729.2006.00170.x.
Moreno, R. (2007). Optimizing learning from animations by minimizing cognitive load:
cognitive and affective consequences of signaling and segmentation methods.
Applied Cognitive Psychology, 21, 765–781. http://dx.doi.org/10.1002/acp.1348.
Moreno, R. (2009). Learning from animated classroom exemplars: the case for guid-
ing student teachers’ observations with metacognitive prompts. Journal of
Educational Research and Evaluation, 15, 487–501. http://dx.doi.org/10.1080/
13803610903444592.
Moreno, R., & Durán, R. (2004). Do multiple representations need explanations? The
role of verbal guidance and individual differences in multimedia mathematics
learning. Journal of Educational Psychology, 96, 492–503. http://dx.doi.org/
10.1037/0022-0663.96.3.492.
Moreno, R., Mayer, R. E., Spires, H., & Lester, J. (2001). The case for social agency in
computer-based teaching: do students learn more deeply when they interact
with animated pedagogical agents? Cognition and Instruction, 19, 177–213.
http://dx.doi.org/10.1207/S1532690XCI1902_02.
Moreno, R., & Park, B. (2010). Cognitive load theory: historical development and
relation to other theories. In J. Plass, R. Moreno, & R. Brünken (Eds.), Cognitive
load theory (pp. 9–28). New York: Cambridge University Press.
Morris, C. C. (1990). Retrieval processes underlying confidence in comprehension
judgments. Journal of Experimental Psychology: Learning, Memory, and Cognition,
16, 223–232. http://dx.doi.org/10.1037/0278-7393.16.2.223.
Park, B., Moreno, R., Seufert, T., & Brünken, R. (2011). Does cognitive load moderate
the seductive details effect: a multimedia study. Computers in Human Behavior,
27, 5–10. http://dx.doi.org/10.1016/j.chb.2010.05.006.
Plass, J. L., Chun, D. M., Mayer, R. E., & Leutner, D. (1998). Supporting visual and ver-
bal learning preferences in a second language multimedia learning environ-
ment. Journal of Educational Psychology, 90, 25–36. http://dx.doi.org/10.1037/
0022-0663.90.1.25.
Plass, J. L., Heidig, S., Hayward, E. O., Homer, B. D., & Um, E. (2013). Emotional design
in multimedia learning: effects of shape and color on affect and learning.
Learning and Instruction, 29, 128–140.
Plass, J., Moreno, R., & Brünken, R. (Eds.). (2010). Cognitive load theory. New York:
Cambridge University Press.
Robinson, D. (2010). Editor’s note: remembering Roxana Moreno. Educational
Psychology Review, 22, 363–374. http://dx.doi.org/10.1007/s10648-010-
9147-2.
Seufert, T., Schütze, M., & Brünken, R. (2009). Memory characteristics and modality
in multimedia learning: an aptitude-treatment-interaction study. Learning and
Instruction, 19, 28–42. http://dx.doi.org/10.1016/j.learninstruc.2008.01.002.
Um, E. R., Plass, J. L., Hayward, E. O., & Homer, B. D. (2012). Emotional design in
multimedia learning. Journal of Educational Psychology, 104, 485–498. http://
dx.doi.org/10.1037/a0026609.
Babette Park*
Saarland University, Department of Education, Campus A 4.2,
66 123 Saarbrücken, Germany
Jan L. Plass
New York University, CREATE Lab/Games for Learning Institute,
Steinhardt School of Culture, Education, and Human Development,
196 Mercer St., Room 804, New York, NY 10003, USA
Roland Brünken
Saarland University, Department of Education, Campus A 4.2,
66 123 Saarbrücken, Germany
* Corresponding author. Tel.: þ49 681 302 2531; fax: þ49 681 302
4373.
E-mail address: b.park@mx.uni-saarland.de