Professional Documents
Culture Documents
a r t i c l e i n f o a b s t r a c t
Article history: The present study examined the effectiveness of techniques intended to reduce anxiety as students learn
Received 12 February 2016 mathematical content from a computer-based lesson. In a between-subjects experiment, students
Received in revised form learned statistical rules through worked examples in a computer-based learning environment that either
2 May 2016
did (treatment group) or did not (control group) include anxiety reducing featuresda coping message
Accepted 13 May 2016
Available online 25 May 2016
delivered through the lesson by an online pedagogical agent concerning how to manage feelings of
anxiety, and prompts for expressive writing, in which students summarize their thoughts and feelings.
An independent samples t-test showed that the treatment group, which received added anxiety-
Keywords:
Anxiety
reducing features, showed higher accuracy than the control group on solving practice problems
Statistics learning (d ¼ 0.71) and retention problems (d ¼ 0.63) and reported higher perceived effort on learning the
Computer-based learning multimedia lesson (d ¼ 0.66). In addition, a standard multiple linear regression found that anxiety, self-
Multimedia instruction efficacy, and cognitive load as a set predicted performance (R2 ¼ 0.56), with self-efficacy as the strongest
predictor (b ¼ 0.63). Adding anxiety-reducing features to an online lesson may encourage greater effort,
which leads to better learning outcomes.
© 2016 Elsevier Ltd. All rights reserved.
1. Introduction evoke high anxiety for many learners (Beilock & Willingham, 2014;
Cargnelutti, Tomasetto, & Passolunghi, in press; Ramirez, Chang,
Learning with an online lesson is a complex process that in- Maloney, & Levine, & Beilock, 2016). This line of research has
volves cognitive and affective processes. Although there is a sub- generated considerable discussion about the need to develop
stantial body of research and evidence-based theory concerning strategies to reduce learner anxiety dealing with mathematical
cognitive processes in learning and instruction that can guide the problems (Beilock & Willingham, 2014; Eden, Heine, & Jacobs,
instructional design of computer-based instruction, much less is 2013; Finlayson, 2014), but rigorous experimental studies investi-
known concerning affective processes (Clark & Mayer, 2016; Mayer, gating the effectiveness of such strategies in academic settings are
2014). For example, Fiorella and Mayer (2015) found substantial less common (Brunye et al., 2013), especially within the context of
evidence for eight cognitive-based learning strategies that promote computer-based instruction. As high anxiety is associated nega-
learner understandingdsuch as summarizing, mapping, or dra- tively not only with performance, but other affective outcomes that
wingdbut much less research work on learning strategies aimed at influence a learner’s academic success (Ashcraft & Krause, 2007),
promoting affective processes. In light of the considerable work on more empirical research on effective and scalable anxiety-reducing
cognitive factors in learning from online instruction, we focused on inventions for online learning is needed.
the less-studied issue of affective factors. The goal of the present study is to examine whether adding
One important affective factor that has been shown to nega- research-based anxiety-reducing features to a computer-based
tively impacts academic outcomes is learner anxiety, especially in multimedia lesson on statistics can have an impact on learner
subject matter areas involving mathematical skills that tend to performance (including on solving problems during learning and
on posttests), as well as on internal characteristics such as learner
perceived anxiety level, self-efficacy, cognitive load, and perceived
* Corresponding author. effort. Applying anxiety-reducing features in a computer-based
E-mail addresses: xiaoxia.huang@wku.edu (X. Huang), rich.mayer@psych.ucsb. setting offers a promising approach for scaling up the
edu (R.E. Mayer).
http://dx.doi.org/10.1016/j.chb.2016.05.034
0747-5632/© 2016 Elsevier Ltd. All rights reserved.
294 X. Huang, R.E. Mayer / Computers in Human Behavior 63 (2016) 293e303
interventions in online learning environments. Furthermore, to our 2.2. Learner anxiety and self-efficacy, cognitive load, and attitude
knowledge, no studies have tested the impact of anxiety-coping
strategies incorporating the broad set of outcome measures In addition to leaning performance, learner anxiety can impact
included in this study. Although some previous research involving motivational factors such as self-efficacy, i.e., perceived confidence
anxiety-reducing interventions examined their impact on learner in successfully performing a specific task (Bandura, 1997). The
anxiety or performance (e.g., Park, Ramirez, & Beilock, 2014; Sharp, relation between anxiety and self-efficacy has been extensively
Coltharp, Hurford, & Cole, 2000), we seek to expand the scope of established both theoretically and empirically. Anxiety is consid-
inquiry to include self-efficacy, cognitive load, and perceived effort, ered a physiological or affective source of self-efficacy (Bandura,
which may be affected by anxiety reduction. These latter outcome 1997). Increased anxiety tends to be associated with lowered self-
variables are important to include in order to more holistically efficacy. This proposal is consistent with a number of empirical
evaluate the impact of anxiety-reducing strategies, and further studies that found a negative correlation between the two con-
explore the connection among these variables (Bandura, 1997; structs (e.g., Griggs et al., 2013). Despite the association between
Salomon, 1984). anxiety and self-efficacy, empirical research testing the effect of
In addition, as previous research has shown the link among anxiety-reducing interventions on self-efficacy is limited (Im,
learner anxiety, self-efficacy, cognitive load, and performance, we 2012).
aim to examine the relations among these intended outcome var- Another construct associated with anxiety is cognitive load,
iables and how anxiety, self-efficacy and cognitive load as a set which is defined as the overall mental activity demand, exerted at a
predict learning performance. Previous research on the relations particular time, on the limited capacity of working memory
among anxiety, self-efficacy and performance is abundant (Griggs, (Sweller, Ayres, & Kalyuga, 2011). Literature distinguishes three
Patton, Rimm-Kaufman, & Merritt, 2013; Pajares & Barich, 2005; types of cognitive load, i.e., intrinsic cognitive load concerning the
Pajares, 1996), and the relation of cognitive load and performance inherent difficulty level of the learning materials, germane cognitive
has been established (e.g., Paas, Renkl, & Sweller, 2004); however, load concerning mental activities directly contributing to learning,
relatively few studies have included cognitive load in the model and extraneous cognitive load concerning mental activities irrele-
together with learner anxiety and self-efficacy in predicting per- vant to learning (Sweller, Van Merrie €nboer, & Paas, 1998; Sweller,
formance. It is worthwhile to investigate these relations in one 1994). Cognitive load has been commonly measured by the
model involving an authentic statistical learning task. amount of mental effort learners invest on a task (Paas, Van
Merrie €nboer, & Adams, 1994). Salomon (1983) considered mental
effort containing cognitive as well as motivational attributes and
2. Literature review defined it as “the employment of nonautomatic elaborations per-
formed on the material” (p. 45). Salomon also claimed that mental
2.1. Learner anxiety and performance effort was in essence related to the “depth,” “mindfulness,” or
“thoughtfulness” during the learning process (1983, p. 42). In this
Anxiety is defined as “a state of anticipatory apprehension over sense, the concept of mental effort is aligned more with the defi-
possible deleterious happenings” (Bandura, 1997, p.137). Learner nition of germane cognitive load. Although previous research in
anxiety in academic settings is a phenomenon that concerns many foreign language learning has shown a positive correlation be-
researchers due to its detrimental effects during the learning pro- tween anxiety and cognitive load, which in turn negatively related
cess. Literature has identified two components of anxiety: the to performance (Chen & Chang, 2009), the exact relation between
worry component concerning the cognitive aspect of anxiety, e.g., anxiety and cognitive load, both concerning working memory,
performance concerns, and the emotionality component concerning might be complex, depending on the amount of anxiety aroused. A
the affective aspect of anxiety, e.g., unpleasant feeling of tension high level of anxiety can be considered a source of extraneous
(Morris, Davis, & Hutchings, 1981). cognitive load, while an optimum level of anxiety may contribute to
The concept of anxiety is closely related to working memory, in germane cognitive load. Overall, it is reasonable to posit that
the sense that thinking about one’s anxiety can waste limited anxiety-reducing interventions should lower the extraneous
cognitive processing resources in working memory. For complex cognitive load of high-anxious learners while enabling them to
learning tasks that demand resources of working memory, the invest more mental effort directly on the learning task (increased
performance of highly anxious learners will “suffer the most” as germane cognitive load).
their working memory resources are wasted on dealing with their Furthermore, Bandura (1997) established the link between
anxiety rather than the task on hand (Ashcraft & Krause, 2007, p. mental effort and self-efficacy, indicating that self-efficacy would
246). affect the amount of effort spent on a particular task: learners who
Researchers have also argued that depending on the amount of feel more efficacious are more likely to invest more effort in
anxiety that is aroused, the impact of anxiety could exert a debili- completing a task. Empirical research has observed positive cor-
tating or facilitating effect on learning (Little & Wuensch, 2015). In relation between self-efficacy and cognitive load (Vasile, Marhan,
other words, high anxiety that exceeds the optimum level will Singer, & Stoicescu, 2011). Salomon (1984) indicated that the cor-
negatively affect student performance, while a moderate level of relation between mental effort and self-efficacy could be positive or
anxiety may motivate learners to achieve better performance. negative, depending on whether the instructional materials are
Nevertheless, for subject matter areas involving mathematical skills perceived as demanding or not. He observed a positive correlation
that tend to provoke high anxiety for learners, developing in- between mental effort and self-efficacy for demanding materials
terventions to alleviate their anxiety is essential. For example, the while a negative correlation for easy materials.
negative impact of math anxiety on math performance has been In sum, learner anxiety can exert an impact on performance as
well documented in the literature (Ashcraft, 2002; Hembree, 1990). well as on self-efficacy and cognitive load. In addition, anxiety, self-
As reported in Hembree’s (1990) classic meta-analysis, higher math efficacy and cognitive load are linked to learner performance.
anxiety and lower math performance are consistently associated Previous research has also reported that high anxiety is normally
with each other and that anxiety-reducing interventions can help associated with negative attitudes and perceptions about problems
increase the performance of students with high math anxiety to the and problem-solving (Dugas & Robichaud, 2007). Hence, devel-
performance level of students with low math anxiety. oping effective interventions to reduce learner anxiety is
X. Huang, R.E. Mayer / Computers in Human Behavior 63 (2016) 293e303 295
in example-based learning research that have demonstrated evi- by a pedagogical agent that students would receive before they
dence of learning benefits (Leppink, Paas, Van Gog, Van der started the example-problem pair practice (as exemplified in
Vleuten, & Van Merrie€nboer, 2014; Renkl, 2011, 2014; Van Gog, Fig. 3a), and the second strategy was an expressive writing activity
2011). We are interested in exploring whether integrating after students completed the example-problem practice before
anxiety-reducing features into an example-based multimedia they started the posttest (as exemplified in Fig. 3b).
learning environment could further aid the learning process as The agent-delivered anxiety coping message was developed
compared to when such features are not included. Based on pre- based on previous research (Carver et al., 1989; Dugas & Robichaud,
vious research on anxiety-reducing features in other contexts, we 2007; Im, 2012; Shen, 2009; Zettle, 2003), focusing on coping
predict that adding anxiety-reducing features to the statistical through statements that acknowledge or accept the student being
multimedia lesson will result in: anxious when learning new content (e.g., “If you are feeling
anxious, just know that when we learn anything new, it is normal to
a lower level of perceived anxiety during the practice activity feel anxious or nervous. You don’t need to focus on getting rid of
and during the test (Predictions 1e2), it.”) as well as the encouragement of engaging in active coping (e.g.,
better performance on practice exercises, retention test, and the “Whether studying the example or solving the paired problem, just
transfer test (Predictions 3e5), focus on what has to be done, one step at a time. You will not feel as
a higher level of self-efficacy (Prediction 6), anxious if you concentrate your efforts on what needs to be done.”).
a higher level of mental effort (i.e., indicator of cognitive load) For example, below is the transcript of the agent-delivered message
during the practice exercises and the transfer test (Predictions presented at the beginning of the treatment lesson, which was
7e8), and adapted from Carver et al. (1989) and Dugas and Robichaud (2007):
a more positive attitude at the end of the program (Prediction 9)
For this practice, you will work on 5 example-problem pairs. For
each pair, you will study the example first, and then solve the
We also predict that anxiety, cognitive load (indicated by mental
paired problem. You will get immediate feedback for your
effort), and self-efficacy are interrelated and contribute as a set to
problem solving. If you are feeling anxious, just know that when
the prediction of performance (Prediction 10).
we learn anything new, it is normal to feel anxious or nervous.
The present study adds to the limited research base on inte-
You don’t need to focus on getting rid of it. After all, we all learn
grating anxiety-reducing features to computer-based learning en-
to live with our anxiety. If you think about anything new that
vironments (e.g., Im, 2012; Shen, 2009; Wei, 2010), and expands
you have learned in your life, such as the first time you drove a
the outcome measures of previous research to include learners’
car, you were probably quite anxious. However, with time and
cognitive load and attitude in addition to their perceived anxiety,
practice, that discomfort went away. It is through repeated
self-efficacy, and performance. The study also contributes to the
practice that we improve our skills and feel confident while
literature on learner anxiety, cognitive load, and self-efficacy as
using them. The 5 example-problem pairs to be presented next
predictors of performance with the involvement of an authentic
focus on just that: engaging you in the repeated practice. Your
statistical learning task in a computer-based learning environment.
goal is to study an example first, and then solve the paired
problem following the similar procedure. Whether studying the
4. Method
example or solving the paired problem, just focus on what has to
be done, one step at a time. You will not feel as anxious if you
4.1. Participants and design
concentrate your efforts on what needs to be done. Now, click
the Next button to start working on the example-problem pairs.
The participants were 54 undergraduates across various disci-
plines at a midwest university in the United States (mean
age ¼ 18.55, SD ¼ 1.26; females ¼ 37, males ¼ 17). The participants The second anxiety coping strategy was developed based on
were novice learners on the instructional topic (i.e., only students previous research on the strategy of expressive writing (Park et al.,
who reported that they had not learned the topic were included) 2014), which is similar to journal writing (Sgoutas-Emch &
and were randomly assigned to one of the two conditions, with 28 Johnson, 1998). Below is the instruction adapted from Park et al.
participants in the control group and 26 in the treatment group. (2014, p. 106) provided to the students in the treatment condition
before the posttest:
4.2. Materials and apparatus
Next, you will work on a set of tests related to the two statistical
rules you just learned. Please take the next 5 minutes to write as
The instructional materials consisted of a treatment version and
openly as possible about your thoughts and feelings regarding
control version of a computer-based instructional lesson on two
the statistical problems you are about to perform. In your
statistical rules that are related to standard deviation: the Empirical
writing, just really let yourself go and explore your emotions
Rule and Chebyshev’s Rule. The goal of the instruction was for
and thoughts as you are getting ready to start the set of prob-
students to be able to state the two rules, distinguish the conditions
lems. You might relate your current thoughts to the way you
for applying the two rules as well as use the two rules to find data
have felt during other similar situations at school or in other
values and draw conclusions about a data set. The learning content
situations in your life. Please try to be as open as possible as you
for both versions of the lesson was organized in the following or-
write about your thoughts at this time. Remember, your writing
der: (a) introduction and review of mean (around 532 words), (b)
will be confidential. Type in the text box below when you are
introduction to standard deviation (around 1650 words), (c) prac-
ready.
tice (on 5 pairs of standard example-problems; a screenshot is
shown in Fig. 1a for a standard worked example and Fig. 1b for the
corresponding paired problem, respectively), and (d) posttest. Fig. 2 The posttest consisted of two tests of student performance (6-
summarizes the content organization of the instructional lessons. item retention test, and 11-item transfer test, with embedded
The treatment version of the lesson was identical to the control measures of cognitive load and task anxiety), followed by a 10-item
version, but had two anxiety coping strategies added into the self-report survey with three subscales (motivation, perceived
lesson. The first strategy was an anxiety coping message delivered
X. Huang, R.E. Mayer / Computers in Human Behavior 63 (2016) 293e303 297
Fig. 1. a. A screenshot of the lesson depicting a standard worked example that students received during practice. b. A screenshot of the lesson depicting a practice problem paired to
the corresponding standard worked example that students received during practice.
difficulty, and perceived effort). In addition, perceived self-efficacy covered in the program (Cronbach’s a ¼ . 88), e.g., “Per Empirical
was measured both before the lesson and at the end of the lesson Rule, approximately __% of the data in a distribution fall within 1
(before the posttest). All performance measures included multiple- standard deviation from the mean.”
choice or fill-in-the-blank questions. The transfer test consisted of 6 near transfer questions and 5 far
The retention test consisted of 6 questions requiring students to transfer questions (Cronbach’s a ¼ 0.92). Each near transfer ques-
recall or recognize statements concerning the two statistical rules tion contains 3 sub-questions, similar to the practice problems that
298 X. Huang, R.E. Mayer / Computers in Human Behavior 63 (2016) 293e303
Fig. 2. Visual display of the content organization of the instructional program. Anxiety coping information in the shaded boxes were presented to the treatment group only.
required students to apply one of the two rules (Cronbach’s regarding the statistical problems helpful in reducing my anxiety in
a ¼ 0.88), e.g., “A data set of size N ¼ 100 with a bell-shaped dis- performing the statistical tasks in the module.’ Why or why not?”
tribution has mean ¼ 5 and standard deviation ¼ 1.5. Please In addition, a demographic survey was included in the study
determine the following: How many standard deviations are 3.5 consisting of questions on student age, gender, self-reported
and 6.5 away from the mean?” The far transfer questions consisted knowledge level of statistics (5-point rating scale, with 1 ¼ least
of 5 real-life scenario problems (5 problems contained 3 sub- skilled, and 5 ¼ most skilled; “Rate your knowledge level of statis-
questions and 1 problem was a multiple-answer question) tics.”), and comfort level with computer-based instruction (5-point
requiring students to apply the rules in a context different than rating scale, with 1 ¼ least comfortable, and 5 ¼ most comfortable;
those presented in the practice problems (Cronbach’s a ¼ 0.85), e.g., “Rate your comfort level with computer-based instruction.”).
“200 students took an IQ test. The scores showed a bell-shaped The apparatus consisted of 30 Dell desktop computers with 20-
distribution with a mean of 100 and a standard deviation of 17. inch color monitors and Windows 7 operating systems. Handheld
Please determine the following: Approximately how many people calculators and 3.5 mm Jack earphones were also used for the study.
have an IQ score between 83 and 117?”
Self-efficacy, in line with previous research (Bandura, 2006;
Frank; Pajares, Hartley, & Valiante, 2001), was measured by a 100- 4.3. Procedure
point rating scale (0 ¼ not confident at all, and 100 ¼ extremely
confident) on perceived confidence on performing 6 tasks related to The study was conducted in a 30-station computer lab at the
the knowledge and application of the two statistical rules (Cron- university, with multiple participants tested in each session.
bach’s a ¼ 0.98 and 0.97 for Time 1 measure and time 2 measure, Participant consent was obtained after the experimenter’s brief
respectively), e.g., “On a scale from 0 (cannot do at all) to 100 (highly description of the study at the beginning of each session. Partici-
certain can do), please rate how certain you are that you can pants were then instructed to access the computer-based program
perform each of the statistics skills listed on the page.” and follow the directions on the screen to complete the program
Cognitive load was measured by a one-item, 9-point Likert scale (including practice problems), posttests, and survey at their own
(1 ¼ very, very low mental effort, and 9 ¼ very, very high mental effort) pace (see Fig. 2 regarding the structure of the program). The
focusing on the mental effort invested during the practice, near average time participants spent on the self-paced program was
transfer test, and far transfer test (Paas et al., 1994), e.g., “Indicate about 50 min. Each participant received $15 monetary incentive
the amount of mental effort you spent on completing the previous after completing the study. We followed guidelines for treatment of
exercise.” Anxiety was measured by a similar one-item, 9-point human subjects.
Likert scale (1 ¼ very, very low anxiety, and 9 ¼ very, very high
anxiety), e.g., “Indicate the amount of anxiety you felt during the 5. Results
process of studying the example-problem pairs.”
In addition, student attitude towards the computer-based 5.1. Did the groups differ on basic characteristics?
instructional program was measured by a 5-point Likert scale
(1 ¼ strongly disagree, and 5 ¼ strongly agree) adapted from Olina The first step is to determine the equivalence of the two groups
et al. (2006), which included 10 items targeting three sub- in terms of age, gender, self-reported knowledge level of statistics,
constructs: motivation (items 1e4, Cronbach’s a ¼ 0.89), e.g., “I comfort level with computer-based instruction, and the initial self-
liked studying about the Empirical Rule and the Chebyshev’s Rule”, efficacy level before the multimedia lesson. Independent samples t-
perceived difficulty (items 5e7, Cronbach’s a ¼ 0.79), e.g., “I had a tests indicated that there was no significant difference between the
hard time understanding how to apply the Empirical Rule and the two groups on age (Mean control group ¼ 18.56, SD ¼ 1.50; Mean
Chebyshev’s Rule to different contexts”, and perceived effort on
treatment group ¼ 18.54, SD ¼ 0.99, p ¼ 0.96), comfortable level with
learning the multimedia lesson (items 8e10, Cronbach’s a ¼ 0.79), computer-based instruction (Mean control group ¼ 3.70, SD ¼ 1.17;
e.g., “I carefully studied the explanations and examples”. Mean treatment group ¼ 3.62, SD ¼ 1.06, p ¼ 0.78), knowledge level of
To better understand the effect of the anxiety-coping strategies statistics (Mean control group ¼ 1.67, SD ¼ 0.83; Mean treatment
implemented in the treatment version of the module, we asked two
group ¼ 1.69, SD ¼ 0.74, p ¼ 0.91), or initial perceived self-efficacy
open-ended questions to the participants in this group whether level (Mean control group ¼ 5.24, SD ¼ 11.55; Mean treatment
they found each of the strategies helpful in reducing their anxiety
group ¼ 0.87, SD ¼ 1.92, p ¼ 0.06). In addition, chi-square tests (with
during the learning process: 1) “Do you agree with the statement ‘I Yates’ correction for continuity) indicated that there was no sig-
found the message delivered by the animated virtual instructor nificant difference in the proportion of males and females between
helpful in reducing my anxiety in performing the statistical tasks in the control group (females ¼ 18; males ¼ 10) and the treatment
the module.’ Why or why not?” and 2) “Do you agree with the group (females ¼ 19; males ¼ 7), c2 (1, N ¼ 54) ¼ 0.161, p ¼ 0.69. We
statement ‘I found writing about my thoughts and feelings conclude that the groups were equivalent on basic characteristics.
X. Huang, R.E. Mayer / Computers in Human Behavior 63 (2016) 293e303 299
Fig. 3. a. A screenshot depicting the pedagogical agent delivering the anxiety coping message included in the treatment version, which was presented before the example-problem
practice. b. A screenshot depicting the expressive writing strategy (adapted from Park et al., 2014) included in the treatment version, which was presented immediately after the
example-problem practice while before the posttest.
5.2. Did adding anxiety-reducing features affect student anxiety the anxiety coping strategies would report a significantly lower
level (predictions 1e2)? level of anxiety during the practice (Prediction 1) and the test
(Prediction 2). As shown in lines 1 and 2 of Table 1, and contrary to
One of the main research questions of the study concerns the our predictions, independent-samples t-tests detected no signifi-
effect of the two anxiety-coping strategies on student anxiety level cant difference between the treatment group and the control group
during their learning process. We predicted that students receiving on the measures of either anxiety during practice (Mean control
300 X. Huang, R.E. Mayer / Computers in Human Behavior 63 (2016) 293e303
Table 1 the control group on the measures of cognitive load during practice
Means and standard deviations for the outcome measures. (Mean control group ¼ 5.54, SD ¼ 1.73; Mean treatment group ¼ 5.73,
Outcome measure Control group Treatment group SD ¼ 1.66), t(52) ¼ 0.42, p ¼ 0.68, d ¼ 0.11, or cognitive load during
M SD M SD d
the transfer test (Mean control group ¼ 5.00, SD ¼ 1.77; Mean treatment
group ¼ 5.40, SD ¼ 1.83), t(52) ¼ 0.82, p ¼ 0.414, d ¼ 0.22. Therefore,
Anxiety ePractice (9) 5.29 1.80 4.92 2.23 0.18
contrary to our prediction, adding the anxiety coping strategies did
Anxiety eTest (9) 5.18 2.06 4.89 2.34 0.13
Practice (20) 11.68 5.08 15.04* 4.36 0.71 not have any significant impact on student cognitive load.
Retention (6) 3.54 2.28 4.81* 1.70 0.63
Transfer (66) 39.11 16.74 44.04 18.87 0.28 5.6. Did adding anxiety-reducing affect student attitudes
Self-Efficacy (100) 44.30 34.58 54.19 31.15 0.30
(Prediction 9)?
Cognitive Load e Practice (9) 5.54 1.73 5.73 1.66 0.11
Cognitive Load e Transfer (9) 5.00 1.77 5.40 1.83 0.22
Attitude e motivation (5) 3.04 1.16 3.48 0.91 0.42 Finally, with regard to student attitude as measured by the three
Attitude e perceived difficulty (5) 3.69 0.90 3.31 1.27 0.35 sub-scales shown in lines 9e11 of Table 1 (i.e., motivation,
Attitude e perceived effort (5) 4.15 0.68 4.54* 0.48 0.66 perceived difficulty, perceived effort), the results indicated that
Notes. Maximum score indicated in parentheses; asterisk (*) indicates treatment students in the treatment condition reported higher perceived
group significantly outscored control group at p < 0.05. effort (M ¼ 4.54, SD ¼ 0.48) on learning the module than did their
counterparts in the control condition (M ¼ 4.15, SD ¼ 0.68),
t(52) ¼ 2.37, p ¼ 0.02, d ¼ 0.66. No significant difference was
group ¼ 5.29, SD ¼ 1.80; Mean treatment group ¼ 4.92, SD ¼ 2.23),
detected between the groups on motivation (Mean control
t(52) ¼ 0.66, p ¼ 0.51, d ¼ 0.18, or anxiety during the test (Mean
group ¼ 3.04, SD ¼ 1.16; Mean treatment group ¼ 3.48, SD ¼ 0.91),
control group ¼ 5.18, SD ¼ 2.06; Mean treatment group ¼ 4.89, SD ¼ 2.34),
t(52) ¼ 1.56, p ¼ 0.13, d ¼ 0.42, or perceived difficulty (Mean control
t(52) ¼ 0.49, p ¼ 0.63, d ¼ 0.13. Thus, there is no evidence that the
group ¼ 3.69, SD ¼ 0.90; Mean treatment group ¼ 3.31, SD ¼ 1.27),
anxiety-reducing features reduced self-reported levels of anxiety
t(52) ¼ 1.29, p ¼ 0.20, d ¼ 0.35. Overall, our prediction on student
during learning and testing.
attitude was partially supported.
5.3. Did adding anxiety-reducing features facilitate practice and 5.7. What worked and what didn’t? A qualitative look at anxiety-
test performance (predictions 3e5)? reducing features
As far as student performance was concerned, we predicted that Student responses to the two open-ended questions in the
the treatment group would perform better on practice problems treatment group were analyzed to identify emerging themes
(Prediction 3), the retention test (Prediction 4), and the transfer test regarding their perceptions of the agent-delivered anxiety coping
(Prediction 5) as compared to the control group. As shown in line 3 message and the expressive writing activity. A summary of the
in Table 1, and consistent with our prediction 3, a t-test showed that results is presented in Table 2.
students in the treatment condition performed significantly better Overall, for each of the two anxiety coping strategies, about half
on the practice exercise (M ¼ 15.04, SD ¼ 4.36) than those students of the participants reported that they found the strategy helpful
in the control condition (M ¼ 11.68, SD ¼ 5.08), t(52) ¼ 2.60, while the other half participants did not agree. With regard to the
p ¼ 0.012, d ¼ 0.71. Similarly, as shown in line 4 of Table 1, and in agent-delivered anxiety coping message, the main themes emerged
line with prediction 4, students in the treatment condition per- why it worked include: (1) the message was encouraging (e.g., “…
formed significantly better on the retention test (M ¼ 4.81, reminded me that several other people in several other instances
SD ¼ 1.70) than their counterparts in the control condition has experienced the same anxious feeling when encountering
(M ¼ 3.54, SD ¼ 2.28), t(49.68) ¼ 2.33, p ¼ 0.024, d ¼ 0.63. something new or challenging.”); and (2) the message provided
However, as shown in line 5 of Table 1, no significant difference was detailed information over what to do (e.g., “… because when you’re
found between the treatment group (M ¼ 44.04, SD ¼ 18.87) and sitting here at a computer by yourself about to learn something
the control group (M ¼ 39.11, SD ¼ 16.74) on the transfer test, completely new, hearing a voice telling you what to expect and do
t(52) ¼ 1.02, p ¼ 0.314, d ¼ 0.28. Overall, our predictions on is helpful!”). With regard to the expressive writing strategy, the
student performance were partially supported. main themes why it worked include: (1) it helped participants to
relax or feel calm (e.g., “… because it let me relax for a little and
5.4. Did adding anxiety-reducing strategies affect student self- actually think about my feelings.”); (2) it allowed venting of feel-
efficacy (Prediction 6)? ings (e.g., “… because writing out my thoughts about the problems
allowed me to vent any feelings about being tested and that
We also predicted that the treatment group would report higher reduced my stress levels”); and (3) writing was something partic-
self-efficacy than the control group at the end of the module. As ipants enjoyed (e.g., “… because I am naturally a writer … doing
shown in line 6 of Table 1, this prediction was not supported. No something I really enjoy helped me relax a little bit and forget the
difference was detected between the treatment group and the stress of doing something that is not my forte”).
control group on the measure of post-self-efficacy (Mean control For participants who did not find the strategies helpful, the main
group ¼ 44.30, SD ¼ 34.58; Mean treatment group ¼ 54.19, SD ¼ 31.15), themes emerged include: (1) participants had low anxiety to start
t(52) ¼ 1.10, p ¼ 0.28, d ¼ 0.30. The results did indicate a trend with (e.g., “I don’t have big problems with anxiety.”); (2) the coping
that the treatment condition had slightly higher self-efficacy at the message delivered by the agent was known information (e.g., “…
end of the module. because I already know … that anxiety is common and all the other
things she talked about.”); (3) participants had difficulty writing
5.5. Did adding anxiety-reducing features affect cognitive load about their feelings (e.g., “I had to think really hard to come up with
(Predictions 7e8)? an example to relate my feelings to.”); (4) one of the strategies
actually triggered their anxiety (e.g., “I hadn’t thought about being
As shown in lines 7 and 8 of Table 1, with regard to cognitive anxious or apprehensive until she pointed it out.”); (5) one of the
load, no difference was detected between the treatment group and strategies served as an distraction (e.g., “… I felt as though it made
X. Huang, R.E. Mayer / Computers in Human Behavior 63 (2016) 293e303 301
Table 2
Themes that emerged from the open-ended questions.
Agent-delivered message Message was encouraging (4) Message presented known information (3)
Message provided details on what to do (2) Participant had low anxiety to start with (3)
Message triggered anxiety (2)
Message was not strong enough due to high math anxiety (2)
Expressive writing Writing helped participant to relax or feel calm (6) Activity triggered/increased anxiety (4)
Writing allowed venting or expression of feelings (4) Participant had difficulty writing about feelings (2)
Participant enjoyed writing about feelings (2) Activity distracted participant (2)
Notes: Numbers in the parentheses indicate the frequency of occurrence for the corresponding themes; Single occurrences were not reported in the table.
lem.”); and (6) strategies not strong enough due to math content Variables b t p R2
(e.g., “I always get nervous when it comes to math.”)
0.56
Post self-efficacy 0.63 6.23 0.000
5.8. How did test performance, anxiety, self-efficacy, and cognitive Cognitive load during test 0.22 2.05 0.046
load correlate with each other? Anxiety during test 0.21 2.08 0.042
Measures 1 2 3 4 The pattern of results found in this study suggest that tech-
1. Self-efficacy e 0.02 0.34* 0.71** niques intended to reduce anxiety may cause an increase in student
2. Anxiety during test e 0.35** 0.15 effortdperhaps because students feel more supported and val-
3. Cognitive load during test e 0.36** ueddwhich is reflected in better performance on practice items
4. Performance on test e and on the retention test. More work is needed to pinpoint an
Note: *p < 0.05 **
p < 0.01. explanation for the lack of effect on perceived anxiety.
302 X. Huang, R.E. Mayer / Computers in Human Behavior 63 (2016) 293e303
One possible explanation is that, as some participants noted, (2013). Learning to relax: evaluating four brief interventions for overcoming the
negative emotions accompanying math anxiety. Learning and Individual Differ-
they did not have anxiety to start with when completing the
ences, 27, 1e7.
multimedia lesson; thus, the anxiety-reducing strategies may have Cargnelutti, E., Tomasetto, C., & Passolunghi, M.C. How is anxiety related to per-
mainly been interpreted as showing encouragement and concern formance in young students? A longitudinal study of Grade 2 to Grade 3 chil-
for the learner which translated into greater effort. dren. Cognition and Emotion. in press.
Carver, C. S., Scheier, M. F., & Weintraub, J. K. (1989). Assessing coping strategies: a
theoretically based approach. Journal of Personality and Social Psychology, 56(2),
6.3. Practical implications 267e283. Retrieved from http://www.ncbi.nlm.nih.gov/pubmed/2926629.
Chen, I., & Chang, C. (2009). Cognitive load theory: an empirical study of anxiety
and task performance in language learning. Electronic Journal of Research in
On the practical level, these results encourage the idea that Educational Psychology, 7(56), 729e746.
instructional designers should consider how to incorporate Clark, R. C., & Mayer, R. E. (2016). e-Learning and the science of instruction (4th ed).
anxiety-reducing features in computer-based lessons. Although Hoboken, NJ: Wiley.
Derakshan, N., & Eysenck, M. W. (2009). Anxiety, processing efficiency, and cogni-
there are well-established guidelines for how to design lessons tive performance. European Psychologist, 14(2), 168e176. http://doi.org/10.1027/
based on cognitive features (Mayer, 2009, 2014), guidelines based 1016-9040.14.2.168.
on affective features are less well established. This research shows Dove, A., & Dove, E. (2015). Examining the influence of a flipped mathematics
course on preservice elementary teachers’ mathematics anxiety and achieve-
the potential benefits of incorporating features that address affec- ment. Electronic Journal of Mathematics & Technology, 9(2).
tive processes as well as cognitive processes. Dugas, M. J., & Robichaud, M. (2007). Cognitive-behavioral treatment for generalized
anxiety disorder: From science to practice. New York, NY: Routledge.
Eden, C., Heine, A., & Jacobs, A. M. (2013). Mathematics anxiety and its development
6.4. Limitations and future directions in the course of formal schooling d A review. Psychology, 4(6), 27e35.
Finlayson, M. (2014). Addressing math anxiety in the classroom. Improving Schools,
It would be useful to determine how anxiety-reducing tech- 17(1), 99e115. http://doi.org/10.1177/1365480214521457.
Fiorella, L., & Mayer, R. E. (2015). Learning as a generative activity: Eight learning
niques work within the context of an actual online course with a strategies that promote understanding. New York: Cambridge University Press.
large sample size, in order to gauge how well the effects hold up Gan, S. K.-E., Lim, K. M.-J., & Haw, Y.-X. (2015). The relaxation effects of stimulative
beyond a short-term small-scale lab study. Further, the present and sedative music on mathematics anxiety: a perception to physiology model.
Psychology of Music, 1e12. http://doi.org/10.1177/0305735615590430.
study was conducted with undergraduates on the topic of statistics Griggs, M. S., Patton, C. L., Rimm-Kaufman, S. E., & Merritt, E. G. (2013). The
in one university in the US. Future research is needed to examine responsive classroom approach and fifth grade students’ math and science
whether the demonstrated benefits of anxiety-reducing features anxiety and self-efficacy. School Psychology Quarterly, 28(4), 360e373.
Hembree, R. (1990). The nature, effects, and relief of mathematics anxiety. Journal of
can be applied to other academic subject matters or to younger
Research in Mathematics Education, 21(1), 33e46.
learners in K-12 settings. Huang, Y. M., Huang, S. H., & Wu, T. T. (2014). Embedding diagnostic mechanisms in
Finally, it might be useful to explore ways to measure both trait a digital game for learning mathematics. Educational Technology Research and
anxiety as a personality dimension and state anxiety as a transient Development, 62(2), 187e207. http://doi.org/10.1007/s11423-013-9315-4.
Im, T. (2012). The effects of emotional support and cognitive motivational messages on
mood state (Derakshan & Eysenck, 2009). Measuring participants’ math anxiety, self- efficacy, and math problem solving. Doctoral dissertation.
trait anxiety at the beginning may provide us additional insights Florida State University.
with regard to the effect of the anxiety-reducing features on the Jansen, B. R. J., Louwerse, J., Straatemeier, M., Van der Ven, S. H. G., Klinkenberg, S., &
Van der Maas, H. L. J. (2013). The influence of experiencing success in math on
intended learning and motivational outcomes, as state anxiety is math anxiety, perceived math competence, and math performance. Learning
influenced by both trait anxiety and situational stress (Derakshan & and Individual Differences, 24, 190e197. http://doi.org/10.1016/j.lindif.2012.12.
Eysenck, 2009). Future research should consider including this 014.
Leppink, J., Paas, F., Van Gog, T., Van der Vleuten, C. P. M., & Van Merrie €nboer, J. J. G.
additional measure of anxiety, including behavioral and physio- (2014). Effects of pairs of problems and examples on task performance and
logical measures, and possibly focusing on students with high trait different types of cognitive load. Learning and Instruction, 30, 32e42. http://doi.
anxiety when conducting an experiment to examine the effec- org/10.1016/j.learninstruc.2013.12.001.
Lester, J. C., Towns, S. G., Callaway, C. B., Voerman, J. L., & Fitzgerald, P. J. (2000).
tiveness of anxiety-reducing features integrated in the instruction. Deictic and emotive communication in animated pedagogical agents. In
J. J. Cassell, S. P. Sullivan, & E. Churchill (Eds.), Embodied conversational agents
Acknowledgement (pp. 123e154). Cambridge, MA: Cambridge University Press.
Little, G. B., & Wuensch, K. (2015). Is the relationship between anxiety and creativity
moderated by other emotional states? PSI CHI Journal of Psychological Research,
This research was supported by a Research & Creative Activities 20(3), 143e150.
Program grant (RCAP#14-8014) at Western Kentucky University. Mayer, R. E. (2009). Multimedia learning (2nd ed.). New York: Cambridge University
Press.
Mayer, R. E. (2014). The Cambridge handbook of multimedia learning (2nd ed.). New
References York: Cambridge University Press.
Morris, L. W., Davis, M. a, & Hutchings, C. H. (1981). Cognitive and emotional
Ashcraft, M. H. (2002). Math anxiety: personal, educational, and cognitive conse- components of anxiety: literature review and a revised worryeemotionality
quences. Current Directions in Psychological Science, 11(5), 181e185. http://doi. scale. Journal of Educational Psychology, 73(4), 541. http://doi.org/10.1037/0022-
org/10.1111/1467-8721.00196. 0663.73.4.541.
Ashcraft, M. H., & Krause, J. A. (2007). Working memory, math performance, and Olina, Z., Reiser, R., Huang, X., Kim, H., Lee, H., Lim, J., … Son, C. (2006). Strategies for
math anxiety. Psychonomic Bulletin & Review, 14(2), 243e248. http://doi.org/10. maximizing learning from worked examples to facilitate acquisition of comma
3758/BF03194059. rules. Paper presented as part of a symposium on worked examples research at
Atkinson, R. K., Mayer, R. E., & Merrill, M. M. (2005). Fostering social agency in 2006 AERA annual meeting, San Francisco, CA.
multimedia learning: examining the impact of an animated agent’s voice. Paas, F., Renkl, A., & Sweller, J. (2004). Cognitive load theory: instructional impli-
Contemporary Educational Psychology, 30(1), 117e139. http://doi.org/10.1016/j. cations of the interaction between information structures and cognitive ar-
cedpsych.2004.07.001. chitecture. Instructional Science, 32, 1e8. http://dx.doi.org/10.1023/B:
Bandura, A. (1997). Self-efficacy: The exercise of control. New York: Freeman. TRUC.0000021806.17516.d0.
Bandura, A. (2006). Guide for constructing self-efficacy scales. In F. Pajares, & Paas, F., Van Merrie €nboer, J. J. G., & Adams, J. J. (1994). Measurement of cognitive
T. Urdan (Eds.), Self-efficacy beliefs of adolescents (pp. 307e337). Greenwich, CT: load in instructional research. Perceptual and Motor Skills, 79, 419e430.
Information Age Publishing. Pajares, F. (1996). Self-efficacy beliefs in academic settings. Review of Educational
Baylor, A. L., & Kim, Y. (2005). Simulating instructional roles through pedagogical Research, 66(4), 543e578. http://dx.doi.org/10.3102/00346543066004543.
agents. International Journal of Artificial Intelligence in Education, 15(2), 95e115. Pajares, F., & Barich, J. (2005). Assessing mathematics self-efficacy: are skills-specific
Beilock, S. L., & Willingham, D. T. (2014). Math anxiety: can teachers help students measures better than domain-specific measures? Psychology, 12(3).
reduce it? American Educator, Summer, 28e33. Pajares, F., Hartley, J., & Valiante, G. (2001). Response format in writing self-efficacy
Bitner, J., Austin, S., & Wadlington, E. (1994). A comparison of math anxiety in assessment: greater discrimination increases prediction. Measurement and
traditional and nontraditional developmental college students. Research & Evaluation in Counseling and Development, 33, 214e221.
Teaching in Developmental Education, 35e43. Park, D., Ramirez, G., & Beilock, S. L. (2014). The role of expressive writing in math
Brunye, T. T., Mahoney, C. R., Giles, G. E., Rapp, D. N., Taylor, H. A., & Kanarek, R. B. anxiety. Journal of Experimental Psychology: Applied, 20(2), 103e111. http://doi.
X. Huang, R.E. Mayer / Computers in Human Behavior 63 (2016) 293e303 303
org/10.1037/xap0000013. messages on math anxiety, learning, and motivation (Doctoral dissertation). Flor-
Ramirez, G., Chang, H., Maloney, E. A., Levine, S. C., & Beilock, S. L. (2016). On the ida State University.
relationship between math anxiety and math achievement in early elementary Sweller, J. (1994). Cognitive load theory, learning difficulty, and instructional design.
school: the role of problem-solving strategies. Journal of Experimental Child Learning and Instruction, 4(4), 295e312. http://doi.org/10.1016/0959-4752(94)
Psychology, 141, 83e100. 90003-5.
Renkl, A. (2011). Instruction based on examples. In R. E. Mayer, & P. A. Alexander Sweller, J., Ayres, P., & Kalyuga, S. (2011). Cognitive load theory. New York: Springer.
(Eds.), Handbook of research on learning and instruction (pp. 272e295). New Sweller, J., Van Merrie €nboer, J. J. G., & Paas, F. (1998). Cognitive architecture and
York: Routledge. instructional design. Educational Psychology Review, 10(3), 251e296.
Renkl, A. (2014). The worked-example principle in multimedia learning. In Van Gog, T. (2011). Effects of identical exampleeproblem and problemeexample
R. E. Mayer (Ed.), The Cambridge handbook of multimedia learning (2nd ed., pp. pairs on learning. Computers & Education, 57(2), 1775e1779. http://doi.org/10.
391e412). New York: Cambridge University Press. 1016/j.compedu.2011.03.019.
Salomon, G. (1983). The differential investment of mental effort in learning from Vasile, C., Marhan, A.-M., Singer, F. M., & Stoicescu, D. (2011). Academic self-efficacy
different sources. Educational Psychologist, 18(1), 42e50. http://doi.org/10.1080/ and cognitive load in students. Procedia - Social and Behavioral Sciences, 12,
00461528309529260. 478e482. http://doi.org/10.1016/j.sbspro.2011.02.059.
Salomon, G. (1984). Television is “easy” and print is “tough”: the differential in- Veletsianos, G. (2010). Contextually relevant pedagogical agents: visual appearance,
vestment of mental effort in learning as a function of perceptions and attri- stereotypes, and first impressions and their impact on learning. Computers &
butions. Journal of Educational Psychology, 76(4), 647e658. http://doi.org/10. Education, 55(2), 576e585.
1037/0022-0663.76.4.647. Wei, Q. (2010). The effects of pedagogical agents on mathematics anxiety and math-
Sgoutas-Emch, S. A., & Johnson, C. J. (1998). Is journal writing an effective method of ematics learning. Doctoral dissertation. Utah State University.
reducing anxiety towards statistics? Journal of Instructional Psychology, 25(1), Zettle, R. D. (2003). Acceptance and commitment therapy (ACT) vs. systematic
49e57. desensitization in treatment of mathematics anxiety. The Psychological Record,
Sharp, C., Coltharp, H., Hurford, D. P., & Cole, A. K. (2000). Increasing mathematical 53, 197e215.
problem-solving performance through relaxation training. Mathematics Edu- Zimmerman, B. (2000). Self-efficacy: an essential motive to learn. Contemporary
cation Research Journal, 12(1), 53e61. http://doi.org/10.1007/BF03217074. Educational Psychology, 25(1), 82e91. http://doi.org/10.1006/ceps.1999.1016.
Shen, E. (2009). The effects of agent emotional support and cognitive motivational