Learning and Instruction 61 (2019) 45–59

Contents lists available at ScienceDirect

Learning and Instruction

journal homepage: www.elsevier.com/locate/learninstruc

Student-perceived teaching quality: How is it related to different T

achievement emotions in mathematics classrooms?
Rebecca Lazaridesa,∗, Janine Buchholzb
a
University of Potsdam, Department of Education, Karl-Liebknecht Straße 24-25, 14476, Potsdam, Germany
b
DIPF | Leibniz Institute for Research and Information in Education, Rostocker Straße 6, 60323, Frankfurt am Main, Germany

ARTICLE INFO ABSTRACT

Keywords: Achievement emotions are important prerequisites for academic outcomes and well-being, yet little is known
Teaching quality about their relation to teaching quality. This study examines the relation between student-perceived teaching
Achievement emotions quality in mathematics classrooms in grade 9 and enjoyment, anxiety, and boredom in grade 10, at both the
Mathematics classrooms student and classroom levels. The original data set included 6020 students who participated in the German
Multilevel analyses
national extension of the Programme for International Student Assessment (PISA). Multilevel regression analyses
showed that teacher support and classroom management were negatively related to student-level anxiety and
boredom. Teacher support was positively related to enjoyment and negatively related to anxiety at the classroom
level. Cognitive activation was positively related to enjoyment and negatively related to boredom at the class-
room level. Classroom management was negatively related to classroom-level boredom. These results provide
insight into differential classroom processes regarding the role of teaching quality in various aspects of student
achievement emotions.

1. Introduction
Students' achievement emotions are important antecedents to suc-
cessful learning processes because they are closely related to learning
strategies, cognitive and behavioral engagement, and academic
achievement (Pekrun, 2006, 2016). However, longitudinal studies have
shown that while the average anxiety level of students increases over
their school years (Pekrun, 2016), their enjoyment in learning declines
(Hagenauer & Hascher, 2010; Vierhaus, Lohaus, & Wild, 2016). Most
studies investigate the relation between perceived teaching quality and
achievement emotions from a cross-sectional perspective (for excep-
tions see Frenzel, Goetz, Lüdtke, Pekrun, & Sutton, 2009; Maulana,
Opdenakker, & Bosker, 2016). Further, examinations of the relation
between teaching characteristics and students' emotions at the student
and classroom levels are rare (Frenzel et al., 2009; Goetz, Lüdtke, Nett,
Keller, & Lipnevich, 2013). Considering individual and classroom cli-
mate effects as well as the directionality of these effects can provide
insight into how classroom teaching can support an adaptive emotional
development of both the individual and the group. This study, there-
fore, investigates the relation between student-perceived teaching
quality in mathematics classrooms in grade 9 and students’ achieve-
ment emotions in grade 10, at both the student and classroom levels.
We focus on mathematics because motivation in this domain is a critical
filter for career choices related to science and mathematics (Watt et al.,
2017).
1.1. Achievement emotions in classroom learning environments
The control-value theory of achievement emotions provides an in-
tegrative theoretical perspective on the antecedents and effects of
emotions experienced in achievement and academic settings (Pekrun,
2000, 2006; Pekrun, Goetz, Titz, & Perry, 2002). The theory defines
achievement emotions as those directly tied to achievement activities
(activity emotions) or achievement outcomes (outcome emotions). Ac-
tivity emotions include enjoyment, frustration and boredom experi-
enced during learning. Outcome emotions comprise joy, hope, pride,
anxiety, hopelessness, shame, and anger related to success or failure
(Pekrun & Stephens, 2010). Presenting a three-dimensional taxonomy
of achievement emotions, Pekrun and Stephens (2010) further differ-
entiate these emotions into activating emotions (e.g., enjoyment or an-
xiety) and deactivating emotions (e.g., boredom).
The literature describes two groups of appraisals as the proximal
determinants of achievement emotions: subjective control over and sub-
jective values of achievement activities and outcomes (Pekrun, 2006). It
is assumed that different types of achievement emotions are related to
various control and value appraisals (Pekrun & Stephens, 2010).

∗
Corresponding author.
E-mail addresses: rebecca.lazarides@uni-potsdam.de (R. Lazarides), buchholz@dipf.de (J. Buchholz).

https://doi.org/10.1016/j.learninstruc.2019.01.001
Received 12 October 2017; Received in revised form 20 October 2018; Accepted 15 January 2019
0959-4752/ © 2019 Elsevier Ltd. All rights reserved.
R. Lazarides, J. Buchholz
Activity emotions are assumed to depend on a combination of po-
sitive competence and value appraisals related to an action (e.g.,
learning in class) and its reference object (e.g., concrete tasks) (Pekrun
& Stephens, 2010). For example, a student who perceives him- or
herself to be competent in an activity or task and values this task as
important is likely to experience feelings of enjoyment in class, whereas
a student who perceives low competence on tasks that he or she judges
to be irrelevant is assumed to perceive high levels of boredom. Conse-
quently, classroom practices that enhance students’ feelings of compe-
tence and the subjective task value may reduce boredom and increase
enjoyment, while a lack of such classroom practices or classroom
practices that reduce perceived competence and value are assumed to
lead to high levels of boredom and low levels of enjoyment.
Outcome emotions such as anxiety in class are assumed to be
strongly associated with the perceived controllability of success and
failure, as well as with the subjective value related to tasks (Pekrun,
2006; Pekrun & Stephens, 2010). For example, anxiety is assumed to be
triggered when students perceive a lack of control over achievement
outcomes in tasks that are important to them. Thus, in classrooms in
which students perceive a high value of learning but low controllability
of success, students are likely to experience high levels of anxiety.
Empirical studies that systematically investigate the relations be-
tween the characteristics of the learning environment and achievement
emotions show that the following factors deliver information about the
subjective control over and subjective values of activities and their
outcomes: cognitive quality of tasks, value induction in instruction,
autonomy support, classroom goal structures, expectations, teachers'
presentation style, and achievement-related feedback (Bieg et al., 2017;
Gläser-Zikuda & Fuβ, 2008; Goetz, Lüdtke, et al., 2013). Most of these
factors are also captured by theoretical models of teaching quality, such
as that of Klieme, Pauli, and Reusser (2009), who describe three basic
dimensions of teaching quality: cognitive activation, classroom man-
agement and a supportive climate. However, relatively little is known
about the relation between teaching characteristics and students’
achievement emotions.
1.2. Teaching quality and students’ achievement emotions
In their theoretical framework of teaching quality, Klieme and col-
leagues (Klieme et al., 2009; Klieme, Schümer, & Knoll, 2001) describe
three basic dimensions of teaching quality: cognitive activation, class-
room management and a supportive climate, which are related in dif-
ferent ways to the development of knowledge and understanding as
well as to the development of motivation and emotion. The three basic
dimensions of teaching quality correspond to the characteristics of ef-
fective teaching outlined by other researchers (e.g., Danielson, 2011;
Pianta & Hamre, 2009; Seidel & Shavelson, 2007; van de Grift, 2007).
Pianta and Hamre (2009), for example, describe emotional support,
classroom organization and instructional support as dimensions of
teaching quality. Other scholars (Maulana, Helms-Lorenz, & van de
Grift, 2015; van de Grift, 2007; van de Grift, Chun, Maulana, Lee, &
Helms-Lorenz, 2017) have identified six domains of teaching quality:
safe and stimulating learning climate, efficient classroom management,
clear and structured instruction, intensive and activating teaching,
teaching learning strategies, and differentiated instruction. Despite
their differences (e.g. regarding the number of identified overarching
dimensions of teaching quality), some similarities shared by all of these
frameworks include classroom management, a supportive climate and
activating teaching as important components of teaching quality. In this
study, we refer to the framework of teaching quality described by
Klieme and colleagues (Klieme et al., 2001, 2009) because it proposes
specific theoretically derived interrelations among the dimensions of
teaching quality and students’ motivational-affective and cognitive
characteristics described in the following paragraphs.
Cognitive activation refers to challenging tasks, activation of prior
knowledge, and support for conceptual understanding (Hiebert &
46
Learning and Instruction 61 (2019) 45–59
Grouws, 2007; Lipowsky et al., 2009), as well as to encouragement for
students to attempt multiple solution pathways (Baumert et al., 2010;
Pauli, Drollinger-Vetter, Hugener, & Lipowsky, 2008). In this study, we
operationalized cognitive activation in the form of cognitively chal-
lenging tasks that allow for multiple solutions, as well as in the form of
teachers' encouragement for students to think about mathematical
concepts and transfer knowledge to new content areas (Baumert et al.,
2009). While cognitive activation is assumed to enhance knowledge
gain, findings regarding its effects on motivation and emotion are am-
biguous. While some studies have not found significant effects of stu-
dent-perceived classroom-level cognitive activation on students' en-
joyment in mathematics (Kunter, Baumert, et al., 2013; Schiepe-Tiska,
Heine, Lüdtke, Seidel, & Prenzel, 2016), others have shown that stu-
dent-perceived classroom-level cognitive activation does positively
impact students' interest in science (Fauth, Decristan, Rieser, Klieme, &
Büttner, 2014). Consequently, there is some empirical evidence in-
dicating that cognitively challenging activities and tasks enhance stu-
dents' enjoyment in learning mathematics. One potential underlying
mechanism is that students who experience themselves as being able to
solve challenging tasks may perceive high levels of autonomy and
competence, which, according to theoretical models such as self-de-
termination theory (Deci & Ryan, 2002) and control-value theory
(Pekrun, 2006), is an important antecedent of interest, enjoyment and
intrinsic motivation to learn. Interest, that has been shown to be related
to cognitive activation in class (Fauth et al., 2014), is a theoretical
construct that is closely related to enjoyment but that also has cognitive
components. It might also be expected that student-perceived cognitive
activation is related to negative emotions in class such as students’
anxiety or boredom with respect to mathematics. Regarding anxiety, if
students perceive that the level of cognitive challenge overextends their
level of competence they might perceive a loss of control over the
achievement outcome which in turn leads to high levels of anxiety
(Pekrun & Stephens, 2010). If students, however, perceive that the level
of cognitive challenge matches their level of ability they might, in turn,
perceive higher control over the achievement outcome and cognitive
challenge might reduce their anxiety (Pekrun, 2006). Regarding
boredom, the control-value theory proposes that boredom is assumed to
be triggered when students perceive only low levels of control over
achievement activities and low value of these activities (Pekrun, Hall,
Goetz, & Perry, 2014). Consequently, students who perceive a high
level of cognitive challenge in mathematics that exceeds their compe-
tencies and at the same time perceive the lesson content as irrelevant
for themselves might perceive high levels of boredom. However, if the
demands of tasks that students perceive as relevant for themselves
match their level of competence they will experience low levels of
boredom.
Classroom management denotes characteristics of teaching such as
rule clarity and monitoring (Evertson & Emmer, 1982; Kunter, Baumert,
& Köller, 2007), classroom structure (Bergsmann, Lüftenegger, Jöstl,
Schober, & Spiel, 2013), establishment of effective learning routines
(Kounin, 1970) and effective behavior management (Emmer & Stough,
2001). In this study, we operationalized classroom management in the
form of effective behavior management in class (e.g., dealing effectively
with disruptions), which allows students to focus on tasks (Lipowsky,
2015). In their meta-analysis on teaching effectiveness studies, Seidel
and Shavelson (2007) demonstrate that for motivational-affective out-
comes, the highest effect sizes are associated with time for learning,
regulation and monitoring, among other factors. Because students are
enabled to spend time on tasks when teachers effectively regulate dis-
ruptive behaviors in class, they might perceive higher levels of control
over their learning success, a higher value of learning and higher
competence, which in turn leads to greater enjoyment and less anxiety
and boredom.
A supportive classroom climate is characterized by teachers who care
about their students as individuals and as learners (Brophy, 2000),
consider students' perspectives (Pianta & Hamre, 2009), and provide
R. Lazarides, J. Buchholz
adaptive explanations (Kunter, Baumert, et al., 2013). In some studies
teacher support is conceptualized as an overarching dimension of
teaching behaviors that aim to enhance students' feelings of autonomy,
competence and social relatedness in class (Maulana, Helms-Lorenz,
Irnidayanti, & van de Grift, 2016). In this study, however, we oper-
ationalized a supportive climate in the form of teacher support, that is,
the sensitivity and individual learning support provided by the teacher
(Klieme et al., 2009; Pianta & Hamre, 2009). Studies have shown that
student-perceived teacher support is positively related to students' en-
joyment at the individual level (Goetz, Lüdtke, et al., 2013) and class-
room level (Kunter, Baumert, et al., 2013). Student-perceived teacher
support has furthermore been shown to be negatively related to in-
dividuals' anxiety (Ahmed, Minnaert, van der Werf, & Kuyper, 2010)
and boredom (Goetz, Lüdtke, et al., 2013). A psychological process that
may explain the effects of teacher support on student achievement
emotions is provided by self-determination theory (Deci & Ryan, 2002).
Based on this theory, empirical work has shown that students who
experience the fulfillment of basic psychological needs for autonomy,
competence and social relatedness in class are likely to enjoy learning
(Vlachopoulos & Michailidou, 2006) and report less anxiety in
achievement settings (Quested et al., 2011). Student-perceived teacher
support enhances the fulfillment of students' basic psychological needs
(Filak & Sheldon, 2008; Soenens & Vansteenkiste, 2005). In light of this
previous theoretical and empirical work, teacher support can be seen as
a key predictor of students' motivational-affective characteristics
(Klieme et al., 2009) and is therefore an essential factor in classroom
learning environments. We therefore expected that perceived teacher
support would be positively related to students’ enjoyment and nega-
tively related to anxiety and boredom.
1.3. Individual characteristics and achievement emotions: the role of gender
and achievement
Individual characteristics related to achievement emotions include
student gender and achievement. Mathematics is a highly gender-ste-
reotyped domain (Nosek, Banaji, & Greenwald, 2002) and such social
stereotypes affect girls' and boys' motivation and achievement emotions
in mathematics (Eccles, Freedman-Doan, Frome, Jacobs, & Yoon, 2000;
Lazarides, Rubach, & Ittel, 2017; Su, Rounds, & Armstrong, 2009; Watt,
2016). From a theoretical perspective, social stereotypes of mathe-
matics being a male domain shape teachers' gender-related expectations
and beliefs in girls' and boys’ mathematics talent and ability (Eccles,
Wigfield, & Schiefele, 1998). Mathematics teachers have been shown to
believe that boys are more talented in mathematics than girls despite
equal achievement of girls and boys (Jussim & Eccles, 1992). Students
are aware of such math-related gender stereotypes (Gunderson,
Ramirez, Levine, & Beilock, 2012) and accordingly develop gendered
mathematics competence beliefs with girls reporting lower competence
beliefs in mathematics than boys despite their equal achievement in the
domain (Marsh & Yeung, 1998). Their low levels of competence beliefs
in mathematics might lead to the perception of low controllability of
success which – combined with their high valuing of mathematics
achievement – results in their high levels of anxiety (Frenzel, Pekrun, &
Goetz, 2007a). Girls accordingly were shown to report higher levels of
mathematics (trait) anxiety and lower levels of mathematics enjoyment
than boys (Frenzel et al., 2007a; Goetz, Bieg, Lüdtke, Pekrun, & Hall,
2013).
Another individual characteristic that is related to students'
achievement emotions is students' achievement level. Research has
shown that a high level of achievement in mathematics reduces
mathematics anxiety (Ma & Xu, 2004) and is reciprocally and positively
interrelated with mathematics enjoyment (Ma, 1997). One theoretical
explanation for such effects is provided by control-value theory
(Pekrun, 2006) and Eccles’ expectancy-value theory (Eccles et al.,
1983). According to these theoretical approaches, the high success ex-
pectancies of students with high achievement in a specific domain are
47
Learning and Instruction 61 (2019) 45–59
related to feelings of control over the achievement outcome in this
domain (Pekrun, 2006) as well as to high valuing of domain-related
tasks (Eccles et al., 1983) – perceived control and value are in turn
important antecedents of student achievement emotions. High levels of
achievement can thus be expected to be positively related to enjoyment
and negatively related to boredom and anxiety.
1.4. The present study
This study expands on previous research by analyzing the relations
among student-perceived cognitive activation, classroom management,
and a supportive climate in mathematics classrooms in grade 9 and
students’ achievement emotions in grade 10 at both the student and
classroom levels.
We tested the following hypotheses:
(1) Empirical studies have shown that student-perceived teacher sup-
port (Ahmed et al., 2010) and classroom management (Frenzel,
Pekrun, & Goetz, 2007b) are negatively related to students' in-
dividual anxiety in class. We hypothesize that student-perceived
teacher support and classroom management in grade 9 are sig-
nificantly and negatively related to students' individual mathe-
matics anxiety in grade 10. We further assume that a level of cog-
nitive challenge that suits students' abilities might enhance their
feelings of control and thus reduce anxiety. Because we assessed
cognitive activation with respect to adaptive cognitive challenge
that aimed at enhancing students' understanding of the lesson
content, we expect cognitive activation in grade 9 to be negatively
related to anxiety in grade 10.
(2) Drawing on previous research, we expect that high levels of stu-
dent-perceived teacher support (Goetz, Lüdtke, et al., 2013) and
classroom management (Frenzel et al., 2007b) in grade 9 are sig-
nificantly and negatively related to the level of boredom students
experience when learning mathematics in grade 10 at both the
student and classroom levels. Because student-perceived cognitive
activation that matches students' ability level might increase their
feelings of control and value (Pekrun et al., 2014), we expect stu-
dent-perceived cognitive activation in grade 9 to be negatively re-
lated to boredom in grade 10.
(3) In line with previous findings (Fauth et al., 2014; Goetz, Lüdtke,
et al., 2013; Kunter, Baumert, et al., 2013), we expect a student-
perceived teacher support, classroom management and cognitive
activation in grade 9 to be significantly and positively related to
students' enjoyment in grade 10 at the individual and classroom
levels.
The hypothesized relations are depicted in the schematic model in
Fig. 1.
Because of the strong relations among school type, gender, and
student achievement, we considered these variables as covariates in the
present analyses. Because the German school system is characterized by
different school tracks, our data set included students from four school
tracks. In our analyses, we controlled for the effects of these four school
tracks by including three dummy-coded variables using academic-track
schools as the reference category. Previous findings show high levels of
cognitive activation and low levels of teacher support in academic-track
schools (Kunter et al., 2005). In addition, early tracking into academic-
track schools is shown to be related to high levels of anxiety and lower
self-concept (Becker et al., 2014). We focused on student perceptions of
teaching quality in this study because previous studies have shown that
when comparing student ratings and observer ratings of teaching
quality, student ratings more strongly predict students’ academic en-
gagement (Maulana & Helms-Lorenz, 2016) and motivational devel-
opment (Clausen, 2002). Consequently, student ratings of teaching
quality are assumed to be highly relevant predictor variables for the
affective outcomes in this study.
R. Lazarides, J. Buchholz
Fig. 1. Schematic representation of the joint model. Note. All hypothesized relations among the variables are indicated by solid arrows. In addition to the uni-
directional relations, the joint model contains bidirectional relations among all variables within one time point at each level of analysis.
2. Method
2.1. Sample
Data for this study stemmed from the PISA-I-Plus study (Prenzel
et al., 2013) conducted in Germany in 2003 and 2004 as a national
extension to the international survey Programme for International
Student Assessment (PISA) of 2003 with a focus on mathematics. PISA
is a triennial international survey administered to 15-year-old students
and assesses the extent to which adolescents have acquired key
knowledge and skills essential for their participation in modern socie-
ties (OECD, 2013). As a national extension to PISA 2003, PISA-I-Plus
was designed as a study with two measurement time points (Prenzel,
2006). In Germany, a subsample of the students who participated in
PISA 2003 were assessed again one year later, i.e., at the end of the 10th
grade. The original sample that participated at both time points con-
sisted of 6020 students (55.2% girls). This study, however, analyzed
only a subset of data consisting of 4612 students (55.6% girls) from 217
classrooms in 152 schools (average classroom size = 21.79) who par-
ticipated in both time points and who did not experience a change of
mathematics teacher. The students attended different school types:
41.6% (n = 1920) attended a Realschule, which is the middle track that
includes school grades 5 to 9 or 10, 36.8% (n = 1697) attended a
Gymnasium, which is the academic track that offers a collegebound
track, and 21.6% (n = 995) attended a comprehensive school (school
with several educational programs: 12.6% – n = 581; Integrierte Ge-
samtschule: 9.0% – n = 414). The highest educational degree attained
48
Learning and Instruction 61 (2019) 45–59
by the parents of almost half of the students (40.9%) was a high-school
diploma (“A-levels,” in German: Abitur) or a university of applied sci-
ences entrance qualification (Fachhochschulreife). 24.9% of the students
reported that their parents' highest eduactional degree was a middle
school degree (in German: Realschule or Mittlere Reife), 14.3% reported
that their parents' highest degree was from a vocational school, and
10.3% reported that their parents' highest degree was from compulsory
basic secondary schooling (in German: Hauptschule). The remaining
9.6% of students reported that their parents have attained another
school degree (2.5%), no school degree (0.6%), or a degree from a
polytechnic institute/extended secondary school (0.4%), and 6.1%
were missing data. The largest group of students (90.7%) reported that
they were born in Germany. The next largest groups reported that they
were born in Russia (or countries of the former Soviet Union) (3.7%),
Poland (1.3%), Bosnia and Herzegovina (0.2%), Turkey (0.2%), Greece
(0.1%), Italy (0.1%) or another country (1.0%), and 2.7% did not an-
swer.
2.2. Measures
Measures of students’ achievement emotions were used for grades 9
and 10. Measures of student-perceived teaching quality were used for
grade 9. Since this study includes only those items in the scales that
resulted in a good fit of the CFA models to the empirical data, a reduced
number of items was used to form certain scales. The exact wording of
all items employed in the analyses is reported in Appendix C.
R. Lazarides, J. Buchholz
2.2.1. Mathematics achievement
Achievement in mathematics was assessed using the regular stan-
dardized PISA 2003 mathematics test (‘1. PV Mathematik Gesamttest’).
The test measured “the capacities of students to analyse, reason, and
communicate ideas effectively as they pose, formulate, solve, and in-
terpret mathematical problems in a variety of situations” (OECD, 2003,
p. 20). The estimates of achievement are plausible values based on item
response models. The mean of the scale across OECD countries was 500
(SD = 100). In our sample, the mean was M = 538.57 (SD = 74.44).
2.2.2. Mathematics anxiety
Anxiety in mathematics was assessed with a five-item scale based on
Pekrun, Götz, and Frenzel (2005) ranging from 1 (strongly disagree) to
4 (strongly agree). Reliabilities were ɑ = 0.83 (grade 9) and ɑ = 0.84
(grade 10), respectively.
2.2.3. Mathematics boredom
Boredom in mathematics was assessed using a five-item scale based
on Pekrun et al. (2005) ranging from 1 (strongly disagree) to 4 (strongly
agree). Reliabilities were ɑ = 0.86 (grade 9) and ɑ = 0.87 (grade 10),
respectively.
2.2.4. Mathematics enjoyment
Students’ enjoyment in learning mathematics was assessed with a
five-item scale based on Pekrun et al. (2005) ranging from 1 (strongly
disagree) to 4 (strongly agree). Reliabilities were ɑ = 0.89 (both in
grade 9 and grade 10).
2.2.5. Cognitive activation
Student-perceived cognitive activation was operationalized using a
seven-item scale measuring ‘cognitively activating tasks in mathematics
classes’ based on Baumert et al. (2009), ranging from 1 (strongly dis-
agree) to 4 (strongly agree). Reliability was ɑ = 0.75 (grade 9).
2.2.6. Classroom management
Student-perceived classroom management was operationalized
through the lack of disruptions in the classroom (see also Fauth et al.,
2014). It was assessed with a three-item scale based on Ramm et al.
(2006) ranging from 1 (strongly disagree) to 4 (strongly agree). Relia-
bility was ɑ = 0.87 (grade 9).
2.2.7. Teacher support
Student-perceived teacher support in mathematics was assessed
using a five-item scale based on Ramm et al. (2006) ranging from 1
(strongly disagree) to 4 (strongly agree). Reliability was ɑ = 0.86
(grade 9).
2.3. Data analyses
We examined the relations among student-perceived cognitive ac-
tivation, classroom management, and teacher support in mathematics
classrooms in grade 9 and students' achievement emotions in grade
10 at both the student and classroom levels using multilevel regression
analyses. In our analyses, we controlled for students’ achievement
emotions in grade 9 at the student level. Data were assessed in Spring
2003 (grade 9) and again in Spring 2004 (grade 10).
To assess the reliability of the aggregated student variables, in-
traclass correlations (ICC) were computed for all variables in the model
(Raudenbush & Bryk, 2002). An ICC1 value greater than 0.05 reveals
that individual ratings are attributable to group membership (LeBreton
& Senter, 2008). An ICC2 value above 0.70 indicates an acceptable
accuracy of class-mean ratings (Lüdtke, Robitzsch, Trautwein, &
Kunter, 2009). Although the ICC1 values of anxiety at times 1 and 2 are
somewhat below the criterion of ICC1 > 0.05, we decided to retain
these variables because we were interested in the different effects of
student-perceived teaching quality on students' achievement emotions
49
Learning and Instruction 61 (2019) 45–59
and thus needed to examine group-level effects. From a methodological
point of view, taking into account the multilevel data structure and
considering group-level effects for variables with low intraclass corre-
lations is meaningful when dealing with nonzero intraclass correlations
and a high number of individuals per group (Julian, 2001). Because this
study included n = 217 classrooms with an average classroom size of
n = 21.79 students per classroom, and because the intraclass correla-
tion coefficients for anxiety were close to the cut-off value (ICC = 0.04
for anxiety at time 1 and time 2), we decided to retain ‘anxiety’ in the
multilevel model.
Before conducting the regression analyses, we tested measurement
invariance across time (two timepoints) and levels (two levels in the
multilevel analyses: student level and classroom level). In line with
Chen (2007), invariance was not rejected if ΔCFI < −0.01 and
ΔRMSEA < +0.015 for the more restrictive compared to the less re-
strictive model. Results are reported in Appendix A and demonstrate
strong measurement invariance across time and levels of analysis for
each construct (anxiety, boredom, enjoyment), thus indicating that the
same constructs were measured at both time points and were compar-
able across analysis levels. The factor loadings of each of the latent
factors assuming measurement invariance across time and levels are
reported in Appendix B for the student and classroom levels.
After testing measurement invariance, we conducted doubly-man-
ifest multilevel regression analyses as described by Marsh et al. (2009)
in a set of contextual models. In doubly-manifest multilevel regression
models, the constructs at each level are based on (single) manifest in-
dicators of each construct and a manifest aggregation of constructs at
the group level (e.g., classroom level, in our study). Doubly-manifest
multilevel regression models do not control for measurement error or
sampling error. Furthermore, more complex models are prone to non-
convergence under conditions of small sample sizes on the individual
and group levels. Despite the large sample sizes in our study, we
decided to apply doubly-manifest multilevel regression models due to
the great complexity of the tested models. To deal with sampling error
and to make valid estimates and inferences of the population, we in-
cluded the survey weight variable in our analyses.
In the first step, we specified four models for each of the three
outcome variables (anxiety, boredom, enjoyment): Model 1 included
student-perceived cognitive activation at the student and classroom
levels as the predictor variable; Model 2 included student-perceived
classroom management as the predictor variable; Model 3 included
student-perceived teacher support as the predictor variable; Model 4
included the three student-perceived characteristics of teaching quality
(cognitive activation, classroom management, teacher support) as
predictor variables, as well as gender and mathematics achievement as
covariates at the individual level and dummy-coded school tracks as
covariates at the classroom level.
In the second step, we tested a joint model including all three
achievement emotions as outcome variables and all three teaching
quality characteristics as well as the covariates as predictor variables.
This model allowed us to evaluate the unique contribution of each as-
pect of teaching quality to the explanation of achievement emotions. A
schematic representation of the ‘joint model’ is depicted in Fig. 1.
In each of the models, we included achievement emotions at time 2
(grade 10) at both the student and classroom levels as dependent
variables. Achievement emotions at time 1 (grade 9) were included as
predictor variables at the student level only. This approach is in line
with other multilevel studies, such as those of Kunter, Baumert, et al.
(2013) and Kunter, Klusmann, et al. (2013). Because of the high sta-
bility in achievement emotions at the classroom level, we focused on
explaining the changes in students' achievement emotions only at the
student level. Previous studies have shown a substantial decline in
student enjoyment at the student level (Hagenauer & Hascher, 2010;
Vierhaus et al., 2016), raising the question of how teaching quality is
related to such developmental changes. This study accordingly ex-
amined how teaching quality was related to changes in students’
R. Lazarides, J. Buchholz Learning and Instruction 61 (2019) 45–59

achievement emotions at the student level, while we examined the di- series c: enjoyment). The first model of each series included student-
rect effects of student-perceived teaching quality in grade 9 on class- perceived cognitive activation, the second model included student-
room-level achievement emotions in grade 10, but not on changes in perceived classroom management, and the third model included stu-
classroom-level achievement emotions from grades 9 to 10. We used dent-perceived teacher support as the predictor variable at the student
group mean centering for the predictor variables at the student level. and classroom levels. Lastly, we tested a fourth model that included the
Missing data analyses showed that in grade 9 (time 1), 2.5% of the three student-perceived characteristics of teaching quality (cognitive
students (n = 113) did not report data on their achievement emotions activation, classroom management, teacher support) as predictor vari-
(anxiety, boredom, enjoyment). We created a missing data variable ables for achievement emotions, as well as gender and mathematics
coded as 0 = not missing and 1 = missing. Missing data in grade 9 achievement as covariates at the individual level and dummy-coded
(time 1) were weakly but significantly associated with students' school-track as covariates at the classroom level.
mathematics competency (r = −0.046, p = .002). In grade 10 (time 2), Models a-c were just-identified, with path coefficients estimated for
6.0% (n = 279) of the students did not report data on their enjoyment all possible predictors from previous waves and correlations estimated
and boredom (anxiety: 6.1%, n = 280). Missing data in grade 10 (time between all variables assessed at the same wave. Consequently, the
2) were weakly but significantly associated with students’ mathematics models yielded a trivially perfect fit (χ2 (df = 0, N = 4612) = 0.00,
competency (r = −0.068, p < .001), but not with their gender RMSEA = 0.00, CFI = 1.00, TLI = 1.00).
(r = −0.007, p = .703). Missing data analyses showed further that data Anxiety. Models a1–a3 (see Table 3) revealed that when considering
were missing at random from grade 9 to grade 10 for anxiety each characteristic of teaching quality as a unique predictor of
(χ2(2) = 3.64, p = 0.16), boredom (χ2(2) = 4.28, p = 0.12), and en- mathematics anxiety, student-perceived mathematics teacher support
joyment (χ2(2) = 4.70, p = 0.10). Missing data were addressed using in grade 9 was significantly and negatively related to students’ anxiety
full-information maximum likelihood (FIML) estimation. All analyses in mathematics in grade 10 at the student and classroom levels.
were conducted using the maximum likelihood with robust standard When simultaneously accounting for each of the three teaching
errors (MLR) estimation in Mplus Version 7 (Muthén & Muthén, 1998- quality characteristics and covariates as predictors of mathematics an-
2015). Goodness of model fit was evaluated using the following criteria xiety (model a4 in Table 3), girls reported higher levels of mathematics
(Tanaka, 1993): Yuan–Bentler scaled χ2 (YB χ2, mean-adjusted test anxiety than boys, mathematics achievement was negatively related to
statistic robust to non-normality), Tucker and Lewis index (TLI), the students' mathematics anxiety, students in comprehensive schools and
comparative fit index (CFI), and the root mean square of approximation middle-track schools reported higher levels of mathematics anxiety
(RMSEA) with the associated confidence intervals (CIs). In addition, than those in academic-track schools, and student-perceived teacher
standardized root mean residual (SRMR) values were reported. TLI and support was negatively and significantly related to students’ mathe-
CFI values .95 (Hu & Bentler, 1999), RMSEA values .06, and matics anxiety at the student and classroom levels.
SRMR ≤ 0.08 (Hu & Bentler, 1999) were accepted as indicators of a Boredom. Models b1–b3 (see Table 4) showed that student-perceived
good model fit. mathematics teacher support in grade 9 was significantly and nega-
tively related to students' boredom in mathematics in grade 10 at the
3. Results student level. At the classroom level, student-perceived cognitive acti-
vation, classroom management, and mathematics teacher support in
3.1. Descriptive statistics grade 9 were significantly and negatively related to students’ boredom
in mathematics in grade 10. When simultaneously considering each of
Table 1 reports the manifest means and standard deviations along the three teaching quality characteristics and the covariates as pre-
with the intraclass correlation coefficients for the variables and Table 2 dictors of mathematics boredom (model b4 in Table 4), girls reported
shows the manifest intercorrelations. The three characteristics of stu- lower levels of mathematics boredom than boys, and mathematics
dent-perceived teaching quality in grade 9 – cognitive activation, achievement and student-perceived mathematics teacher support were
classroom management, and teacher support in mathematics class- negatively related to mathematics boredom at the student level. At the
rooms – were significantly and negatively associated with boredom and classroom level, student-perceived cognitive activation and classroom
anxiety in grades 9 and 10 and significantly and positively associated management were negatively and significantly related to classroom-
with enjoyment in grades 9 and 10. level boredom.
Enjoyment. Models c1–c3 (see Table 5) showed that student-per-
3.2. Teaching quality and students’ achievement emotions ceived cognitive activation, classroom management, and teacher sup-
port in grade 9 were significantly and positively related to students'
In the first step, we tested four models for each of the three outcome mathematics enjoyment in grade 10 at the classroom level. No sig-
variables (Model series a: anxiety, Model series b: boredom, Model nificant effects of student-perceived teaching quality on students' en-
joyment were found at the student level. When simultaneously con-
Table 1 sidering each of the three teaching quality characteristics and the
Descriptive statistics: Means, standard deviations and intraclass correlation covariates as predictors of mathematics enjoyment (model c4 in
coefficients. Table 5), girls reported lower levels of mathematics enjoyment than
boys, and mathematics achievement was positively related to students’
M Level1 SD Level 1 M Level 2 SD Level 2 ICC1 ICC2
mathematics enjoyment. Classroom-level student-perceived teacher
Enjoyment g9 2.14 0.74 2.14 0.22 .08 .65 support was positively and significantly related to classroom-level en-
Enjoyment g10 2.09 0.74 2.10 0.20 .06 .58 joyment. Classrooms in the middle-track school (Realschule) had higher
Anxiety g9 1.90 0.70 1.90 0.14 .04 .48
levels of mathematics enjoyment than students in the academic track
Anxiety g10 1.90 0.71 1.90 0.14 .04 .46
Boredom g9 2.08 0.75 2.08 0.22 .09 .66 (Gymnasium).
Boredom g10 2.10 0.75 2.11 0.24 .10 .69 Joint model. The model that included all three achievement emo-
Cognitive activation g9 2.82 0.54 2.81 0.27 .10 .70 tions as outcome variables at time 2 and each of the three dimensions of
Classroom management 2.61 0.81 2.58 0.54 .31 .91 teaching quality and the covariates as predictor variables at time 1
g9
Teacher support g9 2.53 0.77 2.53 0.46 .26 .88
showed a good fit to the data: χ2 = 67.54, df = 6, CFI = 0.99,
TLI = 0.91; RMSEA = 0.05, SRMRwithin = 0.02, and
Note. ICC1 and ICC2 = Intraclass correlation coefficients. SRMRbetween = 0.01. Table 6 and Fig. 2 display all standardized re-
g9 = Grade 9; g10 = Grade 10. All variables ranged from 1 to 4. gression coefficients of this model.

50
R. Lazarides, J. Buchholz Learning and Instruction 61 (2019) 45–59

Table 2
Intercorrelations between covariates, motivational variables and teaching quality at the student level.
1 2 3 4 5 6 7 8 9 10

1) Girl
2) Mathematics achievement grade 9 –.17***
3) Anxiety grade 9 .20*** –.48***
4) Anxiety grade 10 .19*** –.50*** .66***
5) Boredom grade 9 –.06*** –.15*** .34*** .25***
6) Boredom grade 10 –.11*** –.12*** .24*** .31*** .59***
7) Enjoyment grade 9 –.14*** .39*** –.46*** –.46*** –.55*** –.40***
8) Enjoyment grade 10 –.16*** .40*** –.41*** –.44*** –.41*** –.49*** .71***
9) Cognitive activation grade 9 –.03 .08** –.15*** –.12*** –.30*** –.20*** .29*** .22***
10) Classroom management grade 9 .04 .08* –.17*** –.15*** –.36*** –.25*** .22*** .15*** .23***
11) Teacher support grade 9 –.07*** –.01 –.14*** –.13*** –.31*** –.25*** .32*** .23*** .51*** .23***

Note. *p < .05, **p < .01, ***p < .001.

Table 3
Standardized regression coefficients from models a1-a4. Dependent variable: Anxiety grade 10.
Model a1 Model a2 Model a3 Model a4

Student level
Anxiety g9 .62*** (0.01) .61*** (0.01) .61*** (0.01) .52*** (0.02)
Cognitive activation g9 –.01 (0.02) – – .02 (0.02)
Classroom management g9 – –.03 (0.02) – –.03 (0.01)
Teacher support g9 – – –.04** (0.01) –.05*** (0.01)
Mathematics Achievement g9 – – – –.19*** (0.01)
Girl – – – .05*** (0.01)
Classroom level
Cognitive activation g9 –.22** (0.08) – – .02 (0.11)
Classroom management g9 – –.09 (0.08) – .02 (0.09)
Teacher support g9 – – –.21* (0.09) –.26* (0.09)
MBS vs. academic track – – – .47*** (0.08)
RS vs. academic track – – – .20* (0.08)
IG vs. academic track – – – .27* (0.11)

Note. g9 = grade 9. *p < .05; **p < .01; ***p < .001. The three dummy-coded variables represent the four school tracks using Gymnasium (academic track) as the
reference group. MBS = Schule mit mehreren Bildungsgängen – comprehensive school (school with several educational programs); RS = Realschule (middle-track
school); IG = Integrierte Gesamtschule – comprehensive school with academic track.

Student-level teacher support and classroom management in grade
9 were both significantly and negatively related to students’ anxiety and
boredom in grade 10.
Classroom-level teacher support in grade 9 was significantly and
negatively related to anxiety in grade 10 and was significantly and
positively related to classroom-level mathematics enjoyment in grade
10. Classroom-level classroom management in grade 9 was significantly
and negatively related to boredom in grade 10. Classroom-level cog-
nitive activation was significantly and negatively related to boredom
and was significantly and positively related to classroom-level
mathematics enjoyment in grade 10.
Student-level mathematics achievement in grade 9 was significantly
and positively related to student-perceived cognitive activation and
teacher support as well as mathematics enjoyment in grade 10 and was
negatively and significantly related to anxiety and boredom in grade
10.
Compared to boys, girls reported lower levels of teacher support and
higher levels of mathematics anxiety, but lower levels of boredom and
enjoyment.
Classrooms in comprehensive schools (schools with various

Table 4
Standardized regression coefficients from models b1-b4. Dependent variable: Boredom grade 10.
Model b1 Model b2 Model b3 Model b4

Student level
Boredom g9 .55*** (0.01) .55*** (0.01) .55*** (0.01) .53*** (0.01)
Cognitive activation g9 –.03 (0.02) – – –.01 (0.02)
Classroom management g9 – –.03 (.02) – –.03 (0.02)
Teacher support g9 – – –.06*** (0.02) –.06*** (0.02)
Math Achievement g9 – – – –.06*** (0.01)
Girl – – – –.10*** (0.02)
Classroom level
Cognitive activation g9 –.53*** (0.06) – – –.21* (0.11)
Classroom management g9 – –.63*** (0.06) – –.47*** (0.07)
Teacher support g9 – – –.46*** (0.06) –.17 (0.09)
MBS vs. academic track – – – .01 (0.07)
RS vs. academic track – – – –.17* (0.07)
IG vs. academic track – – – .03 (0.06)

Note. g9 = grade 9. *p < .05; **p < .01; ***p < .001. The three dummy-coded variables represent the four school tracks using Gymnasium (academic track) as the
reference group. MBS = Schule mit mehreren Bildungsgängen – comprehensive school (school with several educational programs); RS = Realschule (middle-track
school); IG = Integrierte Gesamtschule – comprehensive school with academic track.

51
R. Lazarides, J. Buchholz Learning and Instruction 61 (2019) 45–59

Table 5
Standardized regression coefficients from models c1-c4. Dependent variable: Enjoyment grade 10.
Model c1 Model c2 Model c3 Model c4

Student level
Enjoyment g9 .68*** (0.01) .68*** (0.01) .68*** (0.01) .63*** (0.01)
Cognitive activation g9 .02 (0.02) – – –.01 (0.02)
Classroom management g9 – –.01 (0.01) – –.01 (0.02)
Teacher support g9 – – .01 (0.01) .02 (0.01)
Math Achievement g9 – – – .12*** (0.01)
Girl – – – –.04** (0.01)
Classroom level
Cognitive activation g9 .50*** (0.06) – – .03 (0.08)
Classroom management g9 – .26*** (0.08) – .06 (0.08)
Teacher support g9 – – .54*** (0.07) .49*** (0.08)
MBS vs. academic track – – – –.12 (0.08)
RS vs. academic track – – – .20* (0.08)
IG vs. academic track – – – –.08 (0.06)

Note. g9 = grade 9. *p < .05; **p < .01; ***p < .001. The three dummy-coded variables represent the four school tracks using Gymnasium (academic track) as the
reference group. MBS = Schule mit mehreren Bildungsgängen – comprehensive school (school with several educational programs); RS = Realschule (middle-track
school); IG = Integrierte Gesamtschule – comprehensive school with academic track.

educational programs; Integrierte Gesamtschule) and in middle-track
schools (Realschule) were characterized by higher mathematics anxiety
than academic-track schools (Gymnasium). Classrooms in comprehen-
sive schools (Integrierte Gesamtschule) were characterized by higher le-
vels of classroom-level boredom in mathematics and lower levels of
perceived classroom management than academic-track schools.
Classrooms in middle-track schools (Realschule) were characterized by
higher classroom-level mathematics enjoyment than classrooms in
academic-track schools. At the student level, the model explained
considerable amounts of variance in students’ anxiety (R2 = 0.41),
boredom (R2 = 0.32) and enjoyment (R2 = 0.47) in grade 10. At the
classroom level, the model explained considerable variance in anxiety
(R2 = 0.24), boredom (R2 = 0.52), and enjoyment (R2 = 0.39) in grade
10.
4. Discussion
This study extends previous research by examining the relation
between student-perceived teaching quality in grade 9 secondary
school mathematics classrooms and different aspects of students’
achievement emotions in grade 10 at the student and classroom levels.
The findings revealed that classrooms characterized by high levels of
perceived teacher support in grade 9 were also characterized by low
levels of boredom and anxiety and high levels of enjoyment in grade 10.
Classrooms characterized by high levels of classroom management in
grade 9 were also characterized by low boredom in grade 10. Further,
students who perceived a high degree of support by their teacher in
grade 9 reported lower levels of anxiety and boredom in grade 10.
Students who perceived high levels of classroom management in grade
9 reported low anxiety in grade 10.
4.1. Summary of findings and discussion
In line with our expectations (Hypothesis 1) and with previous re-
search (Ahmed et al., 2010; Frenzel et al., 2007b), our findings showed
that student-perceived teacher support and classroom management
were negatively related to students' anxiety at the individual level.
Going beyond these initial findings, our study indicated that perceived
teacher support is important at the classroom level as well – classrooms
in which students perceive on average high levels of teacher support are
characterized by low levels of overall anxiety in mathematics. The ne-
gative relation between student-perceived learning support, classroom
management and student anxiety are possibly explained by students'
feelings of involvement and autonomy. Teacher support is associated
with feelings of autonomy, competence and social relatedness (Skinner
& Belmont, 1993), which might in turn lead to higher perceptions of

Table 6
Standardized regression coefficients from the joint model including all achievement emotions at time 2 as outcome variables and all dimensions of teaching quality
and covariates as predictor variables.
Cognitive activation g9 Classroom management g9 Teacher support g9 Anxiety g10 Boredom g10 Enjoyment g10

Student level
Math Achievement g9 .06** (0.02) .04 (0.02) .03* (0.02) –.19*** (0.01) –.05*** (0.01) .13*** (0.01)
Girl –.02 (0.02) .02 (0.02) –.06*** (0.02) .05*** (0.01) –.09*** (0.01) –.04*** (0.01)
Anxiety g9 – – – .51*** (0.02) – –
Boredom g9 – – – – .51*** (0.02) –
Enjoyment g9 – – – – – .61*** (0.01)
Cognitive activation g9 – – – .01 (0.02) –.02 (0.03) .03 (0.02)
Classroom management g9 – – – –.04* (0.01) –.04* (0.02) .02 (0.02)
Teacher support g9 – – – –.05*** (0.01) –.06*** (0.02) .01 (0.02)
Classroom level
MBS vs. academic track –.10 (0.08) –.03 (0.07) .10 (0.07) .47*** (0.08) .08 (0.07) –.09 (0.08)
RS vs. academic track –.04 (0.08) .01 (0.08) .12 (0.08) .19* (0.08) –.10 (0.07) .22* (0.08)
IG vs. academic track –.12 (0.07) –.14* (0.07) .06 (0.07) .27* (0.11) .13* (0.06) –.06 (0.06)
Cognitive activation g9 – – – .02 (0.11) –.21* (0.10) .23* (0.09)
Classroom management g9 – – – .02 (0.09) –.47*** (0.07) .02 (0.08)
Teacher support g9 – – – –.25* (0.11) –.13 (0.09) .35*** (0.10)

Note. g9 = grade 9. G10 = grade 10. **p < .01; ***p < .001. The three dummy-coded variables represent the four school tracks using Gymnasium (academic track)
as the reference group. MBS = Schule mit mehreren Bildungsgängen – comprehensive school (school with several educational programs); RS = Realschule (middle-track
school); IG = Integrierte Gesamtschule – comprehensive school with academic track.

52
R. Lazarides, J. Buchholz
Fig. 2. Empirical multilevel regression model for the joint model. aOnly significant standardized coefficients are depicted (p < .05). For the sake of clarity, the
covariates are not depicted in the figure. These coefficients are reported in Table 6 and Appendix C.
control and subjective value. Perceptions of control and value are as-
sumed to reduce students' anxiety (Pekrun & Stephens, 2010). The same
mechanism applies to classroom management, which allows students
and teachers to spend time on tasks (Lipowsky, 2015) and may there-
fore increase students' feelings of control and the value they place on
learning (Pakarinen et al., 2010). Student-perceived cognitive activa-
tion was not significantly related to students' anxiety. This finding
needs further discussion. In our study, we assessed cognitive activation
in terms of adaptive teaching methods that foster students' under-
standing of the lesson content by, for example, providing cognitively
challenging tasks. The effect of cognitive challenge on students’ anxiety,
however, might depend on individual characteristics such as the level of
(perceived) competence. For example, at low levels of (perceived)
mathematics competence, cognitive challenge might trigger anxiety
because students might perceive low controllability of achievement-
related outcomes when working on cognitively challenging tasks. For
students with high levels of (perceived) competence, cognitive chal-
lenge might reduce anxiety because these students perceive that cog-
nitively challenging tasks match their ability level and thus, perceive
high levels of control. The interaction of student characteristics (e.g.,
perceived competence) and perceived teaching quality needs further
investigation in order to provide a better understanding of such me-
chanisms.
In line with Hypothesis 2, perceived classroom management and
cognitive activation were negatively related to students' boredom at the
classroom level. These results correspond to the assumptions of Pekrun
and Stephens (2010) who state that classroom characteristics that en-
hance students’ competence and value appraisals are important for
activity emotions such as boredom. Classroom management was oper-
ationalized in this study by the effective handling of disruptions in
class. Thus, in classrooms that are effectively managed by teachers,
students may report lower levels of boredom because they may be able
to spend more of their time on tasks and thus, perceive higher value of
tasks and feel more competent (Lipowsky, 2015). Student-perceived
cognitive activation was negatively related to class-level boredom.
Consequently, classrooms in which students reported on average that
they were provided with cognitively activating tasks by their teacher in
53
Learning and Instruction 61 (2019) 45–59
grade 9 were characterized by a low level of overall boredom in grade
10. One explanation for this effect may be that by working on tasks that
allow students to make connections and engage with demanding pro-
blems and develop multiple solutions, students are encouraged to
practice reflective, higher-level thinking (Klieme et al., 2009) and thus
experience themselves as competent which leads to an increase of
perceived control and value of the tasks.
As theoretically expected (Hypothesis 3), student-perceived teacher
support and cognitive activation were significantly and positively re-
lated to students' feeling of enjoyment when learning mathematics. Our
findings expand upon Klieme's framework of teaching quality (Klieme
et al., 2009) which proposes indirect effects of cognitive activation on
students' motivational-affective characteristics. However, our findings
are in line with previous studies that also suggest that a supportive
climate (Kunter, Baumert, et al., 2013) and cognitive activation (Fauth
et al., 2014) are positively related to classroom-level enjoyment. Dif-
ferent underlying mechanisms may explain these effects. One ex-
planation for the significant and positive effect of a perceived teacher
support on students' enjoyment might be increased feelings of au-
tonomy, competence, and social relatedness as described by self-de-
termination theory (Deci & Ryan, 2002). According to the theory, stu-
dents who feel supported by their teachers perceive a high degree of
autonomy in learning, feel competent and socially related to the teacher
and, through this process, experience high levels of enjoyment. One
explanation for the positive effect of cognitive activation on classroom-
level enjoyment might be the combination of positive competence ap-
praisals along with high subjective valuing of learning mathematics
(Pekrun & Stephens, 2010) when working on challenging tasks.
Taken together, our findings extend previous theoretical work
(Klieme et al., 2009; Pekrun, 2006) by showing differential relations
between the three basic dimensions of teaching quality defined by
Klieme et al. (2009) and students’ activity and outcome emotions as
defined by Pekrun (2006). Student-perceived teacher support and
classroom management were important for activity emotions (teacher
support: enjoyment, classroom management: boredom) and outcome
emotions (teacher support: anxiety). Cognitive activation was relevant
for activity emotions only (boredom and enjoyment). Activity emotions
R. Lazarides, J. Buchholz
depend on a combination of positive competence and value appraisals
(Pekrun & Stephens, 2010), while outcome emotions such as anxiety
are triggered by low levels of perceived controllability of success and
failure and high value placed on the task. Thus, cognitive activation as
assessed in this study may enhance students' feelings of competence and
value by enhancing conceptual understanding, but may not increase
students' feelings of control over the achievement outcome. More re-
search is needed to examine the mediating processes that underlie the
effects examined in this study in order to understand the mechanisms
that explain the relations between teaching quality and achievement
emotions. It is important to note that items of the cognitive activation
scale that had particular high loadings referred to students' reports of
tasks that enhanced their conceptual understanding – one item, how-
ever, had a low loading on the scale referring to tasks that make stu-
dents ‘stop and think’. We retained the item in the analyses to ensure
that all facets of cognitive activation were included in the scale as this
item seemed to refer to cognitive challenge. However, when conducting
further analyses, researchers might need to reflect about the coherence
of measures that assess cognitive activation.
Another contribution of this study to current research is that it
shows different effects of teaching quality on achievement emotions on
the student and classroom levels. For example, teacher support and
cognitive activation were only relevant to students’ enjoyment at the
classroom level. This might be the case because, in order to enhance the
enjoyment of the individual student, it is necessary to provide students
with meaningful and personally involving activities (Hidi & Renninger,
2006). These instructional approaches are more closely related to
adaptive instruction than to behavioral learning support and effective
management of disruptions. However, for educational practice, our
findings point to the high relevance of support and classroom man-
agement when aiming to enhance overall enjoyment in class.
When interpreting the findings of this study, it must be noted that
missing data analyses indicated that those students who did not report
data about achievement emotions tended to have lower mathematics
competence. Consequently, our findings might apply especially to stu-
dents with relatively high competence in mathematics. Further studies
are needed to test the generalizability of our findings in particular to
groups of low-achieving students. Furthermore, we did not have in-
formation about the number of classrooms per teacher. For the PISA-I-
Plus study, two complete ninth-grade classrooms were assessed per
school in 2003. However, it might be the case that some of the class-
rooms had the same teacher. Given the large sample size, the results
might not be biased even if teachers were teaching in two classrooms
per school. However, further studies are needed to validate the gen-
eralizability of our findings. For example, using the classroom as a
cluster variable in multilevel models when, at the same time, some
teachers are teaching in more than one classroom leads to the question
of whether classrooms which are taught by the same teacher are more
similar to each other in terms of instructional quality than other
classrooms that are taught by different teachers (Goellner et al., 2018).
If this would be the case then it might be useful to also consider the
teacher as a cluster variable – however, it is unclear and cannot be
tested in our study whether the variance of instructional quality that is
attributable to the classroom membership is related to the teaching
style of a specific teacher or if it is explained by the characteristics of
the group of students in a joint classroom – or whether the commu-
nications within the group of students and the relationships with the
teacher form students' joint understanding of characteristics of in-
structional quality in their classrooms. This would imply that the var-
iance of instructional quality that is related to students’ classroom
membership can be explained by the interactions between a specific
teacher with a specific group of students rather than only by a stable
style of teaching of a specific teacher or by the characteristics of the
group. Further research is needed to address these questions and will
have implications for research designs in classroom research.
54
Learning and Instruction 61 (2019) 45–59
4.2. Covariates at student and classroom levels
Classrooms in academic-track schools were characterized by lower
levels of mathematics anxiety than classrooms in comprehensive
schools or middle-track schools. This result can be explained by the age
group of our sample: While previous research (Becker et al., 2014) fo-
cused on the effects of early tracking into high-ability schools, our
analysis focused on students at the end of secondary school in aca-
demic-track, comprehensive and middle-track schools. By the end of
secondary school, students’ mathematics self-concept stabilizes and
students in academic-track schools are possibly more aware of their
high competence, consequently displaying a lower level of mathematics
anxiety than students in other school types.
Corroborating with previous findings (Frenzel et al., 2007a; Goetz,
Bieg, et al., 2013), girls in this study reported higher levels of mathe-
matics anxiety and lower levels of mathematics enjoyment than boys. In
addition, they reported significantly lower levels of boredom in
mathematics classrooms than boys. However, mathematics achieve-
ment in this study was substantially related to student gender, with girls
showing lower levels of mathematics competence than boys. Thus, the
higher level of mathematics anxiety and lower level of enjoyment
among girls in this study may be related to their lower mathematics
competence. A possible explanation for such gender differences is
provided by models for gendered socialization processes (Eccles &
Jacobs, 1986), according to which social gender stereotypes affect girls'
and boys’ motivation and achievement in highly gender-stereotyped
domains such as mathematics.
4.3. Limitations
The study has several limitations that should be considered when
interpreting its findings. First, the study focused on unidirectional re-
lations between student-perceived characteristics of teaching quality in
grade 9 and achievement emotions in grade 10. We did not investigate
bidirectional effects because the data set that we used only included
students’ perceptions of teaching quality, and using self-report data
only would not allow us to examine whether achievement emotions
predict actual teaching behaviors.
Furthermore, this study focused on a sample that was assessed in
2003 and 2004. Over the past decade, school reforms have been im-
plemented in Germany, where this study was conducted, also as a result
of international student assessments such as PISA (Maaz, Baumert,
Neumann, Becker, & Dumont, 2013). Despite such changes in educa-
tional systems, our findings provide insights into the different relations
between teaching quality characteristics and achievement emotions and
lead to further questions about the underlying psychological processes
that may explain these effects. However, because changes in educa-
tional systems may affect such relations, further studies should test the
robustness of the results in recent samples (Kuger, Klieme, Lüdtke,
Schiepe-Tiska, & Reiss, 2017).
Another limitation that needs to be addressed is that our study did
not focus on the mediating role of control-value appraisals between the
characteristics of the learning environment and students’ achievement
emotions that is described in control-value theory (Pekrun, 2006). To
deepen our understanding of classroom processes related to achieve-
ment emotions, future studies are necessary that examine such under-
lying mechanisms and fully test the theoretical propositions of control-
value theory.
In this study, we only focused on student reports of teaching quality
because students’ reports of teaching quality have been shown to be
valid predictors of student engagement and motivation (Clausen, 2002;
Maulana & Helms-Lorenz, 2016). However, it might be beneficial to
include multiple sources of information when assessing teaching quality
– observer ratings are advantageous, for example, because observers are
likely to have knowledge about the theoretical underpinnings of teacher
quality research, and to already have experiences with a range of
R. Lazarides, J. Buchholz Learning and Instruction 61 (2019) 45–59

classrooms and teachers (Muijs, 2006).
We further tested doubly-manifest models and thus did not correct
for sampling error associated with aggregating constructs on the stu-
dent level to form constructs on the classroom level (Lüdtke et al.,
2008). In PISA-I-Plus 2003, however, all available students in class-
rooms were assessed (Carstensen, 2006; OECD, 2005) so that sampling
error might not be a source of parameter bias. Another problem might
be that doubly-manifest models cannot correct for measurement error
that might distort the parameter estimates in terms of an under-
estimation of effects (Marsh et al., 2012). Given that we achieved to
identify effects despite this possible source of underestimation, we think
that this could also be interpreted as a sign of robustness of our find-
ings.
4.4. Conclusions and educational implications
This study contributes to previous research on students' achieve-
ment emotions in class by showing that the different characteristics of
teaching quality are not equally relevant in their relation with students'
emotions in mathematics classrooms. Although it needs to be con-
sidered that students' motivation and achievement emotions can also be
seen as situation-specific (Dietrich, Viljaranta, Moeller & Kracke, 2017;
Salmela-Aro, Moeller, Schneider, Spicer & Lavonen, 2016), we tenta-
tively describe implications of our findings that are relevant at the level
of the classroom more than at the level of the individual student. First,
when aiming for classrooms characterized by high enjoyment, different
educational strategies need to be applied than when aiming to manage
classroom-level boredom. When aiming for a high level of enjoyment in
class, the provision of support and cognitively activating tasks seems to
be efficient. When aiming for a low level of boredom in mathematics
classrooms, a low level of disruptions and a high level of cognitive
activation seems to be beneficial. When aiming to deal with mathe-
matics anxiety in class, it may help to provide a high level of learning
support. The results of this study are important for intervention, par-
ticularly when aiming to identify teaching methods that help motivate
students with high levels of anxiety or low levels of enjoyment in
mathematics classrooms. Consequently, as a first step, teachers need to
be able to identify the affective climate in their classrooms – for ex-
ample, mathematics teachers need to be able identify whether their
classrooms are characterized by a high level of student anxiety or
whether the students in their classrooms on average enjoy learning
mathematics. Recognizing such emotions can be considered to be part
of teachers' diagnostic competencies. Possible implications for educa-
tional policy could be that teacher education needs to convey knowl-
edge about students' achievement emotions in class and needs to pre-
pare future teachers for diagnosing such emotions in their classrooms.
For educational practice, the findings also imply that teachers should
provide additional support to anxious students, for example, by being
particularly sensitive in terms of specific task-related feedback and a
constructive approach to failures. When aiming to increase mathe-
matics enjoyment in class and to decrease students' boredom, challen-
ging tasks that do not overextend students' abilities are suitable peda-
gogical means as our findings showed that perceived cognitive
activation enhanced student enjoyment and reduced students’ boredom
in mathematics. Because this study focused on broad dimensions of
teaching quality, future studies should identify specific educational
strategies related to the creation of such supportive, well-structured and
activating learning environments (Baumert et al., 2010; Voss, Kunter,
Seiz, Hoehne, & Baumert, 2014).
Acknowledgement
We would like to thank the Research Data Centre (FDZ) at the
Institute for Educational Quality Improvement for providing the PISA-I-
Plus dataset.

Appendix A

Table 1
Tests of measurement invariance across time (grade 9, grade 10) and level (student-level, classroom-level) for anxiety

M1 M2 M3 M4

2
χ 961.80 953.16 975.06 912.28
df 66 74 79 83
χ2 −8.64 21.9 −62.78
df 8 5 4
CFI 0.942 0.944 0.942 0.942
ΔCFI 0.002 −0.002 0
RMSEA 0.054 0.051 0.050 0.050
ΔRMSEA −0.003 −0.001 0

Note. M1 = measurement model variant (configural model); M2 = loadings time- but not level-invariant; M3 = loadings time- but not level-invariant,
intercepts time-invariant; M4 = loadings time- and level-invariant and intercepts time-invariant.

Table 2
Level tests of measurement invariance across time (grade 9, grade 10) and level (student-, classroom-level) for boredom

M1 M2 M3 M4

χ2 1074.07 1100.49 1218.28 1196.47

df 66 74 78 82
χ2 26.42 117.79 −21.81
df 8 4 4
CFI 0.951 0.950 0.950 0.946
ΔCFI −0.001 0 −0.004
RMSEA 0.058 0.055 0.055 0.054
ΔRMSEA −0.003 0 −0.001

Note. M1 = measurement model variant (configural model); M2 = loadings time- but not level-invariant; M3 = loadings time- but not level-invariant,
intercepts time-invariant; M4 = loadings time- and level-invariant and intercepts time-invariant.

55
R. Lazarides, J. Buchholz Learning and Instruction 61 (2019) 45–59

Table 3
Tests of measurement invariance across time (grade 9, grade 10) and level (student-level, classroom-level) for enjoyment

M1 M2 M3 M4

2
χ 1162.60 1221.81 1269.41 1333.18
df 68 76 81 85
χ2 59.21 47.6 63.77
df 8 5 4
CFI 0.959 0.957 0.955 0.953
ΔCFI −0.002 −0.002 −0.002
RMSEA 0.059 0.057 0.057 0.057
ΔRMSEA −0.002 0 0

Note. M1 = measurement model variant (configural model); M2 = loadings time- but not level-invariant; M3 = loadings time- but not level-invariant,
intercepts time-invariant; M4 = loadings time- and level-invariant and intercepts time-invariant.

Appendix B

Student- and classroom-level factor loadings for the CFA models assuming invariance across time (grade 9, grade 10) and levels (student-level,
classroom-level).

Model Item Factor loading λ (student level) Factor loading λ (classroom level)

2
Anxiety
grade 9 anx11 .50 .83
anx12 .61 .78
anx13 .80 .98
anx14 .72 .77
anx15 .74 .96
grade 10 anx21 .48 .46
anx22 .61 .82
anx23 .82 .90
anx24 .74 .83
anx25 .76 .87

Boredom3
grade 9 bor11 .74 .92
bor12 .79 .94
bor13 .76 .92
bor14 .67 .94
bor15 .66 .99
grade 10 bor21 .75 .99
bor22 .79 .96
bor23 .77 .90
bor24 .66 .99
bor25 .67 .99

Enjoyment4
grade 9 enj11 .87 .99
enj12 .83 .95
enj13 .78 .95
enj14 .69 .99
enj15 .72 .92
grade 10 enj21 .88 .99
enj22 .83 .94
enj23 .79 .93
enj24 .71 .98
enj25 .73 .78

Cognitive activation grade 95 cog1 .45 .61

cog2 .34 .34
cog 3 .61 .99
cog 4 .47 .87
cog 5 .63 .98
cog 6 .56 .98
cog 7 .46 .55

Classroom management grade 96 class1 .70 .99

class2 .77 .98
class3 .75 .99

Teacher support grade 97 supp1 .61 .96

supp2 .77 .99
supp3 .73 .99
supp4 .64 .93
supp5 .58 .87

56
R. Lazarides, J. Buchholz Learning and Instruction 61 (2019) 45–59

2
CFA Model fit is reported in Annex A, Table 1, Model 4.
3
CFA Model fit is reported in Annex A, Table 2, Model 4.
4
CFA Model fit is reported in Annex A, Table 3, Model 4.
5
CFA Model fit: χ2(29) = 165.44, CFI = 0.95, TLI = 0.92, RMSEA = 0.04.
6
CFA Model fit: χ2(0) = 0.00, CFI = 1.00, TLI = 1.00, RMSEA = 0.01.
7
CFA Model fit: χ2(10) = 60.91, CFI = 0.99, TLI = 0.98, RMSEA = 0.03.

Appendix C

Wording of the Items used in the Analyses. Note: Items with an asterisk were recoded for the analyses.

Cognitive activation (Kognitiv aktivierende Aufgaben)

Introductory sentence: ‘How do you explore new issues in mathematics? Our mathematics teacher … ’ (Einleitungssatz:
‘Wie erarbeitet ihr einen Sachverhalt in Mathematik? Unser Mathematiklehrer/unsere Mathematiklehrerin … ’)
coga2_1 Our mathematics teacher provides us with tasks that do not have a clear solution and lets us explain this. (Unser Mathematiklehrer/unsere Mathematiklehrerin stellt auch
Aufgaben, die keine eindeutige Lösung haben, und lässt dies erklären.)
coga3_1 Our mathematics teacher provides us with tasks that make us stop and think. (Unser Mathematiklehrer/unsere Mathematiklehrerin stellt Aufgaben, für deren Lösung man Zeit
zum Nachdenken braucht.)
coga4_1 Our mathematics teacher varies tasks in a way that allows us to recognize what we have understood. (Unser Mathematiklehrer/unsere Mathematiklehrerin variiert Aufgaben
so, dass man sieht, was man verstanden hat.)
coga5_1 When we practice, we often transfer what we have learned to other topics or issues. (Wenn wir üben, wenden wir das Gelernte oft auf andere Dinge an.)
coga6_1 Our mathematics teacher changes the introduction/context of mathematics tasks in a way that allows us to recognize whether we have really understood the underlying
mathematical concept. (Unser Mathematiklehrer/unsere Mathematiklehrerin wechselt die Einkleidung von Aufgaben so, dass man erkennt, ob man die mathematische Idee wirklich
verstanden hat.)
coga7_1 Among the exercises are often tasks that allow us to see whether we have really understood what we have learned. (Unter den Übungsaufgaben sind oft Aufgaben, bei denen
man wirklich sieht, ob man etwas verstanden hat.)
coga8_1 Our mathematics teacher often provides us with tasks that do not only require calculating but also finding the correct approach. (Unser Mathematiklehrer/unsere
Mathematiklehrerin stellt häufiger Aufgaben, bei denen es nicht allein auf das Rechnen, sondern vor allem auf den richtigen Ansatz ankommt.)

Classroom management (Klassenmanagement)

Introductory sentence: ‘What happens in your mathematics classroom?’ (Einleitungssatz: ‘Wie geht es bei euch im Mathematikunterricht zu?’)

stoe11* In mathematics classes, the lesson often gets disrupted. (In Mathematik wird der Unterricht oft sehr gestört.)
stoe21* In mathematics classes, there is always loud talking. (In Mathematik wird fortwährend laut gequatscht.)
stoe31* In mathematics classes, there is always some nonsense. (In Mathematik wird im Unterricht andauernd Blödsinn gemacht.)

Teacher support in mathematics classrooms (Unterstützung durch den Lehrer im Mathematikunterricht)

Introductory sentence:
‘How often do the following situations happen in your mathematics classroom?’ (Einleitungssatz:
‘Wie oft kommt bei euch im Mathematikunterricht Folgendes vor?’)

supp11 Our teacher is interested in the learning progress of every single student. (Unser Lehrer/unsere Lehrerin interessiert sich für den Lernfortschritt jedes einzelnen Schülers/jeder
Schülerin.)
supp21 The teacher supports us further when we need help. (Der Lehrer/die Lehrerin unterstützt uns zusätzlich, wenn wir Hilfe brauchen.)
supp31 The teacher supports us in the process of learning. (Der Lehrer/die Lehrerin unterstützt uns beim Lernen.)
supp41 The teacher explains something until we understand it. (Unser Lehrer/unsere Lehrerin erklärt etwas so lange, bis wir es verstehen.)
supp51 The teacher gives us the opportunity to say what we think. (Der Lehrer/die Lehrerin gibt uns Gelegenheit, unsere Meinung zu sagen.)

Mathematics anxiety (Mathematikansgt)

ax2_1 Because I am so afraid of mathematics I would rather not go to school. (Aus Angst vor Mathematik würde ich am liebsten nicht in die Schule gehen.)
ax4_1 When doing my mathematics homework, I worry whether I can cope with everything. (Beim Erledigen meiner Mathematikhausaufgaben mache ich mir Sorgen, ob ich das
überhaupt alles schaffe.)
ax7_1 Because I am so afraid I would rather not take mathematics exams. (Vor lauter Angst würde ich Mathematikprüfungen am liebsten gar nicht mitschreiben.)
ax8_1 When I'm about to take a mathematics exam, I get a stomachache. (Wenn eine Mathematikprüfung bevorsteht, bekomme ich Bauchschmerzen.)
ax9_1 During mathematics exams I am so nervous that I cannot concentrate properly. (Während Mathematikprüfungen bin ich so aufgeregt, dass ich mich nicht richtig konzentrieren
kann.)

Mathematics boredom (Langeweile in Mathematik)

bo1_1 I find mathematics to be boring. (Ich finde Mathematik langweilig.)

bo2_1 Because I am so bored I can hardly keep mysels awake in mathematics class. (Vor Langeweile kann ich mich im Mathematikunterricht kaum wach halten.)
bo3_1 I am bored in mathematics lessons. (In den Mathematikstunden langweile ich mich.)
bo4_1 I am so bored with my mathematics homework that I don't want to continue. (Bei den Mathematikhausaufgaben langweile ich mich so, dass ich keine Lust mehr habe,
weiterzumachen.)
bo5_1 When doing mathematics homework I get tired because I am bored. (Bei den Mathematikhausaufgaben werde ich vor Langeweile schnell müde.)

Mathematics enjoyment (Freude an Mathematik)

jo1_1 Mathematics is fun for me. (Mathematik macht mir Spaß.)

jo2_1 I look forward to mathematics lessons. (Ich freue mich auf die Mathematikstunden.)
jo3_1 I find the tasks and materials so exciting that mathematics lessons are fun for me. (Ich finde den Stoff so spannend, dass mir der Mathematikunterricht richtig Spaß macht.)
jo5_1 I am in a good mood when doing mathematics homework. (Wenn ich Mathematikhausaufgaben mache, bin ich gut gelaunt.)
jo6_1 Mathematics exams are fun for me. (Prüfungen in Mathematik machen mir Spaß.)

57
R. Lazarides, J. Buchholz
References
Ahmed, W., Minnaert, A., van der Werf, G., & Kuyper, H. (2010). Perceived social support
and early adolescents' achievement: The mediational roles of motivational beliefs and
emotions. Journal of Youth and Adolescence, 39(1), 36–46. https://doi.org/10.1007/
s10964-008-9367-7.
Baumert, J., Blum, W., Brunner, M., Dubberke, T., Jordan, A., Klusmann, U., ... Neubrand,
M. (2009). Professionswissen von Lehrkräften, kognitiv aktivierender
Mathematikunterricht und die Entwicklung von mathematischer Kompetenz (COACTIV):
Dokumentation der Erhebungsinstrumente [Professional competence of teachers, cogni-
tively activating instruction, and development of students' mathematical literacy
COACTIV): Scale documentation] (2008) Materialien aus der Bildungsforschung Nr. 83
[Educational Research Material No. 83]. Berlin: Max-Planck-Institut für
Bildungsforschung.
Baumert, J., Kunter, M., Blum, W., Brunner, M., Voss, T., Jordan, A., ... Tsai, Y.-M. (2010).
Teachers' mathematical knowledge, cognitive activation in the classroom, and stu-
dent progress. American Educational Research Journal, 47(1), 133–180. https://doi.
org/10.3102/0002831209345157.
Becker, M., Neumann, M., Tetzner, J., Böse, S., Knoppick, H., Maaz, K., ... Lehmann, R.
(2014). Is early ability grouping good for high-achieving students' psychosocial de-
velopment? Effects of the transition into academically selective schools. Journal of
Educational Psychology, 106(2), 555–568. https://doi.org/10.1037/a0035425.
Bergsmann, E. M., Lüftenegger, M., Jöstl, G., Schober, B., & Spiel, C. (2013). The role of
classroom structure in fostering students' school functioning: A comprehensive and
application-oriented approach. Learning and Individual Differences, 26, 131–138.
https://doi.org/10.1016/j.lindif.2013.05.005.
Bieg, M., Goetz, T., Sticca, F., Brunner, E., Becker, E., Morger, V., et al. (2017). Teaching
methods and their impact on students' emotions in mathematics: An experience-
sampling approach. ZDM Mathematics Education, 49(3), 411–422.
Brophy, J. E. (2000). Teaching. Educational practices. Series 1. Geneva: International Buero
of Education.
Carstensen, C. H. (2006). Technische Grundlagen für die Messwiederholung. In M.
Prenzel,, J. Baumert,, W. Blum,, R. Lehmann,, D. Leutner,, & M. Neubrand, (Eds.).
PISA 2003. Untersuchungen zur Kompetenzentwicklung im Verlauf eines Schuljahres (pp.
309–323). Münster: Waxmann.
Chen, F. F. (2007). Sensitivity of goodness of fit indexes to lack of measurement in-
variance. Structural Equation Modeling, 14(3), 464–504.
Clausen, M. (2002). Unterrichtsqualität: Eine frage der Perspektive? [Quality of classroom
learning environments - a question of perspective?], Vol 29. Münster: Waxmann.
Danielson, C. (2011). Enhancing professional practice: A framework for teaching. Virginia,
USA: Association for Supervision and Curriculum Development.
Deci, E. L., & Ryan, R. M. (2002). Handbook of self-determination research. New York:
University Rochester Press.
Dietrich, J., Viljaranta, J., Moeller, J., & Kracke, B. (2017). Situational expectancies and
task values: Associations with students' effort. Learning and Instruction, 47, 53–64.
Eccles, J. S., Adler, T. F., Futterman, R., Goff, S. B., Kaczala, C. M., Meece, J., et al. (1983).
Expectancies, values and academic behaviors. In J. T. Spence (Ed.). Achievement and
achievement motives: Psychological and sociological approaches (pp. 75–146). San
Francisco, CA: Freeman.
Eccles, J. S., Freedman-Doan, C., Frome, P., Jacobs, J., & Yoon, K. S. (2000). Gender-role
socialization in the family: A longitudinal approach. In T. Eckes, & H. M. Trautner
(Eds.). The developmental social psychology of gender (pp. 333–360). Mahwah, NJ, US:
Lawrence Erlbaum Associates Publishers.
Eccles, J. S., & Jacobs, J. E. (1986). Social forces shape math attitudes and performance.
Signs, 11(2), 367–380.
Eccles, J. S., Wigfield, A., & Schiefele, U. (1998). Motivation to succeed. In (5 ed.). N.
Eisenberg (Vol. Ed.), Handbook of child psychology: Vol. 3, (pp. 1017–1095). New
York: Wiley.
Emmer, E. T., & Stough, L. M. (2001). Classroom management: A critical part of educa-
tional psychology, with implications for teacher education. Educational Psychologist,
36(2), 103–112. https://doi.org/10.1207/S15326985EP3602_5.
Evertson, C. M., & Emmer, E. T. (1982). Preventive classroom management. In D. Duke
(Ed.). Helping teachers manage classrooms (pp. 2–31). Alexandria, VA: Association for
Supervision and Curriculum Development.
Fauth, B., Decristan, J., Rieser, S., Klieme, E., & Büttner, G. (2014). Student ratings of
teaching quality in primary school: Dimensions and prediction of student outcomes.
Learning and Instruction, 29, 1–9. https://doi.org/10.1016/j.learninstruc.2013.07.
001.
Filak, V. F., & Sheldon, K. M. (2008). Teacher support, student motivation, student need
satisfaction, and college teacher course evaluations: Testing a sequential path model.
Educational Psychology, 28(6), 711–724. https://doi.org/10.1080/
01443410802337794.
Frenzel, A. C., Goetz, T., Lüdtke, O., Pekrun, R., & Sutton, R. E. (2009). Emotional
transmission in the classroom: Exploring the relationship between teacher and stu-
dent enjoyment. Journal of Educational Psychology, 101(3), 705–716. https://doi.org/
10.1037/a0014695.
Frenzel, A. C., Pekrun, R., & Goetz, T. (2007a). Girls and mathematics - a “hopeless”
issue? A control-value approach to gender differences in emotions towards mathe-
matics. European Journal of Psychology of Education, 22(4), 497–514. https://doi.org/
10.1007/BF03173468.
Frenzel, A. C., Pekrun, R., & Goetz, T. (2007b). Perceived learning environment and
students' emotional experiences: A multilevel analysis of mathematics classrooms.
Learning and Instruction, 17(5), 478–493. https://doi.org/10.1016/j.learninstruc.
2007.09.001.
Gläser-Zikuda, M., & Fuß, S. (2008). Impact of teacher competencies on student emotions:
58
Learning and Instruction 61 (2019) 45–59
A multi-method approach. International Journal of Educational Research, 47(2),
136–147. https://doi.org/10.1016/j.ijer.2007.11.013.
Göllner, R., Wagner, W., Eccles, J. S., & Trautwein, U. (2018). Students’ idiosyncratic
perceptions of teaching quality in mathematics: a result of rater tendency alone or an
expression of dyadic effects between students and teachers? Journal of Educational
Psychology, 110(5), 709–725. https://doi.org/10.1037/edu0000236.
Goetz, T., Bieg, M., Lüdtke, O., Pekrun, R., & Hall, N. C. (2013a). Do girls really ex-
perience more anxiety in mathematics? Psychological Science, 24(10), 2079–2087.
Goetz, T., Lüdtke, O., Nett, U. E., Keller, M. M., & Lipnevich, A. A. (2013b).
Characteristics of teaching and students' emotions in the classroom: Investigating
differences across domains. Contemporary Educational Psychology, 38(4), 383–394.
https://doi.org/10.1016/j.cedpsych.2013.08.001.
van de Grift, W. (2007). Quality of teaching in four European countries: A review of the
literature and application of an assessment instrument. Educational Research, 49(2),
127–152.
van de Grift, W., Chun, S., Maulana, R., Lee, O., & Helms-Lorenz, M. (2017). Measuring
teaching quality and student engagement in South Korea and The Netherlands. School
Effectiveness and School Improvement, 28(3), 337–349.
Gunderson, E. A., Ramirez, G., Levine, S. C., & Beilock, S. L. (2012). The role of parents
and teachers in the development of gender-related math attitudes. Sex Roles, 66(3–4),
153–166.
Hagenauer, G., & Hascher, T. (2010). Learning enjoyment in early adolescence.
Educational Research and Evaluation, 16(6), 495–516.
Hidi, S., & Renninger, K. A. (2006). The four-phase model of interest development.
Educational Psychologist, 41(2), 111–127. https://doi.org/10.1207/
s15326985ep4102_4.
Hiebert, J., & Grouws, D. A. (2007). The effects of classroom mathematics teaching on
students' learning. In F. K. Lester (Vol. Ed.), Second handbook of research on mathe-
matics teaching and learning: Vol. 1, (pp. 371–404). Charlotte, NC: Information Age.
Hu, L., & Bentler, P. M. (1999). Cutoff criteria for fit indexes in covariance structure
analysis: Conventional criteria versus new alternatives. Structural Equation Modeling:
A Multidisciplinary Journal, 6(1), 1–55. https://doi.org/10.1080/
10705519909540118.
Julian, M. W. (2001). The consequences of ignoring multilevel data structures in non-
hierarchical covariance modeling. Structural Equation Modeling: A Multidisciplinary
Journal, 8(3), 325–352. https://doi.org/10.1207/S15328007SEM0803_1.
Jussim, L., & Eccles, J. S. (1992). Teacher expectations II: Construction and reflection of
student achievement. Journal of Personality and Social Psychology, 63(6), 947–961.
https://doi.org/10.1037/0022-3514.63.6.947.
Klieme, E., Pauli, C., & Reusser, K. (2009). The pythagoras study: Investigating effects of
teaching and learning in Swiss and German mathematics classrooms. In T. Janik, & T.
Seidel (Eds.). The power of video studies in investigating teaching and learning in the
classroom (pp. 137–160). Münster: Waxmann.
Klieme, E., Schümer, G., & Knoll, S. (2001). Mathematikunterricht in der Sekundarstufe I:"
Aufgabenkultur" und Unterrichtsgestaltung [Mathematics instruction at secondary
level. Task culture and instructional design]. In E. Klieme, & J. Baumert (Eds.).
TIMSS-Impulse für Schule und Unterricht (pp. 43–57). Bonn: Bundesministerium für
Bildung und Forschung.
Kounin, J. S. (1970). Discipline and group management in classrooms. Oxford, England: Holt,
Rinehart & Winston.
Kuger, S., Klieme, E., Lüdtke, O., Schiepe-Tiska, A., & Reiss, K. (2017).
Mathematikunterricht und Schülerleistung in der Sekundarstufe: Zur Validität von
Schülerbefragungen in Schulleistungsstudien. Zeitschrift für Erziehungswissenschaft,
20(2), 61–98. https://doi.org/10.1007/s11618-017-0750-6.
Kunter, M., Baumert, J., Blum, W., Klusmann, U., Krauss, S., & Neubrand, M. (2013).
Cognitive activation in the mathematics classroom and professional competence of teachers:
Results from the COACTIV project. New York: Springer Science & Business Media.
Kunter, M., Baumert, J., & Köller, O. (2007). Effective classroom management and the
development of subject-related interest. Learning and Instruction, 17(5), 494–509.
https://doi.org/10.1016/j.learninstruc.2007.09.002.
Kunter, M., Brunner, M., Baumert, J., Klusmann, U., Krauss, S., Blum, W., et al. (2005).
Der Mathematikunterricht der PISA-Schülerinnen und-Schüler:
Schulformunterschiede in der Unterrichtsqualität [The mathematics instruction of
the PISA students: Differences in instructional quality across the school types].
Zeitschrift für Erziehungswissenschaft, 8(4), 502–520. https://doi.org/10.1007/
s11618-005-0156-8.
Kunter, M., Klusmann, U., Baumert, J., Richter, D., Voss, T., & Hachfeld, A. (2013).
Professional competence of teachers: Effects on instructional quality and student
development. Journal of Educational Psychology, 105(3), 805–820. https://doi.org/10.
1037/a0032583.
Lazarides, R., Rubach, C., & Ittel, A. (2017). Adolescents' perceptions of socializers' be-
liefs, career-related conversations, and motivation in mathematics. Developmental
Psychology, 53(3), 525–539. https://doi.org/10.1037/dev0000270.
LeBreton, J. M., & Senter, J. L. (2008). Answers to 20 questions about interrater reliability
and interrater agreement. Organizational Research Methods, 11(4), 815–852. https://
doi.org/10.1177/1094428106296642.
Lipowsky, F. (2015). Unterricht [classroom teaching]. In E. Wild, & J. Möller (Eds.).
Pädagogische psychologie (pp. 73–101). Heidelberg: Springer.
Lipowsky, F., Rakoczy, K., Pauli, C., Drollinger-Vetter, B., Klieme, E., & Reusser, K.
(2009). Quality of geometry instruction and its short-term impact on students' un-
derstanding of the Pythagorean Theorem. Learning and Instruction, 19(6), 527–537.
https://doi.org/10.1016/j.learninstruc.2008.11.001.
Lüdtke, O., Marsh, H. W., Robitzsch, A., Trautwein, U., Asparouhov, T., & Muthén, B.
(2008). The multilevel latent covariate model: A new, more reliable approach to
group-level effects in contextual studies. Psychological Methods, 13(3), 203–229.
Lüdtke, O., Robitzsch, A., Trautwein, U., & Kunter, M. (2009). Assessing the impact of
R. Lazarides, J. Buchholz
learning environments: How to use student ratings of classroom or school char-
acteristics in multilevel modeling. Contemporary Educational Psychology, 34(2),
120–131. https://doi.org/10.1016/j.cedpsych.2008.12.001.
Ma, X. (1997). Reciprocal relationships between attitude toward mathematics and
achievement in mathematics. The Journal of Educational Research, 90(4), 221–229.
Maaz, K., Baumert, J., Neumann, M., Becker, M., & Dumont, H. (2013). Die Berliner
Schulstrukturreform: Bewertung durch die beteiligten Akteure und Konsequenzen des neuen
Übergangsverfahrens von der Grundschule in die weiterführenden Schulen [The Berlin
School Reform: Evaluation and consequences of new transitional processes from elemen-
tary school to high school]. Münster: Waxmann.
Marsh, H. W., Lüdtke, O., Nagengast, B., Trautwein, U., Morin, A. J., Abduljabbar, A. S.,
et al. (2012). Classroom climate and contextual effects: Conceptual and methodolo-
gical issues in the evaluation of group-level effects. Educational Psychologist, 47(2),
106–124. https://doi.org/10.1080/00461520.2012.670488.
Marsh, H. W., Lüdtke, O., Robitzsch, A., Trautwein, U., Asparouhov, T., Muthén, B., et al.
(2009). Doubly-latent models of school contextual effects: Integrating multilevel and
structural equation approaches to control measurement and sampling error.
Multivariate Behavioral Research, 44(6), 764–802. https://doi.org/10.1080/
00273170903333665.
Marsh, H. W., & Yeung, A. S. (1998). Longitudinal structural equation models of academic
self-concept and achievement: Gender differences in the development of math and
English constructs. American Educational Research Journal, 35(4), 705–738. https://
doi.org/10.3102/00028312035004705.
Maulana, R., & Helms-Lorenz, M. (2016). Observations and student perceptions of the
quality of preservice teachers' teaching behaviour: Construct representation and
predictive quality. Learning Environments Research, 19(3), 335–357.
Maulana, R., Helms-Lorenz, M., Irnidayanti, Y., & van de Grift, W. (2016a). Autonomous
motivation in the Indonesian classroom: Relationship with teacher support through
the lens of self-determination theory. The Asia-Pacific Education Researcher, 25(3),
441–451. https://doi.org/10.1007/s40299-016-0282-5.
Maulana, R., Helms-Lorenz, M., & van de Grift, W. (2015). A longitudinal study of in-
duction on the acceleration of growth in teaching quality of beginning teachers
through the eyes of their students. Teaching and Teacher Education, 51, 225–245.
Maulana, R., Opdenakker, M.-C., & Bosker, R. (2016b). Teachers' instructional behaviors
as important predictors of academic motivation: Changes and links across the school
year. Learning and Individual Differences, 50, 147–156. https://doi.org/10.1016/j.
lindif.2016.07.019.
Ma, X., & Xu, J. (2004). The causal ordering of mathematics anxiety and mathematics
achievement: A longitudinal panel analysis. Journal of Adolescence, 27(2), 165–179.
https://doi.org/10.1016/j.adolescence.2003.11.003.
Muijs, D. (2006). Measuring teacher effectiveness: Some methodological reflections.
Educational Research and Evaluation, 12(1), 53–74.
Muthén, L., & Muthén, B. (1998-2015). Mplus user's guide. Los Angeles, CA: Muthen &
Muthen.
Nosek, B. A., Banaji, M. R., & Greenwald, A. G. (2002). Math = male, me = female,
therefore math ≠ me. Journal of Personality and Social Psychology, 83(1), 44–59.
https://doi.org/10.1037/0022-3514.83.1.44.
OECD (2003). The PISA 2003 assesment framework. Retrieved from https://www.oecd.
org.
OECD (2005). PISA 2003 technical report. Retrieved from Retrieved from https://www.
oecd.org/edu/school/programmeforinternationalstudentassessmentpisa/35188570.
pdf , Accessed date: 27 June 2017.
OECD (2013). PISA 2012 results in focus. What 15-years-olds know and what they can do
with what they know. Retrieved from http://www.oecd.org/pisa/keyfindings/pisa-
2012-results-overview.pdf.
Pakarinen, E., Kiuru, N., Lerkkanen, M.-K., Poikkeus, A.-M., Siekkinen, M., & Nurmi, J.-E.
(2010). Classroom organization and teacher stress predict learning motivation in
kindergarten children. European Journal of Psychology of Education, 25(3), 281–300.
Pauli, C., Drollinger-Vetter, B., Hugener, I., & Lipowsky, F. (2008). Kognitive Aktivierung
im Mathematikunterricht [Cognitive activation in mathematics instruction].
Zeitschrift für Padagogische Psychologie, 22(2), 127–133.
Pekrun, R. (2000). A social-cognitive, control-value theory of achievement emotions. In J.
Heckhausen (Ed.). Motivational psychology of human development (pp. 143–163).
Oxford, England: Elsevier.
Pekrun, R. (2006). The control-value theory of achievement emotions: Assumptions,
corollaries, and implications for educational research and practice. Educational
Psychology Review, 18(4), 315–341. https://doi.org/10.1007/s10648-006-9029-9.
Pekrun, R. (2016). Academic emotions. In K. R. Wentzel, & D. B. Miele (Eds.). Handbook of
motivation at school (pp. 120–144). New York: Routledge.
Pekrun, R., Goetz, T., Titz, W., & Perry, R. P. (2002). Academic emotions in students' self-
59
Learning and Instruction 61 (2019) 45–59
regulated learning and achievement: A program of qualitative and quantitative re-
search. Educational Psychologist, 37(2), 91–105. https://doi.org/10.1207/
S15326985EP3702_4.
Pekrun, R., Götz, T., & Frenzel, A. C. (2005). Academic emotions questionnaire - mathematics
(AEQ-M) - user's manual. Munich: University of Munich: Department of Psychology.
Pekrun, R., Hall, N. C., Goetz, T., & Perry, R. P. (2014). Boredom and academic
achievement: Testing a model of reciprocal causation. Journal of Educational
Psychology, 106(3), 696–710.
Pekrun, R., & Stephens, E. J. (2010). Achievement emotions: A control‐value approach.
Social and Personality Psychology Compass, 4(4), 238–255.
Pianta, R. C., & Hamre, B. K. (2009). Conceptualization, measurement, and improvement
of classroom processes: Standardized observation can leverage capacity. Educational
Researcher, 38(2), 109–119. https://doi.org/10.3102/0013189X09332374.
Prenzel, M. (2006). Untersuchung zur Kompetenzentwicklung im Verlauf eines
Schuljahres: Die Ergebnisse von PISA-I-Plus im Überblick [Investigations of the de-
velopment of competencies across one school year: Overview of the results of PISA-I-
Plus]. In M. Prenzel,, J. Baumert,, W. Blum,, R. Lehrmann,, D. Leutner,, & M.
Neubrand, (Eds.). PISA 2003. Untersuchungen zur Kompetenzentwicklung im Verlauf des
Schuljahres [PISA 2003. Analyses of competence development in the course of a school
year] (pp. 15–28). Münster: Waxmann.
Prenzel, M., Baumert, J., Blum, W., Lehmann, R., Leutner, D., Neubrand, M., et al. (2013).
PISA-I-Plus 2003, 2004. Version: 1. Datensatz. http://doi.org/10.5159/IQB_PISA_I_
Plus_v1 (Berlin: IQB - Institut zur Qualitätsentwicklung im Bildungswesen).
Quested, E., Bosch, J. A., Burns, V. E., Cumming, J., Ntoumanis, N., & Duda, J. L. (2011).
Basic psychological need satisfaction, stress-related appraisals, and dancers' cortisol
and anxiety responses. Journal of Sport and Exercise Psychology, 33(6), 828–846.
Ramm, G., Prenzel, M., Baumert, J., Blum, W., Lehmann, R., Leutner, D., et al. (2006).
PISA 2003. Dokumentation der Erhebungsinstrumente. Münster: Waxmann.
Raudenbush, S. W., & Bryk, A. S. (2002). Hierarchical linear modeling. Applications and data
analysis methods (2nd ed.). Thousands Oaks: Sage.
Salmela-Aro, K., Moeller, J., Schneider, B., Spicer, J., & Lavonen, J. (2016). Integrating
the light and dark sides of student engagement using person-oriented and situation-
specific approaches. Learning and Instruction, 43, 61–70.
Schiepe-Tiska, A., Heine, J.-H., Lüdtke, O., Seidel, T., & Prenzel, M. (2016).
Mehrdimensionale Bildungsziele im Mathematikunterricht und ihr Zusammenhang
mit den Basisdimensionen der Unterrichtsqualität. Unterrichtswissenschaft, 44(3),
211–225.
Seidel, T., & Shavelson, R. J. (2007). Teaching effectiveness research in the past decade:
The role of theory and research design in disentangling meta-analysis results. Review
of Educational Research, 77(4), 454–499.
Skinner, E. A., & Belmont, M. J. (1993). Motivation in the classroom: Reciprocal effects of
teacher behavior and student engagement across the school year. Journal of
Educational Psychology, 85(4), 571–581. https://doi.org/10.1037/0022-0663.85.4.
571.
Soenens, B., & Vansteenkiste, M. (2005). Antecedents and outcomes of self-determination
in 3 life domains: The role of parents' and teachers' autonomy support. Journal of
Youth and Adolescence, 34(6), 589–604.
Su, R., Rounds, J., & Armstrong, P. I. (2009). Men and things, women and people: A meta-
analysis of sex differences in interests. Psychological Bulletin, 135(6), 859–884.
Tanaka, J. S. (1993). Multifaceted conceptions of fit in structural equation models. In K.
A. Bollen, & J. S. Long (Eds.). Testing structural equation models (pp. 10–39). Newbury
Park, CA: Sage.
Vierhaus, M., Lohaus, A., & Wild, E. (2016). The development of achievement emotions
and coping/emotion regulation from primary to secondary school. Learning and
Instruction, 42, 12–21. https://doi.org/10.1016/j.learninstruc.2015.11.002.
Vlachopoulos, S. P., & Michailidou, S. (2006). Development and initial validation of a
measure of autonomy, competence, and relatedness in exercise: The Basic
Psychological Needs in Exercise Scale. Measurement in Physical Education and Exercise
Science, 10(3), 179–201.
Voss, T., Kunter, M., Seiz, J., Hoehne, V., & Baumert, J. (2014). Die Bedeutung des
pädagogisch-psychologischen Wissens von angehenden Lehrkräften für die
Unterrichtsqualität [The importance of pedagogical-psychological knowledge of
preservice teachers for the quality of teaching]. Zeitschrift für Padagogik, 60(2),
184–201.
Watt, H. M. G. (2016). Gender and motivation. In K. Wentzel, & D. Miele (Eds.). Handbook
of motivation at school (pp. 320–339). New York, London: Routledge.
Watt, H. M. G., Hyde, J. S., Petersen, J., Morris, Z. A., Rozek, C. S., & Harackiewicz, J. M.
(2017). Mathematics—a critical filter for STEM-related career choices? A long-
itudinal examination among Australian and US adolescents. Sex Roles, 77(3–4),
254–271. https://doi.org/10.1007/s11199-016-0711-1.