Measuring Student Engagement

27 Addressing the Challenge
of Measuring Student
Engagement
Jennifer A. Fredricks, Tara L. Hofkens,
and Ming-Te Wang
Abstract: Student engagement is a relatively new construct that describes con-
cepts as varied as classroom behaviors, emotional reactions, motivational beliefs,
self-regulatory processes, metacognitive strategies, school belonging and interac-
tions with instructional materials. This chapter reviews a variety of methods to
measure student engagement including self-report surveys, teacher ratings, inter-
views, administrative data, observations, experience sampling methods, and real-
time measures. The authors outline the strengths and limitations of each method.
Next, we present two examples from our own research on approaches to meas-
uring engagement. The goal of these cases is to illustrate how we have addressed
some of the challenges with measurement, as well as showing the importance of
choosing a measurement technique that aligns with the research questions. First,
we describe the results of a qualitative study to develop a new subject-specific
measure of engagement. Next, information on the predictive validity of an obser-
vational measure to assess engagement at the class-level is presented. The chap-
ter concludes with a discussion of measurement limitations, future directions, and
implications for policy and practice.
The term “student engagement” is used to describe such varied concepts as

classroom behaviors, emotional reactions, motivational beliefs, self-regulatory
processes, metacognitive strategies, school belonging, and interactions with
instructional materials. Over the past two decades, there has been an explosion
of interest in the construct of student engagement. This increase is based on
evidence linking engagement to indicators of positive academic adjustment
(e.g., class attendance) and lower risk behavior (e.g., delinquency; Christenson
et al., 2012; Fredricks et al., 2004; Wang & Fredricks, 2014). Moreover, engage-
ment appears to hold tremendous potential as a locus for interventions, as it is
assumed to be malleable, responsive to changes in social and academic factors,
and influenced by personal characteristics (Appleton et al., 2008; Skinner &
Pitzer, 2012).
As enthusiasm for the construct has grown, so too have concerns about incon-
sistencies in the definition and measurement of engagement (Appleton et al., 2008;
This chapter was supported in part by a grant from the National Science Foundation (1315943).
689
Downloaded from https://www.cambridge.org/core. University of Sussex Library, on 02 Mar 2019 at 09:09:08, subject to the Cambridge Core terms of use, available
at https://www.cambridge.org/core/terms. https://doi.org/10.1017/9781316823279.029
690 jennifer a . fredricks , tara l . hofkens , and ming - te wang
Azevedo, 2015; Fredricks & McColskey, 2012). The literature on engagement

is also hampered by its limited attention to the theory that underpins the
choice of measures (Azevedo, 2015; Sinatra et al., 2015). Too often the method
for assessing engagement has driven the conceptualization of the construct,
rather than the construct definition and theoretical framework determining
the way to assess engagement (Sinatra et al., 2015). This lack of definitional
and theoretical clarity has made it difficult to examine specific hypotheses
about the relation between context and engagement, and to compare findings
across studies.
Engagement has been studied in different hierarchal contexts (i.e., prosocial
institutions, schools, classrooms, and discrete learning activities) and at differ-
ent grain sizes, ranging from engagement at the macro-level (i.e., engagement
in school or specific courses) to engagement at the micro-level (i.e., engage-
ment in a discrete task or learning activity; Sinatra et al., 2015; Skinner &
Pitzer, 2012). Whether there is a best measure of engagement is often unclear,
given these different contexts and grain sizes. In this chapter, we review vari-
ous methods to measure engagement, and we discuss the strengths and limita-
tions of each approach. We then present two examples from our own research
to illustrate different approaches to measuring engagement and the way we
have addressed some of the challenges with measurement. We conclude with
a discussion of limitations, future directions, and implications for policy and
practice.
What Is Engagement?
There is broad agreement that engagement is a multidimensional construct
describing the quality of involvement in an activity or learning context. The
most prevalent conceptualization is that engagement consists of three distinct,
yet interrelated, dimensions (Fredricks et al., 2004). Behavioral engagement
has been defined in terms of involvement in classroom and school contexts,
positive conduct, and absence of disruptive behaviors (Fredricks et al., 2004).
Emotional engagement focuses on positive and negative reactions to teach-
ers, classmates, academics, or school; a sense of belonging; and identification
with school or subject domains (Finn, 1989; Voelkl, 1997). Finally, cognitive
engagement is defined as the level of students’ cognitive investment in learn-
ing, which includes being self-regulated and using deep learning strategies
(Fredricks et al., 2004; Meece et al., 1988).
Recently, additional dimensions to the tripartite model of engagement have
been proposed, though more research is necessary to determine if they are
unique components. For example, some scholars have added social engage-
ment as a fourth dimension, which has been defined by the quality of social
Addressing the Challenge of Measuring Student Engagement 691
interactions with peers and teachers in classroom tasks and in the broader
school context (Rimm-Kaufman et al., 2015; Wang et al., 2016). Additionally,
Reeve and Tseng (2011) proposed agentic engagement – which includes pro
active, intentional, and constructive contributions to learning – as an addi-
tional dimension of engagement. Finally, we note that interest can be a
component of meaningful engagement, and that many of the measurement
issues are the same for both of these constructs (see Renninger & Hidi, 2016).
Most studies include surveys in which engagement and disengagement are
operationalized and measured on a single continuum, with lower levels of
engagement indicating disengagement. However, some researchers have begun
to view engagement and disengagement as separate and distinct constructs
that are associated with distinct learning outcomes (Skinner et al., 2009; Wang
et al., 2015, 2016). Researchers also differ on whether they only include char-
acteristics (i.e., indicators such as effort or enjoyment), or whether they also
include antecedents (i.e., facilitators such as teacher or peer support) and out-
comes (such as grades or discipline) in the operationalization and measure-
ment of student engagement (Lam et al., 2012; Skinner & Pitzer, 2012).
An unanswered question in studies of engagement concerns its relation to
motivation. Motivational scholars examine the question of why people act,
think, and do what they do. In general, motivational theories have explained
these motives in terms of individuals’ underlying beliefs, goals, and values
(Eccles & Wigfield, 2002). In contrast, engagement scholars have placed a
greater emphasis on an individual’s interaction with context (Fredricks et al.,
2004). In other words, an individual is engaged in something (i.e., a task, activ-
ity, or relationship), and their engagement is conceptualized as indistinguisha-
ble from the thing with which they are engaging. Engagement is considered to
be malleable and responsive to the variations in context that schools can target
in interventions (Fredricks et al., 2004; Wang & Degol, 2014). Engagement
also tends to be thought of in terms of action, or the behavioral, emotional,
and cognitive manifestations of motivation (Skinner et al., 2009).
Methods for Studying Engagement

In the next section, we review different methods for studying engagement, at
both the macro- and micro-levels, and highlight the strengths and limitations
of each approach. We also provide some examples of ways the methods can
be employed together.
Self-Reports
The most common way to assess engagement in school and in specific subject
areas is self-reporting. In this methodology, students typically respond to a
series of Likert-type questions about their behaviors, attitudes, and experi-

ences. Surveys, in particular, are practical, and can be given to large and diverse
samples of children in classroom settings at a relatively low cost. This makes
it possible to gather data on both engagement and contextual factors at mul-
tiple times, to compare results across schools, and to test different models of
the relation between contextual factors and engagement over time. Collecting
data on students’ perceptions may be a more valid way of understanding how
students make meaning of their experiences than other methods (e.g., teacher
report) that can be highly inferential (Appleton et al., 2008).
Despite these strengths, there are concerns about researchers’ reliance on
self-report methodologies, which are based on the assumption that engage-
ment is stable and can be measured outside of students’ actual involvement in
learning tasks (Greene & Azevedo, 2010). In responding to survey items, stu-
dents need to aggregate their perceptions and reflect on their experiences over
numerous learning tasks. As a result, self-report methods often do not align
with actual or real-time behaviors or strategy use (Greene, 2015; Winne &
Perry, 2000). The validity of students’ self-report responses also depends
on students’ level of reading comprehension, making it difficult to use this
method to assess engagement for students who read below grade level.
Additional limitations of self-report surveys were outlined in Fredricks and
McColskey’s (2012) review of measures to assess elementary through high
school students’ engagement. They reported that of the 14 self-report sur-
veys identified in the review, nine measured engagement at the school level
and only five measured engagement in specific subject areas. Second, only five
of the measures included items to reflect a multidimensional conceptualiza-
tion of engagement, with most including only one or two of the dimensions.
Third, there was often a mismatch between the choice of items and theoreti-
cal conceptions of engagement, with similar items often being used to assess
different dimensions of engagement. Finally, they noted that there was lim-
ited evidence to support the construct and predictive validity of these survey
measures.
Teacher Ratings
There are a few examples of teacher rating scales that have been developed
by researchers to assess student engagement at the school and classroom level
(Fredricks & McColskey, 2012). Some of these rating scales include items
about behavioral and emotional engagement (Skinner et al., 2009), while
others include items that reflect a multidimensional model of engagement
(i.e., behavioral, emotional, and cognitive; Wang et al., 2016; Wigfield et al.,
2008). Teacher rating scales may be more appropriate than self-report meth-
ods for younger children due to reading demands and their limited literacy
skills (Fredricks & McColskey, 2012). They also are easy to collect and allow
for comparison across students. Ratings of student behavior have also been
used by researchers and practitioners to identify and screen for behaviors that
either support or impede academic and social functioning (e.g., Lane et al.,
2012). For example, the Systematic Screening for Behavioral Disorders is an
evidence-based teacher rating scale that has been used to identify students
who are at risk for internalizing (e.g., not talking to other students) and exter-
nalizing (e.g., displays aggression towards objects and persons) behaviors
(Walker & Severson, 1992).
Another benefit of using teacher ratings is that they allow scholars to
examine the correspondence between students’ and teachers’ perceptions of
engagement in the classroom. Teachers tend to be more accurate reporters
of behavioral engagement because these indicators are directly observable. In
contrast, emotional and cognitive engagement tend to be more difficult for
teachers to assess since students can mask their emotions and teachers have to
infer how students engage cognitively with a task based on behavioral indica-
tors (Skinner et al., 2009). Additionally, there are concerns that teachers may
both overestimate and underestimate actual behaviors about potential rater
bias as a result of both student characteristics (e.g., disability, gender, and
socioeconomic status) and teacher characteristics (e.g., knowledge of disabil-
ity and prior experience; Mason et al., 2014).
Interviews and Focus Groups

A few studies have used qualitative techniques including individual interviews
and focus groups with teachers and students to assess engagement and disen-
gagement in school (e.g., Blumenfeld et al., 2005; Conchas, 2001; Fredricks
et al., 2016). These methods can describe both the “how” (the ways that teach-
ers and students conceptualize and make meaning of engagement) and the
“why” (the process by which psychological and contextual factors influence
engagement over time). Qualitative methods have several strengths including
enabling researchers to (1) collect data based on participants’ own categories
and meanings in their own words, (2) study a limited number of cases in depth,
(3) describe in rich detail engagement as embedded in local contexts, and (4)
identify and describe the contextual factors related to engagement (Johnson &
Onwuegbuzie, 2004).
Despite these benefits, qualitative methods have some limitations. Because
of the larger resources that are necessary for collecting and analyzing inter-
view data, studies using these methods often include only a small number of
participants, which can raise concerns about the generalizability of their find-
ings to other settings. The knowledge, skills, and biases of the interviewer also
can impact the quality, depth, and type of responses. As a result, interviewers
need to be well-trained and have strong interpersonal skills (Tolan & Deutsch,
2015). Finally, there are questions about the reliability (stability and consist-
ency) and validity of interview findings, especially with more open-ended
interview protocols (Fredricks & McColskey, 2012).
Administrative (or Institutional) Data

Another method for collecting data on engagement is to use administrative, or
institutional, data that have already been collected and is available in school
records. Examples include attendance, truancy, credits earned, homework
completion, graduation rates, problem behaviors, and assignment submis-
sions (Appleton et al., 2008; Mandernach, 2015). One benefit of using admin-
istrative data to assess engagement is that it is collected on all students and is
easily accessible. In addition, these indicators tend to be meaningful and easily
understood by practitioners, and are often aligned with district and school
priorities. However, one concern is that there is often not a clear demarcation
between indicators and outcomes of engagement, such as grades, discipline,
and number of credits. This lack of clear demarcation between indicators and
outcomes makes it more difficult to explore the consequences of engagement
(Lam et al., 2012). Additionally, there are concerns about potential biases
in reporting of some indicators of disengagement by student characteris-
tics (e.g., race, socioeconomic status, special education status). For example,
low-socioeconomic-status students; minorities, especially African American
students; and boys have been overrepresented in school disciplinary sanc-
tions (e.g., office referrals, suspensions, and expulsions; Skiba et al., 2002).
Finally, administrative data are typically indicators of behavioral engagement
and disengagement and provide limited insight into the other dimensions of
engagement.
Observational Methods
Another technique to assess engagement is to observe the level of engagement
for individual children or for whole classes of students. The majority of these
observational measures assess whether indicators of behavioral engagement
such as on-task behavior, compliance, attention, participation, and disruptive
behavior are present or absent during a defined time interval (Ponitz et al.,
2009; Rimm-Kaufman et al., 2009; Volpe et al., 2005). Some studies score
the average engagement of students in a class (e.g., Pianta et al., 2007), while
others aggregate individual measures of behavioral engagement to form a
single global indicator of behavioral engagement at the classroom level (e.g.,
Briesch et al., 2015).
Aggregating individual ratings to the classroom level assumes that student
engagement is relatively homogeneous. However, recent studies using person-
centered approaches show that student engagement can be quite different for
each individual in the classroom or school based on personal characteristics
and individual interactions with context (Wang & Decol, 2014; Wang & Peck,
2013). Given that individual differences in student engagement are associ-
ated with different learning outcomes, it might be inappropriate to aggregate
individual measures of student engagement to create a global indicator at the

classroom level.
Rather than assessing engagement with prespecified observational cate-
gories, other studies have used narrative and discourse analysis to measure
engagement (Engle & Conant, 2002; Gresalfi, 2009; Nystrand & Gamarond,
1991; Ryu & Lombardi, 2015). This literature has been informed by sociocul-
tural views of learning, and has focused on the processes by which students
come to engage in groups, activities, and communities through the sharing and
negotiating of norms, values, and resources (Ryu & Lombardi, 2015). These
studies have observed the quality of instructional discourse between the indi-
vidual and the group in a specific course, and have assessed teacher question-
ing and the development of student argumentation as evidence of disciplinary
and substantive engagement.
Observational methods can provide a rich description of both engagement
and learning environments. This information can be used to suggest emerg-
ing patterns of engagement that warrant further exploration, as well as to
identify potential triggers of engagement and disengagement. Because obser-
vations are grounded in practice, these measures of student engagement are
also useful to practitioners and can provide deep insight into a particular case
(Renninger & Bachrach, 2015). Furthermore, these techniques can enhance
our understanding of the unfolding of engagement in real time within both
individual and group contexts (Fredricks & McColskey, 2012).
Despite these benefits, there also are several challenges with observational
methods. Scholars need to make decisions about the appropriate time frame
(e.g., continuous duration, momentary time sampling, or partial or whole
interval recording), the setting in which observations are conducted (e.g.,
whole class, small group work, or seat work), the unit of analysis (i.e., individ-
ual or classroom level), the level of specificity (i.e., predetermined categories
or more descriptive techniques), and the level of obtrusiveness (i.e., low intru-
siveness or high obstructiveness) (Waxman et al., 2004; Wood et al., 2016).
Observations can be time-consuming to conduct and analyze. Since these
techniques tend to involve only a small number of students or contexts, there
are questions about the generalizability of such findings to different social and
cultural contexts (Waxman et al., 2004).
There are also concerns about the reliability of observational methods and
potential observer biases, as the quality of the observation depends on the
observer’s ability to capture and make sense of what was observed (Turner &
Meyer, 2000). Furthermore, individual observational measures focus on
behavioral indicators and provide limited information on the quality of
effort, participation, or thinking (Fredricks et al., 2004). Finally, there is lim-
ited information on the predictive validity of these observational techniques
and the relation between these indicators and achievement-related outcomes
(Fredricks & McColskey, 2012).
Experience Sampling
Another way to collect data on student engagement in specific courses and
learning contexts is to use the experience sampling method (ESM; Shernoff
et al., 2003; Uekawa et al., 2007; Yair, 2000). In this methodology, individuals
carry smart phones or electronic pagers or alarm watches for a set time period.
In response to ESM signals on their devices, students fill out a self-report ques-
tionnaire that asks about their location, activities, and cognitive and affective
responses. ESM grew out of research on “flow,” a high level of engagement
in which individuals are so deeply absorbed in a task that they lose awareness
of time and space (Shernoff & Csikzentmihalyi, 2009). One benefit of ESM is
that it allows researchers to examine engagement as it occurs in real time and
in specific contexts, which reduces problems with recall failure and answering
in socially desirable ways (Hektner et al., 2007). Additionally, this technique
can be used to compare engagement levels both within and across contexts.
For example, using ESM techniques, researchers have found that high school
students report the lowest levels of engagement in classroom settings and the
highest levels of engagement in organized out-of-school contexts (Larson,
2000; Larson & Kleiber, 1993).
Despite these benefits, there are some challenges with this methodology.
ESM requires a large time investment from respondents, and the success
of the method depends largely on participants’ ability and willingness to
comply. Moreover, because this methodology requires frequent responses
to survey items, there are concerns about hasty completion, exaggeration,
and deliberation falsification (Shernoff et al., 2003). Since the data collected
through this technique is relatively limited, it also provides limited insight
into individual characteristics and aspects of classroom context that may
help explain variations in engagement. Furthermore, there are concerns that
the multidimensional nature of engagement may not be adequately cap-
tured by the small number of items included in ESM studies (Fredricks &
McColskey, 2012). Finally, there are questions about the feasibility of using
ESM techniques in classroom settings so that data collection is not disruptive
and distracting.
Real-Time Measures
Engagement is assumed to be dynamic, fluctuating, and context dependent,
although several of the measures (i.e., self-report, teacher report, and adminis-
trative data) assess engagement in one context at one point in time. Recently, a
few scholars have attempted to account for the unfolding of engagement over
time by using real-time measures in the context of discrete learning activities
(Gobert et al., 2015; Miller, 2015; Shen et al., 2009). For example, some scholars
have used log files, or the electronic interactions that occur as students work in
online learning environments, to measure behavioral and cognitive engagement
(Azevedo et al., 2010; Gobert et al., 2015). Some examples of indicators of
engagement collected through log files include (1) the number of posts to a dis-
cussion board, (2) the number of pages viewed in an online resource, (3) the
number of edits made during a writing task, and (4) the number of times read-
ing a text. Additionally, log files can provide information on disengagement in
terms of the extent to which a student takes advantage of the properties in
an online system to complete the task rather than deeply thinking through the
material, and the amount of time a student is off task, such as when a student is
surfing the Internet for material unrelated to the learning task (Azevedo et al.,
2010; Gobert et al., 2015; Henrie et al., 2015). Another real-time method used to
collect data on engagement involves eye tracking techniques, which record pat-
terns of eye movement, such as whether a student fixates on a work or object,
whether a student looks back and forth over a text, or whether a student skips
a word (Boucheix et al., 2013; Duchowski, 2007; Miller, 2015). The assump-
tion with eye tracking data is that people look longer at some words or images
because they are thinking more deeply about these objects or are more cogni-
tively engaged (Miller, 2015).
The electroencephalogram (EEG) technique is another method that has
been used to collect data on engagement in real time. In this neurological test,
electrodes are placed on the scalp to measure electrical activity produced by
the brain during authentic learning activities (Antonenko et al., 2010). Brain
activity detected from EEG data has been positively associated with speed
and accuracy while solving chemistry problems (Stevens et al., 2008) and with
the level of difficulty of a reading passage (Mostow et al., 2011). These stud-
ies suggest that EEG methods can provide an accurate measure of cognitive
effort, which is a key indicator of cognitive engagement.
Other researchers have used devices to measure physiological phenomena
associated with an emotional response or experience. One example is a brace-
let that has been used to measure galvanic skin response (e.g., Arroyo et al.,
2009; McNeal et al., 2014; Poh et al., 2010). Others have used galvanic skin
response techniques in combination with blood pressure readings and electro-
encephalography to measure emotional engagement (Shen et al., 2009).
There are several benefits to using these newer real-time measures to assess
engagement. First, these measures are more precise and give information on
engagement levels as it occurs in real time in the context of a discrete learning
activity. This allows researchers to collect large amounts of data over very
short time periods, and to assess and model changes in engagement over time.
Additionally, these techniques do not require participants to stop an activity
to respond to survey questions (Miller, 2015). There also are potential prac-
tical applications to using these techniques. For example, scientists are using
facial recognition and physiological data on emotional engagement to build
adaptive learning systems that can apply behavioral strategies and emotional
support to bolster learning (Kapoor et al., 2007; Shen et al., 2009). Although
these initial efforts are intriguing, there still remain significant questions about
how to reliably collect real-time data on emotions and use this information to
build adaptive systems that can intervene with students to positively impact
learning.
Despite these benefits, there are some limitations and many methodologi-
cal questions concerning the use of real-time measures. Physiological devices
can be complex and expensive, although technological advances continue to
improve affordability (D’Mello & Graesser, 2012).
Some physiological devices are difficult to use in real educational settings,
either because the devices themselves are cumbersome or because the phys-
iological phenomena they are measuring are affected by other physiological
processes, like sweating or movement (Henrie et al., 2015). Furthermore, data
on nervous system arousal can be difficult to interpret without supplemental
self-report or observational information that indicates whether the physiolog-
ical arousal detected is indicative of positive or negative emotions (Henrie
et al., 2015).
Because these methods are relatively new, there are also questions about
the appropriate sampling frequency, time between observations, and level of
granularity. These methods result in large volumes of data, and there are few
guidelines on how to select the appropriate unit of analysis, how to ensure the
validity of these data, and how to discover patterns and relationships in these
data. Moreover, these methods are best suited to well-structured tasks that are
often presented on a computer (Miller, 2015) or in highly controlled experi-
mental settings (Antonenko et al., 2010). As a result, it is not clear whether,
and if so how, this technique can be used in more complex and less structured
learning environments and classroom tasks. Finally, the way these techniques
can account for individual and contextual factors that may explain variations
in student engagement is unclear.
Case Examples
In the next sections of this chapter, we describe two examples from our own
research to show how we have combined different methods in order to address
some of the methodological challenges involved in measuring engagement.
First, we describe how we used interviews and focus groups to develop and
validate the Math and Science Engagement Scales (Wang et al., 2016). As we
noted earlier, there are only a few multidimensional measures of engagement
that have been developed for use in specific subject areas. This has made it dif-
ficult to examine which aspects of engagement are general and which aspects
of engagement are subject specific. Our goal in this work was to use qualitative
methods to help us better understand the way potential respondents of sur-
vey measures (i.e., teachers and students) conceptualize engagement and the
language they use to describe its indicators. Talking to the potential respond-
ents may increase the validity of the scales by helping determine points of
convergence and divergence between researchers and the target population,

as well as by identifying potential unexplored or untapped aspects of the con-
struct (Gelhbach & Brinkworth, 2011).
In the second example, we present preliminary observational data on
students’ behavioral engagement drawn from a study of secondary school
students’ engagement in mathematics classes. Specifically, we describe how
we used observational approaches to measure behavioral engagement at the
classroom level, and how we tested the predictive validity of these methods by
examining the relation between behavioral engagement and academic achieve-
ment. The aim of this observational study was to examine how students’
behavioral engagement in mathematics classes contributes to mathematics
achievement at the student level. Observational methods are useful for inform-
ing our understanding of how classroom-level processes shape the outcomes
of individual students using data that has a higher degree of face validity than
aggregated student self-reports.
Case 1: Using Qualitative Methods to Develop and Validate

the Math and Science Engagement Scales
In this section, we present an example of how we used both interviews and
focus groups to develop and validate the Math and Science Engagement Scales
(Wang et al., 2016). We used the steps outlined by Gelhbach and Brinkworth
(2011) to develop our student self-report measure of engagement and enhance
the ecological validity of our scales. First, we reviewed the academic literature
for different conceptualizations of student engagement, existing survey instru-
ments of engagement, and related constructs. Next, we conducted in-depth
interviews with 34 middle and high school teachers, and conducted interviews
and focus groups with 106 middle and high school students, to learn how they
conceptualized engagement and disengagement in math and science, as well as
the factors that influenced their levels of engagement and disengagement (see
Fredricks et al., 2016 for more discussion of this method).
We wanted to consider how both teachers and students described engage-
ment and disengagement with little prompting from interviewers. So we next
asked more directed probing questions to see if their responses aligned with
the conceptualizations of engagement as a multidimensional construct that
have been outlined in the research literature and include behavioral, emo-
tional, and cognitive components. We then compared indicators that were
derived from a qualitative analysis of the interviews and focus groups against
findings from our literature review and previous measures. The purpose of
this comparison was to determine whether teachers and students included or
excluded the same dimensions of engagement as researchers, as well as the
language they used to describe this construct.
In our initial list of survey items, we included indicators that were men-
tioned by either students or teachers but were not identified in the research,
as well as items from prior measures of engagement that were not mentioned
in the interviews. To ensure that our list of items corresponded to the con-
struct of engagement, we had eight experts rate the items on both clarity
and relevance. As a result of this feedback, we changed the wording of some
items, added items about basic levels of behavioral engagement and the use of
surface-level cognitive strategies, and dropped items that might be misinter-
preted by students (see Fredricks et al., 2016 for more examples). Finally, we
cognitively pretested the revised items with several focus groups of low- and
high-achieving middle and high school students to assess the validity of these
items. These interviews revealed that students did not understand the wording
of some of our items, and they thought some items did not apply to both math
and science (e.g., memorizing steps of a problem only applies to math). As a
result of this feedback, we deleted some items and changed the wording to
better reflect students’ understanding and language.
We found that many of the indicators developed through the qualitative anal-
ysis overlapped with and validated current conceptualizations of engagement,
including indicators of behavioral engagement (e.g., paying attention), affec-
tive or emotional engagement (e.g., enjoyment), and cognitive engagement (e.g.,
applying or connecting ideas). However, our qualitative analysis also revealed
additional indicators – such as negative emotions like frustration and anger, and
doing extra to learn more – that have tended not to be included in other meas-
ures of engagement. Furthermore, our qualitative analysis supported the inclu-
sion of a social dimension of engagement, which has not typically been included
in engagement measures. Indicators included items that reflect social affective
(e.g., caring about others’ ideas) and social cognitive (e.g., building on others’
ideas) dimensions of group interactions. Our qualitative interviews also revealed
some differences in the way students and teachers conceptualize engagement.
For example, students were more likely to discuss competence (showing their
mastery of math and science) as indicative of engagement, and were less likely
than teachers to talk about engagement in terms of cognitive indicators.
Findings from our qualitative analysis also raise questions about the extent
to which engagement is domain general and the extent to which it varies across
subject areas. For example, many of the indicators (like attention and effort)
were not necessarily unique to math and science and have been included in
other measures of engagement. However, we also noted some aspects of
engagement that have been less likely to be included in prior measures and
may be more specific to math and science (such as being frustrated, solving
problems in different ways, and building on ideas). Finally, our interviews
revealed some domain-specific differences in the indicators of engagement.
For example, students were more likely to discuss negative emotions (such as
boredom, frustration, and anxiety in math) and to include social indicators
(such as sharing and contributing to others’ ideas in science).
In sum, the use of qualitative methods helped us validate the conceptu-
alization of engagement outlined in the literature. Talking to the potential
respondents of a survey increased the ecological validity of the construct by

determining points of convergence and divergence between researchers and the
target population, as well as by identifying potential unexplored or untapped
aspects of this construct. Information from these interviews suggested ways to
revise current measures to include aspects of engagement that are not typically
included in current self-report surveys (e.g., social engagement) and to adapt
wording of items to more closely reflect the language teachers and students
use to describe this construct. Finally, the use of qualitative methods raised
some questions to explore in future work on the potential differences between
raters (i.e., teachers and students) in conceptualizations of engagement, and
differences in the indicators of engagement across domains. For example, it is
not clear whether the more socially focused indicators of engagement would
exist in all other subject areas, or whether they are more common in math and
science because of the instructional emphasis on collaboration, complex prob-
lem solving, abstract reasoning, and argumentation and explanation (Sinatra
et al., 2015).
Case 2: Measuring Engagement with Observational Techniques

In this case, we describe how we developed and used an observational measure
of adolescents’ behavioral engagement at the classroom level in our longitudi-
nal STEM engagement study. Studying engagement at the classroom level can
enhance our understanding of students’ engagement in groups, which may, in
turn, help us understand how students’ engagement as a group contributes to
variance in student outcomes. Since student engagement is usually conceptu-
alized and measured at the student level, researchers interested in measuring
or studying classroom-level engagement often aggregate student self-reports.
However, whether the data produced by aggregating individual self-reports
accurately captures student engagement at the classroom level is unclear.
Students’ reports of their academic behaviors are susceptible to social desir-
ability bias, and students may be limited in their ability to accurately reflect
upon and assess their past experiences. In addition, aggregating student-level
data to predict student-level outcomes does not allow for heterogeneity within
classrooms, or for the fact that student perceptions and aggregated compos-
ites of classroom engagement may differentially predict student outcomes.
Observational assessment has the potential to address these limitations by
providing real-time, in situ assessments of engagement based on behavioral
evidence assessed by an outside observer at the classroom level.
In order to investigate the reliability and validity of using observer reports
to study student outcomes, we included an observational assessment of stu-
dents’ behavioral engagement in mathematics. Specifically, we used the engage-
ment dimension of the Classroom Assessment Scoring System for secondary
school students (CLASS-S), an observational tool that provides an assessment
of classroom quality in terms of emotional climate, classroom management,
and instructional supports for learning processes (Pianta et al., 2007). The
CLASS-S framework has been adapted for secondary classrooms and includes
a global measure of students’ behavioral engagement in class. Specifically, the
measure captures “the extent to which all students in class are focused and
participating in the learning activity presented or facilitated by the teacher”
(p. 109), and is assessed with behavioral indicators of engagement like respond-
ing, asking questions, volunteering, actively listening, and lacking off-task
behavior (Pianta et al., 2012). In large-scale validation studies, the engage-
ment dimension of CLASS-S has been shown to have fair inter-rater agree-
ment (exact or adjacent agreement of 76.6 percent) and has been indicated by
raters as easy to score (Kane et al., 2014). Classroom engagement scores on
CLASS-S have also been linked to middle school teachers’ value-added scores
on state standardized tests in New York (Grossman et al., 2013). Thus, there
is preliminary evidence that observational measures of student engagement
predict important achievement outcomes.
During the 2015–16 academic school year, we videotaped 33 math classrooms
consisting of 33 teachers and 492 fifth, seventh, and ninth grade students. The
student sample was 41 percent Caucasian, 52 percent African American, and
49 percent female. 54 percent of students qualified for free or reduced-price
school lunches. The average class size was 15 students. One morning class
period taught by each math teacher in the study was observed. The videotapes
were scored for students’ behavioral engagement in class by having raters cer-
tified in CLASS-S divide the lesson into two consecutive 20-minute segments,
code the segments separately, and then average the scores from the two coded
segments. All video segments were coded by two raters, and the final score for
each class period was calculated by averaging across the scores assigned by
each rater. The coding team attended weekly reliability meetings and attained
high reliability on the engagement dimension (Interclass correlation = 0.856).
Additionally, we collected student demographics and achievement informa-
tion from school records at the end of the academic year (see Table 27.1 for
descriptive statistics).
To examine the relationship between observational scores of classroom
behavioral engagement and student achievement in mathematics, we ran
multi-level regression models in Mplus (Version 7.3; Muthén & Muthén,
1998–2014), which allowed us to account for the clustering of students within
classrooms. First, we ran a fully unconditional model (Model 1) to estimate
the proportion of variance of students’ mathematics course grades that were
within- and between-class components. Next, in Model 2, we added charac-
teristics at the student level, including student gender, race (binary indicator
of Caucasian or non-Caucasian), socioeconomic status (binary indicator of
ineligibility for free or reduced-price lunch), and prior achievement (students’
grade point average from the previous academic year [2014–15]). Finally, in
Model 3, we added observer ratings of students’ behavioral engagement at the
classroom level.
Table 27.1 Descriptive statistics of student- and classroom-level predictors and

achievement outcomes (N = 492 secondary school students)
Range
Variable N Percent Mean SD from to
Student-level predictors
Female 240 48.78
Race
Caucasian 185 40.57
African American 237 51.97
Asian 8 1.75
Hispanic or Latinx/a 3 0.66
Multi-racial 21 4.61
Other 2 0.44
Free or reduced-price lunch 264 53.66
Previous achievement 362 3.22 0.80 0 4.0
Classroom-level predictor
Observer report of student 33 4.29 1.15 2.0 6.5
(behavioral) engagement
Student Outcomes
Math course grade 2015–16 409 84.57 11.62 19.0 100.0
Our results demonstrate the relation between students’ behavioral engage-

ment and mathematics achievement (see Table 27.2). First, the intra-class cor-
relation of 0.40 in the fully unconditional model (Model 1) confirms that a
significant amount of variance in individual student’s math achievement is
explained at the classroom level. Second, Model 3 reveals that observational
reports of students’ behavioral engagement in mathematics class positively
predict students’ math course grades, even after controlling for student gen-
der, minority status, and indicators of socioeconomic status and previous
achievement. Specifically, every one-point increase in students’ overall behav-
ioral engagement in mathematics class was associated with a 3.83 increase in
mathematics grades.
In sum, we found that observational methods were a useful tool for
assessing classroom engagement and demonstrated good predictive validity.
Observational scores were determined by a trained rater over 40 minutes of a
class period and demonstrated high face validity. In addition to overcoming
the potential for bias or inaccurate assessment inherent in student self-reports,
observational methods had the additional advantage of being amenable to
administrators and teachers because video recording did not result in loss
Table 27.2 Hierarchical linear models predicting math achievement with student covariates
and observer reports of students’ behavioral engagement
Model 1: Fully Model 3: Observational

unconditional Model 2: report of student
model Covariates only behavioral engagement
β (SE) β (SE) β (SE)
Fixed effects 77.23** (2.04) 75.57** (2.07) 75.88** (1.88)
Level 1: Student
Female 3.39* (1.12) 3.38* (1.11)
White 5.68** (1.44) 5.44** (1.45)
Free or reduced-price −3.55** (0.96) −3.58** (0.97)
lunch
Previous GPA −0.06 (0.04) −0.04** (0.04)
Level 2: Classroom
Observational report 3.83** (1.00)
of student behavioral
engagement
Random effects
Between level variance, u0 108.43* (37.17) 77.81** (29.01) 55.20* (22.80)
Within level variance, r 160.31** (28.73) 149.87** (26.97) 150.61** (27.08)
Intra-class correlation 0.403 0.320 0.313
Pseudo R-squared
Level 1 0.065 0.065
Level 2 0.282 0.491
*p < 0.05
**p < 0.001
of instructional time. Despite these strengths, it is worth noting that we do

not know how much the presence of the video camera changed instruction,
teacher–student interactions, or students’ engagement in class. It is also not
clear how typical the class period was for that class or group of students. In
order to address these issues, future studies should conduct observations over
multiple days of instruction.
Concluding Thoughts
In this chapter, we have pointed to the challenges of assessing engagement.
When possible, we recommend using multiple methods to assess engage-
ment, and we recognize that the integration of different methodologies adds
additional complexity, such as the appropriate time frame and the temporal
sequence for collecting different types of data. There are also questions about
how variations in the learning environment influence the measures that are
appropriate for assessing engagement. Much of the research using real-time
measures has been conducted in very structured learning contexts, and there
are questions about whether and how these findings generalize to other learn-
ing tasks or settings. Furthermore, there are practical challenges with assess-
ing engagement in classroom contexts in a way that is unobtrusive and does
not disrupt the flow of learning.
There are also questions about how to best integrate results from differ-
ent methodologies. To date, there are few examples of ways to triangulate
data collected from different methods, or how to reconcile discrepant and
sometimes contradictory information about engagement levels provided by
different methods. For example, prior research has shown only moderate
correlations between teachers’ and students’ reports of students’ engagement,
with higher correlations for behavioral than emotional engagement (Skinner
et al., 2009). Teachers may be better able to report on behavioral engagement
because these indicators are more likely to be directly observable, whereas stu-
dents may be better reporters of their emotional and cognitive engagement.
However, even though students may have “better access” to these internal
states than observers and teachers, they may not always be aware of them –
especially younger children – in which case combining student self-report and
interview data with teacher, observer, or physiological data can help get a more
holistic and accurate assessment of engagement levels.
A continuing challenge in measuring engagement is the lack of definitional
clarity about engagement (Azevedo, 2015; Fredricks & McColskey, 2012).
This makes it difficult to compare and meaningfully interpret findings across
different methodologies. Engagement research has come out of a variety of
theoretical and disciplinary traditions, which has led to large variations in
both the operationalization and the measurement of this construct. In future
research, scholars need to more clearly articulate how they define engagement,
describe how their conceptualization is similar or different from other related
educational constructs, and articulate the theoretical framework underpin-
ning the measurement of this construct. Furthermore, it is critical that both
theory and research questions drive the choice of method, as opposed to the
assessment technique determining the theoretical perspective and questions.
One of the reasons that engagement has had such appeal is that it represents
a shift from a focus on individual characteristics to consideration of contextual
factors that can be targeted in an intervention (Fredricks & McColskey, 2012;
Wang & Degol, 2014). Unfortunately, methods used to assess engagement
do so in one context and at one point in time, which has made it difficult to
answer questions regarding the malleability of this construct and identify the
source of engagement (Sinatra et al., 2015). On one hand, macro-level meas-
ures are much easier to administer, but often measure engagement outside of
a learning context and fail to capture the dynamic and fluctuating nature of
engagement. On the other hand, micro-level measures are more precise and
assess engagement in real time in the context of real learning activities, but are
also more difficult to administer and often lack the contextual information
that could help explain engagement processes (Azevedo, 2015).
Implications for Policy and Practice

This chapter highlights different ways of conceptualizing and measuring stu-
dent engagement, and the strengths and weakness of each. Given the potential
importance of engagement for policy and practice and the variety of options
for measuring engagement, it is important that scholars choose methods that
match the theoretical conception and goals of the study.
More research is needed on which methods and survey instruments can
be most easily adopted for use in policy and practice. One of the challenges
with using measures of engagement to inform policy and practice is that these
methods are not often easily accessible to educators and policymakers in a
way that allows for comparison. To help address this concern, Fredricks et al.
(2011) reviewed 21 survey and observational measures of engagement, and
they provided information on the usage, administration, populations studied,
measurement, and psychometric properties of these measures in a way that is
more accessible to practitioners and policymakers.
Presently, many school districts use administrative data collected on student
disengagement as part of an early warning system. Their goal is to identify stu-
dents who are struggling earlier in their school career and use this data to direct
students to appropriate interventions (Balfanz et al., 2007). However, there
are questions about which indicators of disengagement can best identify those
students who are most at risk, how school districts can collect and integrate
this data into a school-based data system, and how educators should interpret
this information and develop appropriate intervention plans (Appleton, 2012).
The measurement approaches outlined in this chapter can also be used to
monitor engagement at the classroom, school, and district levels (Fredricks &
McColskey, 2012). For example, the Institute for Research and Reform in
Education collects survey data on student engagement as part of a structured
process for planning, capacity building, and continuous school improvement
(Connell et al., 2009). The Gwinnett County Public Schools have also been col-
lecting survey data on student engagement as part of efforts to evaluate the effec-
tiveness of district initiatives to increase student engagement (Appleton, 2012).
Furthermore, teachers can use these measures to track students’ engage-
ment in different instructional environments (i.e., whole class, small group,
seat work, or large discussions) and different subject areas. Teachers can use
data on engagement to more effectively design and implement learning expe-
riences for their students, as well as to identify students who may need more
individualized support. For example, the Instructional Practices Inventory is

an observational tool that was developed for teacher leaders to collect data
on engagement at the school level and to facilitate discussion of this data to
influence instructional design and student learning (Valentine, 2005). The field
would benefit immensely from additional examples of the ways educators and
administrators have used engagement data to reflect on and make changes
to their instructional strategies and school policies. Finally, in order for data
on engagement to inform practice, educators will need to be provided with
dedicated time, opportunities to collaborate with their peers, and professional
development related to collecting, analyzing, and using engagement data.
References
Antonenko, P. Paas, F., Grabner, R. & van Gog, T. (2010). Using electroencephalog-
raphy to measure cognitive load. Educational Psychology Review, 22, 425–38.
doi: 10.1007/s10648-010-9130-y.
Appleton, J. J. (2012). Systems consultation: Developing the assessment-to-inter-
vention link with the student engagement instrument. In S. Christenson,
A. Reschly, & C. Wylie (Eds.), Handbook of research on student engagement
(pp. 725–42). New York, NY: Springer.
Appleton, J. J., Christenson, S. L., & Furlong, M. J. (2008). Student engagement with
school: Critical conceptual and methodological issues of the construct.
Psychology in the Schools, 45, 369–86. doi: 10.1002/pits.20303.
Arroyo, I., Cooper, D. G., Burleson, W., Woolf, B. P., Muldner, K., & Christopherson,
R. (2009). Emotion sensors go to school. Conference on Artificial Intelligence
in Education, 200, 17–24.
Azevedo, R. (2015). Defining and measuring engagement and learning in science:
Conceptual, theoretical, methodological, and analytical issues. Educational
Psychologist, 50, 84–94. doi: 10.1080/00461520.2015.1004069.
Azevedo, R., Moos, D., Johnson, A., & Chauncey, A. (2010). Measuring cognitive
and metacognitive regulatory processes used during hypermedia learn-
ing: Issues and challenges. Educational Psychologist, 45, 210–23. doi:
10.1080/00461520.2010.515934.
Balfanz, R., Herzog, L., & MacIver, P. J. (2007). Preventing student disengagement
and keeping students on graduation path in urban middle grade schools:
Early identification and effective interventions. Educational Psychologist, 42,
223–35. doi: 10.1080/00461520701621079.
Blumenfeld, P., Modell, J. Bartko, W. T., Secada, W., Fredricks, J., Friedel, J., & Paris,
A. (2005). School engagement of inner city students during middle childhood.
In C. R. Cooper, C. Garcia Coll, W. T. Bartko, H. M. Davis & C. Chatman
(Eds.), Developmental pathways through middle childhood: Rethinking diver-
sity and contexts as resources (pp. 145–70). Mahwah, NJ: Lawrence Erlbaum.
Boucheix, J. M., Lowe, R. K., Putri, D. K., & Groff, J. (2013). Cueing animations:
Dynamic signaling aids information extraction and comprehension. Learning
and Instruction, 25, 71–84. doi: 10.1016/j.learninstruc.2012.11.005.
Briesch, A. M., Hemphill, E. M., Volpe, R. J., & Daniels, B. (2015). An evaluation
of observational methods for measuring response to class-wide intervention.
School Psychology Quarterly, 30, 37–49. doi: 10.1037/spq0000065.
Christenson, S., Reschly, A., & Wylie C. (Eds.). (2012). Handbook of research on stu-
dent engagement. New York, NY: Springer.
Conchas, G. Q. (2001). Structuring failure and success: Understanding the variability
in Latino school engagement. Harvard Educational Review, 71, 475–504. doi:
10.17763/haer.71.3.280w814v1603473k.
Connell, J. P., Klem, A., Lacher, T., Leiderman, S., & Moore, W. (2009). First things
first: Theory, research, and practice. Howell, NJ: Institute for Research and
Reform in Education.
D’Mello, S. & Graesser, A. (2012). Dynamics of affective states during complex
learning. Learning and Instruction, 22, 145–57. doi: 10.1016/j.learninstruc
.2011.10.001.
Duchowski, A. (2007). Eye tracking methodology: Theory and practice (2nd ed.). New
York, NY: Springer.
Eccles, J. S. & Wigfield, A. (2002). Motivational beliefs, values, and goals. Annual Review
of Psychology, 53, 109–32. doi: 10.1146/annurev.psych.53.100901.135153.
Engle, R. A. & Conant, F. R. (2002). Guiding principles for fostering productive dis-
ciplinary engagement: Explaining an emergent argument in a community of
learners’ classroom. Cognition and Instruction, 20, 399–483. doi: 10.1207/
S1532690XCI2004_1.
Ferguson, R. F. & Danielson, C. (2014). How framework for teaching and tripod 7Cs
evidence distinguish key components of effective teaching. Designing Teacher
Evaluation Systems, 98–143.
Finn, J. D. (1989). Withdrawing from school. Review of Educational Research, 59, 117–
42. doi: 10.3102/00346543059002117.
Fredricks, J. A., Blumenfeld, P. C. & Paris, A. (2004). School engagement: Potential
of the concept: State of the evidence. Review of Educational Research, 74,
59–119. doi: 10.3102/00346543074001059.
Fredricks, J. A. & McColskey, W. (2012). The measurement of student engagement: A
comparative analysis of various methods and student self-report instruments.
In S. Christenson, A. L. Reschy, & C. Wylie (Eds.), Handbook of research on
student engagement (pp. 763–83). New York, NY: Springer.
Fredricks, J., McColskey, W., Meli, J., Mordica, J., Montrosse, B., & Mooney, K. (2011).
Measuring student engagement in upper elementary through high school: A
description of 21 instruments. (Issues & Answers Report, REL 2010–No.
098). Washington, DC: US Department of Education, Institute of Education
Sciences, National Center for Education. Available at http://ies.ed.gov/ncee/
edlabs/projects/project.asp?projectID=268.
Fredricks, J. A., Wang, M. T., Schall, J., Hofkens, T., Snug, H., Parr, A., & Allerton,
J. (2016). Using qualitative methods to develop a survey of math and
science engagement. Learning and Instruction, 43, 5–15. doi: 10.1016/
j.learninstruc.2016.01.009.
Gelhbach, H. & Brinkworth, M. E. (2011). Measure twice: cut down error: A process
for enhancing the validity of survey scales. Review of General Psychology, 15,
380–7. doi: 10.1037/a0025704.
Gobert, J. D., Baker, R. S., & Wixon, M. B. (2015). Operationalizing and detecting
disengagement within online science microworlds. Educational Psychologist,
50, 43–57. doi: 10.1080/00461520.2014.999919.
Greene, B. (2015). Measuring cognitive engagement with self-report scales: Reflections
from over 20 years of research. Educational Psychologist, 50, 13–40. doi:
10.1080/00461520.2014.989230.
Greene, J. A. & Azevedo, R. (2010). The measurement of learners’ self-regulated
cognitive and metacognitive processes while using computer-based learn-
ing environments. Educational Psychologist, 45, 203–9. doi: 10.1080/
00461520.2014.989230.
Gresalfi, M. S. (2009). Taking up opportunities to learn: Constructing dispositions in
mathematics classrooms. Journal of the Learning Sciences, 18, 327–69. doi:
10.1080/10508400903013470.
Grossman, P., Loeb, S., Cohen, J. & Wyckoff, J. (2013). Measure for measure: The
relationship between measures of instructional practice in middle school
English language arts and teachers’ value-added scores. American Journal of
Education, 50, 4–36. doi: 10.1086/669901.
Hektner, J. M., Schmidt, J. A., & Csikzentmihalyi, M. (2007). Experience sampling
method: Measuring the quality of everyday life. Thousand Oaks, CA: Sage
Publications.
Henrie, C. R., Halverson, L. R., & Graham, C. R. (2015). Measuring student engage-
ment in technology-mediated learning: A review. Computers and Education,
90, 36–53. doi: 10.1016/j.compedu.2015.09.005.
Johnson, R. W. & Onwuegbuzie, A. J. (2004). Mixed methods research: A research
paradigm whose time has come. Educational Researcher, 33, 14–26. doi:
10.3102/0013189X033007014.
Kane, T., Kerr, K., & Pianta, R. (2014). Designing teacher evaluation systems: New
guidance from the measures of effective teaching project. New York, NY: John
Wiley & Sons.
Kapoor, A., Burleson, W., & Picard, R. W. (2007). Automatic prediction of failure.
International Journal of Human Computer Studies, 65, 724–6. doi: 10.1016/
j.ijhcs.2007.02.003.
Lam, S., Wong, B. P. H., Yang, H. & Liu, Y. (2012). Understanding student engage-
ment with a contextual model. In S. Christenson, A. L. Reschy, & C. Wylie
(Eds.), Handbook of research on student engagement (pp. 403–19). New York,
NY: Springer.
Lane, K. L., Menzies, H. M, Oakes, W. P., & Kalberg, J. R. (2012). Systematic screen-
ings of behavior to support instruction: From preschool to high school. New
York, NY: Guilford Press.
Larson, R. W. (2000). Toward a psychology of positive youth development. American
Psychologist, 55, 170–83. doi: 10.1037/0003-066X.55.1.170.
Larson, R. W. & Kleiber, D. (1993). Daily experiences of adolescents. In P. H. Tolan &
B. J. Cohler (Eds.), Handbook of clinical research and practice with adolescents
(pp. 125–45). Oxford: John Wiley.
Mandernach, J. (2015). Assessment of student engagement in higher education: A syn-
thesis of literature and assessment tools. International Journal of Learning,
Teaching, and Educational Research, 12, 1–14.
Mason, B., Gunersel, A. B., & Ney, E. (2014). Cultural and ethnic bias in teacher rat-
ings of behavior: A criterion-focused review. Psychology in the Schools, 51,
1017–30. doi: 10.1002/pits.21800.
McNeal, K. S., Spry, J. M., Mitra, R., & Tipton, J. L. (2014). Measuring student
engagement, knowledge, and perceptions of climate change in an introduc-
tory environment geology course. Journal of Geoscience Education, 62, 655–
67. doi: 10.5408/13-111.1.
Meece, J., Blumenfeld, P. C., & Hoyle, R. H. (1988). Students’ goal orientation and cog-
nitive engagement in classroom activities. Journal of Educational Psychology,
80, 514–23. doi: 10.1037/0022-0663.80.4.514.
Miller, B. W. (2015). Using reading times and eye-movements to measure cogni-
tive engagement. Educational Psychologist, 50, 31–42. doi: 10.1080/00461
520.2015.1004068.
Mostow, J., Chang, K. M., & Nelson, J. (2011, June). Toward exploiting EEG input in
a reading tutor. In International conference on artificial intelligence in educa-
tion (pp. 230–7). Berlin and Heidelberg: Springer.
Muthén, L. K. & Muthén, B. O. (1998–2014). Mplus user’s guide. Seventh edition.
Los Angeles, CA: Muthén & Muthén.
Nystrand, M. & Gamoran, A. (1991). Instructional discourse, student engagement,
and literature achievement. Research in the Teaching of English, 25, 261–90.
Pianta, R. C., Hamre, B. K., Haynes, N. J., Mintz, S. L., & La Paro, K. M. (2007).
Classroom Assessment Scoring System Manual, Middle/Secondary Version.
Charlottesville, NC: University of Virginia Press.
Pianta, R. C., Hamre, B. K., & Mintz, S. L. (2012). Classroom Assessment Scoring
System (CLASS): Secondary class manual. Charlottesville, VA: Teachstone.
Poh, M., Swenson, N. C., & Picard, R. W. (2010). A wearable sensor for unobstru-
sive, long-term assessment of electrodermal activity. IEE Transactions on
Biomedical Engineering, 57, 1243–57.
Ponitz, C. C., Rimm-Kaufman, S. E., Grimm, K. J., & Curby, T. W. (2009). Kindergarten
classroom quality, behavioral engagement, and reading achievement. School
Psychology Review, 38, 102–20.
Reeve, J. M. & Tseng, C. (2011). Agency as a fourth aspect of students’ engagement
with learning activities. Contemporary Educational Psychology, 36, 357–67.
doi: 10.1016/j.cedpsych.2011.05.002.
Renninger, K. A. & Bachrach, J. E. (2015). Studying triggers for interest and engage-
ment using observational methods. Educational Psychologist, 50, 58–69. doi:
10.1080/00461520.2014.999920.
Renninger, K. A. & Hidi, S. (2016). The power of interest for motivation and learning.
New York, NY: Routledge.
Rimm-Kaufman, S. E., Baroody, A. E., Larsen, R. A., Curby, T. W., & Abruy, T.
(2015). To what extent do teacher-student interaction quality and student
gender contribution to fifth graders’ engagement in mathematics learning?
Journal of Educational Psychology, 107, 17–185. doi: 10.1037/a0037252.
Rimm-Kaufman, S. E., Curby, T. W., Grimm, K. J., Nathanson, L., & Brock, L.
(2009). The contribution of children’s self-regulation and classroom quality
to children’s adaptive behaviors in the kindergarten classroom. Developmental
Psychology, 45, 958–72. doi: 10.1037/a0015861.
Ryu, S. & Lombardi, D. (2015). Coding classroom interactions for collective

and individual engagement. Educational Psychologist, 50, 70–83. doi:
10.1080/00461520.2014.1001891.
Shen, L., Wang, M., & Shen, R. (2009). Affective e-Learning: Using “emotional”
data to improve learning in pervasive learning environment. Educational
Technology & Society, 12(2), 176–89.
Shernoff, D. J. & Csikszentmihalyi, M. (2009). Flow in schools: Cultivating engaged
learners and optimal learning environments. In R. Gilman, E. S. Huebner, &
M. Furlong (Eds.), Handbook of positive psychology in schools (pp. 131–45).
New York, NY: Routledge.
Shernoff, D. J., Csikzentmihalyi, M., Schneider, B., & Shernoff, E. S. (2003). Student
engagement in high schools from the perspective of flow theory. School
Psychology Quarterly, 18, 158–76. doi: 10.1521/scpq.18.2.158.21860.
Sinatra, G., Heddy, B. C., & Lombard, D. (2015). The challenge of defining and meas-
uring student engagement in science. Educational Psychologist, 1, 1–13. doi:
10.1080/00461520.2014.1002924.
Skiba, R. J., Michael, R. S., Nardo, A. C., & Peterson, R. L. (2002). The color of disci-
pline: Sources of racial and gender disproportionality in school punishment.
The Urban Review, 34, 317–42. doi: 10.1023/A:1021320817372.
Skinner, E. A., Kindermann, T. A., & Furrer, C. J. (2009). A motivational perspec-
tive on engagement and disaffection. Conceptualization and assessment of
children’s behavioral and emotional participation in academic activities in
the classroom. Educational and Psychological Measurement, 69, 493–525. doi:
10.1177/0013164408323233.
Skinner, E. A. & Pitzer, J. R. (2012). Developmental dynamics of student engagement,
coping, and everyday resilience. In S. Christenson, A. L. Reschy, & C. Wylie
(Eds.), Handbook of research on student engagement (pp. 21–45). New York,
NY: Springer.
Stevens, R. H., Galloway, T. L., Berka, C., Johnson, R., & Sprang, M. (2008). Assessing
student’s mental representations of complex problem spaces with EEG tech-
nologies. In Proceedings of the Human Factors and Ergonomics Society Annual
Meeting (Vol. 52, no. 3, pp. 167–71). Thousand Oaks, CA: SAGE Publications.
Tolan, P. H. & Deutsch, N. L. (2015). Mixed methods in developmental science. In
R. Lerner (Ed.), Handbook of child psychology and developmental science
(Vol. 1, 7th ed., 1–45), Hoboken, NJ: Wiley.
Turner, J. C. & Meyer, D. K. (2000). Studying and understanding the instructional
context of classroom: Using our past to forge our future. Educational
Psychologist, 35, 69–85. doi: 10.1207/S15326985EP3502_2.
Uekawa, K., Borman, K., & Lee, R. (2007). Student engagement in the U.S. urban high
school mathematics and science classrooms: Findings on social organization,
race, and ethnicity. Urban Review, 39, 1–43. doi: 10.1007/s11256-006-0039-1.
Valentine, J. (2005). The instructional practices inventory: A process for profiling stu-
dent engaged learning for school improvement. Columbia, MO: University of
Missouri, Middle Level Leadership Center. Retrieved at http://mllc.missouri
.edu/Upload%20Area-Docs/IPI%20Manuscript%2012-07.pdf.
Voelkl, K. E. (1997). Identification with school. American Journal of Education, 105,
204–319. doi: 10.1086/444158.
Volpe, R. J., DiPerna, J. C., Hintze, J. M., & Shapiro, E. S. (2005). Observing students
in classroom settings: A review of seven coding schemes. School Psychology
Review, 34(4), 454–74.
Walker, H. & Severson, H. (1992). Systematic Screening for Behavioral Disorders
(SSBD). (2nd ed.). Technical Manual. Longmont, CA: Sopris West.
Wang, M. T., Chow, A., Hofkens, T., & Salmela-Aro, K. (2015). The trajectories of
student emotional engagement and school burnout with academic and psy-
chological development: Findings from Finnish adolescents. Learning and
Instruction, 36, 57–65. doi: 10.1016/j.learninstruc.2014.11.004.
Wang, M. T. & Degol, J. (2014). Staying engaged: Knowledge and research needs in
student engagement. Child Development Perspectives, 8, 137–43. doi: 10.1111/
cdep.12073.
Wang, M. T. & Fredricks, J. A. (2014). The reciprocal links between school engage-
ment and youth problem behavior during adolescence. Child Development,
85, 722–37. doi: 10.1111/cdev.12138.
Wang, M. T., Fredricks, J. A., Ye, F., Hofkens, T., & Schall, J. (2016). The math science
engagement scale: Development, validation, and psychometric properties.
Learning and Instruction, 43, 16–26. doi: 10.1016/j.learninstruc.2016.01.008.
Wang, M. T. & Peck, S. C. (2013). Adolescent educational success and mental health
vary across school engagement profiles. Developmental Psychology, 49, 1266–
76. doi: 10.1037/a0030028.
Waxman, H. C., Tharp, R. G., & Hilberg, R. S. (2004). Future directions for class-
room observation research. In H. C. Waxman, R. S. Hilberg, & R. G. Tharp
(Eds.), Observational research in U.S. classrooms: New approaches for under-
standing cultural and linguistic diversity (pp. 266–77). Cambridge: Cambridge
University Press.
Wigfield, A., Guthrie, J. T., Perencevich, K. C., Taboada, A., Klauda, S. L., McRae,
A., & Barbosa, P. (2008). Role of reading engagement in mediating the effects
of reading comprehension instruction on reading outcomes. Psychology in
the Schools, 45, 432–45. doi: 10.1002/pits.20307.
Winne, P. H. & Perry, N. E. (2000). Measuring self-regulated learning. In M. Boekaerts,
P. Pintrich, & M. Zeidner (Eds.), Handbook of self-regulation (pp. 531–66).
San Diego, CA: Academic Press.
Wood, B. K., Hojnoski, R. L., Laracy, S. D., & Olson, C. L. (2016). Comparison
of observational methods and their relation to ratings of engagement in
young children. Topics in Early Childhood Special Education, 35(4), 211. doi:
10.1177/0271121414565911.
Yair, G. (2000). Educational battlefields in America: The tug of war over students’
engagement with instruction. Sociology of Education, 73, 247–69. doi:
10.2307/2673233.

Measuring Student Engagement

Uploaded by

Document Information

Original Description:

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Measuring Student Engagement

Uploaded by

Copyright:

Available Formats

27 Addressing the Challenge

The term “student engagement” is used to describe such varied concepts as

Azevedo, 2015; Fredricks & McColskey, 2012). The literature on engagement

Methods for Studying Engagement

series of Likert-type questions about their behaviors, attitudes, and experi-

Interviews and Focus Groups

Administrative (or Institutional) Data

individual measures of student engagement to create a global indicator at the

convergence and divergence between researchers and the target population,

Case 1: Using Qualitative Methods to Develop and Validate

respondents of a survey increased the ecological validity of the construct by

Case 2: Measuring Engagement with Observational Techniques

Table 27.1 Descriptive statistics of student- and classroom-level predictors and

Our results demonstrate the relation between students’ behavioral engage-

Model 1: Fully Model 3: Observational

of instructional time. Despite these strengths, it is worth noting that we do

Implications for Policy and Practice

individualized support. For example, the Instructional Practices Inventory is

Ryu, S. & Lombardi, D. (2015). Coding classroom interactions for collective

You might also like

Measuring Student Engagement

Uploaded by

Document Information

Original Description:

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Measuring Student Engagement

Uploaded by

Copyright:

Available Formats

27 Addressing the Challenge

The term “student engagement” is used to describe such varied concepts as

Azevedo, 2015; Fredricks & McColskey, 2012). The literature on engagement

Methods for Studying Engagement

series of Likert-­type questions about their behaviors, attitudes, and experi-

Interviews and Focus Groups

Administrative (or Institutional) Data

individual measures of student engagement to create a global indicator at the

convergence and divergence between researchers and the target population,

Case 1: Using Qualitative Methods to Develop and Validate

respondents of a survey increased the ecological validity of the construct by

Case 2: Measuring Engagement with Observational Techniques

Table 27.1 Descriptive statistics of student- and classroom-­level predictors and

Our results demonstrate the relation between students’ behavioral engage-

Model 1: Fully Model 3: Observational

of instructional time. Despite these strengths, it is worth noting that we do

Implications for Policy and Practice

individualized support. For example, the Instructional Practices Inventory is

Ryu, S. & Lombardi, D. (2015). Coding classroom interactions for collective

You might also like

series of Likert-type questions about their behaviors, attitudes, and experi-

Table 27.1 Descriptive statistics of student- and classroom-level predictors and