Teaching and Teacher Education: Leadership and Professional Development 1 (2022) 100003

Contents lists available at ScienceDirect

Teaching and Teacher Education: Leadership and

Professional Development
journal homepage: www.elsevier.com/locate/tatelp

Impacting teachers’ reflection on elementary mathematics classroom

videos in online asynchronous professional learning contexts
Michelle Perry a,∗, Meg S. Bates b, Joseph R. Cimpian c, Shereen Oca Beilstein a, Cheryl Moran d
a
Department of Educational Psychology, University of Illinois at Urbana-Champaign, 1310 S. Sixth St., Champaign, IL 61820, United States
b
Illinois Workforce and Education Research Collaborative, Discovery Partners Institute, University of Illinois, United States
c
Department of Applied Statistics, Social Science, and Humanities, New York University, United States
d
UChicago STEM Education, United States

a r t i c l e i n f o a b s t r a c t

Keywords: Video-based learning (VBL) aids STEM teacher professional development. In synchronous, in-person settings,
Professional learning researchers have linked changes in teachers’ reflective comments about videos to changes in their beliefs and
Elementary mathematics classroom practices, which are associated with improved student outcomes. However, questions remain about
Reflection
VBL on asynchronous professional development websites, without facilitator support. This study experimentally
Learning from video cases
examined how prompts, mathematical content, camera focus, and motivation for commenting impacted 300 ed-
ucators’ reflective comments on videos in an online learning community. Results indicated that prompts targeted
at specific focal areas (teachers, students, mathematics) produced more reflection on those focal areas, but less
reflection in non-focal areas.

1. Introduction
In the United States, the development and retention of highly qual-
ified STEM teachers at the elementary level has been, and continues
to be, an issue of major national importance. Although researchers
have quantified the effects of teachers and certain teaching practices
(e.g., Rivkin, Hanushek & Kain, 2005; Stronge, Ward & Grant, 2011), to
confirm both that teaching matters (Wenglinsky, 2001) and that high-
quality teaching in the STEM disciplines has specific elements (Hiebert &
Grouws, 2007), the challenge remains in improving teaching (Bryk, 2015;
Woodland, 2016).
Towards that end, researchers have made intensive efforts to identify
professional learning and professional development (PD) programs that
promote positive improvements, including impacts of PD on classroom
practice and student outcomes in elementary mathematics. A wealth
of studies has found significant effects for active learning, collective
and collaborative participation of teachers, sustained duration, focus
on student learning of subject matter, and supporting teachers’ reflec-
tive thinking about practice through classroom video (Desimone, Porter,
Garet, Yoon & Birman, 2002; Kennedy, 1998; Kersting, Givvin, Sotelo
& Stigler, 2010; Sherin & van Es, 2009; Yoon, Duncan, Lee, Scarloss &
Shapley, 2007). These successful approaches have, of course, primarily
been vetted in traditional, in-person contexts facilitated by teacher edu-
cators. However, the recent pandemic has exacerbated and highlighted
the need for teachers to access professional learning online and at all
times of day, not just when a session might be offered by a facilita-
tor. Thus, it is critical that researchers examine how to transfer the best
aspects of high-quality PD to the asynchronous, online space, to sup-
port teacher learning and ultimately to improve practice (Borko, 2004;
Dede, Ketelhut, Whitehouse, Breit & McCloskey, 2009).
Recently, Bragg, Walsh and Heyeres (2021) noted some general find-
ings for online (OL) PD. Although the findings from the 11 studies
that Bragg et al. reviewed were impressive, these studies varied widely
on methods (including whether the OLPD was synchronous or asyn-
chronous) and whether the PD included a facilitator or not. To capitalize
on the potential accessibility and lower-cost aspects of providing OLPD
compared to when facilitators must also be present, which can also in-
crease costs, we argue that it is important to focus on OLPD that is fully
asynchronous and runs without a facilitator.
Following Brophy (2003) and Shulman (1986, 2005), the study re-
ported here relied on classroom video clips as “case studies” in elemen-
tary mathematics for promoting teacher learning because these have
been a frequently relied-upon aspect of successful PD (Brophy, 2003;
Shulman, 1986, 2005; Stigler & Perry, 2000), which have had posi-
tive impacts on teachers’ reflection on those cases, which, in turn, has
downstream effects on classroom practice and student outcomes (Scott
& Scott, 2010; Sherin & van Es, 2009). Acknowledging that video-based
learning (VBL) in live settings can be quite effective at improving teach-
ers’ reflective decision-making practices—which include first noticing a
practice or behavior that could be improved, then reflecting on what is

∗
Corresponding author.
E-mail address: mperry@illinois.edu (M. Perry).

https://doi.org/10.1016/j.tatelp.2022.100003
Received 5 November 2021; Received in revised form 30 March 2022; Accepted 19 April 2022
Available online 22 April 2022
2667-3207/© 2022 The Authors. Published by Elsevier Ltd. This is an open access article under the CC BY-NC-ND license
(http://creativecommons.org/licenses/by-nc-nd/4.0/)
M. Perry, M.S. Bates, J.R. Cimpian et al.
noticed, then incorporating the reflection to implement change in prac-
tice—we examined how teachers’ reflective decision-making practices
around the teaching of elementary mathematics could be supported and
improved via VBL in the asynchronous online space. The problems we in-
vestigate here stem from the concern that there are many PD websites,
or virtual learning communities, that have been successful at getting
teachers of elementary mathematics to participate asynchronously online,
but it is not clear how best to support learning in this context. Given the
power of live VBL for teachers’ professional learning, it is crucial that
we find ways to refine VBL for the asynchronous online space.
In the following sections, we provide background on VBL and the
use of reflective-decision-making practices as a teacher-level outcome.
We then justify the need to understand VBL within the online space and
introduce our research questions.
2. Background
2.1. VBL: a promising practice for STEM teacher learning
One of the most heralded professional learning practices for STEM
teachers has been the use of classroom video clips as “case studies”
(Brophy, 2003; Shulman, 1986, 2005; Stigler & Perry, 2000). The idea
traces back at least to Shulman’s (2005) pragmatic conception of using
case studies of high-quality teaching practice as an object of teacher
inquiry and the theoretical perspectives of situated learning and the re-
flective practitioner (Brophy, 2003; Schön, 1983). Although teachers
learn from their experiences in real classrooms, such experiences are
often too chaotic and demanding to allow teachers to reflect and con-
sider alternative options for how to manage the situation successfully
(e.g., Miller & Zhou, 2007). By examining and engaging in reflection on
classroom practice in the less-pressured environment of lesson video-
watching, teachers can develop an anticipated set of actions to be called
upon that will allow them to react in the demanding classroom environ-
ment based on their prior reflective practice with videos, thereby build-
ing a connection between reflection-on-action and reflection-in-action
(Schön, 1983). In this way, reflection on video (reflection-on-action)
that leads to observed changes in practice while teaching (reflection-in-
action) can be likened to transfer, as described in other learning contexts
(see, e.g., Bransford & Schwartz, 1999).
In other words, using video clips of classroom practice provides
teachers with opportunities to pause, consider the crucial factors at play
in the particular moment, and think through and reflect on the ramifi-
cations of a range of reactions and next steps. If teachers can connect
something in the video to their own practice, they may then be in the
position to take that as the grounding for thinking about how to link
their existing practice to a new practice, thereby having the behaviors
in the video act to stimulate teachers’ zone of proximal development
(Vygotsky, 1978) around their own developing understanding of teach-
ing and learning. By becoming better at thinking through classroom sit-
uations while outside the classroom, teachers gradually become better
at managing classroom situations inside the classroom.
The subject of intense research (e.g., Sherin, 2003), VBL is
rooted strongly in the features of effective professional learning
(Desimone et al., 2002; Garet, Porter, Desimone, Birman & Yoon, 2001;
Kennedy, 1998; Yoon et al., 2007), particularly in its emphasis on the ex-
amination of artifacts of practice as central to teacher learning (see Ball
& Cohen, 1999; Borko, 2004; Borko, Jacobs, Eiteljorg, & Pittman, 2008;
Brophy, 2003; Darling-Hammond & McLaughlin, 1995; Seago, 2003;
Shulman, 2005). It has been used successfully in pre-service train-
ing (Chval, Lannin, Arbaugh & Bowzer, 2009; McDuffie et al., 2014;
Sorkin & Preston, 2010; Sun & van Es, 2015) and in in-service training
(Borko, Jacobs, Eiteljorg, & Pittman, 2008; LeFevre, 2003; Roth et al.,
2017; Santagata, 2009; Seago, 2003, 2018; van Es & Sherin, 2008),
becoming widely accepted as good practice for teacher training. We
acknowledge that these components—learning for understanding and
presenting information in the context of real problems—have long been
2
Teaching and Teacher Education: Leadership and Professional Development 1 (2022) 100003
acknowledged to provide fertile opportunities both for learning and for
transfer (e.g., Bransford & Schwartz, 1999).
Researchers of video-based interventions have found links between
teachers viewing and reacting to lesson videos and important outcomes.
For example, Sherin, van Es and colleagues (e.g., Sherin & van Es., 2009;
Sherin, Linsenmeier & van Es, 2009) have used an intervention called
“video clubs” to push teachers’ attention to and reflection on important
elements of practice. They have held regular meetings with teachers to
view video and used different facilitation moves (e.g., González, Deal
& Skultety, 2016; van Es, Tunney, Goldsmith & Seago, 2014) to guide
teachers to notice different actors in the videos (students or teachers)
and to produce different analytical responses to video (describing, eval-
uating, and interpreting). They have found links between changes in
teachers’ responses to video and their classroom practices that support
student learning (Sherin & van Es, 2009).
Similarly, Roth et al. (2017) have engaged science teachers in a year-
long, video-based program called STeLLA (Science Teachers Learning
from Lesson Analysis). The program focuses teacher attention on stu-
dent thinking and science content threads. Teachers first watch other
teachers’ videos, then their own and their peers’, and then finally plan
their own lessons. This program has had positive effects on teacher and
student learning in several unique trials.
Of course, developers of VBL have emphasized that teachers sim-
ply watching video likely will not move the needle on their practice.
The features of the PD intervention surrounding the video are key, from
the alignment of video selection, to learning goals, to the skilled use
of prompts and facilitator moves (McDuffie et al., 2014; Roth et al.,
2017; Seago et al., 2018). Some key areas of continued inquiry in-
clude: (a) the technical features of lesson video clips, including ideal length
(LeFevre, 2003; Seago, 2003) and amount of interactivity with or an-
notation of video allowed (Rich & Hannafin, 2009), (b) the content of
the video itself, including how the video focuses on students and stu-
dent thinking versus focuses on the teacher and teacher’s behaviors
(Sherin et al., 2009), (c) the motivation for teachers to view the video,
be it to provide opinions or counter experiences to their current beliefs
(Chval et al., 2009), to prompt attention to important mathematical mo-
ments (Seago, 2003), or to highlight representative classroom practices
in a country (Miller & Zhou, 2007), and (d) the prompts used to promote
teacher discussion about a video, with previous research suggesting that
a healthy mix of open, focused, and probing questions (van Es, 2010),
as well as prompts that are designed to provide context, promote in-
quiry, maintain focus, and create a collaborative ethic (van Es et al.,
2014), are important. These areas of inquiry are, of course, expanded
and challenged by taking VBL to scale in the online space.
2.2. Improving teacher practice: a focus on reflective decision-making
practices
In general, effective professional learning opportunities support
teachers thinking deeply about aspects of their practice, including
thinking about how students learn particular content. Such reflective
decision-making practices are critical to all professional practices and
learning, well beyond the specific realm of VBL (Schön, 1983), but these
concepts have more recently been applied to mathematics education via
VBL for teachers. Recent research on VBL homes in on the idea that
teachers’ thinking processes—rather than their content knowledge or
repertoire of pedagogical practices—are critical levers for improvement.
Acknowledging the situated nature of teaching, recent PD approaches
are rooted in the idea that teacher competence involves decision-making
in complex environments (Santagata & Yeh, 2015).
Researchers have sought to identify the mental processes that have
the most positive effects on teacher practice, from pedagogical content
knowledge (Shulman, 1986) in specific disciplines (Hill, Rowan & Ball,
2005), to “usable knowledge for teaching” (Kersting, Givvin, Thompson,
Santagata & Stigler, 2012), to professional vision (Sherin et al., 2009). In
this section, we describe some of this ground-breaking work that has ex-
M. Perry, M.S. Bates, J.R. Cimpian et al.
amined this crucial step—the step wherein teachers engage in thinking
that can set the stage for changes in practice.
Sherin and van Es’s (2009) “professional vision” metric examines
teacher commentary around video for what it indicates about what teach-
ers notice most in a classroom video clip (e.g., students, teachers, or
something else) and how teachers respond to what they notice (e.g.,
describing, evaluating, or interpreting the activity). Changes in these
indicators have been linked to changes in classroom practices that are
associated with improved student outcomes in mathematics. For exam-
ple, as teachers moved from describing pedagogy to interpreting student
thinking in videos, they became more supportive of and responsive to
student thinking in the classroom (Sherin & van Es, 2009).
The “usable knowledge” measure (Kersting et al., 2012) goes beyond
theories of teacher content knowledge and pedagogical content knowl-
edge (e.g., Ball, 1996; Shulman, 2005) to take on the problem that there
are teachers who may be knowledgeable with respect to both mathemat-
ics and pedagogy, but whose students may not demonstrate success in
learning, in ways expected by high scores on these measures. To explain
and then to make further predictions, Kersting et al. introduced the idea
that teachers must have this knowledge accessible when needed, which
they called usable knowledge. This knowledge was explored in how teach-
ers respond to video, with teachers who provided suggestions for im-
provement of the pedagogical steps based on the mathematical content
and thinking in the video having more positive student outcomes than
teachers who did not do this spontaneously (e.g., Kersting et al., 2010).
Bates et al. (2019) have incorporated aspects of Sherin and van Es’s
(2009) as well as Kersting et al.’s (2010) theories into their model, which
pivots on the idea that reflection is the key to bringing about change in
practice. In this model, reflection can take place both outside of classroom
practice and in the moment (while teaching) (borrowing from Schön’s,
1987, classic distinction). Although they suspected that what matters
most for impacting student learning is reflection in the moment, they pro-
posed that reflection outside of classroom practice should also have an
effect by providing crucial preparation—much like when we rehearse
to build a routine—thereby impacting reflection in the moment by in-
creasing the likelihood of access when needed. From this model, finding
ways to help teachers develop reflective practices, even on video records
of other teachers’ classrooms, should inform teachers’ ability to reflect
in the moment, and thus on their classroom teaching and, ultimately,
on their students’ outcomes.
What many of these frameworks have in common is an interest in
changing what teachers “notice” in classroom interactions as a first step,
then how they reflect on, interpret, or make sense of what they no-
tice, and finally how they respond to those interactions in their class-
rooms based on their interpretations (Santagata & Yeh, 2015; Sherin
& van Es, 2009). In each of these models, in the second phase, teach-
ers’ mental processes around and thinking about students and classroom
practice are important, whether they come before, during, or after actu-
ally interacting with students. These mental processes are also theorized
to exhibit themselves during (and can be improved via) the analysis of
classroom video, although undoubtedly these processes can be devel-
oped through examination of other artifacts like student work and high-
quality curriculum materials (or through general professional practices,
e.g., Schön, 1983).
We summarize these mental processes under the term “reflective
decision-making practices,” knowing that others may use different ter-
minology. We intentionally use “decision-making practices” because
the goal is to change how teachers perceive and react to situations
in context, which is aligned with what researchers in many other
fields call “decision-making” (e.g., Antonietti, Borsetto & Iannello, 2016;
Barberis, 2013; Frydman & Camerer, 2016; Lee & Koh, 2016). We use
the term “reflective” because the kinds of ways researchers seek to mea-
sure and influence these practices is similar to Schön’s (1983) clas-
sic conception of professionals as engaging in reflection-in-action and
reflection-on-action. Given how fundamental these practices are to the
profession of teaching, and given that they have been shown to be mal-
3
Teaching and Teacher Education: Leadership and Professional Development 1 (2022) 100003
leable through VBL, we focus on reflection as a key outcome in this
study.
2.3. The need to move live VBL to asynchronous online environments
Live VBL has had powerful effects due to the presence of skilled fa-
cilitators who support teachers in noticing new elements of practice and
in making more actionable inferences about student thinking. But con-
sidering the recent pandemic, as well as the move to online learning
in general, it is imperative to understand how we can achieve some of
the power of live VBL in an asynchronous online space. Although some
might argue that VBL simply cannot be effective in such spaces, espe-
cially without live facilitation, we maintain that we must do our best to
try to find ways to do it effectively in such a format. Beyond the simple
reality that learning opportunities are moving online (which was true
even before the pandemic), we believe the online environment also pro-
vides three key advantages for making a long-term impact on teacher
learning.
First, even the best live PD currently lacks reach—both in general
and in relation to the highest-need teachers (Desimone, Smith & Ueno,
2006). Precisely because of its sustained and intensive nature, high-
quality PD has substantial costs in terms of the money, time, and broader
infrastructure needed to make it happen. In the United States, in partic-
ular, the educational system simply does not have the consistent struc-
tures to make such a commitment (Darling-Hammond, Wei, Andree,
Richardson & Orphanos, 2009), nor is sustained funding generally avail-
able for such long-term initiatives, especially if results require a large
time commitment.
One built-in benefit of online opportunities for professional learning
is that these can reach a wide group of teachers in a shorter period of
time with less cost than traditional PD programs. This benefit potentially
mitigates the concern that large-scale studies of teacher PD have found
that PD needs to be of sufficient duration, over months or even years to
yield its intended impact (Yoon et al., 2007). Furthermore, many teacher
educators and professional developers argue that PD programs simply
cannot be effective without highly skilled facilitators and supportive
infrastructure in the schools of participants (Roth et al., 2017). Such
precursors, however, are highly unlikely to happen, especially during
and following a worldwide pandemic that will have profound effects on
school budgets and capacity not only during the crisis, but also for years
to come. Websites such as Teaching Channel, Inside Mathematics, and
the Everyday Mathematics Virtual Learning Community (VLC), the latter
of which serves as the context for the current study, have endeavored
to take PD to scale. Free to join, these websites have the potential to
provide teachers with high-quality PD resources whenever they need
them, thereby making strides towards extending the reach of PD and
potentially overcoming some of the pragmatic concerns with duration
and the need for a live facilitator.
Second, live PD often lacks connectedness. Although most well-
designed PD efforts draw on a theory of situated cognition
(Brophy, 2003; Lave & Wenger, 1991), which seeks to connect pro-
fessional learning to teachers’ lived perspectives, it is still—by its very
structure—typically disconnected from teachers’ own learning goals and
the nature in which they learn as professionals. PD often takes place out-
side the school day and focuses on learning goals that are determined
not by the teachers, but by the PD facilitators. But teachers have their
own learning needs and goals that may not map readily onto those of
PD providers or those focused on during a PD session.
Online communities can meet—and connect—with teachers in spaces
where they are already engaging in self-directed learning. Teach-
ers are professional learners (Webster-Wright, 2009). They are con-
stantly engaging in informal learning activities to help them meet their
goals—from reflecting on what just happened in their classroom, to talk-
ing to another teacher in the hall, to listening to a podcast about how
kids learn, to searching for lesson ideas on Pinterest. Conceptions of
teachers as self-directed professional learners (Webster-Wright, 2009),
M. Perry, M.S. Bates, J.R. Cimpian et al. Teaching and Teacher Education: Leadership and Professional Development 1 (2022) 100003

rather than as subjects in need of PD by others, have shown that even

well-designed PD is not reaching teachers where they are, in terms of
what they want to learn and how they engage in learning every day.
Every day, however, teachers can be found on PD websites. Indeed,
currently with over 63,000 members, the VLC has substantial reach and,
with teachers coming to the website to view lesson video and other ar-
tifacts for the past decade, the VLC has substantial connectedness. Gen-
erally, by embedding the elements of effective PD programs in teacher
learning experiences online, teacher educators and professional devel-
opers can support teacher learning in a setting teachers have already
sought out as a trusted, goal-aligned professional-learning resource.
Finally, even the best live PD rarely transfers. Research has shown
that PD efforts that have effects in one context may not be able to trans-
fer their effects to others (Opfer & Pedder, 2011), due to both varying
needs and organizational structures in different schools, as well as vary-
ing needs among teachers in one setting (Guskey, 2002). Many effective
PD programs admittedly require highly trained facilitators and course
materials that are deeply interwoven with the needs of a specific, niche
group of teachers, making the programs difficult to replicate or ineffec-
tive outside the initial purpose.
Online communities have the potential to affect teacher learning,
and by extension, to achieve sustained, transferrable effects on prac-
tice. Now that teachers are signed on and tuned in, online professional
learning communities can leverage—and also test—successful features
of good PD, affecting teacher learning and achieving sustained, trans-
ferrable effects on practice. One such way is to create interventions that
target the mental processes of teachers, like reflective decision-making
practices.
Long story short: Although we would never argue that intensive, rich,
live PD is anything but the gold standard, it is also expensive, hard to
sustain, and often lacks the prerequisite school-level infrastructure to
make it work. Online PD may not be as desirable as the gold standard
of live PD, but it is often a better fit to the constraints of schools and
the lived experiences of teachers. In addition, online PD is happening,
whether it is the gold standard or not. We have a responsibility to make
it the best it can be. Therefore, in this investigation, we examined how
to leverage VBL to support teacher reflection and learning online, which
might ultimately have important and downstream effects on reach, con-
nectedness, and transfer.

2.4. Remaining issue: how to move VBL to the online setting

It is clear that VBL has great promise for STEM-teacher training. The
research base provides several clear elements on which to build, along
with indicators of success that could help a trainer measure progress
in teacher learning. Because of the strong foundation for VBL, a natu-
ral interest in scaling up this learning quickly and widely has emerged.
Indeed, Shulman (2005) called for web databases of case studies that
could be used for teacher learning. Although these websites seemingly
operate with an assumption that the principles of live VBL can be ef-
fectively ported to the web to facilitate asynchronous VBL, previous re-
search on online VBL suggests challenges for teachers to navigate their
own learning on these sites (Barab, MaKinster, Moore & Cunningham,
2001; Bates, Phalen, & Moran, 2016). These relatively lackluster find-
ings provoke the need for empirical research to probe ways to enrich
and improve teacher learning from video in asynchronous learning con-
texts, in which teachers with potentially diverse experiences and needs
intend to access and learn from these websites (e.g., Dede et al., 2009;
Moon, Passmore, Reiser & Michaels, 2014).
Given the need to determine whether teacher reflection on classroom
video can be impacted online and asynchronously, as a crucial step in
supporting teacher professional learning, we developed an experimental
intervention to test which features of videos—and how we framed the
viewing of those videos with prompts to attend to certain features of the
videos—might impact teachers’ reflective decision-making practice, as
measured by their depth of reflection on video. Ideally, by determining
Fig. 1. Flowchart showing each step of the experimental procedure. Note: Each
teacher was randomly assigned to 1 of 4 experimental conditions.
which features lead to greater depth of reflection, these features can be
leveraged to bring teacher professional learning to scale. Specifically,
we asked:
1 Can we impact the depth of teacher reflection on crucial aspects of
mathematics teaching and learning (specifically, the teaching, stu-
dent learning, and the mathematics) with prompts designed to focus
teachers’ attention on these aspects of classroom video clips? And are
these targeted prompts more effective than an open-ended prompt
(see van Es, 2010)?
2 What features of the video clips and prompts are successful at en-
couraging teachers to consider changing their practice?
3 If teachers agreed that they were willing to change their practice,
how do they explain their reasons for changing their practice?
3. Method
3.1. Participants
We recruited 300 educators who were enrolled members of the VLC,
a free professional learning website. The website contains hundreds of

4
M. Perry, M.S. Bates, J.R. Cimpian et al.
lesson video clips regularly viewed by teachers. We opened the website
for participation in this specific study in mid-December 2017, accepting
participants on a first-come, first-served basis, and closed participation
after successfully recruiting the planned 300 teachers for participation,
almost exactly three months later (mid-March 2018).
All participants completed a background survey as part of the exper-
iment (see the Appendix for the information requested in the survey).
The participants had a mean of 13.72 (SD = 7.68) years of teaching ex-
perience, and 93% identified as female (7% male). Regarding education,
220 participants majored in elementary education, 4 in secondary edu-
cation, 23 in education, and 18 in mathematics. In addition, 228 partic-
ipants reported that they pursued additional education and/or training
in graduate school. Regarding race and ethnicity, 4 participants identi-
fied as Asian, 3 identified as Black, 8 identified as Hispanic, 7 identified
as multiracial, 277 identified as White, and 1 preferred not to say. We
note that, according to NCES (2017) data, compared to the U.S. ele-
mentary teaching population, our sample had comparable amounts of
teaching experience (the average U.S. elementary teaching population
had between 10 and 20 years of experience), had a higher percentage of
White teachers (U.S. elementary teaching population is 80% White), and
had a higher percentage of female teachers (U.S. elementary teaching
population is 89% female).
3.2. Experimental procedure
Each participant viewed four different VLC video clips. Each set
of video clips was tailored either to the lower elementary grades (K–
second-grade teachers watched videos from first-grade lessons, seen
by 48.33% of participants) or to the upper elementary grades (third–
sixth-grade teachers watched videos from fourth-grade lessons, seen by
51.67% of participants). We tailored the video to lower or upper grades
because we realized that seeing clips of lessons with mathematical con-
tent vastly different from what they typically considered might not yield
reflection on how teachers might improve their own practice. We pro-
vide a flowchart, outlining how each teacher interacted with the exper-
imental interface in Fig. 1.
The four experimental conditions each manipulated the following
four variables:
1 prompt of what aspect of the video on which to focus (4 levels: stu-
dent thinking, teaching, or the mathematical focus of the lesson, and
open-ended). The prompt for a video was shown both at the end of the
video (for 5 s) and on the study page itself, above a text box requir-
ing a response to the prompt. Participants saw a different prompt for
each of the 4 video clips (i.e., was a within-subjects variable). See
Fig. 2 for screenshots of the experimental interface for two of the
videos.
2 content of the mathematics (2 levels: operations or fractions, which
are two of the most important and widely covered topics in elemen-
tary mathematics),
3 focus of the video [2 levels: the camera was focused on the teacher
leading the whole class (teacher-focused) or on individual learn-
ers (student-focused)], which we included because we realized that
videos of students might be more compelling to lead to a consider-
ation of student thinking than videos of teachers, which has been
linked to improved classroom teaching (Sherin & van Es, 2009) and,
4 motivation for providing commentary (3 levels: for their own learn-
ing, for forming community, or open-ended; see Fig. 3 for a screen-
shot of the open-ended motivation), which we included because we
wanted to highlight the authentic reasons for leaving commentary
and wondered if different motivations might provoke more or less re-
flective commentary. The motivation for watching was shown at the
beginning of each video for 8 s, which allowed enough time to read
it carefully. Participants were assigned such that they saw the same
motivation on all four of their videos (i.e., was a between-subjects
variable).
5
Teaching and Teacher Education: Leadership and Professional Development 1 (2022) 100003
After watching each video, participants then commented on each of
the four videos.
After answering a given prompt for a video and before moving on
to the next video, participants also indicated, via a drop-down menu,
how likely they were to change their practice based on what they saw.
Depending on their reported likelihood (i.e., choosing an option either
that they were, or choosing that they were not, likely to change their
practice), participants were prompted to select a reason (including sev-
eral options and an “other” response) that they were likely or unlikely to
change their practice. Other questions were asked after the video-based
portion of the study that are not utilized in this analysis and are not
reported here.
3.3. Dependent variables
3.3.1. Reflection on video
To get at teachers’ reflection on video, we coded all responses to
video prompts based on a depth-of-reflection rubric (see Bates et al.,
2019). Other researchers (Kersting et al., 2010; Sherin & van Es, 2009)
have designed similar rubrics to capture similar constructs (see
Section 2.2) and thus we adapted others’ rubrics to align more closely
with reflection, the construct we were intending to capture. The purpose
of this scheme was to capture what the teachers wrote about in response
to the video prompts, that may indicate what they noticed and chose to
write about. Using this rubric, we assigned a level 0 to responses that
either did not mention or merely referred to, but did not describe or
explain the dimension of interest (e.g., for student thinking, wrote “I
like what the students did”); we assigned a level 1 to responses that de-
scribed the dimension (e.g., for student thinking, wrote “Students are
exploring through trial and error, how to equally share an item by in-
terpreting the remainder”); and we assigned a level 2 to responses that
provided an inference or interpretation of the dimension, beyond a sim-
ple description (e.g., for student thinking, wrote “Students knew how to
divide the fruit bars between four brothers, but [were] unsure how to
divide the three bars left over between the 4 brothers. From the short
clip that I viewed I would say that the students do not understand the
concept of fractions being equal parts of a whole.”). We achieved good
reliability (Cohen, 1960) for agreement on the depth of reflection on all
3 dimensions, 𝜅 = 0.79 for reliability on depth of reflection on Teaching
(simple agreement = 0.90); 𝜅 = 0.70 for reliability on depth of reflec-
tion on Student Thinking (simple agreement = 0.81); and 𝜅 = 0.70 for
reliability on depth of reflection on Mathematical Content (simple agree-
ment = 0.81).
3.3.2. Other outcomes
Besides reflection, we also examined participants’ selections from the
drop-down menu of whether they were or were not likely to change their
own practice based on the video (and their selected reason, or whether
they wrote in their own reason, of why or why not).
3.4. Analytic plan
In our study, teachers responded to 4 different randomly assigned
prompts. Because the teachers responded multiple times, we need to
have a statistical design that allows for a teacher’s response in one con-
dition to be correlated with their response in another condition. For
example, a highly reflective teacher may be more reflective across all
conditions than a teacher who tends to be less reflective. To account for
these relationships properly, we fit hierarchical linear models (HLMs;
Raudenbush & Bryk, 2002) of responses (level 1) nested within partici-
pants (level 2) to estimate the effects of the experimental conditions on
the participant’s reflection on teaching, student thinking, and mathe-
matical content (to answer Research Question 1), on agreeing that they
were likely to change their practice (to answer Research Question 2),
and on reasons for either endorsing that they likely would or would not
change their practice (to answer Research Question 3). In each model,
M. Perry, M.S. Bates, J.R. Cimpian et al.
responses were clustered within teachers in a 2-level random-intercept
HLM, calculated with robust standard errors. This modeling ensures
that the four responses provided by each teacher (1 response in each
of 4 prompt conditions) are not treated independently when calculat-
ing effects or standard errors, which could have led to biased conclu-
sions. In addition to the various condition manipulations (i.e., prompt,
camera focus, motivation, and math content), all models hold constant
all other factors: grade and prompt order, and include teacher random
effects.
6
Teaching and Teacher Education: Leadership and Professional Development 1 (2022) 100003
Fig. 2. The experimental interface, (a) show-
ing a 1st-grade, teacher-focused video about
fraction content, with the generic “what did
you notice?” prompt and (b) showing a 1st-
grade, teacher-focused video about operations
content, with the teacher-focused prompt.
4. Results
4.1. Effects on reflection
In this section we address RQ1: Can we impact the depth of teacher
reflection on crucial aspects of mathematics teaching and learning (the
teaching, student learning, and the mathematics) with prompts designed
to draw teachers’ attention to these aspects of classroom video clips?
And are these targeted prompts more effective than the typically used
M. Perry, M.S. Bates, J.R. Cimpian et al.
Fig. 3. Screenshot showing the open-ended motivation to comment on the videos with the open-ended prompt.
prompt: “What do you notice?”? Details from our analyses, which we
describe in the following subsections, can be found in Table 1.
4.1.1. Effects of prompts on reflection
Prompting the teachers to attend to the teaching aspect of the video
produced more reflective comments on teaching, but less reflective
comments on student thinking and mathematics relative to the generic
“What do you notice?” prompt. Specifically, directing participants’ at-
tention to teaching increased reflective comments by 𝛽 = 0.136 points
(SE = 0.044), p < .01, on a scale from 0 to 2; in standard deviation units,
the effect converts to an effect size of d = 0.251 relative to the generic
prompt. Additionally, prompting teachers to attend to teaching reduced
reflection on student thinking, 𝛽 = −0.258 (0.049), p < .001, d=−0.410,
and on mathematical thinking, 𝛽 = −0.123 (0.049), p < .05, d = −0.198,
relative to the generic prompt.
Prompting the teachers to attend to the student thinking in the video
produced more reflective comments on student thinking, 𝛽 = 0.206
(0.052), p < .001, d = 0.382, and mathematical reasoning, 𝛽 = 0.201
(0.050), p < .001, d = 0.324, but less reflective comments on teaching,
𝛽 = −0.310 (0.041), p < .001, d = −0.575, relative to the generic prompt.
Video-specific prompts (which typically directed participants’ atten-
tion to the mathematical content of the video) led to less reflection on
teaching, 𝛽 = −0.177 (0.045), p < .001, d = −0.328, but more reflec-
tion on mathematical content, 𝛽 = 0.228 (0.052), p < .001, d = 0.361,
relative to the generic prompt.
It is worth noting that when asked to attend to teaching rather than
to student learning, teachers produced more reflective comments on
teaching (d = 0.827) but less reflective comments on student think-
ing (d = 0.736) and less reflective comments on mathematical thinking
(d = 0.522), all relatively large effects and all ps < 0.001.
7
Teaching and Teacher Education: Leadership and Professional Development 1 (2022) 100003
4.1.2. Effects of mathematical content on reflection
Videos that included lessons about operations, compared to videos
about fractions, led to more reflective comments on student thinking,
𝛽 = 0.140 (0.035), p < .001, d = 0.223, and on the mathematics, 𝛽 =
0.214 (0.036), p < .001, d = 0.346, but no difference in reflection on
teaching.
4.1.3. Effects of the camera’s focus of the video (teacher- vs.
student-focused) on reflection
Teacher-focused videos (where the video captured the whole class,
as opposed to student-focused videos, which captured individual stu-
dents and not the teacher) produced much more reflective comment-
ing on teaching, 𝛽 = 0.656 (0.034), p < .001, d = 1.217, but student-
focused videos produced more reflective comments on student thinking,
𝛽 = 0.251 (0.038), p < .001, d = 0.400. Neither focus led to significant
differences in reflection on the mathematical content.
4.1.4. Effects of motivation on reflection
The participants tended not to pay attention to the motivation we
provided for them to participate: 65% of all participants selected re-
flecting on their own practice as their reason for participating, despite
this being the provided motivation for only 33% of the participants;
and 5% selected community, but this was the motivation for 33% of
the participants. Overall, the motivation condition assigned to partici-
pants did not predict the motivation they reported in our validity checks
(p = .18). Given how infrequently participants accurately paid attention
to the motivation that we provided for them to participate, it is not a
surprise that the motivations to provide reflective commentary (for aid-
ing the community or for aiding one’s own learning) did not produce
more reflective comments than the generic motivation.
M. Perry, M.S. Bates, J.R. Cimpian et al. Teaching and Teacher Education: Leadership and Professional Development 1 (2022) 100003

Table 1
Experimental effects on reflection.

Reflection scales Other outcome

Teaching Student Learning Math. Thinking Reported likelihood of changing own practice

Prompt effect (omitted: Generic)

Teaching focused 0.136∗ ∗ −0.258∗ ∗ ∗ −0.123∗ 0.048
(0.044) (0.049) (0.049) (0.098)
Student learning focused −0.310∗ ∗ ∗ 0.206∗ ∗ ∗ 0.201∗ ∗ ∗ 0.001
(0.041) (0.052) (0.050) (0.096)
Video-specific −0.177∗ ∗ ∗ 0.028 0.224∗ ∗ ∗ 0.106
(0.045) (0.054) (0.052) (0.094)
Video subject (omitted: Students are main subject)
Teacher is main video 0.656∗ ∗ ∗ −0.251∗ ∗ ∗ 0.016 0.654∗ ∗ ∗
subject
(0.034) (0.038) (0.037) (0.073)
Video lesson focus (omitted: Fractions)
Operations −0.025 0.140∗ ∗ ∗ 0.214 ∗∗∗
0.032
(0.029) (0.035) (0.036) (0.069)
Motivation condition (omitted: No explicit motivation)
Community motivation −0.031 0.062 0.080 0.119
(0.048) (0.064) (0.072) (0.152)
Reflection motivation 0.036 0.031 0.060 0.166
(0.049) (0.062) (0.073) (0.148)
Order effects (omitted: order 1)
Order 2 −0.088 −0.011 −0.008 0.072
(0.051) (0.071) (0.081) (0.176)
Order 3 −0.296∗ ∗ ∗ 0.258∗ ∗ ∗ 0.254∗ 0.271
(0.054) (0.078) (0.093) (0.180)
Order 4 −0.174∗ ∗ 0.216∗ ∗ 0.043 0.402∗
(0.058) (0.074) (0.088) (0.170)
Control covariate
Grade-level 4 video −0.058 0.242∗ ∗ ∗ 0.209∗ ∗ ∗ 0.497∗ ∗ ∗
(omitted: grade-level 1) (0.039) (0.052) (0.060) (0.121)
Constant 0.348∗ ∗ ∗ 0.588∗ ∗ ∗ 0.339∗ ∗ ∗ 2.720∗ ∗ ∗
(0.062) (0.076) (0.083) (0.191)
N 1199 1199 1200 1200

Notes. ∗ p < .05, ∗ ∗ p < .01, ∗ ∗ ∗ p < .001. All models are 2-level hierarchical linear models with responses nested within teachers. Robust standard errors appear
in parentheses below the point estimates.

4.1.5. Effects of prompts on reflection: producing the highest levels of
reflection
The models discussed above look at how the experimental conditions
affected reflection treated as a continuous measure. However, one might
be particularly interested in whether the experimental manipulations
could produce higher rates of the highest level of reflection (level 2 in
our coding scheme), rather than participants just going from barely men-
tioning a focus area (level 0) to describing it (level 1). To test this, we
estimated multilevel logistic regressions predicting whether the teacher
provided a response that was coded in the “most reflective” category or
not (i.e., either not reflective at all, level 0, or somewhat reflective, level
1).
The results of these models are presented in Table 2, and they are
largely consistent with the results discussed above. That is, not only did
the experimental conditions shift reflectivity generally, but those shifts
corresponded to more or less of the highest levels of reflectivity. For exam-
ple, Table 2 shows that teachers prompted to focus on teaching provided
1.879 times as many highest-level comments versus lower-level com-
ments on teaching as did teachers in the generic prompt condition (p <
.05); that is, being asked to focus on teaching increased their highest-
level reflectivity on teaching. When prompted to focus on student learn-
ing, teachers provided 0.46 times as many highest-level comments ver-
sus lower-level comments on teaching, compared to the generic prompt
(p < .01); that is, being asked to focus on student learning decreased
their highest-level reflectivity on teaching.
One last notable finding from this analysis of highest-level reflect-
ing is that videos with a teacher present (e.g., whole-class instruction)
were 28.4 times as likely to receive a highest-level reflectivity comment
versus a lower-level comment on teaching as were videos with students
working with each other (p < .001). This is understandable given that
teachers were not present in the student-focused videos. Because the re-
sults of this analysis are consistent with those of the main analysis, we
refer readers to Table 2 for specific results, rather than discussing this set
of findings further here. However, we note that these findings suggest
that the enhancements in reflectivity based on prompt were substantive
and meaningful, not simply participants going from barely mentioning
a focus area to describing it.
4.2. Effects on perceived likelihood of (or of not) changing own practice
Our second research question asked: What features of the video clips
and prompts are successful at encouraging teachers to consider changing
their practice? To answer this question, we asked teachers to rate how
likely they were to change their own practice after watching each video
(see last column in Table 1). Teachers reported being more likely to
change their own practice if they watched a fourth-grade video (vs. first-
grade), 𝛽 = 0.497 (0.121), p < .001, d = 0.419, or if the video was focused
on the teacher (vs. student), 𝛽 = 0.654 (0.073), p < .001, d = 0.552.
The prompt, however, did not affect reported likelihood of changing
practice.
4.3. Reasons for perceived likelihood of (or of not) changing own practice
Our third research question followed up on the second question by
asking: If teachers agreed that they were willing to change their prac-
tice, how do they explain their reasons for changing their practice? We
addressed this question by asking teachers to choose a reason for why
they expected to change or not change their practice due to the video.
The top reason teachers chose for why they expected to change their
own practice was “I try to use practices like this already, and this gave

8
M. Perry, M.S. Bates, J.R. Cimpian et al. Teaching and Teacher Education: Leadership and Professional Development 1 (2022) 100003

Table 2
Experimental effects on highest-level of reflection.

Reflection scales

Teaching Student Learning Math. Thinking

Prompt effect (omitted: Generic)

Teaching focused 1.879∗ 0.456∗ ∗ ∗ 0.714
(0.577) (0.108) (0.203)
Student-learning focused 0.303∗ ∗ 2.014∗ ∗ 2.231∗ ∗
(0.112) (0.450) (0.558)
Video-specific 0.607 1.203 2.462∗ ∗ ∗
(0.229) (0.287) (0.645)
Video subject (omitted: Students are main subject)
Teacher is main video subject 28.461∗ ∗ ∗ 0.479∗ ∗ ∗ 0.866
(10.311) (0.087) (0.158)
Video lesson focus (omitted: Fractions)
Operations 0.893 1.589∗ ∗ 2.680∗ ∗ ∗
(0.201) (0.257) (0.484)
Motivation condition (omitted: No explicit motivation)
Community motivation 0.835 1.240 1.575
(0.305) (0.335) (0.543)
Reflection motivation 1.147 1.017 1.388
(0.407) (0.279) (0.479)
Order effects (omitted: Order 1)
Order 2 0.965 0.930 0.765
(0.375) (0.295) (0.306)
Order 3 0.726 2.284∗ 2.403∗
(0.333) (0.748) (0.953)
Order 4 1.038 1.815 1.119
(0.452) (0.603) (0.466)
Control covariate
Grade-level 4 video (omitted: grade-level 1) 1.277 2.457∗ ∗ ∗ 3.030∗ ∗ ∗
(0.376) (0.565) (0.856)
Constant 0.007∗ ∗ ∗ 0.071∗ ∗ ∗ 0.021∗ ∗ ∗
(0.004) (0.025) (0.010)
N 1200 1200 1200

Notes. ∗ p < .05, ∗ ∗ p < .01, ∗ ∗ ∗ p < .001. All models are 2-level hierarchical logistic regression models with
responses nested within teachers. All estimates are reported as odds ratios. Robust standard errors appear
in parentheses below the odds ratio estimates.
Table 3
Reasons reported for likelihood for changing (not changing) practice.

Student-focused Teacher-focused Total

Reasons unlikely to change

The video clip did not give me any new ideas about how to improve my teaching. 19.83 13.17 16.50∗ ∗
I liked these practices, but I feel good about how I currently teach this content. 10.83 6.17 8.50∗ ∗
I didn’t really like the teaching practices here. 7.00 2.50 4.75∗ ∗ ∗
Other (written in) 3.67 4.50 4.08
Just looking at one video isn’t enough to change my practice on this topic. 4.17 2.00 3.08∗
The classroom in this video is too different from mine, so I don’t think these practices would work in my classroom. 2.33 1.17 1.75
Reasons likely to change
I try to use practices like this already, and this gave me some concrete ideas how to do it. 22.50 40.67 31.58∗ ∗ ∗
The video reveals how students think about this content, and I feel like I can change my practice in response to it. 20.33 21.83 21.08
The video shows a new practice I want to try. 5.33 6.83 6.08
Other (written in) 4.00 1.17 2.58∗ ∗

Note. ∗ p < .05, ∗ ∗ p < .01, ∗ ∗ ∗ p < .001. Column totals sum to 100%, but may not sum exactly due to rounding. Significance indicated on Total column
indicates a significant difference between teacher- and student-focused responses, based on multilevel (2-level) logistic regressions accounting for responses
nested within teachers.

me some concrete ideas how to do it,” selected 31.58% of the time (see
Table 3). The top reasons for not expecting to change their own prac-
tice were “The video clip did not give me any new ideas about how to
improve my teaching,” (16.50%) and “I liked these practices, but I feel
good about how I currently teach this content” (8.50%).
Interestingly, the top reason varied significantly by whether the
teacher viewed a teacher-focused or student-focused video. When view-
ing a teacher-focused video, teachers were much more likely to choose
“I try to use practices like this already, and this gave me some concrete
ideas how to do it” (40.67% vs. 22.50%, p < .001), as the primary reason
for why they perceived a higher likelihood of changing their own prac-
tice. When viewing a teacher-focused video, they were also less likely
to choose “The video clip did not give me any new ideas about how
to improve my teaching,” (13.17% vs. 19.83%, p < .01) and “I liked
these practices, but I feel good about how I currently teach this content”
(6.17% vs. 10.83%, p < .001). There were some significant differences
in other response-options too (shown in Table 3), but they were smaller
in magnitude and selected less often regardless of condition; thus, we
do not discuss them in greater detail here.
5. Discussion
5.1. Supporting teacher learning
We began by considering whether we could find evidence for ways
to make improvements to the reach and connectedness of PD efforts
by understanding ways in which teachers could be supported in pro-
ducing reflective commentary on mathematics lesson video clips. This

9
M. Perry, M.S. Bates, J.R. Cimpian et al.
study took advantage of the existing Everyday Mathematics VLC by recog-
nizing that this website already has substantial reach, with over 63,000
subscribed members, and connectedness, in that teachers already access
this website to achieve their own learning goals, when they need this
support. However, to improve the transfer from the visits to the web-
site to improving teaching practice, we reckoned that teachers needed
to reflect on the videos in productive ways. This study investigated how
the presentation of the videos (the prompts for reflection and focus of
the videos) might impact this crucial reflective engagement with the
mathematics lesson videos.
We also acknowledged that little is known about whether online
PD communities that primarily operate asynchronously are effective in
ways we have come to expect from live VBL in promoting teacher re-
flection, which is linked to teacher learning. Previous research on live
PD with a facilitator has provided robust features that positively impact
teachers’ beliefs and practices, but these are difficult to translate to on-
line, asynchronous PD experiences. Thus, we set out to examine how we
might affect successful reflection, which may be crucial for improving
teacher practice.
5.2. Impacting reflection not ‘In the moment’
Our first research question asked if we could impact teacher reflec-
tion on crucial aspects of mathematics teaching and learning. Overall,
the experimental manipulations substantially shifted reflection in the in-
tended directions, and the focus of the video (on teachers and the whole
class or on individual students) influenced the sorts of comments partic-
ipants made about the videos. These results suggest both that teachers can
be guided to attend to and reflect on certain aspects of classroom practice
and that this can be done asynchronously, thereby allowing the possibility
for teacher PD to be scaled online. This overall finding links back to prior
research on the VLC, where we found that differences in prompts could
elicit deeper analytical commentary from pre-service teachers offline
(Beilstein, Perry, & Bates, 2017). The results that we report here, how-
ever, were more nuanced than these previous findings. We turn to the in-
terpretation of these more complex results—and their implications—in
the remainder of this discussion.
5.3. Supporting teachers to be reflective about teaching, and to encourage
them to anticipate changing their practice
Focusing participants’ attention on the teaching in the selected video
clips, by way of the prompts and by way of the people in the video
(teacher-led whole-class discussion, in this case), led to the most reflec-
tive commentary about teaching and to the most likely endorsement
of changing teaching practice, which was the focus of our second and
third research questions. Although it is not a surprise that asking the re-
spondents to focus on the teaching led to reflective commentary about
teaching, we wonder why it was that watching a teacher resulted in re-
spondents thinking that they would change their practice, but watching
students working did not lead to the same result of respondents thinking
that they would change their practice.
Our thoughts on this matter stem, at least in part, from Vygotsky’s
(1978) theoretical framing of the Zone of Proximal Development: if
teachers can connect something in the video to their own practice, they
may be in the position to take that as the grounding for thinking about
how to link their existing practice to a new practice. In part, we come to
this interpretation because the most-cited reason for anticipating chang-
ing practice was “I try to use practices like this already, and this gave me
some concrete ideas how to do it.” We also note that this reason is quite
similar to what other teachers claim is the reason they go to the VLC in
the first place: Beilstein et al. (2021) found that the VLC members they
interviewed said that they learned from the VLC by getting to see how
a particular lesson was enacted or to see a model lesson. In this way, if
teachers go to the VLC to see how a lesson unfolds and they see that the
teacher’s practices in the video are like their own, they can envision the
10
Teaching and Teacher Education: Leadership and Professional Development 1 (2022) 100003
link between their own and a new practice and also become empowered
to expand their repertoire to try new practices. Without both seeing the
new practice and being able to link it to a current practice, we imagine
that learning about—and expectations to try—new practices would be
hindered.
5.4. Manipulations that had no meaningful impact on reflection
It was notable that the motivation provided for participants to com-
ment had no impact on the depth of their reflective commentary. We
have long known (e.g., Mayer, 1979) that advance organizers support
learning, and we assumed that providing motivations (to impact your
own reflection or to share reflection with others) would act as advance
organizers. However, given that our validity checks indicated that par-
ticipants did not accurately recall the motivation condition in which
they were placed, it is not possible to speculate on the effects of mo-
tivation on reflective commentary. For participants in our experiment,
it seems that the motivation condition was simply not salient, so we
cannot determine whether there was fault in our design, precluding the
prompts for motivations from having an effect or whether teachers sim-
ply do not care about the motivations we offered for them. In future
work, we hope to consider both blunter (e.g., repetition of the motiva-
tion) and more innovative (e.g., requesting active engagement with the
motivations) ways to make the motivations more salient, to see if teach-
ers who understand why they are commenting actually influences the
reflection in the commentary they provide after watching video clips of
mathematics teaching and learning.
5.5. Limitations and future directions
Besides the concerns and future directions already raised, we ac-
knowledge several others that warrant consideration. First, we real-
ize that our measure of reflection captured our participants’ reflection
that they wrote out in a particular moment. This means that they may
have had additional reflections that they did not write down or that
they may have had at another moment in time. Of greater concern
is that the levels of reflection expressed during our task may not be
available when needed (cf., Kersting et al.’s 2010 measure of “usable
knowledge”) and may not transfer to teaching when it would be helpful
(Bransford & Schwartz, 1999). Mitigating this concern, and following
from Dewey (1933), who claimed that “reflection involved not simply
a sequence of ideas [such that] each determines the next as its proper
outcome” (p.4), is the acknowledgement that if educators can be re-
flective on practice, they may be prepared to be reflective in practice
(Schön, 1987). In this way, even when educators are reflective outside
of the moment when reflection is most productive (i.e., while teaching),
it should be that engaging in reflection, at any point, positions a teacher
to be prepared, compared to not reflecting.
Second, we recruited our sample from one online professional devel-
opment website. Although we selected this website not only for conve-
nience but also for principled reasons, we understand that the results
reported here may not generalize to teachers who are members of this
website but did not volunteer to participate or to teachers using other
mathematics curricula or to teachers who prefer other online access to
video cases. Understanding how to attract elementary teachers of math-
ematics to engage productively and easily in reflective practice is a goal
worth pursuing, beyond those educators who are enrolled members of
the VLC.
Third, a corollary of our second concern, is that our sample was not
as diverse as we would have liked. We are aware that the educators who
participated in this study were geographically diverse and had compara-
ble years of experience as the general population of elementary teachers
in the United States, but their racial makeup was less diverse than the
general population’s. Finding ways to reach all teachers is important,
as is understanding how to support teachers who bring diverse perspec-
tives.
M. Perry, M.S. Bates, J.R. Cimpian et al.
Finally, we note that this was an experiment, wherein we purpose-
fully manipulated aspects of the video-viewing context to examine the
effects of these manipulations on educators’ reflective commentary and
their willingness to consider changing their practice. This has at least
two implications. First, we recognize that lack of attention to prompts, as
well as lack of attention to aspects of the video that were not prompted,
means that teachers may miss aspects of the video clips that are cru-
cial for them to get to the next step in their professional learning and
to improve their practice. For example, seasoned teachers may be more
responsive to prompts guiding them to attend to student thinking than
novice teachers, but if all teachers get the same prompt to consider the
teaching in the video, the seasoned teacher will have missed the oppor-
tunity to consider student thinking. Second, we do not know how these
manipulations translate to their learning from the actual VLC website.
More particularly, we ask: If we implement this on the VLC website (i.e.,
outside the context of an experiment), can we get teachers to provide
this sort of reflective commentary where others can see it, thereby mov-
ing to fulfill the promise of “community” (the C in VLC)? We took the
first step to see if we could guide teacher reflection in ways found to be
productive, but we can still probe more deeply into reflection’s link to
teaching practice and its relation to student learning outcomes.
5.6. Concluding remarks
This study endeavored to explore teachers’ reactions to lesson video
clips in the context of an experiment that could manipulate many fea-
tures of the videos and their associated prompts at once. The results
indicated that prompts targeted at specific focal areas (teachers, stu-
dents, mathematics) produced not just more comments, but more deep
reflection on the respective focal areas than on areas not targeted in
the prompt. We see this as significant, given that in-person PD typically
takes significantly more time to engage teachers to become more deeply
reflective (e.g., Roth et al., 2017).
This study also produced a challenging result: although targeted
prompts could increase reflection on a primary focal area, they could
also decrease reflection on other focal areas. Teaching-focused prompts
resulted in decreased reflection on student thinking and mathemat-
ics; student thinking-focused and video-specific prompts resulted in de-
creased reflection on teaching. This is perplexing because a common
goal of many professional development efforts is to help teachers im-
prove their understanding of student thinking in a subject matter and to
root their subsequent teaching moves in that understanding. If focusing on
student thinking or teaching makes educators think less about the other,
this is a divide that clearly needs bridging. This issue is compounded by
the fact that study participants reported more likelihood to change their
practice based on teacher-focused, rather than student-focused, videos;
this divide also needs to be overcome.
In general, our findings have implications for professional developers
and those who provide teacher-learning opportunities and who use VBL
in either in-person or online spaces. This study makes clear the impor-
tance of prompts (and, by extension, facilitation) for teacher reflection.
It also illustrates that video content—including camera focus, mathe-
matical topic, and grade level—can have profound effects on teachers’
reflective outcomes. However, it is also clear that professional develop-
ers must match their video content and prompts to their professional
learning objectives. Seemingly subtle features of videos can promote
different reflection outcomes. Although we clearly face the challenge,
going forward, to find effective ways to promote a number of critical
outcomes with lesson video, we are encouraged that we have shed light
on impacting teacher reflection, which has the potential to be imple-
mented at scale.
Appendix
Text of the Background Survey Completed by Participants
11
Teaching and Teacher Education: Leadership and Professional Development 1 (2022) 100003
What was your undergraduate (college) major(s)? Select all that ap-
ply.
• Elementary education
• Secondary education
• Education
• Mathematics
• Other [text box]
How many math classes—including statistics and engineering classes
that were math intensive—did you take during your undergraduate col-
lege years? [0 to >10]
If you have any post-college (graduate) training, please give your
major and the degree/certificate earned: [text box]
If you have any post-college (graduate) training, how many math
classes did you take at the graduate level? [0 to >10]
How would you describe your teacher certification program?
• Undergraduate college (e.g., B.A. in Elementary Education)
• Graduate college (e.g., M.A. in Teaching)
• Alternative Certification (e.g., Teach for America)
• Other [text box]
How many years have you been a classroom teacher? [0 to >25]
If you have had other roles besides classroom teacher, please specify:
[text box]
How would you best describe your school context?
• Large City (i.e., you teach in the inner city of New York, Chicago, St.
Louis, etc.)
• Suburban (i.e., you teach in a separate city just outside a large city)
• Mid-Size City (i.e., you teach in a city with 50,000+ population that
is not a large city and is not just outside a large city)
• Rural (i.e., you teach in a city smaller than 50,000 people and are
not outside a large city)
• Other
On a scale of 1 (low) to 5 (high), how comfortable do you feel with
technology?
Gender [Male, Female]
What is your race/ethnicity? Select all that apply.
• American Indian or Alaska Native
• Asian
• Native Hawaiian or Other Pacific Islander
• Hispanic or Latino
• Black or African American
• White
• Other race/ethnicity _______
Please select one answer to fill in the blank. Based on the directions,
I responded to each video so I could:
• comment on what I paid attention to
• reflect on my own practice, which can improve my teaching
• let other teachers learn from me, which will help build community
• write something positive about the video
Declaration of Competing Interest
The authors declare that they have no known competing financial
interests or personal relationships that could have appeared to influence
the work reported in this paper.
Funding
This material is based upon work supported by the National Science
Foundation under Grant No. 1621253. Any opinions, findings, and con-
clusions or recommendations expressed in this material are those of the
author(s) and do not necessarily reflect the views of the National Science
Foundation.
M. Perry, M.S. Bates, J.R. Cimpian et al.
References
Antonietti, A., Borsetto, A., & Iannello, P. (2016). A metacognitive approach to financial
literacy. In C. Aprea, E. Wuttke, K. Breuer, N.K. Koh, P. Davies, B. Fuhrmann, & J.S.
Lopus (Eds.), International handbook of financial literacy (pp. 57-68). Springer: Singa-
pore. 10.1007/978-981-10-0360-8_5.
Ball, D. L. (1996). Teacher learning and the mathematics reforms: What we
think we know and what we need to learn. Phi Delta Kappan, 77, 500–508
https://www.jstor.org/stable/20405624.
Ball, D. L., & Cohen, D. K. (1999). Developing practice, developing practitioners. In L. Dar-
ling-Hammond, & G. Sykes (Eds.), Teaching as the learning profession: Handbook of pol-
icy and practice (pp. 3–32). San Francisco: Jossey-Bass.
Barab, S., MaKinster, J. G., Moore, J., Cunningham, D., & the ILF Design Team (2001).
Designing and building an online community: The struggle to support sociability in
the Inquiry Learning Forum. Educational Technology Research and Development, 49(4),
71–96. https://www.jstor.org/stable/30221137.
Barberis, N. C. (2013). Thirty years of prospect theory in economics: A review and assess-
ment. Journal of Economic Perspectives, 27(1), 173–196. 10.1257/jep.27.1.173.
Bates, M., Perry, M., Cimpian, J., Moran, C., Jay, T., Beilstein, S., & Henricks, G. (2019).
An exploratory study of the relationship between teachers’ implicit theories and re-
flection. Toronto: American Educational Research Association.
Bates, M. S., Phalen, L., & Moran, C. (2016). If you build it, will they reflect? Exam-
ining teachers’ use of an online video-based learning website. Teaching and Teacher
Education, 58, 17–27. 10.1016/j.tate.2016.04.004.
Beilstein, S. O., Henricks, G. M., Jay, V., Perry, M., Bates, M. S., Moran, C. G.,
& Cimpian, J. R. (2021). Teacher voices from an online elementary mathematics
community: Examining perceptions of professional learning. Journal of Mathematics
Teacher Education, 24(3), 283–308. 10.1007/s10857-020-09459-z.
Beilstein, S. O., Perry, M., & Bates, M. S. (2017). Prompting meaningful analysis from pre-
service teachers using elementary mathematics video vignettes. Teaching and Teacher
Education, 63, 285–295. 10.1016/j.tate.2017.01.005.
Borko, H. (2004). Professional development and teacher learning: Mapping the terrain.
Educational Researcher, 33(8), 3–15 https://www.jstor.org/stable/3699979.
Borko, H., Jacobs, J., Eiteljorg, E., & Pittman, M. E. (2008). Video as a tool for foster-
ing productive discussions in mathematics professional development. Teaching and
Teacher Education, 24, 417–436. 10.1016/j.tate.2006.11.012.
Bragg, L. A., Walsh, C., & Heyeres, M. (2021). Successful design and delivery of online pro-
fessional development for teachers: A systematic review of the literature. Computers
& Education, 166, Article 104158. 10.1016/j.compedu.2021.104158.
Bransford, J. D., & Schwartz, D. L. (1999). Rethinking transfer: A simple pro-
posal with multiple implications. Review of Research in Education, 24(1), 61–100.
10.3102/0091732X024001061.
Brophy, J. (2003). Introduction. In J. Brophy (Ed.), Advances in research on teaching: Vol.
10. Using video in teacher education (pp. ix-xxiv). Bingley, England: Emerald Group.
10.1016/S1479-3687(03)10014-4.
Bryk, A. S. (2015). 2014 AERA distinguished lecture: Accelerating how we learn to im-
prove. Educational Researcher, 44, 467–477. 10.3102/0013189X15621543.
Chval, K. B., Lannin, J. K., Arbaugh, F., & Bowzer, A. D. (2009). Videos and prospective
teachers. Teaching Children Mathematics, 16(2), 98–105. 10.5951/TCM.16.2.0098.
Cohen, J. (1960). A coefficient of agreement for nominal scales. Educational and Psycho-
logical Measurement, 20, 37–46. 10.1177/001316446002000104.
Darling-Hammond, L., & McLaughlin, M. W. (1995). Policies that support pro-
fessional development in an era of reform. Phi Delta Kappan, 76, 597–604.
10.1177/003172171109200622.
Darling-Hammond, L., Wei, R. C., Andree, A., Richardson, N., & Orphanos, S. (2009). State
of the profession. Journal of Staff Development, 30(2), 42-44, 46-50.
Dede, C., Ketelhut, D. J., Whitehouse, P., Breit, L., & McCloskey, E. M. (2009). A research
agenda for online teacher professional development. Journal for Teacher Education,
60, 8–19. 10.1177/0022487108327554.
Desimone, L. M., Porter, A. C., Garet, M. S., Yoon, K. S., & Birman, B. F. (2002). Ef-
fects of professional development on teachers’ instruction: Results from a three-
year longitudinal study. Educational Evaluation and Policy Analysis, 24, 81–112.
10.3102/01623737024002081.
Desimone, L. M., Smith, T. M., & Ueno, K. (2006). Are teachers who need sustained, con-
tent focused professional development getting it? An administrator’s dilemma. Edu-
cational Administration Quarterly, 42(2), 179–215. 10.1177/0013161X04273848.
Dewey, J. (1933). How we think: A restatement of the relation of reflective thinking to the
educative process. Chicago: Heath.
Frydman, C., & Camerer, C. F. (2016). The psychology and neuroscience of financial deci-
sion making. Trends in Cognitive Sciences, 20(9), 661–675. 10.1016/j.tics.2016.07.003.
Garet, M., Porter, A., Desimone, L., Birman, B., & Yoon, K. S. (2001). What makes profes-
sional development effective? Results from a national sample of teachers. American
Educational Research Journal, 38, 915–945. 10.3102/00028312038004915.
González, G., Deal, J. T., & Skultety, L. (2016). Facilitating teacher learning when using
different representations of teaching. Journal of Teacher Education, 67(5), 447–466.
10.1177/0022487116669573.
Guskey, T. R. (2002). Analyzing lists of the characteristics of effective professional
development to promote visionary leadership. NASSP Bulletin, 87(637), 4–20.
10.1177/019263650308763702.
Hiebert, J., & Grouws, D. A. (2007). The effects of classroom mathematics teaching on
students’ learning. In F. K. Lester (Ed.), Second handbook of research on mathematics
teaching and learning (pp. 371–404). Greenwich, CT: Information Age.
Hill, H., Rowan, B., & Ball, D. L. (2005). Effects of teachers’ mathematical knowledge for
teaching on student achievement. American Educational Research Journal, 42, 371–
406. 10.3102/00028312042002371.
12
Teaching and Teacher Education: Leadership and Professional Development 1 (2022) 100003
Kennedy, M. (1998). Form and substance of inservice teacher education. (Research Mono-
graph No. 13.) Madison. Madison, WI: National Institute for Science Education, Uni-
versity of Wisconsin-Madison. https://files.eric.ed.gov/fulltext/ED472719.pdf
Kersting, N. B., Givvin, K., Sotelo, F., & Stigler, J. W. (2010). Teachers’ analysis of
classroom video predicts student learning of mathematics: Further explorations of
a novel measure of teacher knowledge. Journal of Teacher Education, 61, 172–181.
10.1177/0022487109347875.
Kersting, N. B., Givvin, K. B., Thompson, B. J., Santagata, R., & Stigler, J. W. (2012).
Measuring usable knowledge: Teachers’ analyses of mathematics classroom videos
predict teaching quality and student learning. American Educational Research Journal,
49, 568–589. 10.3102/0002831212437853.
Lave, J., & Wenger, E. (1991). Situated learning: Legitimate peripheral participation. Cam-
bridge University Press. 10.1017/CBO9780511815355.
Lee, C. B., & Koh, N. K. (2016). Children’s monetary decision making: The role of metacog-
nition in everyday problem solving. In C. Aprea, E. Wuttke, K. Breuer, N. K. Koh,
P. Davies, B. Fuhrmann, & J. S. Lopus (Eds.), International handbook of financial liter-
acy (pp. 415–424). Singapore: Springer. 10.1007/978-981-10-0360-8_28.
LeFevre, D. M. (2003). Designing for teacher learning: Video-based curriculum de-
sign. In J. Brophy (Ed.), Advances in research on teaching: Vol. 10. Using
video in teacher education (pp. 235–258). Bingley, England: Emerald Group.
10.1016/S1479-3687(03)10009-0.
Mayer, R. E. (1979). Can advance organizers influence meaningful learning? Review of
Educational Research, 49(2), 371–383 https://www.jstor.org/stable/1169964.
McDuffie, A. R., Foote, M. Q., Drake, C., Turner, E., Aguirre, J., Bartell, T. G., et al. (2014).
Use of video analysis to support prospective K-8 teachers’ noticing of equitable prac-
tices. Mathematics Teacher Educator, 2(2), 108–140. 10.1007/s10857-013-9257-0.
Miller, K. F., & Zhou, X. (2007). Learning from classroom video: What makes it com-
pelling and what makes it hard. In R. Goldman, R. Pea, B. Barron, & S. Derry
(Eds.), Video research in the learning sciences (pp. 321–334). Mahwah, NJ: Erlbaum.
10.4324/9780203877258.
Moon, J., Passmore, C., Reiser, B. J., & Michaels, S. (2014). Beyond comparisons of
online versus face-to-face PD: Commentary in response to Fishman et al., “Com-
paring the impact of online and face-to-face professional development in the con-
text of curriculum implementation. Journal of Teacher Education, 65(2), 172–176.
10.1177/0022487113511497.
National Center for Education Statistics (2017). Number and percentage distribution of
teachers in public elementary and secondary schools, by instructional level and se-
lected teacher and school characteristics: 1999-2000 and 2015-16. Retrieved from
https://nces.ed.gov/programs/digest/d17/tables/dt17_209.22.asp
Opfer, V., & Pedder, D. (2011). Conceptualizing teacher professional learning. Review of
Educational Research, 81(3), 376–407. 10.3102/0034654311413609.
Raudenbush, S. W., & Bryk, A. S. (2002). Hierarchical linear models: Applications and data
analysis methods (2nd Edition). Thousand Oaks, CA: Sage.
Rich, P. J., & Hannafin, M. (2009). Video annotation tools: Technologies to scaffold, struc-
ture, and transform teacher reflection. Journal of Teacher Education, 60(1), 52–67.
10.1177/0022487108328486.
Rivkin, S. G., Hanushek, E. A., & Kain, J. F. (2005). Teachers, schools, and aca-
demic achievement. Econometrica : Journal of the Econometric Society, 73(2), 417–458.
10.1111/j.1468-0262.2005.00584.x.
Roth, K. J., Bintz, J., Wickler, N. I. Z., Hvidsten, C., Taylor, J., Beardsley, P. M., et al.
(2017). Design principles for effective video-based professional development. Interna-
tional Journal of STEM Education, 4(1), 31. 10.1186/s40594-017-0091-2.
Santagata, R. (2009). Designing video-based professional development for mathemat-
ics teachers in low-performing schools. Journal of Teacher Education, 60, 38–51.
10.1177/0022487108328485.
Santagata, R., & Yeh, C. (2015). The role of perception, interpretation, and decision mak-
ing in the development of beginning teachers’ competence. ZDM Mathematics Educa-
tion, 48, 153–165. 10.1007/s11858-015-0737-9.
Schön, D. A. (1983). The reflective practitioner: How professionals think in action. New York:
Basic Books, Inc.
Schön, D. A. (1987). Educating the reflective practitioner. San Francisco: Jossey-Bass.
10.4324/9781315237473.
Scott, D. E., & Scott, S. (2010). Innovations in the use of technology in teacher professional
development. In J.O. Lindberg & M. D. Olofsson (Eds.), Online learning communities and
teacher professional development: Methods for improved education delivery (pp. 169-189).
Hershey, PA: IGI Global. 10.4018/978-1-60566-780-5.ch010.
Seago, N. (2003). Using video as an object of inquiry for mathematics teaching
and learning. In J. Brophy (Ed.), Advances in research on teaching: Vol. 10. Us-
ing video in teacher education (pp. 259–286). Bingley, England: Emerald Group.
10.1016/S1479-3687(03)10010-7.
Seago, N., Koellner, K., & Jacobs, J. (2018). Video in the middle: Pur-
poseful design of video-based mathematics professional development.
Contemporary Issues in Technology and Teacher Education, 18(1), 29–49
https://www.learntechlib.org/primary/p/174330/.
Sherin, M. G. (2003). New perspectives on the role of video in teacher education. In J. Bro-
phy (Ed.), Advances in research on teaching: Vol. 10. Using video in teacher education
(pp. 1–27). Bingley, England: Emerald Group. 10.1016/S1479-3687(03)10001-6.
Sherin, M. G., Linsenmeier, K. A., & van Es, E. A. (2009). Selecting video clips to promote
mathematics teachers’ discussion of student thinking. Journal of Teacher Education,
60(3), 213–230. 10.1177/0022487109336967.
Sherin, M. G., & van Es, E. A. (2009). Effects of video club participation on teachers’ profes-
sional vision. Journal of Teacher Education, 60, 20–37. 10.1177/0022487108328155.
Shulman, L. (2005). The signature pedagogies of the professions of law, medicine, engineer-
ing, and the clergy: Potential lessons for the education of teachers. Speech delivered at
the Math Science Partnerships (MSP) Workshop: “Teacher Education for Effective
Teaching and Learning,” hosted by the National Research Council’s Center for Ed-
M. Perry, M.S. Bates, J.R. Cimpian et al.
ucation, Irvine, CA. http://www.taylorprograms.com/wpcontent/uploads/2018/11/
Shulman_Signature_Pedagogies.pdf.
Shulman, L. S. (1986). Those who understand: Knowledge growth in teaching. Educational
Researcher, 15(2), 4–14. 10.3102/0013189X015002004.
Stigler, J.W., & Perry, M. (2000). Developing classroom process data for the improvement
of teaching. In N.S. Raju, J.W. Pellegrino, M. W. Bertenthal, K. Mitchell, & L. Jones
(Eds.), Grading the Nation’s Report Card: Research from the evaluation of NAEP (229-
264). Washington, DC: National Academy Press.
Stronge, J. H., Ward, T. J., & Grant, L. W. (2011). What makes good teachers good? A cross-
case analysis of the connection between teacher effectiveness and student achieve-
ment. Journal of Teacher Education, 62(4), 339–355. 10.1177/0022487111404241.
Sun, J., & van Es, E. A. (2015). An exploratory study of the influence that analyzing teach-
ing has on preservice teachers’ classroom practice. Journal of Teacher Education, 66,
201–214. 10.1177/0022487115574103.
Sorkin, J.E., & Preston, M.D. (2010). Designing a video library and a web environment
for learning about early childhood mathematics education. In J. Stigler (Chair), Video
analysis as a method for developing preservice teachers’ beliefs about teaching and their
understanding of children, pedagogy, and assessment. Symposium presented at the Amer-
ican Educational Research Association annual meeting, Denver, Colorado.
van Es, E. A. (2010). A framework for facilitating productive discussions in video clubs.
Educational Technology, L(1), 8–12. https://www.jstor.org/stable/pdf/44429757.pdf.
van Es, E. A., & Sherin, M. G. (2008). Mathematics teachers’ “learning to notice”
in the context of a video club. Teaching and Teacher Education, 24, 244–276.
10.1016/j.tate.2006.11.005.
13
Teaching and Teacher Education: Leadership and Professional Development 1 (2022) 100003
van Es, E. A., Tunney, J., Goldsmith, L., & Seago, N. (2014). A framework for the facil-
itation of teachers’ analysis of video. Journal of Teacher Education, 64(4), 340–356.
10.1177/0022487114534266.
Vygotsky, L. S. (1978). Mind in society: The development of higher psychological processes.
Harvard University Press.
Webster-Wright, A. (2009). Reframing professional development through understand-
ing authentic professional learning. Review of Educational Research, 79, 702–739.
10.3102/0034654308330970.
Wenglinsky, H. (2001). Teacher classroom practices and student performance: How schools
can make a difference (Report Number RR-01-19). Princeton, NJ: Educational Testing
Service. 10.1002/j.2333-8504.2001.tb01861.x.
Woodland, R. H. (2016). Evaluating PK–12 professional learning communities: An
improvement science perspective. American Journal of Evaluation, 37, 505–521.
10.1177/1098214016634203.
Yoon, K. S., Duncan, T., Lee, S. W. Y., Scarloss, B., & Shapley, K. (2007). Review-
ing the evidence on how teacher professional development affects student achieve-
ment (Issues & answers report, REL 2007-No. 033). Washington, DC: U.S. Depart-
ment of Education, Institute of Education Sciences National Center for Education
Evaluation and Regional Assistance, Regional Educational Laboratory Southwest.
https://files.eric.ed.gov/fulltext/ED498548.pdf.