Educational Researcher

http://er.aera.net

''Where Is the Action?'' Challenges to Studying the Teaching of Reading in Elementary Classrooms
Robert G. Croninger and Linda Valli
EDUCATIONAL RESEARCHER 2009 38: 100
DOI: 10.3102/0013189X09333206

The online version of this article can be found at:

http://edr.sagepub.com/content/38/2/100

Published on behalf of

American Educational Research Association

and

http://www.sagepublications.com

Additional services and information for Educational Researcher can be found at:

Email Alerts: http://er.aera.net/alerts

Subscriptions: http://er.aera.net/subscriptions

Reprints: http://www.aera.net/reprints

Permissions: http://www.aera.net/permissions

Downloaded from http://er.aera.net by guest on August 23, 2010

“Where Is the Action?” Challenges to Studying the
Teaching of Reading in Elementary Classrooms
Robert G. Croninger and Linda Valli
The authors argue that part of the difficulty in studying the teaching
of reading in elementary classrooms is determining where “the
action” occurs in reading instruction. Based on their 5-year longitu-
dinal study of fourth- and fifth-grade teachers in moderate- and high-
poverty elementary schools, they describe three challenges:
(a) determining key factors in the complex instructional environment
that should be the focus of study, (b) determining who teaches read-
ing to specific students, and (c) determining the boundaries of reading
instruction or when it occurs during the school day and year. The
authors argue that these challenges, although not insurmountable, are
difficult to resolve and that they are becoming more pervasive
because of current policies that promote high-stakes accountability.
Keywords: high-stakes accountability; reading instruction; research
methodology; teacher effectiveness
M
any educators and policy makers who are eager to
improve teaching focus on reading instruction in par-
ticular, because reading generally is seen as fundamen-
tal to learning other skills and subjects. However, there is less
consensus about what constitutes high-quality reading instruc-
tion, or even what is the best way of studying it (Fuhrman,
Cohen, & Mosher, 2007). A range of reasons are given for the
difficulty of developing a reliable, agreed-upon knowledge base
from which to fashion instructional strategies and education
policies in this area. They highlight the multiplicity of scholarly
perspectives that characterize the field of reading research, includ-
ing different definitions of what constitutes reading; examina-
tions of different populations of students, for whom different
reading approaches may be more or less effective; and interests in
different aspects of reading, such as genre, comprehension,
achievement, motivation, and engagement (Allington &
Cunningham, 1996; Allington & Johnston, 2002; Fenstermacher &
Richardson, 2005; Lampert, 2001).
Although we acknowledge the difficulty of arriving at
consensus, we identify three methodological and practical
challenges that complicate attempts to develop a reliable
knowledge base about the nature of reading instruction and its
Educational Researcher, Vol. 38, No. 2, pp. 100–108
DOI: 10.3102/0013189X09333206
© 2009 AERA. http://er.aera.net
100 EDUCATIONAL RESEARCHER
Downloaded from http://er.aera.net by guest on August 23, 2010
relationship with desirable outcomes for students. We argue that
part of the difficulty in studying the teaching of reading is
determining, or delimiting in our research designs, where
“the action” actually occurs in reading instruction—which
instructional interactions actually constitute the teaching
of reading. The three challenges we wish to discuss are
(a) determining the key factors in the complex instructional
environment that should be the focus of study; (b) determining
the sources of instruction—who teaches reading to specific
students within a school; and (c) determining the boundaries of
reading instruction—that is, determining when it occurs during
the school day and year. Although each of these challenges has
been identified to varying degrees by others (see, e.g., Good &
Brophy, 2003; Raudenbush, Martinez, & Spybrook, 2007), we
argue that they are more difficult to resolve than is often
acknowledged and that they are becoming progressively more
difficult in the current, high-stakes accountability environment.
Background
We draw our observations from a 5-year longitudinal study of
teaching that we conducted in moderate- to high-poverty ele-
mentary schools. The primary research goals for the study were
to understand (a) how teachers in fourth- and fifth-grade class-
rooms help students acquire foundational skills in reading and
mathematics1 and (b) how various aspects of a school’s organiza-
tion and culture can facilitate or hinder a teacher’s ability to
engage in positive forms of teaching in the classroom. For the
purposes of this article, we focus primarily on the first goal of the
study. Using a cognitive framework, we define reading as the abil-
ity of students to “understand what is in text” (Pressley, 2000,
p. 545), and we consider some of the challenges that we experi-
enced studying teaching within this framework.
All of the schools that participated in the study were part of a
single school system, the Stevenson Public School District,2 a
large and diverse district that was facing a wide range of
educational challenges associated with significant demographic
changes and heightened expectations for student performance. In
the past two decades, the enrollment of low-income students had
doubled in Stevenson. At the time of our study, the district
enrolled nearly 140,000 students and employed roughly 19,000
employees, making Stevenson one of the largest school districts
in the nation. Approximately 30% of the students in Stevenson
participated in free and reduced-price meals services (FARMS),
and 12% participated in English programs for speakers of other
languages (ESOL). We selected the school district because of its
racial and economic diversity, its reliable database on students
and schools, and its well-publicized commitment to standards-
based reforms.
The study design called for identifying a group of moderate-
to high-poverty elementary schools with greater-than-expected
achievement gains and then following these schools and their
fourth- and fifth-grade teachers for 3 to 4 years.3 When a school
agreed to participate in the study, we attempted to recruit all of
the teachers in those two grades. When identified schools declined
to participate or withdrew from the study, we replaced them with
schools in the district that had similar demographics and, when
possible, reputations for good teaching. During the 4-year period
for which we collected data, approximately 20 elementary
schools, 160 teachers, and 4,300 students participated in the
study. Most of the data used in this article to support our
observations came from the 2003–2004 school year. As a group,
the participating schools had higher-than-average achievement
levels in reading and mathematics on the state’s mandated fourth-
and fifth-grade assessments—higher than those of other schools
with comparable FARMS enrollments in the district, although
the range in achievement levels varied substantially across
participating schools.
We developed a series of instruments and research protocols
to capture the occurrence and consequences of a wide range of
pedagogical practices using a complementary mixed-methods
research design (Tashakkori & Teddlie, 2002). Our instruments
included a time-sampling protocol in which observers recorded
what teachers and students were doing every 3 minutes during a
reading lesson. The 3-minute intervals served two functions.
First, they gave observers enough time to record seven core
dimensions of instruction (teacher activity, student activity,
instructional organization, teacher attention, lesson content,
context of episode, classroom behavior). Second, they provided
enough data points (approximately 20 per lesson) to capture
patterns and sequences of instructional interactions. We used this
instrument, along with pre- and post-lesson interviews, to observe
6–8 reading lessons each year per teacher. In the pre-lesson
interviews, observers asked the teacher about lesson goals,
content, planning, and materials to situate and help focus the
observation. In the post-lesson interviews, teachers were asked if
they had accomplished or departed from their goals and why.4
Our conceptualization of high-quality reading instruction
shaped our instruments and our study design, as well as the
challenges that we experienced in studying teaching.5 Most
pertinent to our study was the research in the areas of reading and
cognitive psychology, which depicts high-quality teaching as a
complex, multidimensional phenomenon best studied through a
variety of overlapping, complementary strategies. This research
encompasses a set of general theories about how children learn
and, correspondingly, how successful teachers teach. What
distinguishes these theories about teaching is that they are based
on a body of cognitive research about learning (Bransford, Brown,
& Cocking, 1999) that is linked to rich descriptions of classroom
practice (Ball & Cohen, 1999; Lampert, 2001). We used a set of
relevant principles, the American Psychological Association’s
learner-centered psychological principles (Alexander & Murphy,
Downloaded from http://er.aera.net by guest on August 23, 2010
1998), to develop our instruments and research design. These
principles indicate that good teaching promotes deep, principled
learning of content; encourages the development of cognitive and
metacognitive skills; motivates students to engage deeply in
subject matter; addresses individual and developmental differences
among students; and creates inclusive, affirming, and successful
learning environments.
We recognized that differences in subject areas make different
demands on teachers’ content and pedagogical expertise
(Alexander & Fives, 2000; Shulman, 1986). Although the
learner-centered psychological principles complement standards
derived from reading research (see, e.g., International Reading
Association & National Council of Teachers of English, 1996),
the manifestations of “exemplary” teaching are shaped in part by
lesson topics, students, and curricular materials. They may also
differ according to the goals and values that drive the instruction
(Chambliss & Graeber, 2003). In general, however, much of the
research on exemplary reading instruction at the fourth- and
fifth-grade levels suggests the importance of large amounts of
reading and writing, connections across the curriculum, and
skills instruction embedded in meaningful interaction with text.
According to this body of research, high-quality instruction
provides students with choices about interesting and challenging
texts associated with a variety of genres and encourages students
to respond personally to those texts. Teachers also promote
dialogue about reading and writing through teacher questioning,
grouping practices, cooperative learning, and informative
feedback (Morrow, Wamsley, Duhammel, & Fittipaldi, 2002;
Pressley, Yokoi, Rankin, Wharton-McDonald, & Mistretta,
1997; Taylor, Pearson, Clark, & Walpole, 2002).
One indicator of high-quality teaching evident in both the
learner-centered psychological principles and the reading
standards that we used is high cognitive demand. As noted in the
standards, teachers should help students critically analyze texts
and engage in purposeful and challenging language experiences
(International Reading Association & National Council of
Teachers of English, 1996). We defined high cognitive demand as
teachers’ press for student reasoning, reflection on learning, and
higher order thinking, as well as the extent to which students are
exposed to new and challenging content. In particular, we looked
at (a) teacher requests for complex forms of thinking, (b) student
responses that signaled cognitive complexity, and (c) the
prevalence of content that required greater cognitive engagement
by students. To be considered high cognitive demand, the reading
content had to focus on genre characteristics, themes, or main
ideas and supporting details. We use this construct of high
cognitive demand as an example of one of the challenges associated
with studying the quality of reading instruction in classrooms
and schools discussed below.
Although we studied reading instruction only in the fourth
and fifth grades and only in the schools that participated in our
study, we have no reason to believe that the challenges we describe
are unique to either these classes or these schools. On the contrary,
as we explain elsewhere (Valli, Croninger, & Buese, 2006; Valli,
Croninger, Chambliss, Graeber, & Buese, 2008; Valli, Croninger,
& Walters, 2007), there is good reason to believe that these
challenges have been magnified by the demands of high-stakes
testing. Nonetheless, we acknowledge that such challenges may
MARCH 2009 101
vary with variations in the press to meet annual achievement
goals, the resources available to teachers and principals to meet
those goals, and beliefs about what constitutes good and effective
teaching of reading.
Determining Key Factors in the Complex
Instructional Environment
One of the greatest challenges in studying the quality of reading
instruction is understanding how “quality” can be dependent on
multiple and simultaneous factors (Cohen, Raudenbush, & Ball,
2003; Good & Brophy, 2003; Fenstermacher & Richardson,
2005). Some researchers have framed this challenge as under-
standing how teacher actions interact with students and content
(Good & Brophy, 2003; Lampert, 2001); others have discussed
how teacher actions interact with available resources (broadly
conceived) to realize desirable student outcomes (e.g.,
Raudenbush, 2008). In either case, teaching is portrayed as a
complex process that requires an investigation of how instruc-
tional effects vary across multiple factors, such as prior knowl-
edge, the difficulty of text, a teacher’s instructional repertoire,
and student expectations. Capturing this complexity was a fun-
damental goal and challenge for our study.
Quality of Teacher–Student Oral Exchanges
By way of illustration, we sought to determine what teachers do
to encourage students to engage with reading content in greater
cognitive depth. We examined the results of the time-sampling
protocol for 412 reading lessons that we observed in 62 reading
classrooms (roughly 7 lessons per teacher). We then coded obser-
vations that characterized the quality of students’ oral exchanges
with their teachers about reading content. We coded an oral
exchange as reflecting greater cognitive demand when students
posed a hypothesis about the text, provided an alternative answer
to a question, justified or explained an answer, or elaborated on
an answer already given. When students provided a simple answer
or made simple statements about text, we coded these responses
as reflecting less cognitive demand and complexity. In a similar
fashion, we coded teachers’ oral exchanges with students during
lessons. When teachers asked students to reflect on text or an
answer, assess a statement, or elaborate on an answer, or responded
to a question with an alternative question, we coded these
exchanges as having greater cognitive demand. Simple statements
or a failure to respond we coded as exchanges having less cogni-
tive demand and complexity.
We then used these new codes to examine the relationship
between the cognitive complexity of teacher requests and the
cognitive complexity of student responses as a characteristic of
reading instruction. Based on our conceptual framework and
related research (Good & Brophy, 2003; Nystrand & Gamoran,
1991), we assumed that students were more likely to engage in
cognitively complex discussions of what they were reading if
teachers encouraged them to do so. We also assumed, however,
that the effectiveness of these requests might vary with the
characteristics of lessons, curriculum materials, students, and
other teacher practices. To this end we considered whether aspects
of classroom and school composition (e.g., percentage ESOL
students, percentage FARMS students) and average teacher
practices (e.g., organization of classes, difficulty of content
102 EDUCATIONAL RESEARCHER
Downloaded from http://er.aera.net by guest on August 23, 2010
presented to students, and orientation of classroom management)
moderated the relationship between teacher requests and student
responses. To examine these relationships, we constructed a three-
level hierarchical model (lessons nested within classrooms, nested
within schools).6
Methodological and Practical Challenges
Our analysis of these data helps to characterize the methodologi-
cal and practical challenges associated with capturing the com-
plexity of reading instruction. First, when we examined variation
in the quality of teacher–student oral exchanges across observa-
tions, we discovered that most of the variance in these exchanges
occurred among lessons enacted by the same teachers. Nine per-
cent (9%) of the variation in the quality of oral exchanges was
associated with schools, and 7% was associated with classes; by
contrast, 84% was associated with the lessons conducted by the
same teachers. Although schools and teachers varied in the extent
to which we were able to observe more cognitively demanding
exchanges surrounding texts, the preponderance of variation was
associated with lessons taught by the same teacher in the same
classroom. One aspect of complexity, therefore, has to do with
the lesson content itself and the extent to which content lends
itself to the study of particular teacher practices—in this case,
fostering cognitively rich dialogue between teachers and students
about text.
Methodologically, this means that studies of reading must
observe multiple lessons by the same teacher to determine the
extent to which practices occur and their effects on student
outcomes. In our study we attempted to observe 6–8 reading
lessons per teacher, a relatively heavy burden for data collection,
given the number of teachers in our study. However, our own
experience suggests that this range for number of observations
may represent the lower limit of the number required to portray
accurately the practices of teachers in reading classrooms. Even
relatively straightforward and prevalent practices, such as the use
of whole-group instruction, varied more among the lessons
taught by the same teachers than among teachers or schools.
Rarer and more complex interactions between students and
teachers require observations across even more lessons to portray
accurately the nature of reading instruction in classrooms (see
the Rowan & Correnti article in this issue of Educational
Researcher).
As we had anticipated, we also found a very strong relationship
between the quality of teacher requests and the quality of student
responses about reading content. The more teachers encouraged
students to think deeply about text, the more students appeared
to do so, at least as measured by the nature of their oral exchanges
with their teachers. The effect of teacher requests on student
responses was nearly half (0.48) a standard deviation. Moreover,
the effect of teacher requests varied among teachers and schools.
The range in effects associated with teachers was nearly a full
standard deviation, whereas the range in effects associated with
schools was approximately half a standard deviation. In other
words, teachers, even when they made the same types of requests
for students to engage text, had different levels of success in
soliciting more cognitively complex responses from students.
Other factors in and outside the classroom moderated the
effects.
 We examined a range of possibilities that might help us better
understand how different factors interacted to produce different
outcomes—in this case, cognitively rich dialogue about text in
classrooms. The strongest influence that we found was teachers’
classroom management style. The more teachers focused on
organizing tasks and materials during a lesson, the more effective
they were at soliciting cognitively complex discussions about text;
the more teachers focused on student behaviors, the less successful
teachers were in soliciting these oral exchanges with students
(controlling for students’ FARM status, English language
proficiency, and a measure of prior knowledge). The differences
in the effects of teachers’ requests for greater cognitive engagement
ranged from 0.61 standard deviation for teachers who focused on
managing materials and activities to 0.35 for teachers who
focused on managing student behavior.
The above illustration demonstrates one of the challenges
associated with the examination of the teaching of reading in
elementary classrooms. Practices vary substantially across lessons,
often more so than they do across teachers or classrooms. As a
result, studies of reading need to examine what teachers do across
multiple lessons (we aimed at 6–8 lessons per teacher in the
examples that we have provided), especially if the practices of
interest are relatively rare or complex. The number of observations
required to accurately portray instruction poses a considerable
practical burden on studies of reading, often forcing reading
researchers to choose between enhancing the internal validity of
a study (by increasing the number of lessons observed) and
enhancing the external validity of a study (by increasing the
number of teachers and classrooms included in a study).
Moreover, although it is well known that teachers who engage
students more deeply in text are more successful in facilitating
both comprehension and the development of literacy skills, less
is known about how teachers do so effectively (Allington &
Cunningham, 1996; Allington & Johnston, 2002). The examples
that we provided suggest that teachers are most effective when
efforts to elicit cognitively demanding dialogue from students are
combined with a classroom management style that focuses on
managing instructional activities more than on student behaviors.
But certainly there are other factors that may influence these
relationships. For example, the moderating effects of classroom
management style may be an indication of how successful a
teacher is in forming normative expectations for student discourse
and engagement in a classroom. Alternatively, it may be an
indication of how fortunate a teacher is to have a classroom of
students who share his or her expectations, possibly as the result
of reading instruction experienced in prior grades or other
settings. In either case, the complexity of reading instruction
poses a serious challenge to researchers. Even when we as
researchers seek to develop designs that capture complexity, our
theories and methods often prove inadequate to the task (Ball &
Lampert, 1999; Cohen et al., 2003; Raudenbush, 2008).
Determining the Sources of Reading Instruction
Education researchers and policy makers generally are aware of
many of the challenges associated with isolating teacher effects on
student learning (see, e.g., McCaffrey, Lockwood, Koretz, &
Hamilton, 2003), including the recognition that students experi-
ence reading instruction across multiple grades and adults.7 In
Downloaded from http://er.aera.net by guest on August 23, 2010
conducting our study, we anticipated that students would inter-
act with multiple adults in the process of reading instruction even
within a single grade, but we did not anticipate the extent and
variety of these student–adult interactions. A major challenge for
the study was delineating and characterizing these more complex
instructional interactions in and outside the classroom. Identifying
who is teaching reading required the consideration of multiple
sources of instruction that occurred during the school day and
across the school year.
Unintentional Shared Designs
As we watched students and adults move in and out of class-
rooms, we began to categorize the different ways in which mul-
tiple adults shared responsibility for teaching reading (Valli et al.,
2007). We first divided these patterns broadly into two categories:
unintentional and intentional shared instructional designs.
Unintentional designs included typical accommodations made in
response to disruptions in the daily routines of students and
teachers. For example, teacher absences require some sharing of
teaching responsibility. Because teachers are absent for a wide
range of reasons, from staff development sessions to family cir-
cumstances, they may require sharing instruction for one class
period in the school day or for a significant part of a school year.
Studies have demonstrated a strong link between teacher absen-
teeism and student achievement, particularly at the school level
(Miller, Murnane, & Willett, 2008); less is known, however,
about how routine accommodations for absences affect teaching
and learning. The data from our study indicate that teachers, on
average, had assistance covering reading instruction 7% of the
time, or for roughly 2.5 weeks of instruction during the school
year. Although these instances of shared instruction are modest,
there is no reason to believe that they have no effect on instruc-
tion or student learning.
Even when teachers carry out their instructional responsibilities
without major absences, student mobility can interfere with the
one-to-one relationship between students and teachers,
necessitating some unintentional sharing of teaching responsibility
(Lash & Kirkpatrick, 1990). The average student mobility rate
was 20% per year in the schools that we studied. This meant that
a significant number of students had a teacher from another
school responsible for part of their instruction during any
particular year of the study (and, of course, some classrooms had
higher rates of student mobility than others). Although it is
tempting to drop students who enter classrooms during the
school year from achievement gain models, doing so undoubtedly
distorts any estimations of instructional effects because changes
in classroom membership can also affect the achievement of
nonmobile students (see Monk & Ibrahim, 1984, for an
interesting study of how changes in classroom membership can
affect achievement, although from the perspective of student
absenteeism).
A third form of unintentional shared design involved the
public notification mandate introduced by the No Child Left
Behind Act of 2001. According to this mandate, parents must be
given the results of testing prior to the end of the school year so
that families can make “informed” decisions about school
transfers and tutoring. In the district that we studied, state tests
were given in March to satisfy the end-of-year notification
MARCH 2009 103
requirement. Based on this single factor, every teacher in the
study shared responsibility for achievement gains (or losses) with
at least one teacher from the previous year. Given the district’s
9.5-month school calendar, this amounted to roughly one quarter
of a student’s “tested” instructional time. Although there is
growing interest in examining instructional effects across multiple
years and classrooms (see Raudenbush, 2008), very few studies
have actually done so.
Intentional Shared Designs
We also observed teachers using a range of more complex instruc-
tional designs during their reading lessons, quite different from
those of the traditional “egg-crate” classroom, where one teacher
works with her class of students apart from other adults (Lortie,
1975). Instead of having one teacher responsible for their yearly
progress in reading, many students had multiple adults who
shared responsibility for their instruction, especially if the stu-
dents were considered to belong to “at-risk” populations, such as
special education students or English language learners. These
more complex instructional designs were neither rare nor unin-
tentional in the schools that participated in our study; rather,
they represented deliberate and systematic attempts by teachers
and principals to reallocate instructional resources to address per-
ceived instructional challenges, particularly those posed by high-
stakes testing.
As we examined these multiple influences on student learning,
we began to see three broad patterns of intentional shared designs.
We labeled these intentional designs supplemental, simultaneous,
and sequential. Although all three patterns are, to varying degrees,
implicitly or explicitly recognized in the literature, there has been
little discussion about their prevalence or their consequences for
studies of teaching. In the 2003–2004 school year, every school
in our study (n  18) practiced at least one of the forms of shared
instruction, and all but two reading teachers (n  68) reported
using some form of supplemental, simultaneous, or sequential
instruction in their classrooms.
Supplemental shared instruction. The most prevalent form of
shared responsibility was supplemental reading instruction.
Nearly three quarters of the reading teachers reported that one or
more of their students received supplemental reading instruction
outside their classrooms. Although the number of affected stu-
dents varied substantially among classrooms, roughly one quarter
of the teachers reported that at least one third of their students
received additional reading instruction from a different teacher
in their school.
In this form of shared teaching, the originally assigned teacher
continues to influence student learning. However, that influence
is supplemented with additional instruction outside the regular
classroom, for individuals or small groups, through homework
clubs, tutorials, remedial instruction, and a variety of intervention
programs. Although classroom and supplementary teachers may
collaborate on instructional goals, the structural pattern itself
distributes responsibility for teaching reading between two or
more teachers. The more pervasive the use of supplemental
instruction, the more difficult it is to disentangle the effects
of instruction associated with different teachers. Study designs
that focus exclusively on interactions that occur in the regular
104 EDUCATIONAL RESEARCHER
Downloaded from http://er.aera.net by guest on August 23, 2010
classroom may miss important instructional influences occurring
elsewhere in school.
Simultaneous shared instruction. The second most prevalent form
of shared teaching that we observed was simultaneous shared
instruction. In this kind of design, teaching responsibility is
directly shared within the boundaries of a lesson in a particular
time and place. More than one adult in the classroom influences
student learning. For example, in team-taught classes, two teach-
ers, sometimes with equal status and expertise, are assigned the
same group of students during the same time period. But respon-
sibility can also be shared among adults with varying degrees of
expertise, such as a classroom teacher and instructional assistants,
resource teachers, student teachers, or parent volunteers. These
shared arrangements can include full-time, intensive support for
one student; occasional help for a small group of students; or the
occasional modeling of instruction by a reading specialist. Nearly
two thirds of the teachers in the study used an instructional
design that involved at least one additional adult in the classroom
(e.g., a teaching assistant, student teacher, or resource teacher).
What these designs have in common is that the instructional
responsibility for the lesson is shared by two or more adults
within the same classroom and class period. Although assigned
teachers may vary in the extent to which they share or delegate
instructional responsibilities in the classroom, it was not
uncommon in the classrooms that we observed for someone
other than the assigned teacher to assume major responsibility for
reading instruction during a lesson, especially for an individual
student or group of students with similar instructional needs.
Even when a study’s design captures the relevant instructional
interactions in classrooms, simultaneous instruction raises
questions about who represents the primary source of instruction,
the answer to which may vary by students, and how to characterize
the mix of instructional opportunities provided to students in the
same classroom. Based on our experience, answers to these
questions are neither simple nor straightforward.
Sequential shared instruction. In the third pattern of shared
responsibility, sequential shared instruction, a teacher is assigned,
temporarily or permanently, to students who were previously
assigned to a different classroom teacher. Nearly one fifth of the
teachers reported using a sequential design during the 2003–
2004 school year. Although this form of shared teaching was
observed less often in the classrooms in our study, specific schools
used this instructional design more often than other schools did.
For example, one school reassigned nearly half of its fourth- and
fifth-grade students for reading instruction during the year;
another school reassigned more than one quarter of its students.
In short, although the usage of specific instructional designs var-
ied within and among schools, teachers were far more likely to
share instructional responsibilities with at least one other adult
than to have sole responsibility for the students in their class-
rooms. As a result, we found our ability to isolate the sources of
reading instruction in schools far more complicated than we had
anticipated.
Conclusions on intentional shared designs. These patterns of shared
instructional responsibility also affected a large proportion of the
1,541 students who participated in the study that year. Two thirds
of the students experienced one or more of these instructional
designs during the course of the year. In other words, only one
third of the students could be said to be solely linked to an indi-
vidual teacher for reading instruction—the typical assumption in
many study designs. The most prevalent form experienced by
students was simultaneous instruction involving an instructional
assistant (30%), student teacher (17%), staff developer/resource
teacher (15%), and/or in-class help assigned specifically to them
(8%). Nearly one fifth of the students received additional reading
instruction outside classrooms, and 7% were reassigned to a dif-
ferent teacher during the course of the year.
Although these practices could be regarded as trivial variations
in a typical instructional day, we found little evidence to warrant
such a stance. When viewed cumulatively and across the course
of a year, they underscore the potential complexity of linking
teacher actions to student learning in classrooms and schools.
Moreover, we suspect that determining the sources of reading
instruction has become increasingly difficult in the current high-
stakes accountability environment, where teachers and principals
look for ways to meet accelerating expectations for student
achievement. The main reasons given by principals for using
these more complex instructional designs were to provide extra
support to both students and teachers, particularly in classes
where numerous students were below grade level in reading, and
to maximize the distribution of the staff ’s reading expertise in
ways that would improve student test scores (Valli et al., 2007).
As a result, in an era of high-stakes accountability, we should
anticipate more rather than less sharing of responsibility for
teaching in tested subject areas.
Determining the Boundaries of Reading Instruction
The third challenge in studying reading instruction that we dis-
cuss in this article is determining when and where it occurs dur-
ing the school day.8 Although all school subjects can intersect
with each other, even in curricula that are not explicitly interdis-
ciplinary, the boundaries between reading instruction and other
subject areas are particularly porous (Allington & Johnston,
2002; Chambliss & Graeber, 2003). Students read mathematics
problems, directions for science experiments, current events, the
calendar of daily events, and so forth. During these times, the
reading teacher, homeroom teacher, instructional assistant, par-
ent volunteer, or even another student may give students explicit
instruction in reading. Even when the instruction is not explicit,
assistance in reading or practice in reading may also influence
student literacy skills and knowledge (Byrnes, 2000; Taylor et al.,
2002), underscoring the importance of identifying these various
sources of instruction in our research designs and analytical
models.
Because of the diffuseness of reading throughout the school
day, simply locating “reading” on the week’s instructional calendar
is not a sufficient strategy for selecting times to study the teaching
of reading. Researchers need to gather more data in situ before
delimiting their research designs. That is, they should determine
whether reading classes or clubs systematically meet during other
parts of the school day, who teaches them, for what purposes,
with what programs, and for what students. Based on our
experience, these preliminary data collection efforts are critical
because both the amount and the type of reading instruction can
Downloaded from http://er.aera.net by guest on August 23, 2010
vary substantially during an instructional day and in the course
of the school year.
Scope of Reading Instruction
Because of its correlation with achievement, the amount of time
students have for reading instruction has long been of interest to
reading researchers (Allington & Cunningham, 1996; Barr &
Dreeben, 1983; Byrnes, 2000). At the beginning of the school
year, administrators typically establish a simple schedule that
shows which teachers teach which subjects, at which grade levels,
and at what times of the day and week. But we soon discovered
that we could not rely on official documents to determine the
times set for reading lessons. There were official times and unofficial
times, and, in some schools, the unofficial times accounted for
more instructional time than the official times.
Variations across schools and changes over time in reading/
language arts instruction made consistency in identifying and
recording lessons more difficult than we had imagined. Most of
our schools set aside between 90 and 120 minutes per day for a
solid block of reading/language arts instruction. In a few schools,
60 minutes were set aside for reading during one part of the day
and another 60 minutes for writing in a separate part of the day.
Teachers, however, also engaged in practices that extended
instructional time in reading/language arts into other areas of the
curriculum or school day. In many schools, teachers incorporated
reading instruction into social studies or science classes, sometimes
emphasizing reading more than the scheduled subject and skill
set. Homeroom teachers also supplemented reading/language
arts instruction with what teachers called “bell work”: generally, a
review of grammar and sometimes oral reading while administrative
tasks such as lunch count took place.
In schools at greatest risk of failing to meet annual
performance standards, principals circumvented the official
curriculum to add supplementary reading instruction during
some part of the day. These supplementary blocks for reading
typically involved populations of students thought to be most
at risk of failing to make proficiency in reading, with some
students receiving two (“double dipping”) and three (“triple
dipping”) times as much reading instruction as called for by the
official schedule. In one school that was on the state “watch
list,” reading instruction could take up to half the day for all
students, with specific students getting even more instruction
during the time typically allocated for specials (e.g., music,
chorus, physical education).
As testing time drew near, more explicit teaching of reading
occurred beyond the boundaries of the official schedule in most
schools. In addition to scheduled supplementary classes, some
homeroom teachers would conduct “test prep” reading sessions
during bell work time, drilling students on content expected to
appear on the state reading assessment tests. Although the school
district encouraged schools to develop local strategies for
addressing the instructional needs of students, none of the
principals recorded these practices as part of the official schedule
for reading instruction. As the time for the state-mandated
assessments approached, practically all of the students received
some form of additional instruction, and one fifth of the students
received intensive supplemental instruction outside the official
reading/language arts block. In some schools these extended
MARCH 2009 105
forms of instruction occurred daily; in others they occurred two
or three times a week.
The Nature of Reading Instruction
If the nature of reading instruction remained constant across all
time points, determining the scope of reading instruction would
be less important (although not inconsequential) to reading
research. However, the schools in the school district we studied
offered a variety of reading programs or special reading interven-
tions that departed, often quite dramatically, from the district’s
curriculum frameworks and preferred instructional practices.
The conflict between preferred reading instructional practices
and supplemental instructional practices became even more per-
vasive 2 months into the 2003–2004 school year, when a new
associate superintendent encouraged the school district to sup-
plement the reading curriculum with a reading intervention pro-
gram used in her former school district. The schools with the
lowest reading test scores received a mandate to incorporate the
program; other schools were “encouraged” to use it. Most note-
worthy was how the implemented program deviated from the
official curriculum and preferred practices, which tended to
encourage a cognitively demanding, constructivist approach to
reading instruction. The methodological challenge created by the
intervention program was not only that it created an extended
block of time for reading instruction but also that it created
expectations for a set of instructional interactions very different
from those experienced by students and teachers in their regular
classrooms.
To illustrate the importance of studying instruction as it
occurs outside the official time blocks, we draw on data from
three schools in which we conducted in-depth case studies during
the 2004–2005 school year (Valli et al., 2008). In these schools
we observed 11 intervention classes, 8 of which were taught by a
regular classroom teacher. A comparison between 143 regular
reading lessons and 56 intervention lessons indicated that the
intervention classes were less likely to engage students in
cognitively demanding work, a practice encouraged in the district
and encouraged by the reading theories that informed our study
(Allington & Johnston, 2002; Taylor et al., 2002). Teacher–
student interactions that occurred during the regularly scheduled
instructional time were nearly twice as likely to involve cognitively
demanding tasks that required students to construct understanding
as were the teacher–student interactions that occurred during
unofficial, supplemental instruction (17% as opposed to 10% of
lesson time). During the unofficial instructional times, students
were more likely to be engaged in repetitive drill and practice
work designed to prepare them for the state reading assessments
(40% as opposed to 26% of lesson time).
Our growing awareness of reading instruction outside the
classroom made us realize the inadequacy of our original data
collection design, which was to record the 60- to 90-minute
reading lessons presented to students during the officially
scheduled reading block. If we were going to get a comprehensive
picture of reading instruction, we would also need to study what
went on outside the official block designated for reading,
especially as the state reading assessments approached. Although
we were able to capture many aspects of instructional experiences
occurring outside the official instructional schedule, our efforts
106 EDUCATIONAL RESEARCHER
Downloaded from http://er.aera.net by guest on August 23, 2010
highlighted the manner in which many studies overlook
substantial amounts of reading instruction that students
experience (and perhaps not always to their benefit). An important
challenge that we face as reading researchers, then, is to understand
how informal instruction interacts with official instruction to
influence reading outcomes.9
Concluding Remarks
The data from our study, along with our experiences in collecting
the data, indicate that it is more difficult to determine “where the
action” resides in reading instruction than typically is acknowl-
edged by many researchers and the designs that inform their
studies. Many research designs assume a simple one-to-one rela-
tionship among teachers, students, and reading instruction. From
this perspective, the teaching of reading occurs during regularly
scheduled blocks of time. At those times students meet with a
single teacher responsible for their reading instruction, and they
rely on that teacher for instruction during the course of the year.
If teaching is constructed as a routine, isolated activity, such a
research design makes sense—it is straightforward and undoubt-
edly a cost-effective way to study the teaching of reading. But our
own work suggests that such a construction is unwarranted, espe-
cially in a high-stakes accountability environment where notifica-
tion requirements complicate fall–spring comparisons of test
scores and principals redeploy resources in ways that increasingly
make the teaching of reading a shared responsibility.
Data from our study also indicate that the temporal boundaries
of reading instruction are porous and become increasingly more
difficult to identify as state-mandated assessments approach
(Valli et al., 2006; Valli et al., 2008). The official instructional
schedule blurs as principals and teachers seek additional
opportunities to prepare students for the reading assessments.
Research designs that rely solely on the official schedule to
identify temporal frameworks for data collection run the risk of
missing a substantial proportion of the reading instruction
experienced by students. For some students in our study, informal
instructional time exceeded formal instructional time during the
school day. The informal instructional experiences of students
warrant further investigation, for they may well influence the
effectiveness of practices that occur during the formally scheduled
time for teaching reading.
Our experiences also underscored the challenges associated
with the complexity of reading instruction in classrooms. We
were especially interested in understanding how teachers and
students interacted in relation to text, so we developed a mixed-
methods design that allowed us to capture classroom interactions,
curriculum coverage, and teachers’ beliefs about what constituted
exemplary forms of reading instruction. Even though we
acknowledged the importance of capturing the complexity of
reading instruction, we were left with a fundamental challenge:
Which constellation of factors and conditions should we
investigate to advance our knowledge about reading instruction?
As Raudenbush (2008) has remarked, more elaborate models of
instruction that attempt to incorporate all of the possible
interaction effects associated with the use of resources by teachers
ultimately collapse under the weight of their own ambitions. The
task of identifying and assessing all of the ways in which the
consequences of reading instruction may potentially vary with
other known and unknown factors is overwhelming. Without a
thoughtful, systematic approach to examining the complexity of
reading instruction, we are not likely to build a very reliable or
useful knowledge base, at least not in the foreseeable future.
A final challenge, therefore, and one not unique to under-
standing the importance of complexity, is to develop a systematic
agenda for assessing the status of our knowledge of teaching, first
reviewing the strategies that have been developed to measure
reading instruction and then setting our priorities for future
research. We suggest at least three priorities for research:
v Characterize the temporal boundaries of reading instruction,
and determine how best to examine the fluidity of these
boundaries in terms of teacher practices and reading
outcomes.
v Characterize more fully the multiple sources of reading
instruction that students receive, and determine how best to
incorporate these sources into our theories and analytical
models.
v Develop strategies for examining more systematically the
complexity of reading instruction and the constellation of
factors and conditions that influence reading outcomes.
These are some of the priorities that we have set for ourselves as
we continue to analyze data on the quality of reading instruction
that we observed in the moderate- to high-poverty elementary
schools that participated in our study.
NOTES
This article is based on a paper that was originally presented at the
symposium “Measuring Classroom Instruction: The State of the Art,” at
the annual meeting of the American Educational Research Association
in New York City, March 2008. The work reported herein was sup-
ported by the Interdisciplinary Educational Research Initiative (IERI
Grant No. 0115389), a combined effort of the National Science
Foundation, the U.S. Department of Education, and the National
Institutes of Health. The opinions expressed in this article are our own
and do not reflect the positions or policies of the National Science
Foundation, the U.S. Department of Education, or the National
Institutes of Health. Please direct all correspondence to Robert G.
Croninger at croninge@umd.edu.
1
Although in this article we focus on the challenges that we experi-
enced in studying reading instruction, very similar challenges applied to
the study of mathematics instruction in these classrooms and schools.
2All names used in this article are pseudonyms.
3We decided to study fourth- and fifth-grade classrooms because
these grades represent an important transition point for students in
terms of curricular demands. Students who fall behind in core subject
areas in these grades, particularly reading, are at considerable risk of
failure in subsequent grades (Chall & Jacobs, 2003; Chall, Jacobs, &
Baldwin, 1990).
4We also developed an attribution scale for characterizing lessons, a
curriculum log for teachers in which to describe the daily content of
lessons, and a series of individual and focus-group interviews to capture
teachers’ beliefs about teaching and the policy environment. We do not
discuss them here because they are not directly relevant to this article.
5Although we believe that the challenges we discuss in this article are
prevalent, their importance varies with one’s beliefs about what consti-
tutes “high-quality” reading instruction. For example, if high-quality
instruction is seen as increasing time on task, then some of the challenges
that we describe may be thought of as indicators of high quality. On the
Downloaded from http://er.aera.net by guest on August 23, 2010
other hand, if desirable forms of instruction are seen as enhancing stu-
dents’ depth of understanding or comprehension of text, then the chal-
lenges that we describe complicate our ability to understand how
teachers facilitate such outcomes.
6We demonstrate the challenges of complexity by using a quantita-
tive paradigm; however, we do not believe that those challenges are
unique to quantitative methods. Both quantitative and qualitative
researchers are concerned with making reliable and accurate observa-
tions about phenomena (Tashakkori & Teddlie, 2002).
7For a more detailed discussion of the data presented in this section,
see “Who (Else) Is the Teacher?” (Valli, Croninger, & Walters, 2007).
8We limit ourselves to the complexities of studying reading instruc-
tion within the official school day. However, outside the scope of this
article, there is a broad range of scholarship from sociocultural perspec-
tives on the importance of literacy development in out-of-school com-
munities. This literature extends back at least to Heath’s (1983) Ways
With Words and includes work by James Gee (1996), Carol Lee (2001,
2007), and Luis Moll and colleagues (Moll, Amanti, Neff, & Gonzalez,
1992). For a useful conceptualization of multiple literacies (school, com-
munity, and personal), see Gallego and Hollingsworth (2000).
9Although there is a body of work that examines the use of “pull outs”
in schools (i.e., pulling students out of classrooms to receive specialized
instruction), particularly schools that receive Title I funding (see, e.g.,
Chambers et al., 2000), we believe that additional work needs to be done
to fully understand how different instructional designs create both infor-
mal and formal opportunities for reading instruction and how those
opportunities influence reading outcomes, including student motivation
and interest in reading.
REFERENCES
Alexander, P., & Fives, H. (2000). Achieving expertise in teaching read-
ing. In L. Baker, M. Dreher, & J. Guthrie (Eds.), Engaging young
readers: Promoting achievement and motivation (pp. 285–308). New
York: Guilford.
Alexander, P., & Murphy, P. (1998). The research base for APA’s learner-
centered psychological principles. In N. Lambert & B. McCombs
(Eds.), How students learn: Reforming schools through learner-centered
education (pp. 25–60). Washington, DC: American Psychological
Association.
Allington, R., & Cunningham, P. (1996). Schools that work: Where all
children read and write. New York: HarperCollins.
Allington, R., & Johnston, P. (2002). Reading to learn: Lessons from exem-
plary fourth-grade classrooms. New York: Guilford.
Ball, D., & Cohen, D. (1999). Developing practice, developing practi-
tioners: Toward a practice-based theory of professional education. In
L. Darling-Hammond & G. Sykes (Eds.), Teaching as the learning
profession: Handbook of policy and practice (pp. 3–32). San Francisco:
Jossey-Bass.
Ball, D., & Lampert, M. (1999). Multiples of evidence, time, and per-
spective: Revising the study of teaching and learning. In E. Lagemann
& L. Shulman (Eds.), Issues in education research (pp. 371–398). San
Francisco: Jossey-Bass.
Barr, R., & Dreeben, R. (1983). How schools work. Chicago: University
of Chicago Press.
Bransford, J., Brown, L., & Cocking, R. (1999). How people learn: Brain,
mind, experience, and school. Washington, DC: National Academy
Press.
Byrnes, J. (2000). Using instructional time effectively. In L. Baker,
M. Dreher, & J. Guthrie (Eds.), Engaging young readers: Promoting
achievement and motivation (pp. 188–208). New York: Guilford.
Chall, J., & Jacobs, V. (2003). Poor children’s fourth-grade slump.
American Educator, 27(1), 14–17.
MARCH 2009 107
Chall, J., Jacobs, V., & Baldwin, L. (1990). The reading crisis: Why poor
children fall behind. Cambridge, MA: Harvard University Press.
Chambers, J., Lieberman, J., Parrish, T., Kaleba, D., Van Campen, J., &
Stullich, S. (2000). Study of education resources and federal funding:
Final report. Washington, DC: U.S. Department of Education,
Planning and Evaluation Service.
Chambliss, M., & Graeber, A. (2003, April). Does subject matter matter?
Paper presented at the annual meeting of the American Educational
Research Association, Chicago.
Cohen, D., Raudenbush, S., & Ball, D. (2003). Resources, instruction,
and research. Educational Evaluation and Policy Analysis, 25, 119–142.
Fenstermacher, G., & Richardson, V. (2005). On making determina-
tions of quality in teaching. Teachers College Record, 107, 186–213.
Fuhrman, S., Cohen, D., & Mosher, F. (2007). The state of education
policy research. Mahwah, NJ: Lawrence Erlbaum.
Gallego, M., & Hollingsworth, S. (Eds.). (2000). What counts as literacy:
Challenging the school standard. New York: Teachers College Press.
Gee, J. (1996). Social linguistics and literacies: Ideology in discourses (2nd
ed.). London: Falmer.
Good, T., & Brophy, J. (2003). Looking in classrooms (9th ed.). Boston:
Allyn & Bacon.
Heath, S. (1983). Ways with words: Language, life and work in communi-
ties and classrooms. Cambridge, UK: Cambridge University Press.
International Reading Association & National Council of Teachers of
English. (1996). Standards for the English language arts: A project of
International Reading Association and National Council of Teachers of
English. Newark, DE, and Urbana, IL: Authors.
Lampert, M. (2001). Teaching problems and the problems of teaching.
New Haven, CT: Yale University Press.
Lash, A., & Kirkpatrick, S. (1990). A classroom perspective on student
mobility. Elementary School Journal, 91, 177–191.
Lee, C. (2001). Is October Brown Chinese? A cultural modeling activity
system for underachieving students. American Educational Research
Journal, 38, 97–142.
Lee, C. (2007). Culture, literacy, and learning: Taking bloom in the midst of
the whirlwind. New York: Teachers College Press.
Lortie, D. (1975). Schoolteacher: A sociological study. Chicago: University
of Chicago Press.
McCaffrey, D., Lockwood, J., Koretz, D., & Hamilton, L. (2003).
Evaluating value-added models for teacher accountability. Santa Monica,
CA: RAND.
Miller, R., Murnane, R., & Willet, J. (2008, August). Do teacher absences
impact student achievement? Longitudinal evidence from one urban
school district. Educational Evaluation and Policy Analysis, 30, 181–200.
Moll, L., Amanti, C., Neff, D., & Gonzalez, N. (1992). Funds of knowl-
edge for teaching: Using a qualitative approach to connect homes and
classrooms. Theory Into Practice, 31, 132–141.
Monk, D., & Ibrahim, M. (1984). Patterns of absence and pupil achieve-
ment. American Educational Research Journal, 21, 295–310.
Morrow, L., Wamsley, G., Duhammel, K., & Fittipaldi, N. (2002). A case
study of exemplary practice in fourth grade. In B. Taylor, P. Pearson,
K. Clark, & S. Walpole (Eds.), Teaching reading: Effective schools, accom-
plished teachers (pp. 289–307). Mahwah, NJ: Lawrence Erlbaum.
Nystrand, M., & Gamoran, A. (1991). Instructional discourses, student
engagement, and literature achievement. Research in the Teaching of
English, 25, 261–290.
108 EDUCATIONAL RESEARCHER
Downloaded from http://er.aera.net by guest on August 23, 2010
Pressley, M. (2000). What should comprehension instruction be the
instruction of? In M. L. Kamil, P. B. Mosenthal, P. D. Pearson, &
R. Barr (Eds.), Handbook of reading research (Vol. 3, pp. 545–561).
Mahwah, NJ: Lawrence Erlbaum.
Pressley, M., Yokoi, L., Rankin, J., Wharton-McDonald, R., & Mistretta,
J. (1997). A survey of the instructional practices of Grade 5 teachers
nominated as effective in promoting literacy. Scientific Studies of
Reading, 1, 145–160.
Raudenbush, S. (2008). Advancing policy by advancing research on
instruction. American Educational Research Journal, 45, 206–230.
Raudenbush, S., Martinez, A., & Spybrook, J. (2007). Strategies for
improving precision in group randomized experiments. Educational
Evaluation and Policy Analysis, 29, 5–29.
Rowan, B., & Correnti, R. (2009). Studying reading instruction
with teacher logs: Lessons from the Study of Instructional
Improvement. Educational Researcher, 38, 120–131.
Shulman, L. (1986). Those who understand: Knowledge growth in
teaching. Educational Researcher, 15(2), 4–14.
Tashakkori, A., & Teddlie, C. (2002). Handbook of mixed methods in
social and behavioral research. Thousand Oaks, CA: Sage.
Taylor, B., Pearson, P., Clark, K., & Walpole, S. (Eds.). (2002). Teaching
reading: Effective schools, accomplished teachers. Mahwah, NJ: Lawrence
Erlbaum.
Valli, L., Croninger, R., & Buese, D. (2006, April). Studying high-quality
teaching in a high-stakes policy environment. Paper presented at the
annual meeting of the American Educational Research Association,
San Francisco.
Valli, L., Croninger, R., Chambliss, M., Graeber, A., & Buese, D.
(2008). Test driven: High-stakes accountability in elementary schools.
New York: Teachers College Press.
Valli, L., Croninger, R., & Walters, K. (2007). Who (else) is the teacher?
Cautionary notes on teacher accountability systems. American Journal
of Education, 113, 635–662.
AUTHORS
ROBERT G. CRONINGER is an associate professor in the Department
of Education Policy Studies, University of Maryland, 2110D Benjamin
Building, College Park, MD 20742-1175; croninge@umd.edu. His
research focuses on research methods, the social context of schooling,
teaching, teacher qualifications, and the distribution of educational
opportunities.
LINDA VALLI is the Jeffrey and David Mullan Professor in Teacher
Education–Professional Development in the Department of Curriculum
and Instruction, University of Maryland, 2311 Benjamin Building,
College Park, MD 20742-1175; lrv@umd.edu. Her research focuses on
professional development, teacher learning, teaching practices, school
improvement, and cultural diversity.
Manuscript received June 3, 2008
Revision received November 20, 2008
Accepted December 24, 2008