EDUCATION AND TREATMENT OF CHILDREN Vol. 38, No.

1,2015

Comparing the Efficiency of Repeated Reading

and Listening-While-Reading to Improve
Fluency and Comprehension
Renee O. Hawkins, Richard Marsicano
University of Cincinnati

Ara J. Schmitt, Elizabeth McCallum

Duquesne University

Shobana Musti-Rao
Pace University
Abstract
An alternating treatments design was used to compare the effects of two
reading fluency interventions on the oral reading fluency and maze accu
racy of four fourth-grade students. Also, by taking into account time spent
in intervention, the efficiency of the two interventions was compared. In the
adult-m ediated repeated reading (RR) condition, students read a grade-level
passage aloud to an adult. The adult provided the students with error correc
tion of oral reading miscues. In the listening-while-reading (LWR) condition,
students read along aloud w ith audio recorded readings of passages using an
MP3 player. The RR and LWR conditions had similar effects on reading flu
ency for three participants and the RR was more effective for one participant.
When accounting for instructional time, the LWR condition was more efficient
at im proving reading fluency for three of the four participants. The same pat
tern of results was evident in Maze comprehension data. Discussion will em
phasize the need to consider instructional time w hen selecting interventions.

ince the reauthorization of the Individuals with Disabilities Edu

cation Improvement Act (IDEIA) in 2004, schools are increasingly
focused on providing students with empirically supported instruc
tion and intervention services (IDEIA, 2004). At the same time, fund
ing to support the implementation of high quality instructional and
intervention support has become increasingly limited. Many schools
are severely limited financially and are struggling to meet the high
expectations for student performance with fewer resources.
There is general consensus among researchers and practitio
ners on components that constitute effective reading instruction (i.e.,

Address correspondence to Renee O. Hawkins, University of Cincinnati, PO Box

210068, Cincinnati OH, 45221; renee.hawkins@uc.edu; (513) 556-3342

Pages 49-70

50

HAWKINS et al.

phonem ic aw areness, alphabetic principle, fluency, vocabulary, com

prehension; International Dyslexia Association, 2010; N ational R ead
ing Panel [NRP], 2000). Reasonably established are effective proce
d u res to strengthen reading fluency an d enhance com prehension (see
Daly, Chafouleas, & Skinner, 2005; Shapiro, 2004, for exam ples of such
evidence-based techniques). O ne w idely im plem ented intervention
for increasing reading fluency is repeated reading (RR). This curricular
supplem ent requires students to reread passages w ith error correction
u n til they achieve a desired level of decoding accuracy and fluency. A
m eta-analysis of RR studies concluded that RR effectively im proves
the reading fluency and com prehension of students, regardless of dis
ability status (Therrien, 2004). Overall, gains in reading fluency were
m ore significant than gains in com prehension. A nother reading fluen
cy intervention is listening-w hile-reading (LWR). The LWR interven
tion requires a student to read a passage (aloud or silently) while the
text is read aloud by a fluent m odel. Research indicates that LWR m ay
be applied to im prove oral reading fluency (Daly & M artens, 1994;
Lionetti & Cole, 2004; Rasinski, 1990; Rose, 1984; Rose & Sherry, 1984;
Skinner, Cooper, & Cole, 1997). In addition, som e researchers have
found positive intervention effects on overall com prehension (Hale et
al., 2005; Schmitt, McCallum, Hale, Obeldobel, & Dingus, 2009; Skin
ner, Robinson, A dam son, Atchison, & W oodw ard, 1998). A lthough RR
and LWR interventions do no t explicitly target reading com prehen
sion, their resulting positive effects on com prehension m ay be due to
increased reading fluency (M organ & Sideridis, 2006; Therrien, 2004)
and/or repeated exposure to reading m aterial content (Hawkins, M usti-Rao, Hale, McGuire, & Hailley, 2010).

Identifying Efficient Interventions

A lthough it is im portant to identify interventions that can effec
tively im prove student reading perform ance, it is also im portant to
identify interventions that can efficiently address students' skill defi
cits. Two im portant factors to consider w hen evaluating efficiency are
resources and time. Resources can include instructional m aterials need
ed to im plem ent an intervention, m oney required to purchase m ateri
als, and staff needed to im plem ent an intervention. For example, two
interventions m ay have em pirical support indicating that they lead to
com parable perform ance gains for students. However, one interven
tion is a published program that costs $1,000 and m ust be im plem ented
by a teacher, b u t the second intervention uses m aterials readily avail
able in the classroom and can be im plem ented by any adult follow
ing a brief training on a sim ple intervention script. In m ost schools, the
choice w ould be obvious and dictated by available resources.

EFFICIENCY OF READING INTERVENTIONS

51

In addition to considering resources, school teams also should

consider time when selecting interventions. There is only so much
time in a school day and school personnel are responsible for mak
ing sure that this time is spent wisely. That is, schools must deter
mine how to allocate time to result in the maximum learning gains
for students across a range of subjects (e.g., reading, math, science, so
cial studies, art, music, physical education). Time allocation becomes
more complex when students need intervention support. School
teams have to decide from where to reallocate time. As school teams
develop intervention plans for students, it seems that it would be
helpful for them to not only judge the effectiveness of varied interven
tions but also the efficiency. For example, a team may be considering
two evidence-based interventions shown to have positive effects on
student performance. To result in the desired performance gains, one
intervention will need to be implemented every day for 30 min and
the second intervention will need to be implemented every day for 15
min. It is likely that with this information a school team would choose
the intervention that takes less time, allowing the "extra" time to be
used for other academic activities.
Given the limited time available in the school day, it is critical
that school teams take into account the amount of time that will be
needed to implement an intervention in order to achieve the desired
result. The amount of time spent in active instruction or time usage
has been identified as an important variable for student learning (Vannest & Parker, 2010). If the goal of applying a reading fluency and
comprehension intervention is to absolutely remediate the problem,
not only should the degree of reading improvement be considered but
also the amount of improvement relative to the time engaged in the
intervention. This latter concept is commonly referred to as learning
rate (Skinner, 2008).
Learning rate can be calculated in a variety of ways. For exam
ple, the amount of learning divided by instructional time provides
an estimate of learning rate. By examining learning rates, interven
tions may be selected that result in the greatest improvement over a
given time period and efficiently meet academic goals. Elucidating
the importance of learning rate, Skinner, Belfiore, and Watson (2002)
reassessed data from previously conducted trials (Skinner, Smith, &
McLean, 1994) that compared the effect of inter-trial interval length
(i.e., immediate presentation of new words versus five-second delay
before the presentation of new words) had on word mastery per day.
It was originally concluded that Student 1 mastered the same amount
of words independent of condition and Student 2 mastered more
words in the five-second delay condition. Reframing the students'

52

HAWKINS et al.

learning rate as w ords mastered per instructional second yielded dif

ferent results: Student la n d Student 2 both demonstrated a higher
level of w ord mastery in response to the immediate presentation of
new words (Skinner et al., 2002). As demonstrated in this study, prior
to an intervention being qualified as efficient, a change in the depen
dent variable over a more specific unit of time should be examined.
A recent ten-year review of the academic intervention literature con
tained in four journals (Journal of School Psychology, Psychology in
the Schools, School Psychology Quarterly, and School Psychology Re
view) found that only two of 27 studies investigated intervention ses
sion length (Bramlett, Cates, Savina, & Lauinger, 2010).
The overarching expectation of academic interventions, typi
cally expressed as an end-of-the-year goal, is not only to achieve said
goal (e.g., 120 words read correctly per minute [WPM]) but to do so
at a pace that allows the student to be on par w ith his or her peers in
a reasonable am ount of time. A student that m ust make up a 60 WPM
discrepancy over a 10-week period would not be best served by par
ticipating in an intervention that yielded a 3 WPM increase/week. Best
practices, in the above hypothetical circumstance, w ould require a reevaluation of the selected intervention's use and implementation. In
other words, the selection of appropriate interventions m ust consider
the rate by which the student is learning. Interventions that are osten
sibly similar in effectiveness m ay result in different decision-making
outcomes w hen factoring in intervention exposure time (Cates et al.,
2003). Learning rate also is im portant given limited time, materials,
and hum an resources available to implem ent evidence-based inter
ventions. Studies of treatment fidelity also suggest that effective and
efficient protocols increase intervention adherence (Gresham, 1989;
Lentz, Allen, & Ehm hardt, 1996).
Current Study
The purpose of the current study was to compare the effective
ness and efficiency of RR and LWR procedures. Although there is re
search supporting the effectiveness of both intervention procedures,
data regarding the relative efficiency of these are needed. Given the
limited instructional time and resources common in school settings,
problem-solving teams need information regarding both intervention
effectiveness and efficiency to make data-based intervention selec
tions. Concerning efficiency, the interventions differ in their use of re
sources and time. The RR procedures implemented required an adult
to provide feedback to students while the LWR used the MP3 player
to guide students. Reading was selected as the target for interventions
because it is a skill commonly targeted by school teams and it is linked

EFFICIENCY OF READING INTERVENTIONS

53

to significant academic and behavioral outcomes (United States De

partment of Education, Institute of Education Sciences [IES], 2009;
NRP, 2000). The study addressed the following research questions:
Does RR or LWR result in higher levels of oral reading fluency
(effectiveness)?
Does RR or LWR result in higher levels of comprehension, as
measured by Maze task accuracy (effectiveness)?
Does RR or LWR result in higher levels of reading fluency when
accounting for instructional time (efficiency)?
Does RR or LWR result in higher levels of comprehension, as
measured by Maze task accuracy, when accounting for instructional
time (efficiency)?
It was hypothesized that both interventions would lead to simi
lar levels of reading fluency and comprehension but the LWR would
be the more efficient intervention (i.e., require less instructional time
and resources to result in comparable performance).
Method
Participants and Setting

The study was conducted in an urban charter school in the Mid

western United States. The school had an enrollment of 605 K-9 stu
dents, including 581 (96%) African-American, eight (1.3%) Caucasian,
two (0.3%) Hispanic, and 14 (2%) multiracial students. A majority of
the students (i.e., 92%) qualified for free and/or reduced price lunch.
At the time of the study, the school did not have tiered supports in
place for students; however, the primary researcher was consulting
with the school staff to establish universal screening procedures for
reading and math, as opposed to the teacher referral system that was
in place. The principal indicated that he would like to phase in the
universal screening of students and wanted to start with the fourthgrade. The principal indicated that many of the now fourth-grade stu
dents did not perform well on their achievement tests at the end of
third grade. As part of this initiative, the primary researcher collabo
rated with fourth-grade teachers to review the results of the universal
screening and to develop instructional and intervention supports.
Four male African American students from one fourth-grade
classroom participated in the study. Students ranged in age from 9-10
years. Pseudonyms are used in this report for all student participants.
Students were selected for participation following fall grade-wide flu
ency screening using the Dynamic Indicators of Basic Early Literacy
Skills (DIBELS) reading fluency measure (Good & Kaminski, 2007).
The participating students scored in the "Some Risk" (i.e., 71-92
words correct per minute) or "At-risk" (i.e., < 71 words correct per

54

HAWKINS et al.

minute) ranges. Of the 28 students in the class, seven students scored

in the "Some Risk" or "At-risk" categories. Parent permissions to par
ticipate were sent home to all seven of these students. The first four
students returning signed permissions and providing student assent
were included in the research study. The researcher collaborated with
the classroom teacher to develop similar interventions for the three
students who were not included in the study. These interventions
were implemented in the regular classroom by the classroom teacher
and were not part of the current study.
Data were collected to verify the performance of the four par
ticipants from the fall screening. During each assessment session, stu
dents were presented with a Maze assessment and instructed to read
the passage silently and circle the correct word choices. Students were
given 3 m in to complete the Maze task. Students then were asked to
read an intact version of the same passage that was used for the Maze
assessment aloud to the researcher for 1 min. The num bers of words
read correctly were recorded. The average pre-intervention perfor
mance of all four participants was in the "At-risk" range (see table 1;
Good & Kaminski, 2007).
None of the participating students qualified for special educa
tion services, and they were not receiving any remedial services at
the time of the study. Throughout the study, students continued to
receive 60 min of classwide reading instruction using the SABIS cur
riculum (SABIS Educational Services, 2012). The sequence of typical
classroom reading activities included (a) the teacher previewing the
reading material that w ould be the focus of instruction for the day or
week, followed by (b) classwide oral reading of text, w ith students
taking turns reading aloud, followed by (c) independent seatwork,
and sometimes (d) peer review of independent seatwork. Teacher pre
viewing activities included identifying and defining key vocabulary
words from the text and classwide discussion of the reading topic.
Independent seat work activities included answering questions about
the reading, writing summaries of the m ain ideas, and completing
story maps or graphic organizers of the information from the reading.
In addition, 30 minutes each day were spent in silent reading. For the
current study, students were removed from the classroom for assess
m ent and intervention sessions during the silent reading time.
Four school psychology graduate students served as data col
lectors and delivered the interventions. The graduate students were
second- and third-year doctoral students. These graduate students
had been previously trained in DIBELS and Maze administration and
scoring by the primary researcher in required Program coursework on
academic assessment. Graduate students were established as reliable

EFFICIENCY OF READING INTERVENTIONS

55

scorers through co-observations with interscorer agreement at 95% or

higher based on word-by-word analysis for ORF and 100% for Maze
correct word choices. The graduate students worked in collaboration
with the primary researcher to develop the intervention scripts for the
LWR and RR conditions (see appendix for scripts). Once the scripts were
developed, the graduate students met with the primary researcher for a
30 min training session. During the session, each graduate student took
a turn administering the intervention procedures as a peer assumed the
role of student. The researcher provided feedback to the graduate stu
dents and each was observed following the script with 100% adherence.
Materials

During all sessions, students read 200-word passages from Read

ing Fluency: Level D, written at the fourth-grade level (Blachowicz,
2004a). Researchers also converted Reading Fluency passages to Maze
assessments. Maze probes were created by replacing every seventh
word after the first sentence with parentheses containing three choices
in random order: the correct w ord from the passage; a near distracter
that was the same type of word, but did not make sense in context; and
a far distracter that was grammatically incorrect (Shinn & Shinn, 2002).
Given the consistency in passage length (i.e., 200 words), the result
ing Maze passages included a small range of between 27 and 30 items
(opportunities for students to make a word choice). During the LWR
condition, students read along to an audio reading of the passage on an
MP3 player. The audio recording was transferred from the commercial
ly available Reading Fluency: Level D series CD (Blachowicz, 2004b)
to the MP3 player. Stopwatches were used to record session durations.
Experimental Design and Dependent Measures

An alternating treatments design (Cooper, Heron, & Heward,

2007) was used to compare the effectiveness and efficiency of RR and
LWR interventions. The presentation of the RR and LWR conditions
was alternated w ith the order of condition presentation counterbal
anced across participants and intervention sessions.
Oral reading fluency (ORF) (i.e., words read correctly per m in
ute) served as one of the dependent variables for analysis. ORF was
assessed at the end of each intervention session using the passage
read during the session. Maze assessments also were used to estimate
reading comprehension. The num ber of correct w ord choices made
during a 3-min assessment period was recorded. Students completed
the Maze task for the passage practiced in the session.
The efficiency of intervention procedures was estimated by divid
ing the dependent variable (i.e., ORF or correct Maze selections) by the

56

HAWKINS et al.

number of seconds required to complete intervention procedures. These

numbers were multiplied by 60 in order to create efficiency estimates of
(a) words read correctly per minute, per minute of intervention, and (b)
correct Maze selections per minute, per minute of intervention.
Procedure
The study was completed across 12 weeks. Students participated
in experimental sessions 1-2 times per week. The duration of inter
vention sessions ranged from 4 m in 42 sec to 20 m in 57 sec. Proce
dures were implem ented in a quiet room in the school. To reinforce
participation across all sessions and conditions, students were given
a sticker at the end of each session. After earning three stickers, stu
dents were allowed to select a small tangible reward (i.e., mechanical
pencil, mini-notebook, piece of candy). Throughout the experiemental
phase, students continued to receive traditional reading instruction in
their general education classroom as described above. The appendix
includes scripts for the RR and LWR interventions.
Training. Prior to the start of the experimental phase, students
were introduced to the study and intervention procedures through
a 20-min training session. In the first part of the training session, the
researcher explained that the students w ould be working with re
searchers on their reading a couple of times each week. Students were
informed that on some days they w ould be reading a passage aloud
to a researcher and other days they w ould be reading aloud along to
a reading on an MP3 player. The researcher then reviewed the steps
involved in RR. One student was selected to model the procedures
for the rest and students were given the opportunity to ask ques
tions. Then, the researcher reviewed the steps involved in LWR. Each
student was given a MP3 player and instructed as to how to use it.
Another student was selected to model the procedures and students
were given the opportunity to answer questions.
Repeated reading. After starting the stopwatch, the researcher pre
sented the student with a reading passage and instructed the student
to read the passage aloud. If the student made a reading error, the re
searcher said, "Stop. The word i s ____ . Point to the word and read it
three times." The student read the passage aloud to the researcher three
times, receiving error correction as needed during each reading. After
the third reading, the researcher stopped the stopwatch and recorded
the time. The student then was presented with a Maze task based on
the passage they had just read. After completing the Maze task, the
student's ORF was assessed using the passage read during the session.
Listening-while-reading. After starting the stopwatch, the re
searcher presented the students w ith a reading passage and an MP3

EFFICIENCY OF READING INTERVENTIONS

57

player. The researcher instructed the student to read the passage

aloud while listening to the passage being read on the MP3 player.
The student read the passage along with the MP3 player three times.
After the third reading, the researcher stopped the stopwatch and re
corded the time. The same procedures used in the RR condition were
followed to assess the student's comprehension and fluency.
Procedural Adherence and Interobserver Agreement
Treatment integrity was monitored for 25% of all intervention
sessions (i.e., 16/64 sessions), 25% of LWR sessions (8/32), and 25% of
RR sessions (8/32) through direct observation. A second observer used
a treatment integrity checklist to record the occurrence or nonoccur
rence of the steps of intervention procedures. This checklist was based
on the scripts created for the procedures, which are included in the
appendix. Procedural adherence was 100% for each of the observed
sessions.
A second scorer was used to establish interscorer agreement for
ORF, correct selections on the Maze task, and seconds of intervention.
Interscorer agreement was calculated for 25% (20/79) of the ORF and
Maze assessments completed in the study, including both interven
tion assessments (16/64 overall; 8 LWR assessments and 8 RR assess
ments) and the assessments conducted to verify students' benchmark
scores (4/15). Interscorer agreement for intervention time was com
pleted for 28% of intervention sessions (18/64). Agreement was cal
culated by dividing the number of word-by-word (ORF) and word
choice-by-word choice (Maze) agreements by the number of agree
ments plus disagreements. Average interscorer agreement for ORF
was 99% (97-100%) and for Maze selections was 99% (94100%). Av
erage agreement on intervention session length was 99% (98-100%).
Intervention Acceptability
At the conclusion of the study, participants completed a five item
questionnaire, which included the items: (a) I liked doing the reading
activities, (b) It was easy to follow the directions for the reading activi
ties, (c) I think the activities helped me read better, and (d) It is impor
tant for me to be a good reader. For the fifth item, (e), students were
asked which intervention procedure they preferred. Students rated the
first four items on a scale of 1-5, with 1 indicating strong disagreement
and 5 reflecting strong agreement. Average ratings for the first four
items ranged from 4.8-5.00, suggesting that students enjoyed the read
ing interventions and felt their reading skills improved as a result. One
student, Dave, reported a preference for the RR intervention, with the
other three students reporting a preference for the LWR intervention.

58

HAWKINS et al.

Results
Figure 1 displays data for ORF an d ORF efficiency across inter
vention conditions. M aze perform ance an d efficiency data are p re
sented in figure 2. Table 1 includes fluency an d com prehension data
collected to verify the selection of students for intervention an d table
2 reports the m eans and stan d ard deviations for ORF, ORF efficiency,
M aze, an d M aze efficiency by condition.
Based on visual analysis, the tw o intervention conditions had
sim ilar effects on reading fluency for Dan, Mike, and Ray (figure 1),
w ith higher m ean ORF scores in the RR condition for Dan and Ray and
a slightly higher m ean ORF in the LWR condition for Mike. For these
three students, m ean fluency w ith practiced passages was in the "Some
risk" range based on DIBELS w inter benchm arks, im proved from the
"At risk" range during baseline based on fall DIBELS benchm arks (table
2; Good & Kaminski, 2007). In addition, data for these students show ed
a slight increasing trend over the duration of the study. Visual analysis
of D ave's data revealed consistent and m arked higher levels of reading
fluency during the RR intervention as com pared to LWR. D ave's m ean
fluency w ith practiced passages was at benchm ark levels in the RR con
dition and in the "Some risk" range for LWR based on DIBELS w in
ter benchm arks, as com pared to baseline perform ance in the "At risk"
range based on fall DIBELS benchm arks (Good & Kaminski, 2007).
G raphic analysis of ORF efficiency data show th at LWR led to
consistently h igher efficiency estim ates for Dan, Mike, an d Ray (figure
1). C ontrolling for tim e spent in the intervention session, LWR led to
greater ORF in less tim e th an d id RR for these three students. Average
ORF efficiency data reported in table 2 indicate that, given one m inute
of intervention, D an and M ike nearly doubled the nu m b er of w ords
read correctly in the LWR condition as com pared to the RR condition.
For exam ple, on average, each m inute of RR intervention w as estim at
ed to contribute 7.92 w ords read correctly to D an's overall RR session
ORF score, w hereas each m inute of LWR intervention w as estim ated
to contribute 13.66 w ords read correctly to his overall LWR session
ORF score. R ay's m ean ORF efficiency estim ate also w as h igher in the
LWR condition. The grap h of D ave's ORF efficiency data show s som e
overlap betw een the tw o conditions, w ith RR appearing to be slightly
h ig h er overall. D ave's m ean ORF efficiency w as slightly higher in the
RR condition than the LWR condition (table 2).
Visual analysis of M aze data shows similar levels of correct w ord
choices across intervention conditions for D an and Mike. The graph of
Ray's Maze data shows slightly higher perform ance in the RR condi
tion, and D ave's data shows a clear difference in perform ance across the
tw o conditions, w ith the RR intervention resulting in consistently higher

59

EFFICIENCY OF READING INTERVENTIONS

Dan
120
100
2 60
O 40
20

11

13

15

17

Session
Mike
120

20

r :

i t- i

: r

. > ,

11

13

15

17

Session

Dave

Jr"

I 20
iu j 1
5
10
S' 5

LWR
RR

-LW R
-RR

11

13

15

17

Session
Ray
25

h
20

LWR
RR

-LW R

w 10

-R R
7

11

13

15

17

Session

Figure 1.

Students' OFR Scores and OFR Efficiency.

Maze scores. Data from Dan, Mike, and Ray also show a slight increas
ing trend across the intervention phase. Based on condition means, the
RR led to slightly higher Maze performance for all participants (table 2).
Although not presented, it should be noted that the pattern of results
was the same for the percent of correct Maze choices as there was a
limited range of choices (i.e., 27-30) across passages used in the study.
Graphs of Maze efficiency for Dan and Mike indicate that LWR
led to higher comprehension levels as compared to the RR condition,
controlling for the time spent in intervention (figure 2). This visual
analysis is supported by higher mean Maze efficiency scores displayed

60

HAWKINS et al.

Table 1
Baseline Performance
ORF Assessment Session
1

M (SD)

Dan

44

54

Mike

42

43

63

61

55.50 (8.58)

52

49

46.50 (4.80)

Dave

71

81

60

Ray

37

52

66

70.67 (10.50)
44

49.75 (12.45)

Maze Assessment Session

1

M (SD)

Dan

14

12

10.75 (2.75)

Mike

5.00 (1.41)

Dave

16

20

17

Ray

13

17.67 (2.08)
4

8.25 (3.78)

in table 2. Each m inute of RR intervention contributed approximately

1.32 correct w ord choices to Mike's average RR session Maze score,
whereas each m inute of LWR intervention contributed approximate
ly 2.37 correct word choices to his average LWR session Maze score.
There is overlap between the data paths of Maze efficiency across the
two conditions for Dave and Ray, w ith mean Maze efficiency data
slightly higher in the LWR condition for these students (table 2).

Discussion
In addition to identifying interventions that lead to academic
skill improvements (effective interventions), practitioners should also
identify interventions that are able to quickly improve student perfor
mance (efficient interventions; Skinner, 2008). Thus, studies such as
this one that directly compare the effects of two interventions on the
same construct are useful in determining not only which intervention
is more effective but also which is more efficient.
The current study compared the effects of two evidence-based
reading interventions on oral reading fluency (ORF) and reading com
prehension. Fluency data indicated that for three of the four students
(Dan, Mike, and Ray) both LWR and RR led to similar ORF levels.
However, when the amount of time these three students spent en
gaged in each intervention condition was considered, results sug
gested that LWR led to more rapid learning than RR. For these three
students, LWR was more efficient. For the fourth student, Dave, RR
was both more effective and efficient in improving ORF. Reading

EFFICIENCY OF READING INTERVENTIONS

61

Dan

30
N 25

ID c

Z O 20

LWR
RR
1

11

13

15

17

S e s s io n

Mike

Dave
n 25
to c
Z o 20

LWR
RR
0
1

7
9 11
S e s s io n

13

15

17

Figure 2. Students' Maze Scores and Maze Efficiency Estimates by Session.

comprehension data reveal a similar pattern, although Ray had slight

ly higher Maze scores in the RR condition, and his Maze efficiency
data were less clearly differentiated across conditions. Also, Dave's
Maze efficiency estimates were similar across the two conditions.
The findings from the current study are consistent with previous
research documenting the effectiveness of RR and LWR interventions
(e.g., Daly & Martens, 1994; Therrien, 2004). Yet, data from the current
investigation regarding the relative efficiency of the two procedures
indicate that LWR may sometimes be a better intervention option, par
ticularly when time and resources are limited. Researchers have noted

62

HAWKINS et al.

Table 2
Average ORF and Maze Performance and Efficiency across Conditions
OFR
LWR
Dan

OFR Efficiency
RR

LWR

RR

M (SD)

M (SD)

M (SD)

M (SD)

86.88 (14.04)

91.29 (9.12)

13.66 (3.41)

7.92 (2.42)

Mike

86.38 (6.89)

84.11 (6.41)

14.74 (4.09)

7.62 (1.50)

Dave

88.75 (14.41)

111.63 (14.68)

15.00 (2.19)

15.71 (4.34)

Ray

72.25 (7.94)

82.25 (10.74)

12.94 (2.44)

7.40 (1.61)

Correct Maze Selections

Maze Efficiency

LWR

RR

LWR

RR

M (SD)

M (SD)

M (SD)

M (SD)

Dan

22.00 (3.02)

24.00 (4.04)

3.48 (0.86)

2.08 (0.72)

Mike

13.88 (1.73)

14.55 (2.40)

2.37 (0.74)

1.32 (0.35)

Dave

22.00 (3.42)

25.75 (3.62)

3.98 (0.79)

3.52 (0.86)

Ray

10.38 (4.93)

11.88 (3.52)

1.87 (1.04)

1.08 (0.45)

th at the initial reading in an RR intervention can be laborious for the stu

dent, requiring a significant am ount of time for the student to complete
the passage (Rasinski, 1990). By providing a fluent m odel of reading,
LWR strategies m ay decrease the difficulty of the initial reading, which
could decrease the duration of intervention sessions and possibly in
crease efficiency. In the current study, the pace of the LWR intervention
was controlled by the initial speed at w hich the reading was recorded,
contributing to shorter intervention sessions and higher relative inter
vention efficiency as com pared to RR. Also, the LWR did not require the
tim e of a trained adult to w ork one-on-one w ith each student. It should
be noted, however, that an adult was present for all LWR sessions in
the current study. It is unknow n if students w ould accurately follow
the procedures if left w ithout close adult supervision. Regardless, using
technology, such as MP3 players, m ay serve to increase the efficiency of
intervention procedures w hile m aintaining effectiveness.
For one student, how ever, the RR intervention w as bo th m ore
effective and efficient th an LWR. O ne possible explanation involves
stu d en t preference for the tw o interventions. Social validity data re
vealed that Dave reported a preference for RR over LWR. Perhaps stu
d en ts' academ ic engaged tim e an d task perseverance w ere higher for
the interventions they preferred, leading to increased perform ance in
the favored interventions.

EFFICIENCY OF READING INTERVENTIONS

63

Implications for Practice

Results of the current study have practical implications for edu

cators faced with the challenge of choosing effective evidence-based
academic interventions for preventing and remedying students' skills.
First, students with skill deficits need intervention support that may
require additional personnel resources for successful implementation.
Lack of resources for instruction and progress monitoring has been
cited as an obstacle to implementing multi-tiered systems of support
(SpectrumK12,2010). Practitioners can utilize available resources more
effectively by choosing interventions that are less likely to drain per
sonnel resources. For example, in the case of students like Dan, Mike,
and Ray for whom both RR and LWR interventions had similar effect,
teachers can start by choosing interventions that are more efficient
(i.e., LWR with an MP3 player) at first. This will free up some time for
the teacher to work with students who may require adult-mediated
intervention. Second, in addition to considering personnel resources,
the study demonstrates the significance of attending to instructional
time when selecting interventions. With limited time in the school day
that must be allocated across many subject areas, school teams select
ing interventions should consider how much time an intervention will
require in order to achieve the desired result. Given two interventions
that produce similar performance gains, school teams should examine
which intervention is more time efficient to inform their decision. Fi
nally, the results of the study highlight the importance of examining
data from many different angles in order to make the most informed
decision. Looking only at the ORF and Maze accuracy levels from the
current study, one would conclude that the two interventions were
equal in terms of their impact on ORF for three of the participants.
Given only these ORF data, a school team may then decide to imple
ment an adult-mediated RR intervention. However, by accounting for
instructional time and considering efficiency estimates for these three
students, one would conclude that LWR is more efficient and decide
to implement this intervention with a struggling student.
Examining the Efficiency of Reading Interventions

In addition to RR and LWR, there are many other effective read

ing fluency interventions that may also be evaluated in terms of ef
ficiency as well. For example, with the Peer Assisted Learning Strat
egies or Peer Assisted Literacy Strategies (PALS) program, students
work in pairs to improve reading fluency and comprehension (Fuchs,
Fuchs, Mathes, & Simmons, 1997; Mathes & Babyak, 2001). The use of
peers essentially reduces the teacher-student ratio to 1:1 and allows
for increased opportunities for students to respond, with students

64

HAWKINS et al.

taking turns in the roles of tutor and tutee. However, both teachers
and students m ust be trained on the procedures and the teacher is
responsible for monitoring student behavior and adherence to the
intervention procedures during sessions. Intervention sessions are
35 min. in duration and are designed to occur 3-4 times a week as a
supplem ent to the prim ary reading curriculum, which w ould be an
im portant consideration for school teams who w ould need to find this
intervention time in the school day. Required costs for Peer-Assisted
Learning Strategies include $40-44 for each grade-level m anual and
optional costs include $15 for training videos and $1,500 for an on-site,
daylong teacher training session (United States Department of Educa
tion, Institute of Education Science, 2012). Attending to information
about the effectiveness of PALS alone, arguably, w ould not provide
a school team enough information about PALS to make a decision
about implementation. Considerations about implementation time,
staffing, and materials provide a m uch clearer picture and w ould be
critical for a school team to make the best decision given their context.
As another example, Read Naturally is an effective fluency in
tervention designed for students to w ork prim arily independently,
either using CDs and hard-copy texts or a computer program (Han
cock, 2002). Sessions are intended to last 30-45 min and to occur 3-5
times per week. Although students primarily work independently, it
is recommended that the teacher/adult to student ratio be no higher
than 1:8 (Read Naturally, 2012a). The cost of materials for each level
of the CD and printed stories version costs $129. The computer ver
sion of the program costs $125 per level per computer. Schools and
districts also have the option to purchase school server software for
$200 or district server software for $1,000 (story levels are then or
dered separately; Read Naturally, 2012b). Clearly, there is m uch more
to consider than effectiveness alone w hen selecting interventions to
implement in school settings w ith limited time and resources.
Limitations

There are some limitations to the current study that should be

considered w hen interpreting the results. First, a survey level assess
m ent (SLA) was not used to determine the instructional level of stu
dents and to choose appropriate reading material. All students read
passages that were written at grade level (i.e., fourth-grade). Although
after receiving intervention on passages student performance was in
the instructional (i.e., 70-100 words correct per minute) and mastery
(i.e., > 100 words correct per minute) ranges, baseline data indicate
that for three out of the four participants fourth-grade reading mate
rial was at the frustrational level (i.e., < 70 words correct per minute;
Shapiro, 2004). This likely contributed to slow and laborious reading

EFFICIENCY OF READING INTERVENTIONS

65

during the RR condition, possibly leading to an underestimation of

the efficiency of this intervention. This would not be true for the LWR
condition because the pacing of the passages was dictated by the au
dio recording. If intervention passages had been at students' instruc
tional levels, the results may have been different. Future researchers
should include a SLA as part of the methods to ensure that the read
ing material is appropriate for the participants.
Second, students were assessed using the same passages that they
read during intervention conditions; therefore, the effects of the inter
ventions on the reading of unpracticed material are unknown. Students
may have memorized some of the content of the passages, likely lead
ing to higher fluency and comprehension scores than would have been
observed on generalization passages. Future researchers may wish to
implement similar procedures comparing the effectiveness and effi
ciency of LWR and RR on reading passages not included in interven
tion procedures. Another limitation involves using Maze tasks to assess
reading comprehension. Maze tasks may not be the best measures of
reading comprehension because they do not require students to dem
onstrate a depth of understanding that may be better assessed using de
tailed multiple choice or open-ended comprehension questions. Next,
students received corrective feedback from an adult in the RR condi
tion, whereas an explicit error correction procedure was not included in
the LWR condition. In the LWR intervention, students heard the correct
reading of the passage, but if they made reading errors when reading
along with the audio there was no procedure requiring the students
to correct their mistake. This difference between the two interventions
may have contributed to some of the differential effects favoring RR
seen in the current study. When selecting an intervention for a student
who has high reading error rates, an intervention such as adult-mediat
ed RR that includes an error correction procedure may be preferred to
LWR to ensure that students do not repeatedly practice errors.
Despite these limitations, the current study not only adds to the
existing evidence base establishing LWR and RR as effective and ac
ceptable reading interventions (Chard, Ketterlin-Geller, Baker, Doabler, & Apichatabutra, 2009; Hale et al., 2005) but the results also have
practical implications for educators choosing interventions that will
lead to the most rapid achievement gains. As schools are increasingly
focused on prevention and early intervention, educators are faced with
selecting empirically validated interventions that are both effective
and efficient. Researchers should continue to examine the efficiency
of interventions and provide specific information on the time and re
sources needed to deliver an intervention and achieve the desired re
sult to help better inform intervention decisions made by school teams.

66

HAWKINS et al.

Appendix
Intervention Scripts
Repeated Reading Script

1. Have the student sit in a quiet location without too many distrac
tions.
2. Tell the student that today you are going to ask him to read a story
out loud to you.
3. Start the timer.
4. Provide the reading passage to the student.
5. Instruct the student to read the passage aloud.
6. If the student makes an error while reading, say "Stop the word is
. Point to the w ord and read it three times."
7. W hen the student is finished reading the passage, provide praise
and instruct him to read it again from the start.
8. Have the student read the passage 3 times total.
9. After the third read of the passage, stop the timer and record the time.
10. Have the student complete a Maze probe based on the passage he
read that day.
11. Administer 1-minute ORF timing on the passage he read that day.
12. Provide praise and a sticker on the student's rew ard card.
13. If the student earned three stickers, allow them to select a reward.
Listening-While-Reading Script

1. Have the student sit in a quiet location without too m any distrac
tions.
2. Tell the student that today you are going to ask him to read along
with a recording of a story.
3. Start the timer.
4. Provide the reading passage to the student.
5. Instruct the student to read along aloud as they listen to a recording
of the story.
6. Start the audio.
7. W hen the audio is finished, provide praise and instruct him to re
read the passage along with the audio.
8. Have the student read the passage along with the audio 3 times
total.
9. After the third read, stop the timer and record the time.
10. Have the student complete a Maze probe based on the passage he
read that day.
11. Administer 1-minute ORF timing on the passage he read that day.
12. Provide praise and a sticker on the student's reward card.
13. If the student earned three stickers, allow them to select a reward.

EFFICIENCY OF READING INTERVENTIONS

67

References
Blachowicz, C. L. Z. (2004a). Readingfluency reader: Level D. New York:
Jamestown Education.
Blachowicz, C. L. Z. (2004b). Reading fluency audio program: Level D.
New York: Jamestown Education.
Bramlett, R., Cates, G. L., Savina, E., & Lauinger, B. (2010). Assessing
effectiveness and efficiency of academic interventions in
school psychology journals: 1995-2005. Psychology in the
Schools, 47,114-125. doi: 10.1002/pits.20457
Cates, G. L., Skinner, C. H., Watson, T. S., Smith, T. L., Meadows, T,
Weaver, A., & Jackson, B. (2003). Instructional effectiveness
and instructional efficiency as considerations for data-based
decision making: An evaluation of interspersing procedures.
School Psychology Review, 32, 601-616.
Chard, D. J., Ketterlin-Geller, L. R., Baker, S. K., Doabler, C., &
Apichatabutra, C. (2009). Repeated reading intervention for
students with learning disabilities: Status of the evidence.
Exceptional Children, 75, 263-281.
Cooper, J. O., Heron, T. E., & Heward, W. L. (2007). Applied behavior
analysis (2nd ed.). New York: Macmillan.
Daly, E. J., Chafouleas, S., & Skinner, C. H. (2005). Interventions for
reading problems: Designing and evaluating effective strategies.
New York: Guilford Press.
Daly, E. J., & Martens, B. K. (1994). A comparison of three interven
tions for increasing oral reading performance: Application of
the instructional hierarchy. Journal ofApplied Behavior Analysis,
27, 459-469. doi: 10.1901/jaba.l994.27-459
Fuchs, D., Fuchs, L. S., Mathes, P., & Simmons, D. C. (1997). PeerAssisted Learning Strategies: Making classrooms more re
sponsive to diversity. American Educational Research Journal,
34,174-206.
Good, R. H., Ill, & Kaminski, R. A. (Eds.). (2002). Dynamic Indicators
of Basic Early Literacy Skills (6th ed.). Eugene, OR: Institute for
the Development of Educational Achievement.
Gresham, F. M. (1989). Assessment of treatment integrity in school
consultation and prereferral interventions. School Psychology
Review, 18, 37-50.
Hale, A. D., Skinner, C. H., Winn, B. D., Oliver, R., Allin, J. D.,
& Molloy, C. C. M. (2005). An investigation of listening
and listening-while-reading accommodations on reading

68

HAWKINS et al.
comprehension levels and rates in students w ith emotional
disorders. Psychology in the Schools, 42, 39-51. doi: 10.1002/
pits.20027

Hancock, C. M. (2002). Accelerating reading trajectories: The effects of

dynamic research-based instruction. Dissertation Abstracts
International, 63 (06), 2139A. (UMI No. 3055690) Hawkins,
R. O., Musti-Rao, S., Hale, A. D., McGuire, S., & Hailley, J.
(2010). Examining listening previewing as a classwide strat
egy to prom ote reading comprehension and vocabulary.
Psychology in the Schools, 47, 903-916. doi: 10.1002/pits.20513
Individuals w ith Disabilities Education Improvem ent Act of 2004,
Pub. L. No. 108-446. International Dyslexia Association.
(2010). Knowledge and practice standards for teachers of
reading. Retrieved from http://www.interdys.org/ewebeditpro5/upload/KPS3-l-12.pdf
Lentz, F. E., Allen, S. J., & Ehrhardt, K. E. (1996). The conceptual el
ements of strong interventions in school settings. School
Psychology Quarterly, 1 1 ,118-136.
Lionetti, T. M., & Cole, C. L. (2004). A comparison of the effects of two
rates of listening while reading on oral reading fluency and
reading comprehension. Education and Treatment of Children,
2 7 ,114-129.
Mathes, P. G., & Babyak, A. E. (2001). The effects of Peer-Assisted
Literacy Strategies for first-grade readers w ith and w ithout
additional mini-skills lessons. Learning Disabilities Research
and Practice, 16, 28-44.
M organ, P. L., & Sideridis, G. D. (2006). Contrasting the effectiveness
of fluency interventions for students w ith or at-risk for learn
ing disabilities: A multi-level random coefficient modeling
meta-analysis. Learning Disabilities Research & Practice, 21,
191-210.
National Reading Panel (NRP). (2000). Teaching children to read: An ev
idence-based assessment of the scientific research literature on read
ing and its implications for reading instruction. Bethesda, MD:
National Institute of Child Health and H um an Development.
Rasinski, T. V. (1990). Effects of repeated reading and listening-whilereading on reading fluency. Journal of Educational Research, 83,
147-150.
Read Naturally. (2012a). Read Naturally Live Fidelity Checklists.
Retrieved from http://www.readnaturally.com/knowledgebase/documents-and-resources/23/337

EFFICIENCY OF READING INTERVENTIONS

69

Read Naturally. (2012b). Read Naturally Price List. Retrieved from

http://www.readnaturally.com/pdf/m OrderForm .pdf
Rose, T. L. (1984). The effects of two prepractice procedures on oral
reading. Journal of Learning Disabilities, 17, 544-548.
Rose, T. L., & Sherry, L. (1984). Relative effects of two previewing
procedures on LD adolescents' oral reading performance.
Learning Disability Quarterly, 7, 39-M4.
SABIS Educational Services. (2012). Curriculum. Retrieved from
http://w w w .sabis.net/educational-system s/educational-approach/curriculum
Schmitt, A. J., McCallum, E., Hale, A. D., Obeldobel, E., & Dingus, K.
(2009). Can text-to-speech assistive technology improve the
reading comprehension of students with severe reading and
emotional disabilities? Journal of Evidence Based Practices for
Schools, 10, 95-119.
Shapiro, E. S. (2004). Academic skills problems: Direct assessment and in
tervention (3rd ed.). New York: Guilford Press.
Shinn, M. R., & Shinn, M. M. (2002). AlMSweb training workbook:
Administration and scoring of reading maze for use in general out
come measurement. Eden Prairie, MN: Edformation, Inc.
Skinner, C. H. (2008). Theoretical and applied implications of precisely
measuring learning rates. School Psychology Review, 37,309-314.
Skinner, C. H., Belfiore, P. J., & Watson, T. S. (2002). Assessing the
relative effects of interventions in students with mild disabili
ties: Assessing instructional time. Journal of Psychoeducational
Assessment, 20, 346-357.
Skinner, C. H., Cooper, L., & Cole, C. L. (1997). The effects of oral
presentation previewing rates on reading performance.
Journal of Applied Behavior Analysis, 30, 331-333. doi: 10.1901/
jaba.1997.30-331
Skinner, C. H., Robinson, D. H., Adamson, K. L., Atchison, L. A., &
W oodward, J. R. (1998). Effects of different listening-whilereading rates on comprehension in secondary students with
reading deficits. Special Services in the Schools, 13,115-128.
Skinner, C. H., Smith, E. S., & McLean, J. E. (1994). The effects of inter
trial interval duration on sight-word learning rates in children
with behavioral disorders. Behavioral Disorders, 19, 98-107.
Spectrum K12 School Solutions. (2010). Response to intervention (RTI)
adoption survey 2010. Retrieved from w w w .spectrum kl2.
com

70

HAWKINS et al.

Therrien, W. J. (2004). Fluency and comprehension gains as a result of

repeated reading. Remedial and Special Education, 25, 252-261.
United States Departm ent of Education, Institute of Education
Sciences (2009). Assisting students struggling w ith reading:
Response to Intervention (Rtl) and muti-tier intervention in
the prim ary grades. Washington, D.C.: Author.
United States Departm ent of Education, Institute of Education
Sciences (2012). W hat works clearinghouse: Peer-assisted
learning/literacy strategies. Washington, D.C.: Author.
Vannest, K. J., & Parker, R. I. (2010). M easuring time: The stability of
special education teacher time use. Journal of Special Education,
44(2), 44-106.

Copyright of Education & Treatment of Children is the property of West Virginia University
Press and its content may not be copied or emailed to multiple sites or posted to a listserv
without the copyright holder's express written permission. However, users may print,
download, or email articles for individual use.