You are on page 1of 15

A Comparison of Three Instructional Methods for

Teaching Math Skills to Secondary Students With


Emotional/Behavioral Disorders

Glenna Billingsley, Brenda Scheuermann, and Jo Webber


Texas State University-San Marcos

ABSTRACT: The purpose of this study was to determine the most effective of three instructional
methods for teaching mathematics to secondary students with emotional and behavioral disorders. A
single-subject alternating-treatments research design was used to compare three instructional
methods: direct teach, computer-assisted instruction, and a combination of both methods. Disability
label, age, and IQ were related to learning outcomes. In addition, although the combined method
was more effective for some learners, variables such as attendance and motivation, as well as IQ,
comorbid conditions, age, and number of years spent in school, affected learning outcomes.

& The educational prognosis for students sive math instruction for students with EBD are
receiving special education services for emo- still lacking. These authors found only five
tional and behavioral disabilities (EBD) is poor. studies in the past decade that focused on
Compared with other populations of students explicit mathematical instruction for this pop-
with disabilities, students with EBD have ulation.
higher rates of academic failure, grade reten- The research base delineating effective
tion, absences, suspension, and dropping out instruction for students with EBD is limited,
of school (Kauffman, 2005; Wagner, Newman, but generally consistent, indicating that direct,
Cameto, & Levine, 2006; Walker, Ramsey, & teacher-led, explicit instruction is most likely
Gresham, 2004; Webber & Plotts, 2008). to produce desired learning and behavioral
Given the 43% to 56% dropout rate for outcomes for students with EBD (Forness,
students with EBD (Landrum, Brubaker, Kat- Kavale, Blum, & Lloyd, 1997). Specifically,
siyannis, & Archwamety, 2004), a high prob- instruction for students with EBD should
ability for a shortened educational experience, incorporate a direct teach or direct instruction
it is critical that these students receive effective approach that is explicit and clear, presents
educational programs with rigorous, research- material in a structured and systematic fashion,
based teaching practices as mandated by provides daily review of previously learned
Public Law 107-110, the No Child Left Behind concepts, provides sufficient supports in the
Act (NCLB). Furthermore, because school early stages of learning, provides high levels of
failure predicts dropout, careful attention to opportunities to respond to ensure maximum
designing the most effective instructional student engagement in learning activities, and
environments for students with EBD may provides repeated practice opportunities (Gun-
actually result in students remaining in school ter, Hummel, & Venn, 1998; Martella, Nelson,
longer. & Marchand-Martella, 2003; Scott & Shearer-
The NCLB and related state mandates to Lingo, 2002; Sutherland & Wehby, 2001; Yell,
increase participation and accountability in 2009). Academic instruction using a direct
mathematics by students with disabilities have teach approach has been associated with
spurred organizations such as the National increased academic gains for students with
Council of Teachers of Mathematics to con- EBD (Gunter, Coutinho, & Cade, 2002; Pierce,
sider ways in which these students can be Reid, & Epstein, 2004).
accommodated in rigorous, higher-level math Today’s teachers are urged to integrate
acquisition (National Council of Teachers of technology into their lessons to enhance
Mathematics, 2003). Yet, as Templeton, Neel, instructional presentation and increase student
and Blood’s (2008) meta-analysis of math motivation. The use of computers for instruc-
intervention for students with EBD indicates, tion, called computer-assisted instruction
data to help teachers implement comprehen- (CAI), is an appealing concept but has only

4 / November 2009 Behavioral Disorders, 35 (1), 4–18


minimal research support for students with of these students were served in a self-
learning and behavioral difficulties. Hughes contained setting in an urban, public high
and Maccini (1997) recommended CAI for school. Table 1 provides demographic infor-
improving math performance in students with mation for all participants. In addition to
learning disabilities, based on their review of gender and ethnicity, the table provides age,
research in this area, but whether CAI is an enrolled grade level, and a measure of grade
effective instructional medium for students level equivalency (GLE) in math from the
with EBD remains to be seen. Woodcock-Johnson Psycho-Educational Bat-
Certainly, CAI has intuitive appeal, given tery (Woodcock & Johnson, 1977) for each
the structure inherent in CAI, the fact that CAI student. The column titled IDEA denotes the
software can incorporate effective instructional disability by which the student qualified for
design principles, and the potential motiva- special education under the Individuals with
tional aspects of CAI. In a study by Dawson, Disabilities Education Act (IDEA). Finally, IQ
Venn, and Gunter (2000), computer-based information is included where available; IQ
reading models resulted in improved reading information was unavailable for 2 students
performance compared with a no-reading (Chad and Tyrene).
model condition. However, when the reading The first author served as the teacher of
model was presented by the teacher, student record for these students. This author is a certified
results were higher than the computer-based special education teacher with more than 20
approach. Therefore, it is necessary to evaluate years of experience teaching students with EBD
the efficacy of computer-based instruction for in self-contained, resource, and inclusion set-
students with EBD before making blanket tings. To minimize the risk of internal validity
recommendations for use with this population. errors, reliability checks were conducted by
The present study was undertaken after independent observers to ensure reliability of
students in a self-contained high school the dependent measures and fidelity of imple-
classroom for students with EBD were provid- mentation of the instructional conditions.
ed with a computer curriculum with which to
provide instruction. The classroom teacher
Research Design
questioned the efficacy of using CAI with this
population but found minimal research rec- The study occurred in four phases. The first
ommending its use. Given the significant cost phase was the identification of 10 target math
of the computerized instructional system, and objectives that students did not master on the
given the fact that NCLB and the Individuals math subtests of the WRAT-3 and/or three
with Disabilities Education Improvement Act curriculum-based quizzes covering the 10
require evidence-based practices, an evalua- objectives. The three curriculum-based quiz-
tion of the efficacy of computer-based instruc- zes served as a baseline measure. During the
tion was in order. second phase, students from three special
The purpose of this study was to determine education classes were taught nine target math
the most effective of these three instructional skills using three different instructional ap-
methods for teaching mathematics to second- proaches. Each instructional condition was
ary students with EBD: (a) teacher-directed used for 1 week, to teach one target skill.
instruction (direct teach) as recommended in The instructional condition for each group
the literature, (b) CAI yet to be shown effective alternated randomly each week, resulting in
for students with EBD, and (c) a combination students in each of the three groups being
of these approaches. In addition, variables exposed to each instructional method three
such as IQ, disability label, grade level, and
times. During the third phase, the best
age were analyzed for their impact on student
treatment phase, individual student scores for
learning.
each condition were averaged, and the best
treatment was determined to be the condition
Method in which most participants overall had the
highest mean score. The 10th math objective
Participants and Setting was taught to all classes using the best
treatment method. The final phase was a
Participants included 10 special education postinstruction probe, which consisted of a
students in Grades 9 to 11 whose behavioral curriculum-based quiz to determine mastery of
needs necessitated intensive interventions. All the 10 target objectives.

Behavioral Disorders, 35 (1), 4–18 November 2009 / 5


TABLE 1
Student Demographic Information
Age
Student Gender Ethnicity (Years) Grade GLE IDEA IQ
Clay M W 17 11 6.0 TBI, LD 87 (WISC-III; 2/98)
Crane M W 17 11 11.8 ED, OHI, LDa 111 (WASI; 4/03)
Lupita F H 16 11 6.0 ED, LD 83 (WISC-II; 10/02)
Thaddeus M W 16 11 5.5 ED 101 (WISC-III; 10/02)
a
Manny M H 16 10 3.8 ED, LD 92 (WISC-III; 5/00)
Bryan M H 16 9 6.0 ED, LDa 115 (KABC; 4/02)
b
Chad M W 16 10 4.8 ED
Junior M H 16 10 4.8 ED, LDa 84 (WISC-III; 12/02)
c
Tyrene M AA 15 10 6.2 ED
Hank M H 14 9 5.4 LD,a ED 102 (WISC-III; 3/01)

Note. Pseudonyms have been used in place of students’ real names. Ethnicity abbreviations: H 5 Hispanic, W 5 White, AA 5 African
American. IDEA abbreviations: ED 5 emotional disturbance; OHI 5 other health impairment; LD 5 learning disability; TBI 5
traumatic brain injury; WISC II 5 Wechsler Intelligence Scale for Children, 2nd edition; WISC III 5 Wechsler Intelligence Scale for
Children, 3rd edition; WASI 5 Wechsler Abbreviated Scale of Intelligence; KABC 5 Kaufman Assessment Battery for Children.
aStudents who are LD in areas other than mathematics.
bNo testing available; examiner relied on teacher report as ‘‘very bright.’’
cNo testing available; examiner reported ‘‘normal intelligence.’’

Response Measure ATD is one in which two or more treatments or


conditions rapidly alternate to compare their
The dependent measure of math learning effectiveness and is ideally suited to multiple-
was teacher-constructed, curriculum-based as- intervention comparisons when a withdrawal
sessments (CBAs) used across research condi- design is not appropriate.
tions and as the postinstruction probe. First,
students completed three different versions of a Procedures
teacher-constructed, CBA to establish baseline
levels of performance. The baseline versions of A behavior management system was in use
the quiz evaluated students’ proficiency on during the study that reinforced compliance
nine selected math objectives linked to mate- with classroom rules and participation. For
rial that was taught during the 9 weeks of appropriate behavior, students earned points
intervention, plus one additional objective that with which they could purchase items from a
was covered during the best treatment phase. classroom store. Furthermore, students lost
Each baseline measure consisted of 20 ques- privileges for inappropriate behavior.
tions covering 10 math objectives, or two
questions per objective. Students were provid- Baseline
ed one 55-min class period in which to
complete each quiz. Calculators were not The first phase of the study served to
permitted at any time during the study, and establish a baseline, which included adminis-
the only assistance provided on quizzes was tration of the WRAT-3 and three curriculum-
encouragement for students to put forth their based measurements that covered 10 math
best effort. objectives that the WRAT-3 assessment indi-
The independent variable in this study was cated most students had not mastered. The
the method of instructional delivery: direct WRAT-3 was administered on the first day of
teach, CAI, and a combination of those two the baseline phase of the study followed by
methods. An alternating-treatments single-sub- three CBAs; students completed one baseline
ject research design (ATD) was employed to assessment per day.
determine which of the three instructional Students’ pretest results from the WRAT-3
models was most effective for teaching math- were analyzed to identify specific skill areas
ematics to secondary students with EBD. that would become targets for instruction. This
According to Barlow and Hersen (1984), an analysis was accomplished by first ordering the

6 / November 2009 Behavioral Disorders, 35 (1), 4–18


objectives reflected in skills assessed on the OdysseyWareH CAI to facilitate their learning
Written Arithmetic subtest. The ordering was to add fractions. Each respective unit of
done in terms of item difficulty and necessary instruction was designed to be covered within
prerequisite skills. The skill areas that the a period of 3 to 4 class days.
majority of students answered incorrectly were The following procedures used during the
then targeted for instruction. intervention phase were specific to the instruc-
Ninety percent of the students incorrectly tional condition and were rigidly followed.
answered problems dealing with one- and two- Direct teach. The following direct teach
digit division. Fifty percent were unable to components, as described by Scheuermann
simplify fractions, 90% were unable to add or and Hall (2008), were used during the
subtract fractions without regrouping, and 100% teacher-mediated instruction conditions. First,
were unable to subtract fractions requiring the designated math objective was written on
regrouping. No students were able to correctly the board each day. Second, the teacher
solve problems requiring multiplication and introduced the lesson by connecting new
division of fractions. Ninety percent were unable material to prior learning, showing a correctly
to correctly multiply decimals. Finally, no finished problem, and describing how the skill
students were able to convert fractions, decimal, could be used in real-world applications.
and percentages or determine the percent of a Third, the teacher presented the lesson in a
number. Hank did not get any of these questions variety of formats including lecture, modeling,
correct; Crane got the most answers correct, but and interaction with math manipulatives.
he missed 47% of them. Fourth, students practiced the skill by per-
Based on this analysis, 10 math objectives forming problems on the board or individual
were targeted for instruction. Those 10 objec- dry-erase response boards and were asked to
tives in the order they were taught were as verbally demonstrate the process or strategy
follows: division with a one-digit divisor; for performing the work. Fifth, students
division with a two-digit divisor; simplifying completed additional problems on individual
fractions to lowest terms; multiplication of dry-erase response boards while the teacher
fractions; division of fractions; addition and checked for understanding and provided high
subtraction without borrowing of fractions; levels of response opportunities. Finally, the
subtraction of fractions involving regrouping; teacher provided another example of the type
multiplication of decimals; conversion of of problem being studied and left that
fractions, decimals, and percents; and finding problem on the board before closing with a
percent of numbers. summary of the lesson, its relevance, and the
Crane was the only student who correctly importance for building a base for future
answered any problems involving fractions, but learning. Following the direct teach portion
he missed some of the more basic, untargeted of the lesson, students engaged in indepen-
items including column addition and subtrac- dent practice of the skill taught. The number
tion without borrowing. These were perhaps of independent problems assigned matched
careless errors, typical of students with learning the number of questions that were presented
disabilities (LD) and attention deficit hyperac- to students during CAI conditions for the sake
tivity disorder, both conditions with which of consistency. Further assistance in the form
Crane had been diagnosed. of additional explanation or a different strat-
egy such as a flowchart was provided as
Training needed.
Computer-assisted instruction. The follow-
The second phase, the intervention, pro- ing procedures were followed during times the
vided each student exposure to each instruc- class was using CAI. For this study, the
tional modality three times over a period of 9 software curriculum OdysseyWareH was used.
weeks. Interventions were randomly assigned OdysseyWareH is a multimedia-enhanced CAI
across weeks for each of three math classes; all curriculum published by Pathway PublishersH
classes covered a common objective. For that includes diagnostic features; individual-
example, although students in all classes were ized, self-paced instruction; and teacher man-
learning to add fractions, students in the first- agement utilities. After beginning-of-class pro-
and sixth-period math classes were exposed to cedures, students were dismissed to their
the direct teach approach, and students in the assigned computers. Headphones were avail-
fourth-period math class were exposed to the able, but students were reminded to use them

Behavioral Disorders, 35 (1), 4–18 November 2009 / 7


only for listening to the instruction-related board or used interactive videos infused into
material. All activities and assignments for OdysseyWareH. When needed, students were
the week were listed on a chart posted in the assisted at their computer, just as a teacher
room. Students went to the Odyssey menu, would assist students with a problem from a
located their name, selected ‘‘Math,’’ and textbook. The utilities of the program were
participated in the assigned lessons. Although adjusted so that once a student entered the
the OdysseyWareH curriculum offers prepared answer, it was immediately assessed, and
courses appropriate to a particular grade level, missed problems did not cycle through again,
the lessons assigned for this study were as was the case during the CAI condition.
customized based on specific objectives to Grades were taken just as grades would be
be taught. The program’s scope and sequence taken from textbook problems or worksheets.
were used to create customized lessons and On Friday of each week, all treatment
activities on a particular math objective at groups completed a teacher-made, criterion-
grade levels appropriate for each student. The referenced test relating to the specific objec-
module created for each objective was a broad tive covered during the week. If students were
sampling across grade levels and ranges of absent or refused to participate, they were
difficulty. Each module included an introduc- allowed to retake intervention-phase quizzes
tory lesson in which the skill was presented until instruction began on the next objective.
and four to six additional lessons for practice Regardless of instructional method, all students
were provided. Students worked through the took the same pencil-paper quiz. Students’
lessons and activities at their own pace, percentages of correct responses on this
although a recommended schedule suggested weekly quiz (dependent variable) were graph-
the completion of 20 to 30 problems during a ically displayed for each individual. Partici-
50-min class period. Students were encour- pating students’ data from each class produced
aged to work on OdysseyWareH independent- a unique data graph, resulting in 10 single-
ly. Those who became too frustrated were subject studies.
allowed to leave the computer for a brief The instructional condition (or treatment)
break. Students were allowed to get tutoring for each class alternated in a random sequence
from the teacher on problems similar to the for each math objective. Each objective was
ones on which they were working but not the presented for 1 week using a different instruc-
exact problem. The teacher utilities on Odys- tional method for each class; then a new
seyWareH allow the teacher to set the level of objective and new methodology were intro-
feedback the student received. For students duced. This means that students were exposed
participating in CAI, immediate feedback as to to each instructional method three times.
the accuracy of the answer was provided. Instruction continued in this manner for 9
Incorrect problems recycled themselves until a weeks, with each group exposed to each
mastery level of 100% was achieved. If methodology three times on various objec-
students did not finish the assigned lessons tives. Week 10, during which students were
by the end of the period, a completion grade taught the skill of finding the percent of a
was assigned. Omitted problems were counted number, was part of the third phase, the best
as incorrect, just as they would on paper- treatment phase.
pencil tasks. The first-period class was taught simplify-
Combination strategy. In the combination ing fractions, division of fractions, and con-
direct teach and CAI condition, students version of fractions, decimals, and percents
received direct instruction from the teacher using direct teach. This class was taught
on the assigned material. The direct teach division with a two-digit divisor, multiplication
components, as described previously, were of fractions, and subtraction of fractions
used until time for independent practice. At involving regrouping with the CAI method.
this point, students were dismissed to their Finally, they were instructed in division with
computers, where they were instructed as to a one-digit divisor, addition and subtraction
which section of the OdysseyWareH program of fractions without regrouping, and multipli-
was to be used for the day. In this approach, cation of decimals using the combined
students used the OdysseyWareH program as a method.
textbook, with further explanations being The fourth-period class was taught division
offered by the teacher. For example, the with a one-digit divisor, multiplication of frac-
teacher reviewed sample problems on the tions, and addition and subtraction of fractions

8 / November 2009 Behavioral Disorders, 35 (1), 4–18


without regrouping using direct teach. They study where they were told about the nature of
were taught division with a one-digit divisor, the study, their role in checking the fidelity of
division of fractions, and subtraction of frac- quiz grades, and assessing the presence of the
tions involving regrouping using CAI. They critical components of each method.
were taught simplifying fractions, multiplica- The critical components of each instruc-
tion of decimals, and conversion of fractions, tional method were thoroughly explained and
decimals, and percents using the combined modeled. For evaluating fidelity of implemen-
method. tation of each of the instructional methods,
The sixth-period class was taught simpli- observers were provided with a checklist of the
fying fractions, division of fractions, and critical components of the three types of
subtraction of fractions involving regrouping instruction (see the appendix). As critical
using direct teach. They were taught division components were modeled, observers identi-
with a two-digit divisor, multiplication of fied those components on these checklists.
fractions, and multiplication of decimals using Finally, the two observers were provided a
CAI. Lastly, they were instructed in division schedule of times during which study-related
with a one-digit divisor, addition and subtrac- instruction would be occurring. The observers
tion of fractions without regrouping, and selected five occasions to conduct reliability
conversion of fractions, decimals, and percents checks; these observation times were not
using the combined method. communicated to the researcher.
Follow-up. The third phase provided Scoring reliability on dependent measures
instruction in one additional math objective was assessed by having one of the two
using the modality data shown to be the most observers score a random sampling of CBAs.
effective overall across students. The purpose The observer inspected 20% of the baseline
for this best treatment phase, which followed quizzes and 25% of all intervention, best
the 9th week of the intervention, was to treatment, and posttest quizzes. Interobserver
demonstrate the superior effectiveness of one reliability for baseline measures was 100%,
condition over any other. This was determined and checks of weekly quizzes during the
by averaging scores for each condition. The intervention and follow-up phases yielded an
best treatment was determined to be the agreement level of 95%.
condition in which most students overall had During fidelity observations, each observ-
the highest mean score. er used his or her checklist to record those
Following the baseline, intervention, and essential components that they observed dur-
best treatment phases, a postinstruction probe ing the period. There were 15 components for
was given to assess each student’s progress the direct teach method, 6 components for
compared with baseline. The postinstruction CAI, and 16 components for the combined
probe consisted of a teacher-made, curricu- method. The observers attended two sessions
lum-based 20-item assessment, similar to the in which the combined method was in use,
ones used in the baseline phase. two sessions of direct teach, and one occasion
when CAI was used. Thus, there were a total of
Procedural Integrity and Observer 68 opportunities for agreement. The percent-
Reliability age of agreement for critical components of
each method was calculated by dividing the
This study was implemented by the total number of agreements by the total
teacher who was also the researcher. To number of items on the checklist. The observ-
minimize the risk of internal validity errors, ers agreed on 64 of the 68 critical components,
checks were conducted by independent ob- resulting in interobserver agreement of 94.1%.
servers to ensure scoring reliability on the Agreement of 90% or higher is considered a
dependent measures and fidelity of implemen- reliable measure (Alberto & Troutman, 2006;
tation of the instructional conditions. Bailey & Burch, 2002). Next, the observers
The two special education department were instructed to determine which of the
chairs at the campus served as the two three methods was being used for instruction
observers. One was a former teacher with 5 during the observation period by attending to
years of experience teaching students with activities occurring during the observation.
EBD; the other had 3 years of experience They were able to correctly identify the
teaching math in resource settings. Observers’ instructional method being used in five out of
training was held prior to the beginning of the five occasions, or 100%.

Behavioral Disorders, 35 (1), 4–18 November 2009 / 9


Data Analysis used as baseline measures, as well as the mean
of these assessments; scores ranged from 0% on
The most common form of analysis for all assessments to a passing score of 70% or
single-subject design is visual inspection to higher. Clay’s percentage scores for the baseline
examine data paths for lack of overlap. The CBAs were 25%, 30%, and 15%, with a mean of
degree of differential effect produced by two or 23%. Crane scored 70%, 80%, and 60%, with a
more different independent variables is deter- mean of 70%. Lupita scored 5%, 0%, and 10%,
mined by the amount of vertical difference with a mean of 5%. Thaddeus scored 0%, 5%,
between data paths (Cooper, Heron, & Heward, and 0%, with a mean of 2%. Manny scored 0%
2007). Furthermore, according to Parker, Van- on all assessments; thus, his mean score was
nest, and Brown (2009), determining effect size also 0%. Bryan scored 10% on the first
is a useful supplement to visual analysis. Thus, assessment and 5% on the third assessment.
the effect size for each condition was computed He was absent and did not make up the second
using standard mean difference (SMDALL), the assessment, yielding an 8% mean. Chad scored
recommended method to complement visual 5% on the first assessment and 10% on the third
analysis for single-subject research designs assessment, yielding an 8% mean. Chad was
(Olive & Smith, 2005). To calculate (SMDALL), absent and did not make up the second
the baseline mean is subtracted from the mean of assessment. Hank scored 0%, 5%, and 5%,
the intervention phase. This number is then with a mean of 3%. Junior scored 5%, 0%, and
divided by the standard deviation of the baseline 5%, with a mean of 3%. Finally, Tyrene scored
scores. Because the intervention phase was 20%, 5%, and 0%, with a mean of 8%.
composed of three instructional conditions, this Each participant’s individualized educa-
resulted in three distinct calculations to attain an tion plan (IEP) specified 70% as the mastery
effect size for each condition. Lastly, the mean criteria for math skills. Only two students
effect size for each condition overall was found reached mastery on any of the skills assessed
by calculating the mean of all students’ baseline during the WRAT-3 pretest. Clay displayed
scores, subtracting that from the mean for each mastery in multiplication of decimals and in
condition across all students, and dividing by the conversion of fractions, decimals, and per-
standard deviation of all baseline scores. cents. However, he was unable to answer any
Analysis of the CBAs followed the con- questions involving fractions and only 25% of
ventions of single-subject research designs. questions involving division on the CBA used
Quantitative analysis consisted of visual in- for baseline. Crane demonstrated mastery in 6
spection of data to determine which method of the 10 objectives: division of one- and two-
was most effective (i.e., high scores on digit divisors, addition and subtraction of
quizzes) for mastery of math objectives. fractions including those requiring regrouping,
Students’ individual quiz grades during base- division of fractions, and conversion of frac-
line or intervention phases were calculated as tions, decimals, and percents.
a percentage of correct responses and graphed
with each instructional condition plotted as a
Intervention
separate data path. The greater the separation
of data points between data paths, the stronger Results of each student’s performance on
the indication of differentiation of treatment. the CBAs for each treatment condition are
Descriptive statistics, including mean, mode, presented in Figures 1 and 2. The graphs are
and variance, were calculated for individuals, presented in order by class period and
groups, and overall to determine which descending order of age. Clay, Crane, Lupita,
method was associated with the highest scores,
and Thaddeus were in first period. Manny was
as well as to determine whether students
in fourth period, and Bryan, Chad, Junior,
exposed to a particular methodology for a
Tyrene, and Hank were in sixth period.
particular objective received similar scores.
Absences and refusal to work resulted in
a number of missing data points for most of
Results the participants. Students were encouraged to
make up missing quizzes, but they often
Baseline lacked the motivation to do so. This was the
case with Manny, Bryan, and Crane; each
Results were calculated as the percentage missed one intervention phase quiz and did
of correct responses for each of the three CBAs not choose to make up the quiz despite

10 / November 2009 Behavioral Disorders, 35 (1), 4–18


Figure 1. Percentage of correct responses for Clay, Crane, Lupita, Thaddeus, and Manny.

having the opportunity and incentive to do this intervention score was less than his
so. Clay missed 3 consecutive weeks of baseline score, and Manny, whose baseline
instruction and quizzes during this phase mean was 0, making calculation of (SMDALL)
because he was expelled to a disciplinary impossible. Thus, the various interventions did
alternative education placement. Further- have a positive effect over baseline, but
more, some students refused to work even variables other than instructional conditions
though they were aware of behavioral conse- appeared to affect individual student perfor-
quences (e.g., loss of privileges) and academ- mance.
ic consequences (e.g., lowered grades) that None of the instructional conditions were
accompanied that choice. Tyrene had one clearly superior for all students, as evidenced
episode of refusing to take an intervention by the fact that data paths for most students
phase quiz, whereas Thaddeus refused to take showed some degree of overlap across condi-
CBAs for 3 consecutive weeks. tions. Likewise, effect size, which provided
data identical to visual analysis, showed only
Mean Scores for Conditions that the intervention had a very high effect on
student performance. One condition was not
Standard mean difference (SMDALL) was significantly superior to another.
calculated by subtracting the mean of the When mean scores for each condition are
baseline from the mean of each treatment considered, scores for the combined method
condition and dividing by the standard devi- were higher for five students (Clay, Lupita,
ation of the baseline values, as suggested by Manny, Chad, and Hank). Two additional
Olive and Smith (2005). Again, no one students (Crane and Bryan) had a mean that
condition was clearly superior for all students. was highest for the combined method and one
Table 2 shows the mean score for each other method, CAI and direct teach, respec-
condition, as well as the effect size for all tively. Thaddeus and Tyrene had a higher
students across conditions. The effect size mean score during the direct teach method,
ranged from .83 to 1.00, except for Clay and Junior had a higher mean score during the
during the CAI method, where the mean of CAI method.

Behavioral Disorders, 35 (1), 4–18 November 2009 / 11


Figure 2. Percentage of Correct Responses for Bryan, Chad, Junior, Tyrene, and Hank.

A passing rate of at least 70%, the scores. Clay, Crane, Lupita, Bryan, Chad,
minimum standard required on each student’s Hank, and Tyrene had an average score in
IEP, can also be considered a measure of the passing range when the combined method
practical significance. The mean score was was used. Manny did not receive any passing
passing during the direct teach method for 5 grades on quizzes during any condition.
students (Clay, Crane, Thaddeus, Bryan, and Direct teach. Direct teach appeared to be
Tyrene). Only Crane received passing scores more effective for students with above-average
during the CAI method; Junior scored highest IQ than for participants with lower cognitive
during CAI but did not receive any passing levels. Although an IQ score was not available

TABLE 2
Mean Scores, Standard Deviation, and Effect Size (ES) for Each Instructional Condition
Mean
Direct Mean Mean Baseline Direct
Student Teach CAI Combination Mean SD Teach ES CAI ES Combination ES
Clay 70 10 80 23 6.24 .97 2.72 .98
Crane 90 95 95 70 8.16 .77 .83 .83
Lupita 60 53 93 5 4.08 .99 .99 1.00
Thaddeus 70 40 53 2 2.36 1.00 .99 1.00
Manny 5 27 42 0 0
Bryan 73 50 73 8 2.5 1.00 .99 1.00
Chad 58 58 80 8 2.5 1.00 1.00 1.00
Junior 47 58 53 3 2.36 .99 1.00 1.00
Tyrene 95 58 78 8 8.5 .98 .95 .97
Hank 38 67 78 3 2.36 .99 1.00 1.00

12 / November 2009 Behavioral Disorders, 35 (1), 4–18


for Tyrene, the examiner did state that he had 19.94. The mean for each condition across all
average intelligence. Bryan had the highest IQ students was 60.6, 51.6, and 72.5 for direct
of all participants (115), and Thaddeus was in teach, CAI, and the combined method, re-
the average range with a score of 101. Bryan’s spectively. The resulting effect size was found
quiz average for direct teach was the same as to be .696 for CAI, .767 for direct teach, and
his mean average during the combined meth- .83 for the combined method. An effect size of
od, which exceeded his mean during the CAI .2 is generally considered a small effect size, .5
method. is considered to be a medium effect size, and
Furthermore, direct teach appeared to be .8 is considered to be a large effect size.
more effective for students who were identified
only as emotional disturbance. Thaddeus and Best Treatment Phase
Tyrene were the only students who qualified
with the singular diagnosis of emotional The best treatment phase is instituted in an
disturbance. Both had a higher mean score alternating-treatments design to make a stron-
and effect size for direct teach. ger case for the existence of a functional
Computer-assisted instruction. Junior had a relationship (Alberto & Troutman, 2006). In the
higher mean score on quizzes while using the present study, the 10th math objective, con-
CAI method. Although his average was not in the cepts of percents, was taught using the
passing range, he did pass two of the quizzes combined method. The combined method
during the CAI method. Crane scored the same was identified for the best treatment phase
for CAI and the combined method. Crane had a based on mean scores on CBAs, which were
strong preference for the CAI method but highest for the combined method for 7 of the
received passing scores during all conditions. 10 participants (Clay, Crane, Lupita, Manny,
Combined method. The combined method Bryan, Chad, and Hank). Because students
was clearly superior for 2 students (Clay and within class periods could not logistically be
Lupita), the only students labeled emotional taught using differing methods, the 10th
disturbance and LD in math calculation. With objective was taught to all classes using the
the exception of Junior, all students with a combined method.
comorbid learning disability showed the com- Student scores on the CBA at the end of
bination method to be superior either by mean the best treatment phase were as follows: Clay,
score for each condition or higher number of 0%; Crane, 95%; Lupita, 85%; Thaddeus, 5%;
quizzes passed. Crane and Bryan scored equally Manny, 50%; Bryan, 40%; Chad, 0%; Junior,
well during the combined method as when 75%; Tyrene, 70%; and Hank, 85%.
another condition was being used. Chad’s and Of those 7 students for whom the combi-
Hank’s averages for conditions were higher nation method produced the highest mean
during the combined method, and they passed scores, 4 (Crane, Lupita, Manny, and Hank)
more quizzes during the combination method. scored within 10 points of that mean score on
Although Manny received no passing score the 10th objective. For these students, perfor-
during any condition, his quiz average was mance was consistent with previous scores for
higher during the combined method. this instructional method.
The combined method seemed to be more Clay and Chad, for whom the combined
suitable for younger learners as well. Students method had appeared more effective during
who had both higher mean scores and higher the intervention phase, as evidenced by a
passing rates during only one condition higher mean score and as many or more
(Lupita, Hank, and Chad) were younger than passing scores during the combined method,
16 years old and tended to have lower GLEs. did not correctly answer any questions on the
Effect size for all conditions. The following quiz during the best treatment phase. Further-
are the mean scores on quizzes following more, Junior and Tyrene had a higher mean
instruction in direct teach, CAI, and the score during a condition other than the
combination method, respectively: Clay (70, combined method, yet both received passing
10, 80), Crane (90, 95, 95), Lupita (60, 53, 93), scores during this phase. In fact, Junior did not
Thaddeus (70, 40, 53), Manny (5, 27, 42), Bryan pass any quizzes during the intervention phase
(73, 50, 73), Chad (58, 58, 80), Junior (47, 58, when the combined method was used.
53), Tyrene (95, 58, 78), and Hank (38, 67, 78). The postinstruction probe, a teacher-made
The total baseline mean was 13.0, and the quiz similar to the baseline measures, was
standard deviation of the baseline scores was administered 2 weeks following the best

Behavioral Disorders, 35 (1), 4–18 November 2009 / 13


treatment phase. Table 3 provides a compar- TABLE 3
ison of mean baseline scores for each student Comparison of Baseline Scores to
with scores on the postinstruction probe. All Postinstruction Probe
scores are reported as a percentage of correct
Baseline Postinstruction Percentage
responses. Student Mean Probe Increase
All students who took the postinstruction
Clay 23 40 +73.9
probe demonstrated improvement over their
baseline mean; however, only Crane achieved Crane 70 80 +14.2

mastery level of at least 70%, as set forth in his Lupita 5 50 +900


IEP. In this study, methodological conditions Thaddeus 1.7 20 +1,076.4
were adhered to closely, and students were a
Manny 0 5
encouraged only to try their best. In routine
classroom operations, a variety of strategies Bryan 7.5 15 +100

and alternative activities are used until a Chad 7.5 40 +433.3


student demonstrates mastery over a concept. Junior 3.3 25 +657.5
Tyrene 8.3 NA NA
Limitations and Implications for Future
Hank 3.3 15 +354.5
Research
Note. NA 5 not available due to absence.
Certain limitations of this study should be aPercentage increase cannot be calculated because the

considered. Single-subject design results, par- baseline mean was zero.


ticularly alternating-treatments design, can be
affected by how rapidly the dependent vari-
able responds to the intervention (in this case, useful for evaluating multiple interventions,
performance on math quizzes) as well as how may not be well suited for evaluating cumu-
many times the intervention (each instructional lative intervention effects.
method) is applied. In alternating treatments, Effect size data may also have been
the number of intervention occasions for each affected by the time constraints of the study.
intervention is, by virtue of the design, reduced Barlow and Hersen (1984) recommend a
compared with other research designs. In minimum number of three data occasions per
addition, the time constraints of typical class- intervention to determine effect size. For this
rooms pose a challenge for researchers. For study, precisely three data points were pre-
example, in this study, 1 week was lost to state scribed for baseline and for exposure to each
testing, another week to spring vacation, and a instructional method. These applications were
few days to school holidays. Finally, the calculated to fully encompass the entire
looming end to the school year prevented semester of instructional days. However, only
optimal application of the alternating-treat- 3 of the study participants completed all
ments design. Generally, in alternating-treat- prescribed data points for both baseline and
ments design, conditions are alternated at least intervention. Absences, suspensions, and re-
five times after clear differences have emerged. fusals, all common characteristics of this
However, given the reality of the end-of-year population, reduced the number of data points
schedule of this public school, it was not to less than the recommended minimum,
possible to do so. Ideally, future studies will possibly compromising effect size calcula-
allow for more exposure to each instructional tions; thus, results should be cautiously con-
condition. sidered. Although single-subject designs ac-
In addition, this study involved measure- commodate small sample sizes (Cooper et al.,
ment of acquisition of new learning in 2007), a study in an applied setting with this
mathematics, which is most likely dependent population may well present various real-life
on a cumulative effect of exposure to the limitations, especially with regard to time and
intervention. Such an intervention may pro- frequency of intervention occasions. Interven-
duce weak results after one application but tion schedules that allow for more frequent
stronger results after multiple, consecutive instructional applications might produce more
applications. Typically, acquisition of new robust effects. Planning for increased applica-
skills, particularly in math, is the result of tions of each intervention may also address the
repeated exposure to instruction. The nature of problem of student data missing for behavioral,
the alternative treatments design, although health, or other reasons.

14 / November 2009 Behavioral Disorders, 35 (1), 4–18


Intervention randomization posed another in mean quiz passing (70%) scores across all
limitation. Because of the small number of students (combined, 72.5; direct teach, 60.5;
intervention occasions, it would have been CAI, 51.6). Importantly, all students improved
better to randomize sequences of interventions their math quiz scores with the alternating
and assign them to classes rather than ran- treatments; however, only 1 student achieved
domizing the instructional method for each his targeted 70% mastery level. This failure to
class. Subsequently, an identical sequence of obtain mastery levels may not be a function of
interventions occurred for the first 6 weeks of each treatment’s potential effectiveness so
the study, possibly resulting in a weaker much as the research design limitations, time
research design. Future studies should address constraints, and individual student variables.
this limitation. Nevertheless, the combined method (large
Experimenter bias may be considered a effect size 5 .83) for teaching math to
limitation because the teacher conducted her secondary students with EBD seemed to result
own interventions. However, the motivation more often in better test performance than did
for this study was this teacher’s curiosity about the other two methods, although direct in-
which method would provide the best learning struction (medium effect size 5 .767) and CAI
outcomes for her students, and the researcher (medium effect size 5 .696) also showed
had extensive experience and regard for all positive results. CAI is often popular with
three instructional methodologies. To reduce teachers because it reduces teacher prepara-
this limitation, fidelity checks were conducted tion time and works well in self-contained
to monitor the teacher’s adherence to correct settings where students work at multiple
practices for each method. academic levels in a variety of coursework.
Finally, this study further illustrated the CAI can also assist special education teachers
complex interaction of the presenting charac- to coordinate their instruction with general
teristics of students with EBD and learning. education content. However, teachers may do
Comorbid conditions, IQ, age, and motiva- well to reconsider CAI as the sole source of
tional factors seemed to affect results regard- instruction; instead, additional instruction and
less of instructional method. The math perfor-
practice is recommended. Furthermore, stu-
mance results must also be considered in the
dents will have little opportunity to master
context of multiple absences, suspensions,
critical social skills if they spend large amounts
refusal to complete quizzes, and lack of
of time solely interacting with computers.
motivation to perform. Future studies may
In sum, secondary teachers of students
want to address the interactions among in-
with EBD may find, as was found in this study,
structional methods, student characteristics,
that any of these three instructional approach-
and preferences for a particular method.
es will likely result in math content acquisition
for their students given enough instruction over
Conclusions and Implications for a period of time. However, we would caution
teachers to resist generalized applications of
Best Practice
any of the strategies. In this era of widespread
Despite limitations, the results of this study placement of students with EBD into general
seem to support all of these interventions for education classes as a result of NCLB and
teaching math to secondary students with IDEA 2004 mandates for grade-level testing
EBD. However, no single treatment achieved and for highly qualified content teachers
best results for all the students. Therefore, we (Webber & Plotts, 2008), it is likely that these
might assume that application of any one of students may very well be exposed to singular
these strategies across students with EBD will instructional approaches. This study supports
probably not achieve universal positive learn- the notion that individualized determination
ing outcomes. Instead, as is the premise of for instructional interventions may serve stu-
special education, strategies should be chosen dents best. Finally, although the results of this
based on individual learning factors. study did not clearly identify a best method of
In this study, the combined CAI and direct instruction for teaching math to secondary
teach model resulted in higher math quiz students with EBD, it does illustrate that quality
scores for 7 of the 10 students; the direct teach research can be conducted by practicing
model resulted in better scores for 2 students, teachers by using a single-subject design.
and 1 student learned better through the CAI- Results of such research can augment instruc-
alone strategy. These results were also evident tional outcomes as well as contribute to the

Behavioral Disorders, 35 (1), 4–18 November 2009 / 15


sparse body of literature pertaining to effective Pierce, C. D., Reid, R., & Epstein, M. H. (2004).
academic instruction for this population. Teacher-mediated interventions for children
with EBD and their academic outcomes. Reme-
dial and Special Education, 25, 175–188.
REFERENCES Scheuermann, B., & Hall, J. (2008). Positive behav-
Alberto, P. A., & Troutman, A. C. (2006). Applied ioral supports for the classroom. Columbus, OH:
behavior analysis for teachers (4th ed.). Engle- Merrill Prentice Hall.
wood Cliffs, NJ: Prentice-Hall. Scott, T. M., & Shearer-Lingo, A. (2002). The effects
Bailey, J. S., & Burch, M. R. (2002). Research of reading fluency instruction on the academic
methods in applied behavior analysis. Thousand and behavioral success of middle school stu-
Oaks, CA: Sage. dents in a self-contained EBD classroom. Pre-
Barlow, D. H., & Hersen, M. (1984). Single case venting School Failure, 46, 167–175.
experimental designs: Strategies for studying Sutherland, K. S., & Wehby, J. H. (2001). Exploring the
behavior change (2nd ed.). New York: Perga- relationship between increased opportunities to
mon. respond to academic requests and the academic
Cooper, J. O., Heron, T. E., & Heward, W. L. (2007). and behavioral outcomes of students with EBD.
Applied behavior analysis. Columbus, OH: Remedial and Special Education, 22, 13–121.
Merrill. Templeton, T. N., Neel, R. S., & Blood, E. (2008).
Dawson, L., Venn, M. L., & Gunter, P. L. (2000). The Meta-analysis of math interventions for students
effects of teacher versus computer reading with emotional and behavioral disorders. Jour-
models. Behavioral Disorders, 25, 105–113. nal of Emotional and Behavioral Disorders, 16,
Forness, S. R., Kavale, K. A., Blum, I. M., & Lloyd, J. 226–239.
W. (1997). Mega-analysis of meta-analyses. Wagner, M., Newman, L., Cameto, R., & Levine, P.
Teaching Exceptional Children, 29(6), 4–9. (2006). The academic achievement and functional
Gunter, P. L., Coutinho, M. J., & Cade, T. (2002). performance of youth with disabilities: A report of
Classroom factors linked with academic gains findings from the National Longitudinal Transition
among students with emotional and behavioral Study-2 (NLTS2). Menlo Park, CA: SRI Interna-
problems. Preventing School Failure, 46, tional. Internet site: www.nlts2.org/reports/2006_
126–132. 07/nlts2_report_2006_07_complete.pdf.
Gunter, P. L., Hummel, J. H., & Venn, M. L. (1998). Walker, H. M., Ramsey, E., & Gresham, F. M. (2004).
Are effective academic instructional practices Antisocial behavior in school: Evidence-based
used to teach students with behavior disorders? practices (2nd ed.). Belmont, CA: Wadsworth.
Beyond Behavior, 9, 5–11. Woodcock, R. W., & Johnson, M. B. (1977).
Hughes, C. A., & Maccini, P. (1997). Effectiveness of Woodcock-Johnson Psycho-Educational Battery.
computer-based instruction on the mathematics Allen, TX: DLM.
performance of students with learning disabili- Webber, J., & Plotts, C. (2008). Emotional and
ties. Learning Disabilities: A Multi-Disciplinary behavioral disorders: Theory to practice. Boston,
Journal, 8, 155–166. MA: Allyn & Bacon.
Individuals with Disabilities Education Improvement Yell, M. (2009). Teaching students with EBD I:
Act, 20 U.S.C. 1 300.8 (c)(4)(2004). Effective teaching. In M. Yell, N. Meadows, E.
Kauffman, J. M. (2005). Characteristics of emotional Drasgow, & J. Shriner (Eds.), Evidence-based
and behavioral disorders of children and youth practices for educating students with emotional
(8th ed.). Upper Saddle River, NJ: Merrill/ and behavioral disorders (pp. 320–341). Upper
Prentice Hall. Saddle River, NJ: Merrill/Pearson Education.
Landrum, T., Brubaker, D. L., Katsiyannis, A., &
Archwamety, T. (2004). An analysis of place-
ment and exit patterns of students with emo- AUTHORS’ NOTE
tional or behavioral disorders. Behavior Disor-
ders, 29, 140–153. Address correspondence to Glenna Billingsley,
Martella, R. C., Nelson, J. R., & Marchand-Martella, Texas State University-San Marcos, Curriculum
N. E. (2003). Managing disruptive behaviors in and Instruction, 601 University Dr., San Mar-
the schools. Boston, MA: Allyn & Bacon. cos, TX 78666. Phone: (512) 293-3402; Fax:
National Council of Teachers of Mathematics. (512) 245-3158; E-mail: gb28@txstate.edu.
(2003). Principles and standards for school
mathematics. Reston, VA: Author.
Olive, M. L., & Smith, B. W. (2005). Effect size
calculations and single subject designs. Educa-
tional Psychology, 25, 313–324.
Parker, R. I., Vannest, K. J., & Brown, L. (2009). The
MANUSCRIPT
improvement rate difference for single-case
research. Exceptional Children, 75, 135– Initial Acceptance: 5/5/09
150. Final Acceptance: 10/19/09

16 / November 2009 Behavioral Disorders, 35 (1), 4–18


Appendix where needed and has the student solve a
question similar to the problematic one.
Fidelity of Instructional Methods __ Completed work is turned into an assigned
Checklist basket corresponding to the period number.
Teacher grades work and records the grade in
Directions: Put a check mark by each their individual folder.
element you observe. At the end, circle the
instructional method you thought was in use
Computer-Assisted Instruction
during your observation.
__ Students are verbally reminded and visually
Direct Teach prompted with a chart stating the week’s
objective. (A chart that cannot be seen by
__ Students are verbally reminded and visually observers will indicate which method a
prompted with a chart stating the week’s particular class will be using. This chart will
objective. (A chart that cannot be seen by list the computer activities and exercises
observers will indicate which method a relative to the week’s objective that must be
particular class will be using. This chart will completed by the end of the week.)
list the activities and exercises for the day.)
__ Students are sent to their assigned computer
__ Teacher reviews prior learning and ties this stations.
prior learning to today’s lesson.
__ Students log in, click on ‘‘Math,’’ and
__ Teacher establishes relevancy by discussing participate in the assigned lessons and activ-
real-world application. ities at their own pace.
__ Teacher introduces lesson by illustrating __ When students have questions, they are
what a correctly finished problem will look encouraged to go back and study the informa-
like. tion and videos that taught the skill. The
__ Teacher presents the lesson in a variety of teacher may assist the student in locating the
formats including lecture, demonstrations, or answers to their problems, but they cannot re-
manipulatives. teach the information. Students are encour-
aged to participate in the Odyssey work
__ Teacher provides multiple opportunities for
independently.
all students to respond to the learning.
__ Missed questions recycle at the end of the
__ Teacher verbally explains the process or
lesson until all questions are answered with
strategy while simultaneously working the
100% accuracy. Once they accomplish this,
problem, then has students do the same.
they announce that they are through, and the
__ Students practice the skill by performing teacher comes and records their score in their
problems on the board, individual dry-erase individual folder.
response boards, as the teacher solves prob-
__ Students continue to the next lesson until all
lems on the board in a parallel manner.
lessons for the week are complete. (Students
__ Feedback is provided as students work will have completed approximately 80 prob-
through problems. lems after a completed learning unit.)
__ Students perform problems on individual
dry-erase response boards while the teacher Combination
checks for accuracy and understanding.
__ Students are verbally reminded and visually
__ The teacher closes the lesson with another prompted with a chart stating the week’s
example of the type of problem being studied objective. (A chart that cannot be seen by
that will be left on the board. observers will indicate which method a
__ A summarization of the lesson and its particular class will be using. This chart will
importance provide closure. list the computer activities and exercises for
__ Students practice the skill taught by the day.)
completing a worksheet of approximately 20 __ Teacher reviews prior learning and ties this
problems. prior learning to today’s lesson.
__ Individual assistance is provided when a __ Teacher establishes relevancy by discussing
student asks for help. The teacher re-teaches real-world application.

Behavioral Disorders, 35 (1), 4–18 November 2009 / 17


__ Teacher introduces lesson by illustrating __ A summarization of the lesson and its
what a correctly finished problem will look like. importance provide closure.
__ Teacher presents the lesson using videos __ Students are sent to their assigned computer
and explanations offered by the Odyssey- stations.
WareH projected onto a screen. Teacher may __ When students log in and click on ‘‘Math,’’
also present lecture, manipulatives, or addi- the program takes them to a lesson where
tional examples. they begin completing approximately 20 prob-
__ Teacher provides multiple opportunities for lems.
all students to respond to the learning. __ Individual assistance is provided when a
__ Teacher verbally explains the process or student asks for help. The teacher re-teaches
strategy while simultaneously working the where needed and has the student solve a
problem, then has the students do the same. question similar to the problematic one.
__ Students practice the skill by performing __ After students complete a problem, they
problems on the board, individual dry-erase submit their answer for grading. They an-
response boards, as the teacher solves prob- nounce that they are through, and the teacher
lems on the board in a parallel manner. comes and records their score in their individ-
__ Feedback is provided as students work ual folder.
through problems. Date:________________________________
__ Students perform problems on individual Period:_______________________________
dry-erase response boards while the teacher Circle the method you think was in place
checks for accuracy and understanding. during your observation:
__ The teacher closes the lesson with another Direct Teach
example of the type of problem being studied Computer-Assisted Instruction
that will be left on the board. Combination

18 / November 2009 Behavioral Disorders, 35 (1), 4–18

You might also like