Professional Documents
Culture Documents
Michael J. Kennedy, PhD1, Donald D. Deshler, PhD2, and John Wills Lloyd, PhD1
Abstract
The purpose of this experimental study is to investigate the effects of using content acquisition podcasts (CAPs), an
example of instructional technology, to provide vocabulary instruction to adolescents with and without learning
disabilities (LD). A total of 279 urban high school students, including 30 with LD in an area related to reading, were
randomly assigned to one of four experimental conditions with instruction occurring at individual computer terminals over
a 3-week period. Each of the four conditions contained different configurations of multimedia-based instruction and
evidence-based vocabulary instruction. Dependent measures of vocabulary knowledge indicated that students with LD
who received vocabulary instruction using CAPs through an explicit instructional methodology and the keyword
mnemonic strategy significantly outperformed other students with LD who were taught using the same content, but with
multimedia instruction that did not adhere to a specific theoretical design framework. Results for general education
students mirrored those for students with LD. Students also completed a satisfaction measure following instruction with
multimedia and expressed overall agreement that CAPs are useful for learning vocabulary terms.
Keywords
vocabulary, multimedia learning, quantitative, research method
Given pervasive negative outcomes for large numbers of curriculum (King-Sears & Bowman-Kruhm, 2010, 2011;
students with specific learning disabilities (LD; see Zigmond, 2006). Special and general education teachers
Newman et al., 2011), it is important for researchers to frequently face limitations with respect to their own
continue to develop, test, and disseminate theories and content knowledge, pedagogical repertoire, and cognitive
interventions that support the cognitive and academic resources when juggling realities of teaching (e.g., large
learning needs of ado- lescents with LD (Scammacca et class sizes, limited planning time, accountability pressures,
al., 2007; Scruggs, Mastropieri, Berkeley, & Graetz, 2011; behavioral challenges) with the demands of designing and
Swanson, 2009). Specifically, interventions should be delivering individualized instruction (Feldon, 2007;
grounded in theory and empirical findings related to the Kennedy & Ihle, 2012; McKenzie, 2009). Thus, educators
cognitive processes of ado- lescents with LD when may benefit from interventions that are designed
interacting with academic tasks (Compton, Fuchs, Fuchs, specifically to address the unique learning needs of
Lambert, & Hamlett, 2011; Johnson, Humphrey, Mellard, students with LD and that are packaged to deliver
Woods, & Swanson, 2010). In addition, designers of new evidence-based instruction in a wide variety of content
interventions should carefully note the demands of specific areas without adding significantly to the already heavy
content areas and identify the component skills that load that teachers carry (Berkeley, Mastropieri, & Scruggs,
underlie these learning tasks given perceived and known 2011). The purpose of this article is to describe the results
needs of students with LD (Archer & Hughes, 2011; of the first experimental test of an emerging, multimedia-
Deshler & Shumaker, 2006; Pressley & Harris, 2006). based instructional intervention that may help teachers of
Multimedia technology offers an avenue for researchers students with exceptionalities provide
and practitioners to purposefully build instruc- tional
materials that meaningfully convey subject-specific 1
University of Virginia, Charlottesville, VA, USA
content while simultaneously supporting students’ learning 2
University of Kansas, Lawrence, KS, USA
needs and motivation to learn (Kennedy & Deshler, 2010).
Corresponding Author:
The purpose of special education is to provide specially Michael J. Kennedy, University of Virginia, Curry School of
designed instruction to help students with exceptionalities Education, PO Box 400273, Charlottesville, VA 22904, USA.
access and make progress in the general education Email: MKennedy@Virginia.edu
Downloaded from ldx.sagepub.com at GEORGIAN COURT UNIV on February 12, 2015
Kennedy et al. 23
specially designed instruction that supports vocabulary Abbott, 2006; Reed, 2008), and (c) explicitly teaching
learning within the general curriculum. strat- egies for forming connections between semantically
related terms (Baumann et al., 2003; Graves, 2006). Two
categories of vocabulary instruction are necessary to help
Identifying Academic Demands to
translate these themes into practice: (a) teaching the
Inform Instruction for Students definitions of terms and (b) teaching students the skills and
With LD strategies needed to decipher word meanings (Graves,
In this article we define students with LD using the defini- 2006; S. A. Stahl & Kapinus, 2001). These two types of
tion of the National Joint Committee on Learning instruction are referred to as nongenerative and generative
Disabilities (1991). Specifically, students with LD who teaching strate- gies, respectively (Harris, Shumaker, &
have significant difficulty acquiring and using skills and Deshler, 2011).
knowledge related to reading are the subgroup of interest. For students with LD, both direct instruction in word
The rationale for adopting this definition is, in content-area meanings (nongenerative) and building capacity through
classrooms, most content demands are tied to reading pro- the use of strategies (generative) are generally needed for
cesses and the accompanying cognitive processing tasks successful learning (Bryant et al., 2003; Ebbers & Denton,
(Shanahan & Shanahan, 2008). Students who have 2008; Harris et al., 2011; Jitendra et al., 2004).
inherent difficulty learning from texts are at a substantial Unfortunately, in practice, the primary methods used to
disadvan- tage for finding success on typical measures of teach vocabulary terms and concepts to students include
academic proficiency (Roberts, Torgesen, Boardman, & mentioning the defi- nition during lecture, expecting they
Scammacca, 2008). In addition, given traditional general will pick up words’ meanings through reading, and having
education instructional settings at the secondary level, it is students look up terms in the dictionary (Kennedy &
unlikely a student with LD who struggles with reading will Wexler, 2013; Lesaux, Harris, & Sloane, 2012). Although
receive the type and amount of evidence-based reading reasons why teachers do not use evidence-based practices
instruction needed to improve reading skill and make for vocabulary instruction are not well understood (see
progress within the content’s standards (Kennedy & Ihle, Jitendra et al., 2004), we hypothesize the creation and
2012; Mastropieri et al., 2005). It is for this reason that we introduction of instructional technology that contains
explore multimedia as a tool for packaging and delivering embedded evidence-based instruc- tion may provide one
evidence-based vocabulary instruction for students with section of the bridge needed to span the divide between
LD. Multimedia provides educators with an opportunity to research and practice.
shape instruction in any needed configuration to meet the
demands of the content (Clark, 2009) and also the needs of Use of Multimedia in Vocabulary
the learner (Kennedy & Deshler, 2010; Mayer, 2011).
Instruction for Students With LD
Given the cognitive needs of adolescents with LD during
Effective Vocabulary Instruction
content-area learning tasks, instructional materials should
Vocabulary knowledge is an example of a high-leverage aca- not be left to chance with respect to their audio and visual
demic skill (Ebbers & Denton, 2008) and is correlated with makeup. For example, few videos that instructors purchase
comprehension (RAND Reading Study Group, 2002) and or download to use during instruction are likely to adhere
other processing tasks required within secondary-level to a validated instructional model and be appropriate for
coursework (Roberts et al., 2008). Helping students under- stu- dents with LD (Kennedy, 2011; Kennedy & Wexler,
stand the context-based and often multiple meanings of 2013). Multimedia provides educators with a unique
words so they can comprehend and use words when reading opportunity to control every second of instruction by way
or during other language-rich situations is the primary goal of planning and selecting the audio and visuals that a
of vocabulary instruction at the secondary level (Baumann, learner receives (Mayer, 2011). However, this control
Kame’enui, & Ash, 2003). Thus, for students with LD, sig- alone is never suffi- cient to guarantee effective instruction
nificant attention is needed to determine ways to gain sus- (Clark, 2009). Instead, instruction should reflect and
tained engagement with new vocabulary words and concepts incorporate existing evidence-based instruction for a given
embedded within general education courses (Ebbers & content demand, but also scaffold cognitive processes so
Denton, 2008; Jitendra, Edwards, Sacks, & Jacobson, 2004). the learner’s cognitive load capacity is not overwhelmed
Key interventions for vocabulary instruction include (Kennedy & Deshler, 2010). With that said, the largest
(a) helping students become aware of the semantic parts of limitation of the existing research on instructional
words (Bos & Anders, 1990), (b) dedicating instructional technology is the scarcity of detail describing the “looks
time to teaching word parts and meanings (Bryant, and sounds” of instruction (Clark, 2009; Mayer, 2011).
Goodwin, Bryant, & Higgins, 2003; Nagy, Berninger, & To illustrate our point regarding the need to carefully
script and shape the “looks and sounds” of multimedia
instruction, we offer Richard Mayer’s (2009) cognitive
24 Journal of Learning Disabilities 48(1)
theory of multimedia learning (CTML) and accompanying highly bounded and process-based description of learning
instructional design principles (Mayer, 2008) as a specific and cognition. This is an important limitation, as multime-
and applied framework to guide practice. This framework dia can affect learners’ cognitive and emotional interests in
consists of 12 evidence-based practices for instructional unpredictable ways depending on their prior experiences
design (Mayer, 2008) but should not be considered an evi- and knowledge and other variables, such as mood (Harp &
dence-based practice on its own for teaching students with Mayer, 1997). In other words, any motivation that occurs
disabilities. Instead, this framework is a logical starting during multimedia instruction will function to either con-
point for anchoring instructional practices that have evi- sume or enhance a viewer’s limited cognitive processes.
dence for delivering a specific type of instruction (e.g., This is currently unaccounted for in Mayer’s model. Given
vocabulary instruction). that researchers in special education have long reported the
In the following discussion, we use Mayer’s model to impact of motivation on the performance of students with
deconstruct a sample video from the Khan Academy disabilities (e.g., Guthrie & Davis, 2003), this is an impor-
(www. khanacademy.org) to illustrate how validated tant aspect of multimedia instruction to be explored in
instructional design principles, such as those from Mayer, future research.
are commonly violated by multimedia acquired via the In the following discussion we use Mayer’s applied the-
web. In addition, we criticize Mr. Khan’s video for its lack ory to illustrate our call for caution when adopting
of adherence to evidence-based practices that are multime- dia for use in teaching students with LD and
appropriate for individu- als with LD. To be true and fair, demonstrate how the intervention tested in this article
these videos are not in any way intended for teaching reflects cognitive science and relevant evidence-based
students with LD; however, as teachers use them in that practice. In addition, we utilize a simple satisfaction
vein (see Fulton, 2012), the criti- cism is justified, and measure as a proxy for moti- vation to gain preliminary
needed. Following this deconstruc- tion, we detail how perspective on whether students with LD report being
Mayer’s model is jointly leveraged along with evidence- motivated to learn using this new instructional tool.
based practices for vocabulary instruc- tion to create the
intervention tested in this study, content acquisition
podcasts (Kennedy, Lloyd, Cole, & Ely, 2012; Kennedy & Deconstruction of Freely Available
Wexler, 2013). Internet Videos Used to Teach Students
With LD
Designing Quality Instruction Using
As noted, it is important to deconstruct any example of
Multimedia multimedia that a general or special education teacher
Mayer’s CTML might opt to use with students with (and without) LD.
Although deconstruction of videos on the Internet (e.g., the
CTML is a learner-oriented instructional theory and
Khan Academy or YouTube) for this purpose is to some
empiri- cally validated design process intended to guide
extent unfair because of their purpose (or lack thereof), as
the process of creating effective multimedia-based
special educators, it is our job to help shed light on
instruction (Mayer, 2009). Mayer’s CTML is an applied
potential mis- matches between student learning needs and
theory in that he intends instructional designers and other
instructional materials potentially being used to teach them
educators to leverage understandings of how people use
(Kennedy & Wexler, 2013). In addition, we have the
visual and auditory inputs when creating new multimedia
responsibility to ensure students with disabilities receive
for various learning tasks. Specifically, his model is
evidence-based, individualized instruction that addresses
operationalized through 12 principles of instructional
the goals and objectives noted within their individualized
design that function together as a framework to limit
education pro- gram (IEP; Zigmond, 2006). Multimedia
cognitive load and maximize human capacity to learn
that does not meet this standard should not be used.
(DeLeeuw & Mayer, 2008). Each of Mayer’s principles is
supported by at least 3 experimental studies of its Figure 2 lists Mayer’s (2008) 12 instructional design
effectiveness and as many as 17 (Mayer, 2008, 2011). principles and a side-by-side comparison of the extent to
Figure 1 reports the number of studies and mean effect which a sample video from the Khan Academy (http://
size for each principle based on Mayer’s program of www.khanacademy.org/humanities/history/euro-hist/v/
research in this field. For additional detail on each french-revolution--part-1) adheres to these principles.
principle and the empirical research supporting this Figure 1 is the rubric used in this study to evaluate the
framework, please see Mayer (2009). extent to which multimedia adhere to Mayer’s instructional
design principles.
Although Mayer’s theory is well respected, some
After review, to a large extent, the sample video from
researchers have questioned its comprehensiveness. To
the Khan Academy does not adhere to Mayer’s principles
illustrate, Astleitner and Wiesner (2004) point out that the
for creating multimedia. Given its long length (17:05),
CTML does not specifically address the impact of motiva-
signifi- cant number and use of history-specific
tion on a learner’s cognition (p. 11), instead holding to a
vocabulary terms,
Kennedy et al. 25
Figure 1. Mayer’s Instructional Design Principles as Rubric for Evaluating Multimedia Instructional Materials.
Note. The 11 studies for the personalization principle include the voice and image principles; they are separated here for the purpose of guiding
instructional design.
lack of explicit instructional cues, redundant use of on- the broad scope of this multifaceted topic. The narrator
screen text, and constant presence of visuals (pictures and uses a pedagogy of telling, which can be explicit, but falls
text) that do not correspond to the narration, according to short of recommendations for explicit instruction noted by
Mayer’s principles, this video will contribute extraneous Archer and Hughes (2011) and is not strategic, as noted by
cognitive load—especially to a viewer unfamiliar with the Deshler and Shumaker (2006) and others (see Pressley &
content being presented. Harris, 2006). There is a substantial assumption of prior
In addition, this video does not make use of evidence- knowledge without explicit references to other videos that
based practices known to help students with LD learn con- may contain the information. The video lacks any advance
tent. For one, the video contains substantial content about organization that follows a hierarchical ordering system
the French Revolution. To illustrate, the narrator that might help facilitate a learner’s engagement with the
repeatedly uses history-specific language conventions and content (see Dexter, Park, & Hughes, 2011). In summary,
vocabulary terms without explicitly defining them. This this video does not provide individualized instruction as
content may be difficult to organize and internalize for required by stu- dents’ IEPs.
many learners given
26 Journal of Learning Disabilities 48(1)
Figure 2. Mayer’s 12 Instructional Design Principles and Corresponding Evaluation of Sample Video from the Khan Academy.
Innovation for Multimedia-Based Another contrast with the Khan Academy video is that
Vocabulary Instruction each CAP contains evidence-based vocabulary instruc-
tional elements for one critical vocabulary term and lasts
In the present study, an intervention called content acquisi- approximately 2 minutes. An example CAP can be viewed
tion podcasts (CAPs) is tested for the purpose of at www.vimeo.com/39293791. Six specific instructional
evaluating its effect on vocabulary performance of practices, grounded in the empirical literature on vocabu-
adolescents with and without LD. CAPs are multimedia- lary instruction (e.g., Bryant et al., 2003; Ebbers &
based instructional modules created using Mayer’s (2008) Denton, 2008; Jitendra et al., 2004), constitute a menu of
instructional design principles. For example, Figure 3 practices embedded into the instructional routine used
presents Mayer’s model, with a description of how CAPs within CAPs. These include (a) promoting word
reflect each of these 12 instructional design principles. In consciousness (e.g., pronunciation, spelling, syllables,
contrast to the sample video from the Khan Academy, prefix, suffix, root words; Reed, 2008), (b) providing
CAPs ensure all images (text and pictures) correspond to direct instruction of word meanings (Archer & Hughes,
the content being pre- sented, narration is limited to 2011), (c) providing guided practice and scaffolding
essential content, and a for- mula is used with respect to (Dexter et al., 2011),
helping students develop a routine for recognizing the (d) providing instruction that promotes awareness of
structure of each video (see closely related terms (Graves, 2006), (e) using the key-
http://tecplus.org/files/download/19). word mnemonic strategy (Mastropieri, Scruggs, & Levin,
Kennedy et al. 27
Coherence - Instructional materials are enhanced when The CAP only contains information relevant to Yes
irrelevant or extraneous information is excluded the history term/concept being presented
Signaling - Learning is enhanced when explicit cues are There are recurring explicit cues to signal the Yes
provided that signal the beginning of major elements beginning of a new section (e.g., definition,
synonym, antonym, mnemonic)
Redundancy - Inclusion of extensive text (transcription) The CAP only contains carefully selected key Yes
on screen along with spoken words and pictures text
hinders learning. Carefully selected words or short
phrases, however, augment retention
Spatial Contiguity - On screen text and pictures should be The on-screen text and pictures are presented Yes
presented in close proximity to limit eye shifting in close proximity to one another
Temporal Contiguity - Pictures and text should shown on Pictures and text correspond to the audio Yes
screen should correspond to the audio presentation presentation
Modality - People learn better from spoken words and Pictures and narration are used Yes
pictures than they do from pictures and text alone
Segmenting - People learn better when multimedia The CAP is 2:52 long Yes
presentations are divided into short bursts as opposed to
longer modules
Pretraining - People learn better when given an advance The CAP begins with an explicit statement of Yes
organizer or preliminary instruction for the content being purpose and an advance organizer for the
introduced term
Multimedia - People learn better from pictures and spoken The video contains pictures and narration Yes
words than from words alone
Personalization - Narration presented in a conversational The narrator uses a conversational tone Yes
style results in better engagement and learning than more throughout the video
formal audio presentations
Voice - People learn better when narration is clearly spoken The narration is not unusual with respect to Yes
with respect to rate and accent rate or accent
Image - People learn better when images are non-abstract, The pictures shown on screen are not Yes
and clearly represent the content being presented abstract, and seem to represent the core
content
Figure 3. Mayer’s 12 Instructional Design Principles and Corresponding Evaluation of Sample content acquisition podcast (CAP).
1987), and (f) providing a statement of purpose/rationale instruction, to achieve maximum learning effects (Bryant
for why the student needs to learn a given term or concept et al., 2003; Ebbers & Denton, 2008; Jitendra et al., 2004).
(Deshler & Shumaker, 2006). These six elements of effec- One limitation of CAPs that use the keyword mnemonic
tive vocabulary instruction constitute a checklist used to strategy is that there is not an automated rehearsal mecha-
create CAPs (see Kennedy, Lloyd, et al., 2012). When nism built in, which is an important element of this
combined with Mayer’s instructional design principles, strategy (Mastropieri et al., 1987). Although not used in
the use of these evidence-based practices for vocabulary this way during this study, practitioners who use CAPs in
instruction may form the base for a multimedia-based the future should follow up with students to ensure the
practice that can be tested for use with students. rehearsal of the keyword and definition is provided.
The keyword mnemonic strategy is utilized within In this study, CAPs were produced using Microsoft’s
CAPs given its strong empirical record for improving PowerPoint (see Kennedy, Hart, & Kellems, 2011, and
vocabulary performance in various content areas (see Kennedy & Thomas, 2012, for specific production steps).
Mastropieri, Scruggs, Bakken, & Whedon, 1997; Although CAPs differ slightly from the textbook defini-
Mastropieri, Sweda, & Scruggs, 2000; Scruggs et al., tion of podcasts (audio recording or synced audiovisual
2011; Scruggs, Mastropieri, Berkeley, & Marshak, 2010) recording uploaded to the Internet), our rationale for
and use of imagery, which lends itself to multimedia-based referring to this learning tool as podcasts is marketing
instruction (Austin, 2009). This is critical because to general education teachers. In other words, because
empirical evidence in the field of vocabulary instruction most practitioners know what podcasts are, and possibly
shows that teachers should use a blend of methods, use generic podcasts in their teaching, we argue that
including explicit and strategic
28 Journal of Learning Disabilities 48(1)
including podcast in the name of this new tool may facili- assumed that approximately three quarters of students
tate adoption and usage. In addition, although not specifi- received free or reduced-price lunch.
cally used in this way during this study given our research Two groups of students participated: (a) students with
questions, we intend for CAPs to be uploaded to the web, LD in a specific area related to reading (n = 30) and
and downloaded by students for use on their iPad, iPhone, (b) students without disabilities and students who receive
or other portable device. special education services for a reason other than a reading
Thus, the purpose of this experimental study is to disability (n = 248). All students in the LD group had an
explore the use of CAPs as a tool to improve vocabulary IEP stemming from a diagnosis of specific learning
learning for adolescents with LD enrolled in rigorous disability related to reading, which manifests as difficulty
secondary- level coursework. Four configurations of CAPs conducting cognitive processes necessary for reading.
with embedded evidence-based vocabulary instruction These 30 stu- dents were in the 10th grade at the time of
were tested to determine the most efficient and effective the study and had a mean age of 16.9 years. In terms of
combi- nation of multimedia and vocabulary instruction: demographic charac- teristics, 80% were male and 20%
(a) explicit instruction alone, (b) the keyword mnemonic were female; 63.3% were African American, 26.7% were
strategy alone, Hispanic, and 10.0% were Caucasian. The mean Wechsler
(c) combination of explicit and strategy (keyword mne- Intelligence Scale for Children–IV IQ score for the 30
monic strategy) instruction, and (d) explicit instruction students with LD was 93.4 (SD = 9.3). Furthermore, the
without adherence to Mayer’s model. In summary, CAPs students scored homoge- neously on the reading subtests
were used to answer these research questions: To what (Word Reading, Reading Comprehension, and
extent does multimedia instruction that adheres to various Pseudoword Decoding) of the Wechsler Individual
instructional design principles and is combined with evi- Achievement Test–Second Edition. Using grade-based
dence-based vocabulary instruction promote vocabulary norms, the students’ mean standard score for the Word
learning for adolescents with and without LD? Do students Reading subtest was 68 (SD = 4.3), which is the second
who learn using various forms of CAPs report satisfaction percentile. The mean standard score for the Reading
with this learning method? Comprehension subtest was 63 (SD = 9.1), which is the
first percentile; and the mean standard score for the
Method Pseudoword Decoding subtest was 66 (SD = 4.8), which is
also the first percentile. Using grade-based norms, the
Setting and Participants mean reading composite score for these 30 students was
176 (SD = 18.2), which translates into a standard score of
The University Human Subjects Committee, the partici-
51 and less than the first percentile.
pating school district’s research review board, the princi-
pal of the school, parents of all students, and the students All students received daily special education services
gave permission to conduct this research. The school dis- embedded within their core academic content classes
trict is located in an urban, Midwestern community of taught by a general education teacher (e.g., social studies,
146,867 residents. The researchers recruited two world science, mathematics, and language arts) and also had a
history teachers responsible for teaching 12 total sections study skills course taught by a special educator. Although
of world history to approximately 300 students to partici- some students without LD also had an IEP (primarily
pate in the study. A total of 278 urban high school stu- students with an emotional/behavioral disorder diagnosis),
dents (9th to 12th graders) enrolled in a world history they were grouped with the students without disabilities
course participated in the study. African American stu- for the pur- pose of this study’s activities and analyses. The
dents represented the largest ethnic group (67.5%), rationale for this decision is the desire to study the effects
Caucasian students were the next largest group at 21.9%, of the inter- vention on students who have documented
and Hispanic students constituted 8.1%. Of the 278 par- disabilities spe- cific to the demands of reading.
ticipants, 52% were female, 48% were male, and 91% Using a table of random numbers, researchers randomly
were in 10th grade. The mean age of participants was 16.7 assigned students into one of four experimental conditions.
years. At the time of the study, the selected high school Two stratification variables proportionately sorted students
had a student enrollment of 987, 78% of whom received into the four groups: disability status (LD or not LD) and
free and/or reduced-price lunch. Permission to collect achievement status (high, typical, low). The researchers
individual socioeconomic status could not be obtained used students’ first-semester GPA in world history to sort
from the school district’s human subjects review board. students into one of three levels of achievement status: (a)
However, given that nearly every 10th grader in the high achiever—85% or above, (b) typical achiever—84%
school is enrolled in one of the 12 sections of world his- to 70%, and (c) low achiever—69% and below. The
tory participating in this project and 78% of students at researchers strat- ified students by achievement status
this school received free or reduced-price lunch, we given the expectation that high achievers would likely
learn more vocabulary con- tent than typical or low
achievers regardless of how they are
Kennedy et al. 29
ing the given term is important, (b) direct instruction of
taught. Therefore, it was important to proportionately
distrib- ute these students across the four experimental
conditions.
throughout the study, they were asked to note which of the retaught Terms 1 to 20 as a part of their required course
four groups they were in for the experiment. This allowed curriculum. The researcher did not monitor how the terms
survey responses to be sorted by group and analyses of were retaught to students. As a result, Terms 1 to 20 were
responses based on the version of the CAPs that were assessed only at posttest. However, Terms 21 to 30 were
watched. The survey was reviewed by three doctoral stu- not retaught or assigned as part of assignment during this
dents, who were asked to provide feedback on wording of period of time. Thus, the maintenance probe consisted of
questions, order of questions, and overall quality of the an MC test to evaluate student knowledge of Terms 21 to
instrument. Feedback was used to make updates to the sur- 30; the probe is the same as the posttest measure.
vey. The reliability alpha for the survey was .73.
Research Design
Procedure for Watching CAPs and
Researchers conducted main and secondary analyses for
Taking Assessments
this study. There are two parts of the main analysis. In Part
The first author and the two teachers used a checklist to 1, the between-subjects factors for students with LD are
ensure fidelity during research activities. The checklist four approaches to delivering vocabulary instruction with
con- tains procedures for the research activities for each multimedia (EI, n = 7; KMS, n = 8; EI + KMS, n = 7; NM,
day, including the script for directions that was read to n = 8). The four groups are evaluated on a dependent mea-
students. To begin research activities, students completed sure with two levels: performance on the pretest and post-
a pretest during their regularly scheduled class, which is test for Items 1 to 30 (evaluated using a 4 × 2 split-plot,
the same instrument as the posttest and maintenance fixed-factor repeated measures ANOVA). To create the one
probes. Students were informed that the pretest would not dependent variable score, researchers respectively stan-
count toward their grade, but would be used as a class dardized and then averaged student scores on the MC and
work/participation grade. One week later, students OE instruments. In the second part of the main analysis,
watched an orientation CAP that introduced this format the between-subjects variables are the same as above; the
of instruction and cued them to its various features. dependent measure has three levels: performance on the
Students watched CAPs at individual laptop terminals pretest, posttest, and maintenance probes for Items 21 to
while wearing headphones. CAPs were loaded onto the 30 (evaluated using a 4 × 3 split-plot, fixed-factor repeated
school’s intranet and sorted into password- protected measures ANOVA).
folders for each experimental group. Students watched In the secondary analysis, the above analyses are
a total of 10 CAPs during one class period. The first repeated, but for students without LD (between- and
author and teacher circulated the room to ensure stu- within- subjects variables are the same; EI, n = 60; KMS,
dents were watching the CAPs and not navigating to other n = 62; EI
websites or programs. + KMS, n = 63; NM, n = 63). The reason for including this
After finishing the fifth video, students were to raise secondary analysis here is twofold. First, because of the
their hand and close their laptop. The teacher handed small sample size and thus limited statistical power in the
students the OE instrument, which asked students to (a) four groups among students with LD, it is important to
write the defini- tion for each term/concept, (b) provide a con- firm observed results using a larger group (albeit with
synonym/antonym, and (c) provide any other related stu- dents without LD). Second, if there were any effects
information for each term/ concept. When students for any of the types of multimedia instruction when
completed this instrument, they raised their hands and the comparing stu- dents without LD, that intervention would
teacher collected the paper and handed the students the potentially be more attractive to a general education
MC assessment. The decision to give students the OE teacher.
instrument first was purposeful to elim- inate the Finally, results from the satisfaction survey were ana-
possibility of being cued to the answer with the MC lyzed using quantitative data from the Likert-type items.
instrument. After completing the MC items, students again Responses are organized by group assignment and
raised their hands, the paper was collected, and students compared to one another using a series of one-way
watched the next five CAPs, and the process repeated. In ANOVAs for the purpose of evaluating differences and/or
sum, students watched a total of 30 CAPs, in the order of emerging trends that may be attributed to the different
10 per day across 3 days of the experiment. versions of the CAPs.
Table 2. Raw Mean Scores and Standard Deviations for Students With Learning Disabilities (LD) and Without LD (Not LD) on the
Multiple Choice and Open-Ended Pretest and Posttest (Terms 1–30) and Maintenance Instruments (Terms 21–30).
Note. The pretest and posttest MC has a score range of 0–30; the pretest and posttest OE has a score range of 0–60. The maintenance instrument has a
score range of 0–10 for MC and 0–20 for OE. LD = students with an LD in an area related to reading; MC = multiple-choice instrument; Not LD =
students without a learning disability in an area related to reading; OE = open-ended instrument.
the pretest and posttest for Terms 1 to 30. Levene’s test for the NM group Pos = −2.8, SD = 2.3; d = 1.97). No other
t
(M
equality of variances was not significant for this post hoc comparisons were statistically significant at the
evaluation. Given the random assignment of students to .025 level; however, the mean scores show the EI + KMS
experimental
groups, there was no further need to statistically control for group (MPost = 1.45, SD = 2.7) scored higher on the posttest
between-group variance and differences. The raw score than the students with LD in the EI Pos = −0.92, SD = 1.5;
t
(M
means, standard deviations, p values, and effect sizes d = 1.09) and KMS Pos = −1.6, SD = 1.6; d = 1.40)
t
(M
(Cohen’s d) for the pretest, posttest, and maintenance groups. These large effect sizes should be interpreted with
probe are listed in Tables 2 and 3. There was not a caution given the limited statistical power and use of a
significant main effect for group, F(3, 26) = 1.7, p = .187, researcher-created instrument; however, they do provide
ω2 = .03, or time, F(1, 26) = > 1, p = .516, ω2 = .00; preliminary evidence that for students with LD learning
however, the interaction between group and time, F(3, 26) these vocabulary terms using the full CAP model had an
= 13.0, p < .000, ω2 = .26, was significant. Therefore, 27% impact on their learning.
of the variance in this model is explained by the sum of
effects from the predictor variable (EI, KMS, EI + KMS,
NM) in the experiment.
Main Analysis, Students With LD: Part 2
To examine the direction and location(s) of the signifi- In this analysis, researchers conducted a 4 × 3 split-plot,
cant interaction, researchers conducted post hoc pairwise fixed-factor repeated measures ANOVA. The between-
comparisons using Tukey’s honestly significant difference subjects factor was group assignment (group), and the
(HSD) test. A Bonferroni correction was used to evaluate three levels of the within-subject factor (time) were perfor-
the individual group differences at the two time points mance on the pretest, posttest, and maintenance probe for
(.05/2 = .025). These results demonstrate that there was Terms 21 to 30. Levene’s test for equality of variances was
not a significant difference in scores between any group of not significant for this evaluation. However, Mauchly’s
stu- dents with LD on the pretest; however, students with
LD in
the EI +KMS group Pos = 1.45, SD = 2.7) had signifi- test of sphericity was significant, Mauchly’s W = .551, p =
t
(M
cantly higher scores on the posttest than students with LD in .001; thus, Greenhouse–Geisser corrections are used to
Kennedy et al. 33
Table 3. Maintenance Mean Scores and Standard Deviations Secondary Analysis, Students Without LD: Part 1
for Multiple-Choice and Open-Ended Instruments.
To mirror the main analysis but with students without LD
MC instrument OE instrument (to help confirm findings given limited statistical power),
M SD M SD researchers again conducted a 4 × 2 split-plot, fixed-factor
repeated measures ANOVA. The between-subjects factor
LD Items 21–30
was group assignment (group), and the two levels of the
Group 1 (n = 7) 3.4 1.40 4.0 1.15
within-subject factor (time) were performance on the pre-
Group 2 (n = 8) 4.5 1.41 4.1 1.55
test and posttest for Terms 1 to 30. Levene’s test for
Group 3 (n = 7) 6.6 1.90 7.7 2.87
equality of variances was not significant for this
Group 4 (n = 8) 2.2 1.67 1.4 1.51
Not LD Items 21–30
evaluation. Given the random assignment of students to
Group 1 (n = 60) 4.8 2.08 7.6 6.77 experimental groups, there was no further need to
Group 2 (n = 62) 4.5 1.99 7.1 5.80 statistically control for between- group variance and
Group 3 (n = 63) 6.3 2.14 11.7 9.40 differences. The raw score means, stan- dard deviations, p
Group 4 (n = 63) 3.1 1.72 4.4 3.81 values, and effect sizes (Cohen’s d) for the pretest, posttest
and maintenance probe are listed in Tables 2 and 3. There
Note. The MC instrument has a score range of 0–10; the OE instrument was a significant main effect for group, F(3, 243) = 6.7, p
has a score range of 0–20. MC = multiple-choice instrument; NSWD =
< .001, ω2 = .05, and the interac- tion between group and
students without a learning disability (LD); OE = open-ended instrument;
SWD = students with LD. time, F(3, 243) = 25.3, p < .001, ω2
= .05. However, the evaluation for time, F(1, 243) = > 1, p
= .827, ω2 = .00 was not significant. Therefore, 10% of the
variance in this model is explained by the sum of effects
evaluate and interpret results. There was a significant main from the predictor variable (EI, KMS, EI + KMS, NM) in
effect for group, F(3, 26) = 6.3, p < .002, ω2 = .21, and the the experiment.
interaction between group and time, F(4.1, 52) = 8.9, p < To examine the direction and location(s) of the signifi-
.000, ω2 = .18. However, the effect for time, F(2, 52) = > cant interaction, researchers conducted post hoc pairwise
1, p = .460, ω2 = .00, was not significant. Therefore, 39% comparisons using Tukey’s HSD test. A Bonferroni
of the variance in this model is explained by the sum of correc- tion was used to evaluate the individual group
effects from the predictor variable (EI, KMS, EI + KMS, differences at the two time points (.05/2 = .025). In a
NM) in the experiment. repetition of findings from the main analysis, these results
To examine the direction and location(s) of the signifi- demonstrate that there was not a significant difference in
scores between any group on the pretest; however,
students without LD in the EI
cant interaction at maintenance, researchers conducted +KMS group (MPost = 1.8, SD = 2.4) had significantly higher
post hoc pairwise comparisons using Tukey’s HSD test. A scores on the posttest than students with LD in the NM
Bonferroni correction was used to evaluate the individual group Pos = −1.7, SD = 2.6; d = 1.41). No other post hoc
t
(M
group differences at the three time points (.05/3 = .017). comparisons were statistically significant at the .025 level;
Student performance on the posttest was evaluated for the however, the mean scores show the EI + KMS group (M
Post
30 vocabulary terms in the main analysis, and so for the = 1.8, SD = 2.4) scored higher on the posttest than the stu-
sake of brevity the results are not reported here for Items dents with LD in the EI (MPost = 0.30, SD = 2.6; d = 0.61)
21 to 30. and KMS Pos = 0.10, SD = 2.3; d = 0.74) groups. These
t
(M
At maintenance, students with LD in the EI +KMS group medium to large effect sizes should be interpreted with
(M = 1.1, SD = 1.2) had significantly higher scores on cau- tion given the use of a researcher-created instrument,
Maint
the maintenance probe than students with LD in the NM how- ever, they do provide corroborating evidence that
students
group (MMaint = −1.4, SD = 1.3; d = 2.40). No other post who learned these vocabulary terms using the full CAP
hoc
comparisons were statistically significant at the .017 level; model (EI + KMS) made gains in performance relative to
however, mean scores show the EI + KMS group (M = other students who did not receive the full intervention or
Maint
1.1, SD = 1.2) scored higher on the posttest than the stu- a different intervention.
dents with LD in the EI (MMaint = −0.87, SD = 0.75; d =
1.96) and KMS Main = −0.34, SD = 0.71; d = 1.45)
t
Secondary Analysis, Students Without LD: Part 2
(M groups.
These large effect sizes also should be interpreted with nary evidence that students with LD who learned using the
cau- tion given the limited statistical power and use of a full CAP model made gains in vocabulary performance.
researcher-created instrument; however, there is prelimi-
Again to mirror the main analysis for students with LD,
researchers conducted a 4 × 3 split-plot, fixed-factor
repeated measures ANOVA. The between-subjects factor
was group assignment (group), and the three levels of
the
34 Journal of Learning Disabilities 48(1)
within-subject factor (time) were performance on the pre- they do not have a difficult time learning new vocabulary
test, posttest, and maintenance probe for Terms 21 to 30. terms in history (M = 4.97, SD = 2.56); however, students
Levene’s test for equality of variances was not significant generally agreed that the CAPs helped them learn the
for this evaluation. However, Mauchly’s test of sphericity mean- ing of the terms (M = 7.38, SD = 2.06) and that the
was significant, Mauchly’s W = .705, p = .001; thus, CAPs prepared them to do well on the posttest and
Greenhouse–Geisser corrections are used to evaluate and maintenance probes (M = 6.91, SD = 2.16). Finally,
interpret results. There was a significant main effect for students in the EI + KMS and KMS only groups, on
group, F(3, 243) = 14.672, p < .001, ω2 = .10, and the average, agreed that the KMS is helpful in learning the
inter- action between group and time, F(4.6, 375) = 19.2, p meaning of vocabulary terms (M = 7.58, SD = 2.19).
< .000, ω2 = .06. However, the effect for time, F(1.5, Post hoc analyses of student survey responses were
375) = > 1, p = .914, ω2 = .00 was not significant. com- pleted using one-way ANOVAs. Tukey post hoc
Therefore, 16% of the variance in this model is explained compari- sons of the four groups’ survey responses for
by the sum of effects from the predictor variable (EI, Item 3 (The podcasts looked good) indicated that the
KMS, EI + KMS, NM) in the experiment. students in the EI + KMS group (n = 70, M = 8.53, 95% CI
To examine the direction and location(s) of the signifi- = 8.11, 8.95), EI only
cant interaction at maintenance, researchers conducted post group (n = 68, M = 8.41, 95% CI = 7.96, 8.86), and KMS
hoc pairwise comparisons using Tukey’s HSD test. A only group (n = 70, M = 8.66, 95% CI = 8.31, 9.00) all
Bonferroni correction was used to evaluate the individual expressed significantly higher satisfaction with the visual
group differences at the three time points (.05/3 = .017). elements of the CAPs than did students in the NM group
Student performance on the posttest was evaluated for the (n
30 vocabulary terms in the main analysis and so for the = 70, M = 7.26, 95% CI = 6.70, 7.81), p < .000, p < .002, and
sake of brevity will not be reported here for Items 21 to 30. p < .000, respectively. In addition, students in the EI +
At maintenance, students with LD in the EI +KMS group
KMS group (n = 69, M = 7.91, 95% CI = 7.42, 8.41)
expressed significantly higher satisfaction on Item 6 (The
podcasts helped me learn the meanings of the vocabulary
terms) than
(MMain = 1.1, SD = 1.6) had significantly higher scores on students in the NM group (n = 69, M = 2.13, 95% CI = 6.26,
t
the maintenance probe than students without LD in the EI 7.28, p < .006). Finally, students in the EI + KMS group (n
(MMain = 0.10, SD = 1.3; d = 0.66), KMS (MMaint = −0.08, = 70, M = 7.69, 95% CI = 7.26, 8.11) expressed significantly
t
SD = 1.3; d = 0.80), and NM (MMaint = −0.86, SD = 0.99; higher satisfaction on Item 7 (After watching the podcasts I
d = 1.47) groups. In addition, students without LD in the CAPs was interesting (M = 6.32, SD = 2.16). On average,
EI and KMS groups, respectively, significantly students reported that
outperformed students without LD in the NM group (d =
0.81; 0.69) at maintenance. These medium to large effect
sizes should also be interpreted with caution given the use
of a researcher- created instrument; however, this finding
corroborates the observed impact of the CAP intervention
for students with LD noted above.
Discussion
General and special education teachers should seek
oppor- tunities to use various instructional techniques
and strate- gies to support individual learning needs of
students with LD (Swanson, 2001), especially during
rigorous second- ary-level content-area coursework
(Deshler & Shumaker, 2006). Although it can be
difficult to fully understand the complex learning
needs of students with LD and then translate that
understanding into individualized instruction given
competing demands (e.g., curriculum pacing, plan-
ning), this is the meta-space educators must occupy to
sup- port the learning of children with the significant
learning needs (Kennedy & Deshler, 2010; King-
Sears & Bowman- Kruhm, 2010; Zigmond, 2006).
The results of this study confirm the need to leverage
evidence-based teaching practices with delivery
mechanisms that do not solely pres- ent content in a
novel way. Instructional technology, such as CAPs,
may provide a functional mechanism for design- ing
and delivering this type of individualized instruction.
This does not preclude the use of technology in an
assistive role (see King-Sears, 2009, for an excellent
discussion) but does provide compelling rationale to
invest resources into ways instructional technology
can deliver individualized
Kennedy et al. 35
Note. Item scores range from 1 to 10. EI = explicit instruction; KMS = keyword mnemonic strategy; NM = non-Mayer.
a
Does not include Item 8, which was answered only by students in Groups 1 and 3.