Professional Documents
Culture Documents
2 0 1 1 , Vol. 3 4 , No. 1, 5 1 - 7 2
Abstract. T h i s m e t a - a n a l y s i s r e v i e w s e x p e r i m e n t a l a n d q u a s i -
e x p e r i m e n t a l studies in w h i c h upper-elementary, i n t e r m e d i a t e ,
a n d s e c o n d a r y students w i t h l e a r n i n g disabilities l e a r n e d f r o m
g r a p h i c organizers. Following a n e x h a u s t i v e search for studies
m e e t i n g specified design criteria, 5 5 s t a n d a r d i z e d m e a n effect
sizes w e r e e x t r a c t e d f r o m 1 6 a r t i c l e s i n v o l v i n g 8 0 8 p a r t i c i p a n t s .
Students a t levels r a n g i n g f r o m g r a d e 4 t o g r a d e 1 2 u s e d g r a p h i c
o r g a n i z e r s t o l e a r n in c o r e - c o n t e n t classes ( E n g l i s h / r e a d i n g , sci
ence, social studies, m a t h e m a t i c s ) . Posttests m e a s u r e d n e a r a n d far
transfer. Across several c o n d i t i o n s , settings, a n d features, t h e use
of g r a p h i c organizers w a s associated w i t h increases in v o c a b u l a r y
knowledge, comprehension, a n d inferential knowledge. M e a n
e f f e c t sizes v a r i e d f r o m m o d e r a t e t o l a r g e b a s e d o n t y p e o f m e a s
ure, type of g r a p h i c organizer, a n d subject area. Conclusions,
implications for future research, a n d practical r e c o m m e n d a t i o n s
are presented.
DOUGLAS D. DEXTER, Ph.D., College of Education, The Pennsylvania State University, University Park, Pennsylvania.
CHARLES A. HUGHES, Ph.D., College of Education, The Pennsylvania State University, University Park, Pennsylvania.
As students enter the upper-elementary, intermediate, skills (Deshler, Ellis, & Lenz, 1996). These students gen
and secondary grades, academic demands are height erally have difficulty connecting new material to prior
ened as material becomes more complex and the cur knowledge, identifying and ignoring extraneous infor
riculum is driven by higher-order skills and advanced mation, identifying main ideas and supporting details,
concepts (Fletcher, Lyon, Fuchs, & Barnes, 2 0 0 7 ) . drawing inferences, and creating efficient problem-solv
Students at these levels typically receive less individual ing strategies (Baumann, 1984; Holmes, 1985; Johnson,
attention than in primary grades (Hughes, Maccini, & Graham, & Harris, 1997; Kim, Vaughn, Wanzek, & Wei,
Gagnon, 2003), and are often required to learn prima 2004; Williams, 1993). Because many textbooks are
rily through didactic lecture and expository text presen written above grade-level reading ability and lack orga
tation (Gajria, Jitendra, Sood, & Sacks, 2007; Minskoff nizational clarity (Gajria et al., 2007), these learning
& Allsopp, 2003). This shift in learning presentation, difficulties make interpreting and comprehending
rife with abstract concepts, unfamiliar content, and expository text especially challenging (Bryant, Ugel,
technical vocabulary (Armbruster, 1984), may seem Thompson, & Hamff, 1999).
daunting to most students, but especially so to students Students with LD need explicit content enhance
with learning disabilities (LD). ments to assist in verbal (e.g., text or lecture) compre
Students with LD often have difficulty with basic aca hension, and graphic organizers (GOs) have often been
demic skills (e.g., reading) and organizational/study recommended as an instructional device to assist these
51
52 1)1 X I I It & III (,1 IIS
trees
rain
plants earth oceans
30% T 97.5%
clouds
atmosphere |
.001%
students in understanding increasingly abstract con may support and develop organizational strategies
cepts (Bos & Vaughn, 2002; Hughes et al., 2003; Ives & essential to both reading and writing of content area
Hoy, 2003; Kim et al., 2004; Nesbit & Adesope, 2007; material by bridging the gap between idea organization
Rivera & Smith, 1997). and written language structures (Pehrssen & Denner,
1988). Further, Boyle and Yeager (1997) pointed out
W h a t Are Graphic Organizers?
that minimizing sentences and details in the GO is an
GOs are visual and spatial displays that make rela
essential component of cognitive mapping. They rec
tionships between related facts and concepts more
ommend using keywords and simple drawings rather
apparent (Gajria et al., 2007; Hughes et al., 2003; Kim
than complex sentences or elaborate drawings.
et al., 2004). They are intended to promote more mean
Teachers often make cognitive mapping GOs to use as
ingful learning and facilitate understanding and reten
advance organizers for challenging text. Students also
tion of new material by making abstract concepts more
can fill in teacher-prepared blank cognitive mapping
concrete and connecting new information with prior
GOs during or after attending to challenging verbal
knowledge (Ausubel, 1968; Mayer, 1979). While there
material. An example of a cognitive mapping GO is
is inconsistency in the definitions of types of GOs found in Figure 1.
(Rice, 1994), we were able to classify all the studies on
Semantic Mapping. Semantic mapping (SM) is a
students with LD where effects could solely be attrib
heuristic that enables students to recognize relevant
uted to the GO into the following five general cate information from lecture and text (e.g., main ideas,
gories. important supporting details), delete isolated details
Cognitive Mapping. Cognitive mapping assists in that may not be relevant to overall understanding, and
making major ideas and relationships explicit by using highlight key concepts that may have not been fully
"lines, arrows, and spatial arrangements to describe developed in a lecture or text (Bos & Anders, 1990).
text content, structure, and key conceptual relation Unlike cognitive mapping, when using SM, students
ships" (Darch & Eaves, 1996, p. 3 1 0 ) . By graphically and the teacher actively create a visual representation
displaying important information, cognitive mapping (e.g., relationship map or web) to represent the rela-
GRAPHIC ORGANIZERS AND LD S3
Nobles Fiefs
Vassals Farming
tionships among concepts. Typically, concepts are superordinate concept. Finally, subordinate concepts
listed, and the teacher and students make predictions (e.g., concepts representing the coordinate concept) are
about how the concepts can be arranged to demonstrate listed below each coordinate concept (Bos & Anders,
those relationships on the GO (Bos & Anders, 1992). 1990, 1992; Pearson & Johnson, 1978). Figure 2 pro
A well-made GO consists of a superordinate concept vides an example of an SM GO.
(e.g., main idea, topic) placed in an oval in the middle Semantic Feature Analysis. Semantic feature analy
or top of the page. Coordinate concepts (e.g., categories sis (SFA) is similar to SM by helping students recognize
representing related concepts) are placed in ovals sur relevant information from lecture and text. This is
rounding or underneath the superordinate concept and done through a presentation of related concept charac
connected by lines. Coordinate concepts can include a teristics in a matrix form. In SFA, unrelated concepts
variety examples, functions, or characteristics of the can be inferred directly from the chart (Darch &
Energetic - + + +
Weekly Grooming + -
Good with + + ? -
Kids/Other Pets
Gersten, 1986). Typically, a relationship matrix is con ways: temporal (e.g., timeline), spatial (e.g., decision
structed with vocabulary representing the coordinate tree), sequential (e.g., flowchart), hierarchal (e.g., tax
concepts placed along the top of the matrix, and the onomy), or comparative (e.g., Venn diagram). An
vocabulary representing the subordinate concepts example of a visual display GO is found in Figure 5.
placed along the side (Bos & Anders, 1990) The teacher Previous Research
and students can then make predictions and confirma Several groups of researchers have conducted reviews
tions of relationships (e.g., related, not related, not and meta-analyses of the effectiveness of using GOs
sure) between the coordinate and subordinate concepts with nondisabled students (e.g., Ausubel, 1 9 6 8 ;
(Bos &Anders, 1992). The superordinate concept serves Kulhavy, Stock, & Caterino, 1994; Mayer, 1979; Moore
as the title. Figure 3 provides an example of an SFA G & Readence, 1984; Nesbit & Adesope, 2007; Robinson,
S y n t a c t i c / S e m a n t i c Feature Analysis. Syntactic/ Katayama, DuBois, & Devaney, 1998). Based on these
semantic feature analysis (SSFA) is nearly identical to examinations of both the benefits of GOs and the
SFA but with the addition of cloze-type sentences writ effective design of GOs, several key findings are con
ten based on the matrix (Bos & Anders, 1990). Cloze sistently replicated: (a) students with low verbal ability
sentences contain blank spaces replacing new vocabu gain more from GOs than students with high verbal
lary words. Students must use the context of the sen ability; (b) students with little or no prior knowledge
tence and the SFA matrix to fill in the blanks. An in a subject gain more from GOs than students with an
example of an SSFA GO is found in Figure 4. abundance of prior knowledge in a subject; (c) GOs are
Visual Display. Visual displays present concepts or especially helpful in assisting students with far-transfer
facts spatially, in a computationally efficient manner. tasks, in addition to near-transfer tasks and factual
That is, relationships between concepts are made recall; (d) GOs should be explicitly taught to students
apparent and clear by their location on the display. for maximum impact; (e) GOs should spatially group
According to Hughes et al. (2003), in a visual display, together or connect concepts so readers are more likely
facts or concepts are typically presented in one of five to perceive them as being interrelated and to draw
Energetic
Weekly Grooming
Simile Metaphor
Similarities
perceptual inferences about their relationships; (f) GOs Drawing definitive conclusions from these reviews is
should not be clustered with a lot of information; read problematic for several reasons, however. First, the
ers should easily perceive the phenomena or relations reviews did not take sample sizes into consideration
that are important; (g) GOs are effective because of when calculating effect sizes and comparing studies.
their computational efficiency, minimizing stress on Estimations of effect were based on individual study
the working memory; and (h) GOs can be effective characteristics without standardization. The use of
when used before, during, or after a lesson. While meta-analysis allows for statistical standardization of
these findings are promising, the vast majority of the findings, resulting in numerical values that are inter-
studies reviewed used college students as participants, pretable in a consistent pattern across all studies, thus
and few comparisons were made to students identified controlling for an individual study's sample size (Lipsey
as LD. & Wilson, 2001). Furthermore, according to Lipsey and
Two research syntheses of school-aged children with Wilson, meaningful effects and relationships where
LD (Gajria et a l , 2007; Kim et al., 2004) have focused on multiple quantitative studies with varying sample sizes
GOs. In Gajria et al., as part of an examination of several agree, as well as differential effects of study differences,
content enhancements, GOs were found to have large are more likely to be identified by meta-analytic proce
effects for comprehension of expository text for upper- dures than by less systematic and analytic approaches.
elementary, intermediate, and secondary students with Second, the reviews focused solely on factual com
LD. Likewise, Kim et al. found large effects for GOs on prehension measures, not on vocabulary, inference, or
the reading comprehension of upper-elementary, inter relational comprehension. While information on fac
mediate, and secondary students with LD. tual comprehension is important, it may be useful to
56 DEXTER & HUGHES
measure the utility of GOs for students with LD on mul Research & Practice, Remedial and Special Education, and
tiple constructs rather than only on the ability to Reading Research Quarterly. This process yielded a total
answer fact-level questions about a topic. The utility of of 27 published articles to analyze, many (e.g., 20)
an intervention (e.g., GO) on multiple constructs is con including more than one study.
sidered important for high-quality research (Gersten, Inclusion Criteria
Baker, & Lloyd, 2000). Thus, the effectiveness of an We used six criteria to evaluate the appropriateness
intervention should be examined on all possible con of each study. First, the study must have included
structs. a dependent measure of near or far transfer of verbal
Third, no systematic analysis of effects was conducted (e.g., text or lecture) material and a GO as the inde
by type of measure (e.g., near or far transfer), type of pendent variable. Studies with a mnemonic illustration
GO, subject area, or student stage of attending to verbal rather than GO as the content enhancement (e.g.,
material. This lack of clarity makes it difficult to identify Brigham, Scruggs, & Mastropieri, 1995; Mastropieri,
why the GO interventions were so effective. A more sys Scruggs, & Levin, 1 9 8 7 ) were excluded, as effects
tematic analysis of GO studies may allow for more pre could not be attributed solely to the GO. Likewise,
cise explanations of effect based on multiple criteria, studies including GOs associated with Content En
and lead to a more robust understanding of how and hancement Routines (Bulgren, Deshler, Schumaker, &
why GOs are effective for students with LD. Lenz, 2 0 0 0 ; Bulgren, Lenz, Schumaker, Deshler, &
In the present study, we conducted a meta-analysis of Marquis, 2 0 0 2 ; Bulgren, Marquis, Lenz, Schumaker,
GO research to address the following questions: & Deshler, 2009; Bulgren, Schumaker, & Deshler, 1988;
1. What are the overall effects of GOs on the posttest Scanlon, Deshler, & Schumaker, 1996) were excluded,
performance of students with LD? because effects could not be attributed solely to the
2. Do these effects maintain over time? GO, even though GOs were vital components of each
3. Are there differential effects by type of measure of these routines. Further discussion about Content
(near or far transfer)? Enhance- ment Routines is included in the Limitations
4. Are there differential effects by type of GO? section of this meta-analysis.
5. Are there differential effects by subject area?
6. Are there differential effects by stage of attending Second, the study must have taken place in upper-
to verbal material (before, during, after instruc elementary-, intermediate-, or secondary-level class
tion)? rooms (e.g., grades 4-12). This grade range was selected
because it represents a time when curricula typically
Method become more complex and students are required to
Literature Search P r o c e d u r e learn primarily through didactic lecture and expository
A three-step process was used to identify studies text presentation (Fletcher et al., 2007; Hughes et al.,
using GOs with upper-elementary, intermediate, and 2003; Minskoff & Allsopp, 2003).
secondary students with LD. First, we conducted a com Third, based on the recommendations of Rosenthal
prehensive computerized search of Psyclnfo, ERIC, and (1994) and Lipsey and Wilson (1993), only studies
Social Science Citation Index databases for studies from using experimental or quasi-experimental group
1975 (e.g., year LD was officially recognized by passage designs with control groups were included. Therefore,
of P.L. 94-142) to October 2009, using a list of search single-subject research studies (e.g., Gardill & Jitendra,
terms generated from previous reviews of GO studies 1999; Idol & Croll, 1987), repeated-measures studies
(e.g., Horton et al., 1993; Moore & Readence, 1984; (e.g., Lenz, Adams, Bulgren, Poulit, & Laraux, 2007),
Nesbit & Adesope, 2006). W e used the following com and single-group studies (e.g., Bergerud, Lovitt, &
bination of descriptors: graphic organ*, expository, verbal, Horton, 1988; Boon, Fore III, Ayres, & Spencer, 2005;
learning disab*, concept map, cognitive map, adolesc*, Horton, Lovitt, & Bergerud, 1 9 9 0 ; Lovitt, Rudsit,
semantic map, semantic feature analysis, visual display. Jenkins, Pious, & Benedetti, 1 9 8 6 ; Sinatra, Stahl-
Second, we conducted ancestral searches of identified Gemake, & Berg, 1984; Sturm & Rankin-Erickson, 2002)
articles, as well as the two most recent reviews of con were excluded.
tent enhancements used with students with LD (e.g., Fourth, the study must have provided sufficient quan
Gajria et al., 2007; Kim et al., 2004). titative information (e.g., group means and standard
Finally, we conducted hand searches of the following deviations; F statistic) to permit calculation of an effect
journals to locate the most recent literature: Exceptional size (ES). One experimental study (Boyle & Weishaar,
Children, Journal of Educational Psychology, Journal of 1997) was excluded because it provided only multivari
Learning Disabilities, The Journal of Special Education, ate analysis of covariance (MANCOVA) data without
Learning Disability Quarterly, Learning Disabilities group means and standard deviations. According to
GRAPHIC ORGANIZERS AND LD 57
Hunter and Schmidt (2004), there is no algebraically effect size estimate was computed using the following
equivalent method to compute a comparable £ 5 in formulae:
such an instance. f 3 1
Fifth, participants in the experimental and control E S
' s m = 1
" 4N- 9 E S $ m
'
groups must have included students with LD. W e
defined LD the same way as Kim et al. (2004) in their
research review and Swanson, Hoskyn, and Lee (1999) WG1 + G 2
SE = J " -J^ZI ,
in their meta-analysis (e.g., average intelligence and sm
1 n n
+
2(n + n )
G1 G2 G1 G2
poor performance in at least one academic or related
behavioral area). In each of the studies included, all
1 2n
2 z
Finally, the study must have been published in a SE\ m 2{n Gl +n ) G2 + n n (ES' )
G1 G2 sm '
peer-reviewed journal and in English. This excluded
any studies in Dissertation Abstracts International and where N is the total sample size, ES sm is the biased stan
unpublished studies from researchers in the field. dardized mean difference, and
While this criterion ensures that only the highest qual
1
ity research was included in the meta-analysis (Slavin,
1995), it also represents a potential publication bias °'-'sm
n + n n + n
d= t c t c
nn t c n + n -1
t r
is the square root of one divided by the summed inverse sampling variance plus a constant that represents the
variance weight. The z-test for the weighted mean effect variability across the population effects (Lipsey &
size was then computed by dividing the mean effect size Wilson, 2001). The formulae are as follows:
by the standard error of the mean effect size, or in sta
1
tistical notation: Wj = ,
set + vg
ES
sees Qr-k-l
Vg= — ,
Results
A total of 55 unique posttest effect sizes were
The Q statistic is distributed as a chi-square with degrees extracted from studies in 16 published articles meeting
of freedom (df) equaling number of ESs - 1. In our our inclusion criteria. For the purposes of this analysis,
analysis of all studies, the critical value for a chi-square each unique effect size was considered an individual
with df = 47 and p = .05 is 64. Because our calculated Q estimate of effect. (Included articles are marked with an
statistic (57.2) is less than this critical value, we can fail asterisk in the References section.) In addition, eight of
to reject the null hypothesis of homogeneity and the published articles included maintenance data ren
assume a fixed-effects model, under which the variabil dering 29 more unique effect sizes. Table 1 includes
ity across effect sizes does not exceed what would be detailed information on each study, participants, vari
expected based on sampling error. However, Lipsey and ables, measures, and individual effect sizes.
Wilson (2001) warned that a nonsignificant Q statistic
"does not always provide great confidence that a fixed Instructional Context
effects model is justified" (p. 117). Because we do not Each of the studies included instruction on the use of
have a large number of effect sizes and the correspon a GO. The majority of studies (Anders, Bos, & Filip,
ding samples are relatively small, the Q statistic may not 1984; Bos & Anders, 1990, 1992; Boyle, 1996, 2000,
have sufficient statistical power. Therefore, we also fit Darch & Carnine, 1986; Darch, Carnine, & Kame'enui
a random-effects model, which assumes sampling error 1986; Darch & Eaves, 1986; Darch & Gersten, 1986
plus other sources of variability are randomly distrib DiCecco & Gleason, 2002; Englert & Manage, 1991
uted (Lipsey & Wilson). The random-effects model is Griffin, Simmons, & Kame'enui, 1991; Hudson, 1996
also a more conservative estimate than the fixed-effects Ives, 2007; Reyes, Gallego, Duran, & Scanlon, 1989)
model of differences between moderating variables incorporated aspects of direct, explicit instruction (e.g.,
(Hunter & Schmidt, 2004). modeling, prompted practice). The authors of the
remaining study (Bos, Anders, Filip, & Jaffe, 1989)
Whereas the fixed-effects model weights each study reported that written guidelines for teaching the GO
by the inverse of the sampling variance, the random- were developed; however, these guidelines were not
effects model weights each study by the inverse of the included in the article.
GRAPHIC ORGANIZERS AND LD 59
Table 1
Independent Variables, Dependent Measures, and Effect Sizes of Individual Experiments
SOCIAL STUDIES
Semantic Mapping
c o m p r e h e n s i o n test
Semantic Feature Analysis
Visual Display
Table 1 c o n t i n u e d
Independent Variables, Dependent Measures, and Effect Sizes of Individual Experiments
SCIENCE
Semantic Mapping
It
GRAPHIC ORGANIZERS AND LD 61
Table 1 c o n t i n u e d
Independent Variables, Dependent Measures, and Effect Sizes of Individual Experiments
SCIENCE continued
Semantic Feature Analysis
SM/SFA/SSFA Combination
Table 1 c o n t i n u e d
Independent Variables, Dependent Measures, and Effect Sizes of Individual Experiments
SCIENCE/SOCIAL STUDIES
Visual Display
ENGLISH/LANGUAGE ARTS
Cognitive Mapping
Table 1 c o n t i n u e d
Independent Variables, Dependent Measures, and Effect Sizes of Individual Experiments
Visual Display
MATHEMATICS
Visual Display
"'p < .001; "p < .05; *p < .1; all GOs were created by the researchers.
II
64 DEXTER & HUGHES
Generally, instruction for the experimental groups studies were conducted in a resource classroom during
included one to two sessions focused solely on how to or after the regular school day.
use the GO, one to two sessions of prompted practice W h a t Are t h e Overall Effects of GOs o n t h e
using the GO, and independent student use of the GO Posttest P e r f o r m a n c e o f Students W i t h LD?
for the remainder of sessions. During the initial ses After the removal of six outliers, there was a large
sions, the teacher or researcher presented the GO to overall standardized effect of GOs on the posttest per
students and described how it illustrated relationships. formance (e.g., multiple-choice comprehension, vocab
For example, Darch and Carnine (1986) presented their ulary, written recall) of students with LD across all
visual display via an overhead projector, and students studies (£5 = .91, SE = .062) for both random- and fixed-
followed along while the teacher used a script to effects models and a 9 5 % confidence interval of .79,
describe the various cells in the display and the inter 1.03 for the random effects model. Table 2 provides the
relationships between them. full comparison between the random- and fixed-effects
The following sessions generally included the instruc models.
tor explicitly guiding the students in creating or filling
out the GO. For example, Bos and Anders (1990) explic Do These Effects M a i n t a i n Over T i m e ?
itly prompted the students in each step of creating a Twenty-nine studies included maintenance measures.
hierarchical semantic map from a vocabulary list. This In each of the studies, measures consisted of multiple-
level of assistance was gradually faded. For instance, choice comprehension or vocabulary items. These
Darch and Gersten (1986) first presented a visual dis measures were given to students one to four weeks after
play with all the cells labeled, and prompted the entire the conclusion of the intervention.
group in answering questions about specific facts in The test of homogeneity for overall maintenance
the GO. The researchers followed this by guiding the effects produced a nonsignificant Q statistic (Q = 24.49,
students through a visual display that did not provide cv = 35.17). Therefore, similar to the overall posttest
cell labels. Finally, individual students were prompted effects, results of both the random- and fixed-effects
in labeling blank visual displays. Instruction in the models are reported. After removal of five outliers, there
remaining sessions generally focused on independent was a medium overall effect for maintenance across all
use of the GO by the students, in addition to text or lec studies (ES = .56, SE = .074) with a 9 5 % confidence inter
ture presentations. However, in each of the visual dis val of .41, .70 for the random-effects model. Table 3 pro
play studies all of the content was presented solely vides the full comparison between the random- and
through the GO. fixed-effects models.
Each of the interventions lasted between one and Differential Effects
seven weeks, with an additional one to four weeks Based on number of effect sizes and significant Q-val-
between posttest and maintenance measures. All of the ues, the remaining analyses of differential effects are
Table 2
Overall ES for Fixed and Random Models
9 5 % CI
Table 3
Maintenance ES for Fixed and Random Models
9 5 % CI
£5 SE ofES Z-test Lower Upper
Fixed M o d e l .5625 .077 7.305* .41 .71
* p < -001.
reported as the random-effects model. This provides a respectively. Overall mean effect size was 1.07 for
more conservative estimate of effect. posttest and .78 for maintenance.
Are There Differential Effects by Type of Measure Ten individual estimates of effect were calculated for
(Near or Far Transfer)? Forty-five individual estimates far-transfer measures at posttest and two individual esti
of effect were calculated for near-transfer measures at mates of effect for maintenance. These measures tested
posttest and 27 individual estimates of effect for main students' ability to apply knowledge to situations not
tenance. In all but two articles, near-transfer measures directly covered in the text or lecture. For example,
consisted of researcher-generated multiple-choice ques Hudson (1996) included the question "Name one way
tions on material directly covered in the lessons. In the the environment influenced the culture of the Arctic
remaining two articles, Boyle (1996) used a standardized tribes" (p. 81). The teacher never stated the causal rela
measure of reading comprehension and Englert and tion between environment and culture; only facts about
Mariage (1991) used a measure of written free recall, environment and culture were stated.
Table 4
Near- and Far-Transfer - Random-Effects Model
Posttest Maintenance
9 5 % CI 9 5 % CI
ES Lower Upper £5 Lower Upper
Table 5
Type of Graphic Organizer - Random-Effects Model
Posttest Maintenance
95% CI 95% CI
ES Lower Upper ES Lower Upper
Cognitive .8914 .58 1.21 N/A N/A N/A
Mapping n = 7 n = 0
As for near-transfer, far-transfer measures consisted of nance measures, SSFA and SM/SFA/SSFA Combination
researcher-generated multiple-choice questions in all had significantly larger effects than the other GO types
but one study. Boyle (1996) used a standardized meas (e.g., 1.39, 1.01). Table 5 provides the full comparison
ure for far transfer. The overall mean effect size was .61 between types of GOs.
for posttest and .69 for maintenance. Table 4 provides Are There Differential Effects by Subject Area?
the full comparison between near- and far-transfer Posttest effects were calculated for the subject areas of
measures. English/writing/reading, mathematics, science, and
Are There Differential Effects by Type of GO? The social studies. Large posttest effects were found for all
types of GOs used in the studies matched the defini subject (e.g., .96-1.05) areas except mathematics (e.g.,
tions in the introduction to this analysis (e.g., cognitive .59). Mathematics had a moderate posttest effect that
mapping, SM, SFA, SSFA, visual display). However, in was significantly smaller than the other subject areas.
one article containing eight studies (Bos & Anders, Maintenance effects were calculated for mathematics,
1992), the researchers used a combination of SM, SFA, science, and social studies. Science had a large mainte
and SSFA. The method they utilized to present their nance effect (e.g., .80) that was significantly larger than
results prohibited disaggregation of the findings. the moderate effects for mathematics and social studies.
Therefore, a sixth category (SM/SFA/SSFA Combination) Table 6 provides the full comparison between GOs by
was added to the analysis. Large posttest effects (e.g., subject area.
.74-1.2) were found for all types of GOs except visual Are There Differential Effects by Stage of
displays. Visual displays had a moderate effect (e.g., Attending to Verbal Material (Before, During, After
.74). There were no statistically significant differences Instruction)? There was not enough information to
between GOs with large posttest effects. For mainte quantify differential effects by stage of instruction. All
GRAPHIC ORGANIZERS AND LD 67
but one study included GOs before and during instruc consistent with the theories of Ausubel (1968) and
tion, and not enough information was provided to dis Mayer (1979) that GOs may especially assist lower abil
aggregate these data. One study, Englert and Mariage ity learners in both basic and higher level skills by cre
(1991), used GOs after instruction. The unstandardized ating an easier context for assimilating information into
effect size for a near-transfer, written free recall measure their memory.
was large (ES = 1.84). Do These Effects M a i n t a i n Over T i m e ?
There was a moderate mean effect for maintenance
Discussion
(ES = .56, SE = .07) of students with LD. The significant
As was the casae in previous research syntheses (e.g.,
drop-off from posttest to maintenance is consistent
Gajria et al., 2007; Kim et al., 2004; Moore & Readence,
with the findings of the other GO research syntheses.
1984), findings from this meta-analysis indicate that The drop has been attributed to lack of clarity regarding
GOs improve the factual comprehension of upper- the duration and length of intervention sessions needed
elementary, intermediate, and secondary students with to positively affect maintenance (Gajria et al., 2007;
LD. Unlike these previous reviews, this analysis also Gersten, Fuchs, Williams, & Baker, 2001). The relatively
indicates that GOs may improve vocabulary and infer short duration of the intervention studies (e.g., 1-7
ence/relational comprehension for students with LD. weeks) may not have provided sufficient instruction
W h a t Are t h e Overall Effects of GOs o n t h e time for students to use GOs independently. However, a
Posttest P e r f o r m a n c e of Students W i t h LD? closer look at effects by type of GO shows that effect
Overall, there was a large mean effect for posttest per sizes for visual displays and SSFA were larger for main
formance (ES = .91, SE = .06) of students with LD using tenance than posttest. This may lend support to the
both the fixed-effects and random-effects models. The visual argument hypothesis (Waller, 1981), which posits
immediate posttest performance spanned multiple con that GOs that are structured in a way that facilitates
structs, including multiple-choice factual comprehen understanding and perception of concept relationships
sion, vocabulary, and written recall requiring relational are superior to more complicated GOs that may require
comprehension. This suggests that GOs are effective in instruction to recognize conceptual relationships
not only improving basic skills (e.g., factual recall) but (Dexter, 2010). The visual displays and SSFA were more
also higher-level skills (e.g., inference). This finding is computationally efficient than the other GOs. That is,
Table 6
Subject Area - Random-Effects Model
Posttest Maintenance
95% CI 95% CI
ES Lower Upper ES Lower Upper
they were simple enough for students to recognize con groups in factual recall, but the GO groups significantly
ceptual relationships without teacher instruction. This outperformed the outline and text-only groups in iden
may explain why maintenance effects were larger for tifying concept relations and making far-transfer con
these types of GOs. cept comparisons.
Are There Differential Effects b y T y p e o f Are T h e r e Differential Effects b y Subject Area?
Measure (Near o r Far Transfer)? This analysis also examined the effects of GOs by
This meta-analysis also separated results into near- subject area. All subject areas had moderate to large
transfer and far-transfer measures. Near-transfer results effects for posttest and maintenance measures. The
(i.e., measures applying knowledge to situations largest effects were in science (ES = 1.05 for posttest, ES
directly covered in the text or lecture) indicate that = . 8 0 for maintenance) and the smallest effects in
GOs are effective strategies for improving factual recall, mathematics (ES = .59 for posttest, £5 = .46 for mainte
factual and relational comprehension, and vocabulary nance). The large effects in science may be explained by
knowledge. Across all near-transfer studies, the mean the unfamiliar, technical vocabulary and content often
effect size was large (ES = 1.07), and m a i n t e n a n c e based on relationships between concepts (Lovitt et al.,
effects were moderate (ES = .78). Students using GOs 1986). This type of content lends itself to computa
significantly outperformed their peers receiving typical tionally efficient GOs that make relationships explicit
classroom instruction on near-transfer measures. and clear. Also, students may rely more heavily on con
Interestingly, more complicated GOs requiring inten tent enhancements like GOs when c o n t e n t seems
sive teacher instruction (e.g., SM, SFA) resulted in the strange or foreign.
largest effects for near-transfer posttest measures. This The small effects for mathematics may be explained
indicates that while these GOs are difficult to under by the fact that the information was much more
stand independently, with appropriate instruction they abstract than the other subject areas. It may also be
are superior to less complicated GOs for immediate fac explained by the fact that the mathematics study used
tual recall. a visual display, which only had a moderate overall
Far-transfer results (i.e., measures applying knowl effect. Use of GOs with mathematics concepts and solv
edge to situations not directly covered in the text or ing systems of linear equations are only beginning in
lecture) indicate that GOs may also improve the infer the field. Ives (2007) offerred several implications for
ence skills and relational knowledge of secondary stu future research based on his initial study in this field. It
dents with LD. Across all far-transfer studies, the mean will take time and more study to fully understand the
effect size was moderate (£5 = .61), and maintenance effects of GOs on mathematics understanding.
effects were moderate (£5 = .69). Are There Differential Effects b y Stage of
It is interesting to note that for far-transfer measures, Attending t o Verbal Material (Before, During,
maintenance effect sizes were larger than posttest effect After Instruction)?
sizes. Previous research has indicated students with LD Finally, a previous GO research synthesis (Moore &
typically perform poorly on far-transfer tasks due to Readence, 1984) reported that GOs presented as text
their inability to detect underlying concepts in verbal summaries after instruction were more effective than
information due to difficulty assimilating verbal infor GOs presented before or during instruction. This find
mation with previous knowledge (Suritsky & Hughes, ing cannot be corroborated by the present analysis
1991). The results of this analysis demonstrate that because only one study (Englert & Mariage, 1991)
GOs may bridge the gap between verbal information used GOs after instruction. While this study had an
and prior knowledge and assist students with LD in far- extremely large effect size (1.84), more studies are
transfer tasks. This finding supports Mayer's (1979) needed to confirm this finding. The current analysis
assimilation theory, which posits that GOs that assim points to effective instruction and choice of GO to be
ilate material to a broader set of past experiences allow more important than stage of attending to verbal mate
superior transfer to new situations. rial in effectiveness of the intervention.
The finding is also consistent with the research of Methodological Limitations
Robinson and colleagues (Robinson et al., 1 9 9 8 ; There are two methodological limitations to the con
Robinson & Kiewra, 1995; Robinson & Schraw, 1994; clusions of this analysis. First, there is the possibility of
Robinson & Skinner, 1996), comparing visual displays a publication bias. According to Smith (1980), as well as
(e.g., tree diagrams, matrices, network charts) with tra Lipsey and Wilson (1993), published articles have a
ditional, non-graphic outlines. In each of the studies, larger mean effect size than unpublished studies. This
groups of nondisabled college students using GOs and bias, also known as a "file-drawer" effect, happens
traditional outlines equally outperformed text-only because studies with null findings are less likely to be
GRAPHIC ORGANIZERS AND LD 69
published by major journals, often leaving offending tant to note that all but one study used measures that
data in a researcher's file drawer (Lipsey & Wilson, were researcher-created and closely tied to the content.
2001). In this analysis, based on the advice of Slavin While these measures should have good content valid
(1995), we purposefully selected only published studies ity, there is no way to measure broader construct valid
to ensure the highest quality of research designs. While ity. Only Boyle (1996) used a standardized measure for
this eliminated our ability to compare mean effect sizes reading comprehension. This fact may limit the gener
of published studies versus unpublished studies, we alizability of these findings and questions the actual
were able to utilize Rosenthal's (1979) fail-safe N statis level of understanding obtained by students in the GO
tic, which was later adapted by Orwin (1983). This sta conditions.
tistic determines "the number of studies with an effect Finally, studies using Content Enhancement
size of zero needed to reduce the mean effect size to a Routines were not included in this meta-analysis
specified or criterion level" (Lipsey & Wilson, 2 0 0 1 , because their effects could not be attributed solely to
p. 166). the GO, even though GOs are a vital component of
The formula is as follows: each routine. Research on Content Enhancement
Routines has been conducted for over two decades,
focusing primarily on assisting all students, including
*o = * [ S - l " , those with LD, thrive in the often rigorous intermedi
ate and secondary general education content class
0
[ES C J
where k is the number of effect sizes of zero needed to
rooms (Bulgren, 2006; Bulgren, Deshler, & Lenz, 2007
These routines have been shown to improve both basic
reduce the mean effect size to ES (criterioneffect level),
C skills (e.g., factual knowledge, comprehension) and
k is the number of current studies, and ES is the cur
k higher-order skills (e.g., manipulation, extension, gen
rent weighted mean effect size. Using this statistic, eralization) for students with LD (e.g., Bulgren et al.,
reduction of our overall weighted effect size of .91 to 2000; Bulgren et al., 2002; Bulgren et al., 2009; Scanlon
.50 would take 45 additional unpublished studies with et al., 1996). While results of these studies did not fit
an effect size of zero. While this may be a possibility, into our analysis, they do serve as examples of effective
it is unlikely that 45 additional null studies exist in interventions utilizing GOs as a component of a larger
researchers' file drawers. routine.
Second, our 55 unique effect sizes or studies were
culled from only 16 published articles. While this is Implications for P r a c t i c e
an acceptable practice (Lipsey & Wilson, 2001), it does The major implication of this study for applied prac
limit the generalizability of the findings because there tice is consistent with assimilation theory and the
were only 21 distinct samples of students with LD visual argument hypothesis; that is, more instruction-
(total N - 808). Therefore, caution should be exercised intensive types of GOs (e.g., SM, SFA) are better for
in generalizing these findings to all intermediate and immediate factual recall while more computationally
secondary students with LD. efficient GOs (e.g., visual display, SSFA) are better for
maintenance and transfer. This knowledge can help
Individual Study L i m i t a t i o n s teachers in designing GOs for initial instruction and for
Three limitations to the individual studies warrant re-teaching, studying, and retention purposes. For
consideration. First, while all of the effect sizes in the instance, a semantic map for initial instruction, fol
present analysis were based on differences between a lowed by a simpler visual display for review and study
treatment group and a control group, it is not clear if will potentially maximize the effects of recall, mainte
the control conditions constituted an adequate stan nance, and far-transfer for students with LD.
dard to measure the effects of GO interventions Another implication for practice is that, regardless of
(Gersten, Baker, & Lloyd, 2000). The control conditions GO type, a teacher must explicitly teach students how
in the studies used primarily typical classroom practices to use a given GO. Students with LD need explicit
(e.g., dictionary instruction) rather than more closely instruction to understand how concepts are related, to
comparable practices (e.g., outlines, structured over recognize differences between main and subordinate
views). While this provides much evidence for GO ideas, and to put all the pieces together to make a clear
effects over typical classroom practice, it does not yield picture of the content being learned no matter how
information comparing GOs with other researched implicit a GO may seem. A teacher's use of effective
practices (Kim et al., 2004) instruction practices (e.g., modeling, corrective feed
Second, while the results indicate large effects for back, etc.) will positively impact the intervention's
vocabulary, inference, and comprehension, it is impor effectiveness.
70 DEXTER & HUGHES
Minskoff, E., & Allsopp, D. ( 2 0 0 3 ) . Academic success strategies for Rosenthal, R. ( 1 9 9 4 ) . Statistically describing and combining stud
adolescents with learning disabilities and ADHD. Baltimore, MD: ies. In H. Cooper & L. V. Hedges (Eds.), The handbook of research
Brookes Publishing. synthesis (pp. 2 3 1 - 2 4 4 ) . New York, NY: Russell Sage Foundation.
Moore, D. W., & Readence, J . F. ( 1 9 8 4 ) . A quantitative and quali Scanlon, D., Deshler, D. D., & Schumaker, J . ( 1 9 9 6 ) . C a n a strat
tative review of graphic organizer research. Journal of Educational egy be taught and learned in secondary inclusive classrooms?
Research, 78, 1 1 - 1 7 . Learning Disabilities Research and Practice, 11, 4 1 - 5 7 .
Nesbit, J . C , & Adesope, O. O. ( 2 0 0 6 ) . Learning with concept Sinatra, R. C , Stahl-Gemake, J . , & Berg, D. N. ( 1 9 8 4 ) . Improving
and knowledge maps: A meta-analysis. Review of Educational reading comprehension of disabled readers through mapping.
Research, 76, 4 1 3 - 4 4 8 .
The Reading Teacher, 38, 2 2 - 2 9 .
Orwin, R. G. ( 1 9 8 3 ) . A fail-safe iV for effect size in meta-analysis.
Slavin, R. E. ( 1 9 9 5 ) . Best evidence synthesis: An intelligent alter
Journal of Educational Statistics, 8, 1 5 7 - 1 5 9 .
native t o meta-analysis. Journal of Clinical Epidemiology, 48,
Pearson, P. D., & J o h n s o n , D. ( 1 9 7 8 ) . Teaching reading comprehen
9-18.
sion. New York, NY: Holt, Rinehart, & Winston.
Pehrsson, R., & Denner, P. ( 1 9 8 8 ) . Semantic organizers: Smith, M. L. ( 1 9 8 0 ) . Publication bias a n d meta-analysis.
Implications for reading a n d writing. Topics in Language Evaluation in Education, 4, 2 2 - 2 4 .
Disorders, 8, 2 4 - 3 7 . Sturm, J . M., & Rankin-Erickson, J . L. ( 2 0 0 2 ) . Effects of hand-
*Reyes, E. I., Gallego, M. A., Duran, G. Z., & Scanlon, D. J . ( 1 9 8 9 ) . drawn and computer-generated concept mapping o n the expos
Integration of internal concepts and external factors: Extending itory writing of middle school students with learning
the knowledge of learning disabled adolescents. The Journal of disabilities. Learning Disabilities Research & Practice, 17, 1 2 4 - 1 3 9 .
Early Adolescence, 9, 1 1 2 - 1 2 4 . Suritsky, S. K., & Hughes, C. A. ( 1 9 9 1 ) . Benefits of notetaking:
Rice, G. E. ( 1 9 9 4 ) . Need for explanations in graphic organizer Implications for secondary and postsecondary students with
research. Reading Psychology: An International Quarterly, 15, 3 9 - learning disabilities. Learning Disability Quarterly, 14, 7-18.
67. Swanson, H. L., Hoskyn, M., & Lee, C. ( 1 9 9 9 ) . Interventions for stu
Rivera, D. P., & Smith, D. ( 1 9 9 7 ) . Teaching students with learning dents with learning disabilities. New York, NY: Guilford Press.
and behavior problems (3rd ed.). Boston, MD: Allyn & Bacon. Thalheimer, W., & Cook, S. ( 2 0 0 2 , August). How to calculate effect
Robinson, D. H., Katayama, A. D., DuBois, N. F., & Devaney, T. sizes from published research articles: A simplified methodology.
( 1 9 9 8 ) . Interactive effects of graphic organizers and delayed
Retrieved from http://work-learning.com/effect_sizes.htm.
review o n concept acquisition. Journal of Experimental Education,
Waller, R. ( 1 9 8 1 , April). Understanding network diagrams. Paper pre
67, 1 7 - 3 1 .
sented at the annual meeting of the American Educational
Robinson, D. H., & Kierwa, K. A. ( 1 9 9 5 ) . Visual argument: Graphic
Research Association, Los Angeles.
organizers are superior to outlines in improving learning from
text. Journal of Educational Psychology, 87, 4 5 5 - 4 7 6 . Williams, J. P. ( 1 9 9 3 ) . Comprehension of students with or without
Robinson, D. H., & Schraw, G. ( 1 9 9 4 ) . Computational efficiency learning disabilities: Identification of narrative themes and idio
through visual argument: Do graphic organizers c o m m u n i c a t e syncratic text representations. Journal of Educational Psychology,
relations in t e x t t o o effectively? Contemporary Educational 85, 6 3 1 - 6 4 1 .
Psychology, 19, 3 9 9 - 4 1 5 .
Robinson, D .H., & Skinner, C. H. ( 1 9 9 6 ) . W h y graphic organizers * = Studies included in meta-analysis.
facilitate search processes: Fewer words or c o m p u t a t i o n a l l y
efficient indexing? Contemporary Educational Psychology, 21,
166-180. Please address correspondence about this article to: Douglas D.
Rosenthal, R. ( 1 9 7 9 ) . The "file-drawer" problem and tolerance for Dexter, 2 3 0 CEDAR, The Pennsylvania State University, University
null results. Psychological Bulletin, 86, 6 3 8 - 6 4 1 . Park, PA 1 6 8 0 2 ; email:Dddl76@psu.edu