Professional Documents
Culture Documents
Volume 51 Number 3
Summer 2007 272-284
© 2007 National Association for
Gifted Children
10.1177/0016986207302722
A Study of Curriculum Effectiveness in http://gcq.sagepub.com
hosted at
Catherine A. Little
University of Connecticut
Annie Xuemei Feng
Joyce VanTassel-Baska
The College of William and Mary
Karen B. Rogers
University of New South Wales
Linda D. Avery
Private Consultant
Abstract: This quasi-experimental study examines the effects on student performance of a Javits-funded curriculum
designed to respond to the needs of high-ability students in elementary and middle school social studies. The cur-
riculum, implemented with all students in heterogeneous classrooms, addresses state standards while integrating
advanced content, higher level process emphases, and a conceptual orientation. Data collection focuses on student
performance in conceptual reasoning, critical thinking, and content learning and on teacher demonstration of specific
desired teaching behaviors. Results demonstrate significant and important differences between treatment and com-
parison groups in the area of content learning, favoring the treatment group; no significant differences are found for
the small subsample of gifted students. Subanalyses yield differential results for specific units and schools, poten-
tially indicating issues of treatment fidelity. Contextual challenges and implications of the study are discussed,
including issues related to social studies curriculum implementation and differentiation in the current standards-based
environment.
Putting the Research to Use: This study provides evidence useful in practice and in further research, in terms of
both its specific findings and the contextual issues that emerged in the implementation of the study. First, the study
supports the idea that curriculum that integrates higher level processes and specific conceptual thinking activities
with strong content may yield content gains as strong as or stronger than a more direct, knowledge-based struc-
ture for teaching to standards. In an era heavily focused on standards coverage, this finding is an important
reminder that content standards may be used as a basic frame for a larger tapestry of more complex teaching and
learning in a content area. Second, the study demonstrates that such integrated and challenging curriculum and
instruction, designed to respond to the needs of highly able learners, may also promote growth gains for students
not identified as gifted, supporting the need for high expectations for all learners. Third, the study supports claims
from the social studies literature that integration of the habits of mind of the social science disciplines is impor-
tant for content learning.
At the same time, the study reminds practitioners that although challenging curriculum designed to meet the
needs of gifted students may promote learning for nonidentified students, appropriate instructional differentiation
is still necessary to strengthen the possibility that students of varying ability levels will benefit from the learning
experience and that the challenge level for highly able students is not diluted. Focusing on a content area that often
receives limited attention and limited differentiation in the elementary and middle school years, this study sup-
ports greater research and practice attention to the needs of gifted students in their social studies learning.
272
Instrument N M SD M SD N M SD M SD
CNTA 589 11.1 4.7 13.2 4.5 139 11.5 4.6 12.6 4.5
CRTA 563 8.1 2.1 8.9 2.3 141 7.4 1.8 8.7 2.2
Content knowledge 276 18.2 13.7 28.3 15.1 63 10.5 7.7 12.9 8.6
Note: CNTA = conceptual thinking test; CRTA = critical thinking test.
Table 3
One-Way ANCOVA Conducted for Whole Sample
Dependent Variable N df F Significance Effect Size (η2)
Table 4
One-Way ANCOVA Conducted for Identified GT and Non-GT Students
Group Dependent Variable N df F Significance Effect Size (η2)
statistics were used to examine patterns of change in the favoring the treatment group. Using eta squared as the
six teachers’ instructional practices against the nine cat- index for effect size, the treatment effect was moder-
egories of teaching behaviors delineated in the COF. ate in content knowledge (.11).1 No statistically sig-
nificant differences were found in concept reasoning
and critical thinking between the two groups.
Results Additional ANCOVAs were performed to examine
the treatment effect for identified gifted students and
Pre- and post-assessment means for the treatment nonidentified students, respectively. Table 4 summa-
and comparison groups on all three measures are rizes the results. For the gifted students, no signifi-
given in Table 2. ANCOVA tests were run to examine cant differences were detected between the treatment
differences between the treatment and comparison and comparison group in any of the three areas. It
students’ post-assessment scores in concept reason- should be noted that this result was affected by the
ing, critical thinking, and content knowledge, con- limited data available from gifted students (N < 31)
trolling their initial differences using pre-assessment for the analyses. For nongifted students, statistically
scores. As Table 3 summarizes, statistically signifi- significant differences were found in students’
cant differences in students’ post-assessment scores posttest scores in content knowledge, favoring the
were registered in content knowledge (p < .007), treatment group (p < .005). The effect sizes, using eta
Table 6
One-Way ANCOVAs Conducted by Unit
Unit Dependent Variable N df F Significance Effect Size (η2)
squared, was moderate (.11). No significant differ- treatment and comparison students’ post-assessment
ences were found in concept reasoning and critical in concept reasoning.
thinking between the treatment and comparison Further ANCOVAs were performed to examine the
group, with their initial differences controlled. treatment effect for four of the units administered.
ANCOVAs were also conducted to explore treatment Treatment effect could not be measured for the fifth
effect by school. Four schools hosted both treatment and unit (The 1920s), as data were incomplete for one or
comparison students; however, no post-assessment data the other group on all measures. Table 6 shows that sta-
were available for one participating school. Thus, the tistically significant differences were found in
school-level ANCOVAs were conducted for the three students’ posttest performance in content knowledge
schools for which both pre- and post-assessment data within two unit groups, Civilization and Civil War,
were available. Table 5 presents the results. A statis- favoring the treatment group. The implementation of
tically significant treatment effect was found in both units also resulted in moderate to large effect sizes
students’ post-assessment in content knowledge in (eta squared results were .11 and .42, respectively),
two schools, favoring the treatment group; the effect indicating the differences between treatment and com-
sizes were large (eta squared = .16 and .62, respec- parison groups had practical importance. For students
tively), indicating educational importance as well as who were exposed to the Government unit, statistically
statistical significance. At one school (School 2), a significant differences were also found in their
statistically significant difference was found in post-assessment in critical thinking, favoring the treat-
students’ critical thinking post-assessment, favoring ment group. The effect size using eta squared reached
the comparison group, with a moderate effect size. No .12, indicating that the magnitude of treatment
statistically significant differences were found between students’ gains in critical thinking was fairly large and
Planning 3 2.50 0.77 2.25 0.69 2.50 0.84 — — 2.33 0.52 2.17 0.98
Expectations 3 2.30 0.88 2.75 0.61 3.00 0.00 — — 3.00 0.00 2.17 0.98
Accommodating 5 2.33 0.98 2.33 0.61 3.17 0.75 — — 3.67 1.03 3.17 1.17
individual
differences
Curriculum 3 1.33 0.98 1.75 0.88 2.50 0.84 — — 2.17 0.75 1.83 0.98
delivery
features
General strategies 9 3.83 2.70 4.42 0.86 5.50 1.22 — — 6.33 1.51 4.67 2.88
Critical thinking 5 2.25 1.13 2.50 0.63 3.00 1.10 — — 4.00 2.00 2.33 1.03
Problem solving 6 1.58 1.16 2.25 0.52 2.50 1.64 — — 1.00 1.10 0.83 0.98
Metacognition 3 0.58 1.02 1.25 0.88 0.16 0.40 — — 1.50 1.05 0.67 0.52
Extensions 3 0.17 0.26 0.33 0.61 0.17 0.41 — — 0.33 0.52 0.17 0.41
Total 40 16.80 8.05 19.83 2.48 22.50 2.81 24.30 5.61 20.17 7.36
Note: COF = classroom observation form.
educationally meaningful. No significant differences four observations were conducted for each of these
were found between treatment and comparison teachers, with two (pre-observations) conducted
students who used the Colonial unit. No treatment before the implementation of the curriculum but
effect was detected in conceptual reasoning for after the training workshop on how to use the curricu-
students in these four units. lum; two observations (post-observations) followed
during the implementation of the unit. For analysis
purposes, the two pre-observations scores for the par-
Teacher Results ticipants were averaged as a Year 1 pre-observation
score; the two post-observation scores were averaged
Six teachers, out of an original sample of 26, as a Year 1 post-observation score. In Year 2 of the
remained in Project Phoenix for the 3 years of imple- project, only the observation data collected before
mentation. These six teachers were all female and var- the implementation of the unit in that year were
ied in age between younger than 25 and older than available for analysis. In Year 3 of the project, two
50. Four of these teachers taught second graders observations were conducted separately near the
(Civilizations unit) for the 3 years, and two teachers beginning and near the end of implementation.
taught fourth and fifth graders (Colonial unit). All of Because of the small number of cases available for
the teachers held a BA or BS, and one held a master’s analysis (N = 6) for the teacher behavior data, descrip-
degree. All of them reported having taken at least six tive statistics were used to document these six
credits of graduate coursework, except the youngest teachers’ instructional practice changes from Year 1 to
teacher, who had taken only one graduate course; none Year 3 implementation. Table 7 illustrates means and
of them had taken any courses in gifted education. standard deviations of instructional behaviors against
Their teaching experience ranged from 3 years to more the nine behavioral categories. The results showed that
than 20 years. All teachers had received at least 5 days there was a consistent pattern of improvement in eight
of training related to the project curriculum, with the out of the nine categories from initial observations
majority of the training conducted in Year 1 and shorter to later observations in Year 1 implementation; this
follow-up sessions conducted in the subsequent years increasing behavioral pattern continued from Year 1 to
of the project. Year 2 implementation, as recorded by external obser-
Three years of observation data for the six teachers vation results during that year. From Year 2 observa-
were available for analysis. In Year 1 of the project, tion (pre-observation only) to Year 3 pre-observation,