What Makes Professional Development Effective Strategies That Foster Curriculum Implementation

What Makes Professional Development Effective?
Strategies That Foster Curriculum

Implementation
Author(s): William R. Penuel, Barry J. Fishman, Ryoko Yamaguchi and Lawrence P. Gallagher
Source: American Educational Research Journal, Vol. 44, No. 4 (Dec., 2007), pp. 921-958
Published by: American Educational Research Association
Stable URL: http://www.jstor.org/stable/30069418 .
Accessed: 28/06/2014 12:09
Your use of the JSTOR archive indicates your acceptance of the Terms & Conditions of Use, available at .
http://www.jstor.org/page/info/about/policies/terms.jsp
.
JSTOR is a not-for-profit service that helps scholars, researchers, and students discover, use, and build upon a wide range of
content in a trusted digital archive. We use information technology and tools to increase productivity and facilitate new forms
of scholarship. For more information about JSTOR, please contact support@jstor.org.
American Educational Research Association is collaborating with JSTOR to digitize, preserve and extend
access to American Educational Research Journal.
http://www.jstor.org
This content downloaded from 91.238.114.210 on Sat, 28 Jun 2014 12:09:51 PM

All use subject to JSTOR Terms and Conditions
AmericanEducationalResearch Journal
December2007, Vol.44, No. 4, pp. 921-958
DOLT 10.3102/0002831207308221
C 2007 AERA.http://aerj.aera.net
WhatMakesProfessional
DevelopmentEffective?Strategies
ThatFosterCurriculumImplementation
WilliamR. Penuel
SRIInternational
BarryJ. Fishman
Universityof Michigan
RyokoYamaguchi
AbtAssociates
LawrenceP. Gallagher
SRIInternational
Thisstudy uses a sample of 454 teachers engaged in an inquiry science pro-

gram to examine the effectsof different characteristics ofprofessional develop-
ment on teachers' knowledge and their ability to implement theprogram. The
authors analyzed resultsfrom a survey of teachers served by 28 professional
development providers within a hierarchical linear modeling framework.
Consistentwith findings from earlier studies of effectiveprofessional develop-
ment, this study points to the significance of teachers'perceptions about how
coherent theirprofessional developmentexperiences werefor teacher learning
and program implementation. The authors also found that the incorporation
of timefor teachers toplanfor implementation andprovision of technical sup-
port were significant forpromoting program implementation in theprogram.
KEYWORDS: curriculumimplementation,professionaldevelopment,science
education
Policymakers,school and districtleaders,and researchersare all increas-

Policy
ingly concernedwith improvingthe qualityof evidence about the effec-
tivenessof teacherprofessionaldevelopment,especiallyin termsof its impact
on desiredreformoutcomes.At the federallevel, for instance,programsat
the U.S.Departmentof Educationand the NationalScienceFoundation(NSF)
fundstudiesaimedatmeasuringthe impactof effortsto improveteacherqual-
ity on instructionand studentachievement.Withinschools and districts,lead-
ers are increasinglylooking to providersof professionaldevelopment for
evidence thattheiractivitiescontributeto improvedstandardizedtest scores,
especiallyamonglow-performingstudents.Andfortheirpart,researchersare

Penuel et al.
increasingly concerned with describing the linkage between the design and
conduct of professional development and subsequent improvements to both
teacher practice and student learning outcomes (Borko, 2004; Fishman,Marx,
Best, & Tal, 2003).
Significant efforts to develop high-quality curriculum materials aligned
with standards are also under way, particularlyin mathematics and science,
in light of findings that existing materials are not sufficient to support stu-
dent learning (Roseman, Kulm, & Shuttleworth, 2001). In conjunction with
these efforts, research is needed that examines what kinds of professional
development provide support for the implementation of these curricula.
Professional development is widely believed to be required for supporting
implementation (Smylie, 1996; Spillane & Thompson, 1997), and some large-
scale survey studies have shown how professional development can influ-
ence teachers' knowledge and practice (Garet, Porter, Desimone, Birman, &
Yoon, 2001; Supovitz & Turner, 2000). These studies provide a strong basis
for developing hypotheses about what makes professional development
effective, because they rely on nationally representative samples and include
teachers who experienced a wide variety of professional development activ-
ities. However, three aspects of these studies limit their utility for under-
standing curriculum implementation: (a) the very breadth of the programs
studied to date limits the depth of questions about changes to practice that
could be asked of teachers, (b) the models tested in these studies did not
include sources of data from professional development providers them-
selves, and (c) an objective measure of program implementation was not
included as part of these studies. These limitations are driven largely by the
practicalities required of researchers studying such broad programs; there-
fore, new research is needed that can illuminate how particular programs'
designs and requirements of teachers might influence what makes profes-
sional development effective for promoting curriculum implementation.
In this article, we report on evidence of effective professional develop-
ment conditions and practices from a study of 454 teachers taking part in pro-
fessional development preparing them to implement materials from the
WILLIAM R.PENUEL is Directorof EvaluationResearchat the CenterforTechnology

His researchexaminesconditionsforeffectiveimple-
in Learningat SRIInternational.
mentationof technology-supported educationalinnovationsin science education.
BARRY J. FISHMANis an associateprofessorof learningtechnologiesin the School
of Educationat the Universityof Michigan.His researchfocuses on teacherlearning
and its relationshipto studentoutcomes and on models for scalingof technology-
supportedinnovations.
RYOKO YAMAGUCHI researcher
is an educational Herresearch
at AbtAssociates.
focuses on educationpolicy, groupdynamics,and healthbehavior.
LAWRENCE P. GALLAGHERis an educational at SRIInternational.
researcher His
researchfocuses on formaland informallearningenvironments,epistemicdevelop-
ment, and assessmentmethodologies.
922

EffectiveProfessionalDevelopment
GLOBEProgram,an international earth-scienceeducationprogram.We drew
on multiplesources of data for our study:surveysof professionaldevelop-
mentproviderson the design of theirprograms,surveysof teacherson their
perceptionsof professionaldevelopmentand implementationpractices,and
objectivedataon implementationobtainedfroma programdatabase.Unlike
previouslarge-scalestudies of professionaldevelopment,we were able to
takeadvantageof an unusualopportunityto explorewhatmakesprofessional
developmenteffective:a situationin whichteachersarebeing askedto imple-
ment a common programwith a well-articulatedmodel of implementation
fidelityand a readilyavailableobjectivemeasureof implementationin which
the professionaldevelopmentconditionsand practicesare allowed to vary
significantlyfromone settingto another.
We analyzedthe datawithina hierarchical linearmodeling(HLM)frame-
work (Raudenbush& Bryk,2002), and we reporthere on answersto three
researchquestions:(a) Whatkinds of professionaldevelopmentactivitiesin
GLOBEare associatedwith increased levels of programimplementation?
(b) Whatkinds of professionaldevelopmentactivitiesin GLOBEare associ-
atedwith increasedteacherknowledgeand changesto scienceteachingprac-
tice? (c) How do support and follow-up after professionaldevelopment
influenceprogramimplementationand teacherknowledge and changes to
science teachingpractice?
Backgroundto the Study

In recent years, researchershave increasinglyfocused on what makes
professionaldevelopmenteffective.Thistrendis an improvementcompared
withthe decadesin whichlittleattentionwas directedto the outcomesof pro-
fessionaldevelopmentand much to evaluationsof teachersatisfactionwith
professionaldevelopmentexperiences(Frechtling,Sharp,Carey,& Vaden-
Kiernan,1995). Recent researchexplores the complex links between the
design of professionaldevelopment,teachers'learningduringprofessional
developmentactivities,andsubsequentchangesin classroompractice(Borko,
2004). In addition,researchersare beginning to take up the challenge of
designingstudiesthatcan help identifythe linkagebetween the design and
implementationof professionaldevelopmentand studentlearningoutcomes
(Fishmanet al., 2003;Loucks-Horsley & Matsumoto,1999).
One of the most notable recent studies of professionaldevelopment
effectiveness(Garetet al., 2001)was a large-scalestudythatused regression
modelingto examinehow core structuralcomponentsof professionaldevel-
opment funded throughthe EisenhowerMathand Science programled to
self-reportedchanges in teachers'knowledge and practice.The researchers
in this studyfound evidence supportingthe value of the followingstructural
features of professional development: reform orientation(with reform-
oriented activities such as teacher study groups being more effective than tra-
ditional professional development settings such as workshops or college
courses), duration (in terms of both time span and total contact hours), and
923

Penuel et al.
the collectiveparticipationof teachersfromthe same school. The following
core featuresalso contributedto enhancedknowledgeand skillsandchanges
in teachingpractice:a focus on contentknowledge,activeor inquiry-oriented
learningapproachesin the professionaldevelopmentexperience,and a high
level of coherencewith otherreformactivitiesand standardsin the teachers'
local school contexts.
This particularstudy markedan importantadvance within the field,
because it reliedon datafroma nationalprobabilitysampleof teacherswho
had takenpartin professionaldevelopmentactivitiesthatvariedwith respect
to key featuresof interest,such as contentfocus and duration.Manyof the
featuresof professionaldevelopmentthatGaretet al. (2001)foundto be sig-
nificantpredictorsof effectivenesshad alreadybeen identifiedin the litera-
tureas contributingto the qualityof professionaldevelopment(e.g., Hawley
&Valli,1999;Loucks-Horsley, Hewson, Love,& Stiles,1998;Loucks-Horsley
& Stiles,2001), but before the Eisenhowerstudy, empiricalevidence of the
relativevalue of specificprofessionaldevelopmentfeatureswas limited.The
researcherswho conductedthe evaluationof the Eisenhowerprogramtook
advantageof a significantopportunityto advanceknowledgeaboutthe fea-
tures of effective professionaldevelopment in that they had access to a
nationallyrepresentativesample of teachers,for whom professionaldevel-
opment activitiesand self-reportedoutcomesvariedwidely. The content of
teachers'professionaldevelopmentincludedtargetingimprovementsto con-
tent knowledge, pedagogicalstrategies,alignmentof curriculumand assess-
ment,and a rangeof othertopics(Porter,Garet,Desimone,&Birman,2003).
Forsome teachers,theirprofessionaldevelopmentactivitieslastedless than
a day,whereasfor othersit lastedseveraldaysover a span of severalmonths.
Finally,some teachersreportedsignificantchanges to theirknowledge and
practice in multiple areas, although others reportedthat the professional
developmentactivitiesin which they had takenparthad had a limitedeffect
(Garetet al., 2001).
LimitationsAssociatedWithBreadthof
ProfessionalDevelopmentProgramsStudied
The breadthof the Eisenhowerprogrammakes it difficultto drawcon-
clusionsaboutwhat makesprofessionaldevelopmenteffectivewithina sin-
gle domain, such as science, or for a specific programor set of curriculum
materials.The questions that were sensible for Garetet al. (2001) to pose
on a teachersurveytargetingmultipleactivitiesabout,for example,subject
mattercontent or specific programoutcomeswere necessarilygeneraland,
therefore,harderto use to determinesuccess or failureof implementationof
a specific programmodel beyond a rough approximation.Findingsfrom
Supovitzand Turner's(2000) analysisof professionaldevelopmentactivities
within the Local Systemic Change initiative of the NSF offer some insight
about professional development features that matter most in the domain of
science for fostering use of inquiry-oriented instructional materials among
924

teachers.Butthis study,too, had to relyon generalquestionsaboutchanges
to teacherpracticethat could be answeredby teachersinvolved in each of
the 24 differentcurriculumprojectsbeing studied.
Individualcurricular programsand innovationsvarywith respectto their
requirements teachers,however,in ways thatare likelyto influencewhat
of
kindsof professionaldevelopmentmaybe beneficialto teachers.Forinstance,
in scienceeducationspecificcurriculum materialsdifferin how they structure
opportunities for student inquiry. In the AmericanGeological Institute's
Investigating Earth Systems curriculum units for studyingwater,writerspro-
vide a broadframeworkforstudentinvestigations, butteachersmustadaptthe
contentto addresslocalwaterandwatershedissues(Smith,Southard,&Mably,
2001).By contrast,the Whatis theQualityof Waterin OurRiver? unitfromthe
Centerfor HighlyInteractiveComputingin Education(Singer,Krajcik,Marx,
& Clay-Chambers, 2000)is highlyspecifiedin termsof the activitiesand inves-
tigations that teachers conductwith students,highly supportivein termsof
to
pointers resources, and also educative(Davis& Krajcik,2005) in termsof
for
support specificteachingpractices.These and otherscience curriculaalso
differas to the materialsthatthey expect teachersto use, such as consumable
kitsor laboratoryequipment.Itis not surprising,then,thatto use curriculathat
requiremore specializedequipment,teacherswould need supportas partof
theirinitialprofessionaldevelopmentand implementationin orderto suc-
cessfullyadopt curricula,particularly in urbanor otherwiseunderresourced
environments(Means,Penuel,&Padilla,2001).Indeed,researchindicatesthat
the relativelysimpleact of helpingteacherswith theirinitialconfigurationof
equipment,such as calibratingmeasurementequipment,can make a differ-
ence in theirlevel of implementationof programsthatgoes beyond the rou-
tinelyacknowledgedcontributionof well-designedprofessionaldevelopment
activities(Penuel& Means,2004).
LimitationsAssociatedWithModelingApproaches
Thesourcesof dataavailableforlarge-scalestudiescompletedto datehave
also influencedthe extentto which models of effectiveprofessionaldevelop-
ment can be expected to apply to specific programcontexts.For example,
althoughGaretet al. (2001)had conducteda surveyof professionaldevelop-
mentproviders,they reliedsolelyon teacher-leveldatato constructtheirmod-
els. The techniquethat they used--ordinaryleast-squaresregression--was
appropriate forthe teachersurveydataset thattheyusedin theirstudyandpro-
vided a path model with a readilyinterpretablevisualdiagramshowing the
strengthsof particular of featuresof professionaldevelopmentto
relationships
changesin practiceandknowledge.A subsequentanalysisof longitudinaldata
fromthe sameprojectdiduse HLMto analyzeeffects(Desimone,Porter,Garet,
Yoon, & Birman,2002), but the analysisstill used only data fromteachers.
Supovitzand Turner(2000) used an HLMmodel to investigateschool-level
effects separately from the contribution of individual teacher experience in
professional development. However, for their school-level data, they relied
925

Penuel et al.
primarilyon data from a survey of school leaders and collected data from teach-
ers ratherthan from professional development providers on the design of their
activities,even though these designs varied by provider ratherthan by teacher.
As a consequence, from these models we have only a limited understandingof
how particular professional development designs developed by providers
might yield different outcomes from teachers.
A furtherlimitationto the accuracy of the models of effective professional
development that have been analyzed so far is that neither of these two
important studies had available data from a source other than teacher self-
report about changes to teacher knowledge or practice. When teachers are
asked about specific practices and the frequency with which they engage in
them, there is often good agreement between teacher self-report and obser-
vations or regularlycollected teacher logs (Mayer, 1999; Porter,Kirst,Osthoff,
Smithson, & Schneider, 1993). However, within science education at least,
teachers are increasingly aware of and biased favorably toward endorsing
items that ask them about how much "inquiry-oriented"instructionthat they
use, whether or not they in fact engage in those practices (Hollweg & Hill,
2003). Therefore, it is important to validate self-report data on instructional
practice against direct observation or some other independent measure of
practice.
Requirements for Studies ExaminingAspects of Professional Development

Associated With More Effective CurriculumImplementation
We agree with researchers who argue that the most robust inferences
about what makes professional development effective must come from exper-
imental tests of different professional development designs that examine the
impact on student achievement (Borko, 2004; Cohen, Raudenbush, & Ball,
2003). However, in addition to researchon overall effectiveness of professional
development in promoting student achievement, research is needed on the
conditions required for effective scaling of programs,which requires different
research designs and methods in which the focus is on predicting high-quality
implementation of programs(Means & Penuel, 2005). Especially importantare
analytic methods that allow researchers and educational decision makers to
understand how actions at different levels of the system (e.g., district,school,
individual) can influence implementation and scaling processes.
Hierarchicallinear models allow researchers to estimate the contribution
of particularpredictor variables when students and teachers are nested within
particularconditions, such as different professional development experiences
or in different school settings. Standardized coefficients from these analyses
of effects of professional development on curriculumimplementation provide
a rough estimate of what kinds of professional development designs at the
provider level and experiences at the teacher level may be necessary to
achieve particularlevels of scale with a program.
Ideally, there would be measures at each of the levels required for the
study of program implementation. At the professional-development-provider
926

level, data are needed about the design of the activities,specificallythe
extentto which they incorporatethe kinds of featuresthatpast studieshave
identifiedas potentiallyeffectivestrategies,such as opportunitiesfor teach-
ers to spend time planningfor classroomintegrationof materialsinto their
instruction-what is called "activelearning"in the frameworkof Garetet al.
(2001). In addition,some objectivemeasureof programimplementationat
the teacherlevel tied to a model of implementationfidelitythatcan serve as
an independentsource of data on the effectivenessof professionaldevel-
opmentwould be ideal. Observationdata or automatedrecordsthat docu-
ment implementationprovide potentiallythe most reliabledata and could
complementself-reportdataobtainedthroughthe kinds of surveysused in
past studies. Surveyquestionsshould align as closely as possible with the
implementationmodels of curriculain orderto yield more program-specific
information,as opposed to general informationabout teacher beliefs,
knowledge,or instructionalstrategies.
Finally,we argue that studies need to be guided by a theoreticalper-
spectivethatgives centralimportanceto bothlearningprocessesamongteach-
ers and to the particularcurricular
and school contextsin which professional
development takes place. The theoretical
perspectiveshould drawon avail-
able researchin the learningsciences about how people learn (National
ResearchCouncil,1999),but it shouldalso drawon studiesthatexaminethe
conditionsunderwhich innovationsare able to succeed at scale (Means&
Penuel,2005).Below,we describea theoreticalframeworkthatintegratesfind-
ings fromlearningsciences researchon professionaldevelopmentwith spe-
cificfindingsaboutconditionsrequiredforscalingof educationalinnovations,
especiallyin science education.Thisframeworkguided the developmentof
instrumentsfor the studyand specificplan for analyzingresults.
Our Theoretical Framework: The Importance of General

Processes and Specific Contexts for Teacher Learning
The theoreticalconstructsthatwe employbuildfromthose exploredby
Garetet al. (2001), but we have modifiedand extended these constructsto
reflecta more comprehensiveaccountof the role of contextin teacherlearn-
ing fromin-serviceprofessionaldevelopment.Garetet al. reliedon a combi-
nationof in-depthqualitativeresearchfromthe growingbody of researchon
teacherchange(Richardson &Placier,2001)and richwisdomfrompracticein
the formof "bestpractices"researchto identifythe constructsthatcomprised
theirmodelof effectivecore andstructuralprofessionaldevelopmentfeatures.
This body of research,as interpretedby Garetet al., suggests generalized
processesthatarenecessaryto promoteteacherlearning.However,when con-
sideringthe challenge of scaling up specific curricularinnovations,issues
relatedto local contextand implementation fidelityalso emergeas important.
In particular,given that our study is grounded within the context of imple-
menting a curricularinnovation (GLOBE),we found it necessary to highlight
or redefine key constructs from the Garet et al. study to generate a theoretical
927

Penuel et al.
model that more directly informs implementationconcerns. Below, we
describethe constructsthatunderlieour model of teacherlearningfrompro-
fessionaldevelopment,explainingourrationalefor includingeach in the final
model we tested.
ReformVersusTraditionalProfessionalDevelopment
Thereis broadconsensusamongteacherlearningresearchers that"reform
oriented"professionaldevelopmenttends to be more effectivethan "tradi-
tional"professionaldevelopment(Loucks-Horsley et al., 1998;Putnam&Borko,
2000).In describingreform-oriented professionaldevelopment,thesescholars
pointto the need for morein-depthengagementthanis typicallyprovidedin
the standardworkshop given to teachersat the beginningof an initiative.
Reform-oriented professionaldevelopmentactivities,accordingto Garetet al.'s
(2001)definition,includebeing mentoredor coached,participating in a com-
mittee or study group, or engagingin an internship.Some hypothesizethat
these activitiesare more effective than workshops because most reforms
demandmuch of teachers,and workshopsdo not allow teachersto explore
new concepts and teachingstrategiesin enough depth (e.g., Little,1993).
Others argue that these types of activitiesare likely to be more effective
becausethey often are led by currentclassroomteachers,whom otherteach-
ers trustas a source for meaningfulguidance on improvingteaching(e.g.,
Lieberman& McLaughlin, 1992).
But where many scholarsseem to divide the territoryaccordingto type
of professionaldevelopmentactivity(e.g., workshopsversus learningcom-
munities),we preferto focus moreon the designof the activitieswithintype,
acknowledgingthata workshopcan be designedusingreform-oriented prin-
ciples and a coachingrelationshipcan be "traditional." Therefore,our focus
is on proximityto practice,given evidence thathelping teachersto prepare
for theirclassroompracticeyields resultsthatare most directlytranslatableto
practice (Darling-Hammond& McLaughlin,1995; Kubitskey& Fishman,
2006).A numberof researchstudieshave focused on what has been called
"site-based" or "curriculum-linked"professionaldevelopment(Fishmanet al.,
2003;Slotta, 2000). This for
structure professionaldevelopmentis hypothe-
sized to prepareteachersmore effectivelyto enact curriculathan do work-
shops that are distalwith respectto practice,both in termsof instructional
focus and timeof enactment.Site-basedprofessionaldevelopment,for exam-
ple, providesassistanceat school,in the contextof teachers'enactment,using
approachessuch as coaching (Veenman& Denessen, 2001). Curriculum-
linked professionaldevelopmentfocuses specificallyon how to enact peda-
gogical strategies,use materials,and administerassessmentsassociatedwith
particularcurricula.A large-scalestudy of professionaldevelopment con-
ductedin Californiafoundthatthis type of professionaldevelopmentwas far
more effective than workshops that focused on general pedagogical strategies
in promoting change in teacher's practice (Cohen & Hill, 2001).
928

DurationandTimeSpan
A commoncriticismof professionaldevelopmentactivitiesdesignedfor
teachersis thatthey aretoo shortand offerlimitedfollow-upto teachersonce
they begin to teach. Curricular reformsare extremelydemandingon teach-
ers, and the natureof inquiry-orientedreformsin science are such thatthey
requiremost teachersto make big changesto implementthem well (Bybee,
1993;Crawford,2000).Frequently,the resultis thatteacherseitherassimilate
teachingstrategiesinto theircurrentrepertoirewith littlesubstantivechange
or they rejectthose suggested changes altogether(Coburn,2004;Tyack&
Cuban,1995).Thereis growingconsensusthatto makesuch changes,teach-
ers need professionaldevelopmentthatis interactivewiththeirteachingprac-
tice, allowing for multiplecycles of presentationand assimilationof, and
reflectionon, knowledge(Blumenfeld,Soloway,Marx,Guzdial,& Palincsar,
1991;Kubitskey,2006).
Professionaldevelopmentthat is of longer durationand time span is
more likely to contain the kinds of learning opportunitiesnecessary for
teachersto integratenew knowledge into practice(J. L. Brown,2004). For
example, in their study of NSF-fundedLocalSystemicInitiatives,Supovitz
andTurner(2000)foundlongerdurationsof professionaldevelopmentwere
needed to create "investigativecultures"in science classrooms,as opposed
to small-scalechangesin practice.Professionaldevelopmentthatis stretched
acrosslongerperiodsof time, such thatit is sequenced to alignclosely with
enactment,is also likelyto requireclose coordinationwith classroomteach-
ers at the site and thus to requiremore reform-orientedprofessionaldevel-
opmentactivities.Therefore,it is likely thatthere is an interactionbetween
the durationor time span of professionaldevelopmentand other structural
and core features,such as the employmentof reform-orientedprofessional
development.
The Roleof Colleagues

The constructof "collectiveparticipation"
in Garetet al.'s(2001)research
refersto professionaldevelopmentin which teachersparticipatealongside
colleagues from their school and district.Supportingthe notion that this
would be an effectivestrategyfor teacherlearningis a largebody of theory
and researchfocused on the importanceof teachers'professionalcommuni-
ties (Desimone, 2002). Evidencefrom a wide range of studies of schools
engagedin reformsuggeststhatthose thatmakeextensiveuse of teachercol-
laborationare particularlysuccessfulin promotingimplementation,in part
becausereformshavemoreauthoritywhen they areembracedby peers(Bryk
& Schneider,2002). Othershave proposedthat effortsto improveteaching
qualitythroughcollaborationbuildsrelationaltrustin a school building.Such
trustallows leadersand teachersmore latitudeand discretionin makingdif-
ficult decisions, creates clearer understandings of role obligations, and sus-
tains commitment to improving student outcomes (Frank, Zhao, & Borman,
929

Penuel et al.
2004).Stillothersarguethatinteractionsamongteachersconstitutea resource
to teachersin supportof theirimplementationof reforms,which can be con-
sidereda formof social capital(Penuel,Frank,& Krause,2006).
Teacherswho get help fromcolleagueswho are more expertthanthey
are may also gain importantnew informationfrom those interactionsthat
extends what they learnfromformalprofessionaldevelopmentexperiences
(Ball & Cohen, 1996).Havingmultipleparticipantsin professionaldevelop-
mentfroma singleschool helps to buildthese kindsof trustand supportrela-
tionships(Penuel,Riel,Frank,&Krause,in press).Furthermore,teachersoften
as a in can
report participating group professionaldevelopment give focus
that
to collegialinteractionsand motivateworkingthroughproblemsof practice
together(Little,1993).
Focusof ProfessionalDevelopment
Whatis more importantto focus on within professionaldevelopment:
contentor teachingstrategies? A numberof studiessuggestthatteachers'con-
tent knowledge is relatedto the science teaching strategiesthat they use
(Carlsen,1993;Cronin-Jones, 1991;Hollon,Roth,& Anderson,1991).In gen-
eral,when teachersaremorecomfortablewithteachinga particular topic,they
are more likelyto allow for studentquestioningand discussion,an essential
featureof inquiry(NationalResearchCouncil,2000).Butinquiryscienceteach-
ing also demandsthat teachershave specificknowledge of how to support
studentsin developingresearchablequestions,planningan investigation,col-
lecting and interpretingdata, and presenting results (Gess-Newsome &
Lederman1999;Shulman,1986).Therefore,scienceteachersalsoneed knowl-
edge of strategiesforteachingparticular contentareasthroughinquiryor ped-
agogical content knowledge(Kennedy,1999).
Although some mix of focus on content and strategiesis undoubtedly
necessary help supportsuccessfulimplementation,thereis extensivesup-
to
portboth for a focus on contentknowledgein generaland on specificforms
of content that best supportteachingpractice(Hill, Rowan,& Ball, 2005).
Thisis consistentwith evidence in supportof professionaldevelopmentthat
is closely alignedwith practice,as describedabove, helpingteachersto focus
on what studentsare expected to know and the natureof commonstudent
misconceptions.
ActiveLearning
Withinscience education,it is widely believedthatto learnhow to sup-
portstudentinquiryin the classroom,teachersneed firsthandexperiencesof
inquiryeitheras partof theirprofessionaldevelopmentor as partof appren-
ticeshipsto scientists(Gess-Newsome,1999).Thisneed arisesin partbecause
most teachers today learned science from textbooks and tend to hold con-
ceptions of the discipline and of how students learn that are inconsistent with
an inquiry approach (Boone & Kahle, 1998; Marek & Methven, 1991). Some
930

researchstudieshave presentedevidencethat supportsthe strategyof more
hands-on,activelearning,in thatthey have founda relationshipbetweenpro-
fessionaldevelopmentactivitiesin whichteachersengage in inquiryand pos-
itivestudent-achievement outcomes(A. L.Brown&Campione,1996;Fishman
& Krajcik,2003).
Thereare otherways in which it maybe importantto promoteteachers'
activelearningwithinprofessionaldevelopment.Curriculum designersoften
have concerns about the ways in which teachers enact their curriculum,
claimingthat some adaptationsof materialsconstitute"lethalmutations"of
those materials'intent (A. L. Brown & Campione, 1996). It is this lack of
understandingof firstprinciplesthatsome hypothesizepreventseffectiveuse
of curriculummaterialsby teachers(Lieberman& Miller,2001;Singeret al.,
2000;Wiggins&McTighe,1998).Butthe act of planning,enacting,and revis-
ing curricularunitsengagesteachersmoredeeply with theirteaching,so that
they can come to understandmorefullythe principlesof effectivecurriculum
(Spillane,1999,2004).Therefore,professionaldevelopmentthatincorporates
time for instructionalplanning,discussion,and considerationof underlying
principlesof curriculummaybe moreeffectivein supportingimplementation
of innovations.
Coherence
Coherencerefersto teachers'interpretationsof how well alignedthe pro-
fessionaldevelopmentactivitiesarewiththeirown goalsforlearningandtheir
goals for students.These interpretationsare criticalin at least two respects.
First,teachersfilterpolicy demandsand messages fromprofessionaldevel-
opmentaboutteachingthroughtheirown interpretiveframes(Coburn,2001,
2004;Cuban,1986;Cuban,Kirkpatrick, &Peck,2001).Second,the socialcon-
textof schoolshas a stronginfluenceon teachers'interpretive framesandthus
theirdecisions about how to enact (or resist)particularinnovations(Rivet,
2006).If teachersperceivethe demandsto be alignedwiththeirdistrict'sgoals
andwith socialpressureswithinthe schools,they are morelikelyto perceive
professionaldevelopmentfocused on a particularinnovationas congruent
withtheirown goals and thuscommitto adoptingor adaptingthe innovation
(Lumpe,Haney,& Czerniak,2000).
LocalSupportsandBarriers
Teachers,like otherlearners,bringtheircontextswith them to the pro-
fessionaldevelopmentsituation(Penuel,Shear,Korbak,&Sparrow,2005).Not
only do they bringtheirscience knowledgeand framesfor interpretingpoli-
cies as describedabove,butalsotheirschoolschedules,budgetsforequipment
and materials,and time for planningand reflectionare constraintsthatinflu-
ence whetherwhata teacherlearnsin professionaldevelopmentis appliedto
classroom teaching. There are also often school-specific philosophies and ini-
tiatives that compete for attention in schools, making it difficult to focus on
science-oriented innovations (Stein & Kim, 2006). Because teachers face
931

Penuel et al.
a dizzyingarrayof conflictingdemands,theirresponseto pressureto change
tends to be influencedby messages in their local environmentthat certain
kindsof changeare necessaryand not optional(Coburn,2004).
In this context, it is criticalthatmodels of professionaldevelopmentin
science educationconsiderboth necessaryresourcesfor supportingimple-
mentationof a particularinnovationand likely barriersto implementation.
The specificdemandsof the innovationare one set of considerations,partic-
ularlyif specializedteachingstrategiesor materialsareneeded forenactment.
Modelsalso need to capturethe barriersthatteachersperceivein theirlocal
school environmentthatimpedeboth planningfor and enactinginnovations,
as these are proximalinfluenceson theirinstructionaldecisionmaking.
Context for the Current Study

The GLOBEProgramprovidesa contextin which each of the important
conditionsdescribedabove can be met for studyingwhatmakesprofessional
development effective in fosteringcurriculumimplementation.This earth-
science educationprogramexists at a largescale (morethan24,000teachers
have been trainedworldwide),and it has a well-specifiedmodel of imple-
mentationthat calls for teachersto use protocolsdeveloped by scientiststo
engage studentsin authenticdatacollectionactivities,to reportdatato a Web
site for use by studentsand scientists,and to structurestudent-ledinvestiga-
tionsusing datacollectedfor the program.At the same time,the programhas
a distributedmodel for providingprofessionaldevelopmentto teachers,and
thereis wide variationin levelsof programimplementation. In additionto sur-
vey datathatcan be collected from professionaldevelopmentprovidersand
teachers,there is an accessible,independentmeasureof implementation-
data reportingto a centralizedWeb site-that can be obtainedfor teachers
who have receivedprofessionaldevelopmentfromdifferentpartners.Finally,
thereare some specificaspectsof implementingGLOBEcurriculum materials
thathave provendifficultforlargenumbersof teachersto implementthathave
been documentedin evaluationstudies.These implementationbarriersare
relatedto the contentof the materialsitself,and they also interactwith local
school contexts,includingperceivedalignmentto standards.
ImplementationModel
GLOBE's
GLOBE'sprogramdesign emphasizesboth science and education.The
programprovides curricularmaterialsfor use in classroomsas well as an
online databasethat is directedat supportingthe work of scientistsinvesti-
gatingaspects of the global environment.At the beginningof the program,
scientistsfunded as GLOBEprincipalinvestigatorsby the NSFdeveloped a
set of protocolsfor studentsto use in fourdistinctinvestigationareasrelated
to earth systems: atmosphere,hydrology, soils, and land cover/biology.
GLOBE"schools,"which include a small number of other kinds of organiza-
tions such as science museums and senior centers, collect the data according
to the protocols.
932

The programexpects GLOBEteachersto implementthe protocolsfor
the program,usingspecializedequipmentthatteacherscan purchasedirectly
or that GLOBE'sprofessionaldevelopmentproviderscan sometimesmake
available.Forexample,atmosphereprotocolsincludecollectingdataon pre-
cipitation,maximumand minimumtemperature,and cloud cover on a daily
basis. To collect these data, studentsare expected to use GLOBE-certified
equipment(availablethroughmajorscientificequipmentcompanycatalogs)
set up in a way specified by scientistsin the program.Schoolsmust set up
an instrumentshelterin a site thatis in a grassyarea,farfromany potential
shadows that could affect temperaturereadings. For other investigation
areas,studysites also have specialrequirementsand may need to be off the
school propertyto accommodatescientists'specifications(e.g., at a stream
to collectwater-qualitydata).
In additionto collectingdata,the programexpectsteachersto reportdata
to the GLOBEWeb site. Reportingdatabenefitsscientistsin the programby
makingstudent-collecteddataavailablefortheirown investigations(see, e.g.,
Brooks&Mims,2001;Mims,1999).Forstudentsin classrooms,reportingdata
thatthey have collectedprovidesthem with immediateaccess to chartsand
graphsof theirdatathatthe Web site produces;these chartsand graphs,in
turn,can be used as partof classroomdiscussionsof patternsin theirschool's
dataandexplanationsforthem.Pastresearchon GLOBE's effectson students
suggeststhathigherlevels of studentlearningin GLOBEare associatedwith
both morereportingand morefrequentdiscussionsof student-collecteddata
(Coleman& Penuel, 2000;Penuelet al., 2002).
In the past 4 years,GLOBEhas begun to encourageclassroomsto pose
questions using student-collectedGLOBEdata as part of extended, local
investigationsof earthsystems(The GLOBEProgram,2005). The visualiza-
tions of school-reporteddata,exemplarsof studentprojects,rubricsfor judg-
ing the qualityof studentresearchprojects(availableon the GLOBE Website)
and conferenceshighlightingstudentinquiryare justa few examplesof the
kindsof resourcesthatGLOBEhas made availableto supportinquiryin the
classroom.Inquiryprojectsareexpectedto focus on student-generated ques-
tions tied to local issues or concerns,to use and interpretGLOBEdata,and
to communicatefindingsin a formattypicalof scientificresearchreports(e.g.,
with Questions,Hypotheses,Methods,Results,Discussion,and Conclusions
sections).As partof its effortsto promotestudentinquiry,GLOBEhas also
providedits professionaldevelopmentpartnerswith new formatsfor orga-
nizingworkshops,organizedthe identificationof earth-sciencequestionsthat
could be investigatedat the local professionaldevelopmentsite, and pre-
sented protocols and learningactivitiesin such a way as to illustratehow
these can be used to addressthese questions.
The GLOBE
Program'sProfessionalDevelopmentActivities
GLOBEhas a distributednetwork of professional development partners.
The rapid growth of the program in its first 3 years of existence (1995 to
933

Penuel et al.
1998) demonstratedthe need for a differentway to organizeteacherprepa-
rationactivitiesthanthe centralized,headquarter-administered trainingpro-
gramthatGLOBEinitiallydeveloped.To offera broadernetworkof support
for teachers,headquartersstaffinstituteda programto engage officiallydes-
ignated regional GLOBEpartnersto recruit,develop, and supportGLOBE
teacherson a local level. The GLOBEProgramhas a smallstaffthatfunctions
as partnersupportat its headquarters,but the programdoes not fund part-
ners directlyfor theiractivities.Partnersgeneratefinancialresourcesfroma
varietyof sources,includingfederalgrants,foundationgrants,stateand local
educationagencies, and theirown institutions.
Althoughpartners'professionaldevelopmentactivitiesinitiallyfocusedon
trainingteachersto conductprotocolswith students,in generaltheirmodels
of professionaldevelopmenthave evolved to reflectthe emergingconsensus
fromresearchand practiceaboutprinciplesof effectiveprofessionaldevelop-
ment.Partnerstypicallydevote extensivetime for teachersto plan classroom
implementationand use activelearningstrategiesin theirinitialtraining.In
addition,a numberof GLOBEpartnershave developed practicesto support
teachersin theirclassroomsaftertheirinitialtrainingexperience,such as help-
ing themto alignGLOBEactivitieswith local standards,modelingactivitiesin
the classroom,or helpingwith equipmentsetup (Penuelet al., 2005).Many
schools and districts,for theirpart,have sent groupsof teachersfromschools
to participatein professionaldevelopment,sometimesas partof schoolwide
strategiesto encourage the implementationof GLOBEor inquiry-oriented
instructionin theirschools. Hence, the partnersin this studyofferedprofes-
sionaldevelopmentthatreflectsfeaturesof the mostsuccessfulprogramsstud-
ied by researchersin the past(Garetet al., 2001;Supovitz& Turner,2000).
At the same time, thereare significantvariationsin what GLOBE's part-
ners offer and what teachersexperience.Limitedfundingof some partner-
ships,for example,makesit difficultfor all to offerfollow-upto teacherswho
participate.Some partnerschoose to modularizetheirinitialprotocol-training
sessions for teachers,teachingone protocolat a time. Some partnersoffer
extensive opportunitieswithininitialprofessionaldevelopmentactivitiesfor
teachersto plan for curriculumintegration;othersofferjusta half hour to 1
hourforthistype of activity.Thisvariabilityin offeringsandexperiencesmakes
GLOBEa good context for studyinghow particularfeaturesof professional
developmentare associatedwith differentimplementationoutcomes.
DataArchive
The GLOBE
Whatmakes GLOBEa particularly usefulcontextfor studyingthe effec-
tiveness of professionaldevelopmentactivitiesis the availabilityof an inde-
pendentmeasureof programimplementationfor use by researchers.All data
collectedand reportedby studentsforthe GLOBEProgramis storedin a com-
mon database. The database, known as the GLOBEData Archive, is available
to anyone visiting the GLOBEWeb site. Researchers can download the data
easily to measure patterns in data reporting by investigation area and for any
934

specifiedduration.In our study,we have used the DataArchiveas a source
of dataon programimplementationto complementsurveydatawe obtained
fromteachersabouttheirreportedlevel of protocolimplementation.
DocumentedBarriersto GLOBE
Implementation
Evaluationresearchershave studiedGLOBEthroughoutits 10-yearhis-
tory using a variety of methods. These include case studies of particular
schools'effortsto implementthe program(Meanset al., 1999;Penuel et al.,
2005), large-scalesurveysthat have investigatedbarriersto implementation
(Meanset al., 2001),and multimethod,correlationalstudiesof factorsassoci-
atedwith higherand lower levels and breadthof implementation(Penuel&
Means,2004). Evidencefromthese studiessuggeststwo ways in which the
design of the program,its content,and local standardscontributesignificant
barriersto implementation thatprofessionaldevelopmentactivitieshavebeen
to
designed help overcome.
An example of how the curriculumcontentof GLOBEhas shaped pro-
fessionaldevelopmentactivitiesin the programis the emphasisthat some
partnershave given to supportingteacherswith settingup equipmentin their
schools. The specific protocolsfor settingup equipmenthave proven diffi-
cultformanyteachersto follow over the years;the programhas had to adapt
its requirementsto fit the realitiesof schoolyards.Forexample,few school-
yardshave a 100 m2areaof land on which to build a weatherstationthatis
neverobstructedby shadows.Professionaldevelopmentpartnersdiscovered
thatthey had to consultwith teachersand oftentimesvisitschools in orderto
findan acceptable,if not perfect,place for a weatherstationthatmet GLOBE
specifications.
A second barrierrelatedto the designof the programpertainsto the cur-
ricularcontentof GLOBE. AlthoughGLOBEdevelopersintendedits protocols
andlearningactivitiesto be used at anygradelevel, districtandstatestandards
typicallyalign with only a few of those activities.Hence, a big reasonwhy
some teachersreportthatthey do not implementa broadarrayof protocolsis
thatthey do not alignwell with topicsthatthey are requiredto teach(Means,
Coleman,&Lewis,1998).Manyelementaryschoolteachersalso havereported
thattheirschools'emphasison improvingtest scores in readingand mathe-
maticstake away fromtime thatthey can dedicateto GLOBEand othersci-
ence activities(Means,Penuel,Crawford, et al., 2001).As a consequence,some
professionaldevelopmentpartners have soughtto createdocumentsdemon-
strating how GLOBE aligns with specific content in state standards(Penuel
et al., 2005).Just how much teachersreferto such documentsis not well
known,however,and some teachershave decidedto implementthe program
in contextsthat are not particularly supportiveand in which alignmentwith
standardsis poor, becausethey were committedto the goals of the program
(Penuel et al., 2005). Taken together, these findings suggest that curricularalign-
ment mattersbut thatteachers'judgmentsabout coherence of the programwith
their own goals for students influence their implementation decisions.
935

Penuel et al.
Method
The sources of data for the studywere a surveyof GLOBEpartnersin
theircapacityas professionaldevelopmentproviders,a surveyadministered
to GLOBEteachers,and an independent measure of programimplemen-
tation, GLOBEdata reporting(to view the survey instruments,see http://
The partnersurvey
ctl.sri.com/projects/displayProject.jsp?Nick=evalglobe).
included items relatedto the design of professionaldevelopmentactivities.
The teacher survey included items used by Garetet al. (2001) to measure
changes in teacherknowledge and teacherpracticeand measuresto assess
GLOBEimplementationand informationaboutequipmentand supportpro-
vided to teachers.Becausethe teachersin thisstudycouldbe associatedwith
particularpartnersas professionaldevelopmentproviders,we conductedour
analysesof these questionswithinan HLMframework.
Sample
Our sample included454 teacherswho receivedinitialGLOBEprofes-
sional developmentand protocoltrainingfrom28 GLOBEpartnerswithin a
2-yearperiod (2002 to 2004).A medianof 11 teacherswas representedper
GLOBEpartner(rangingfrom2 to 141teachersper providerwithinthe study
population, although providersin general trainedfar greaternumbersof
teachersper partnerthanthose representedin our sample).
The 28 GLOBEpartnerswere diverse geographicallyas well as in the
types of professionaldevelopment offered to teachers. For example, the
majorityof partnersused about six differenttypes of active learningstrate-
gies as partof theirinitialprotocoltraining,such as presentingexamplesor
informationon ways to integrateGLOBEwith teachers'own curriculumor
classroomactivities(54%of providers),discussingthe scientificsignificance
of students'data-collectionactivities(62%of providers),and discussinghow
data sets (GLOBEor other)can be used to illustratemathematicalconcepts
(38%of providers).The partnersalso provided,on average,five different
types of activitiesthatfocus on studentinquiry,such as discussingdatafrom
protocolsto supportstudentinquiry(51%of providers),offeringexamples
of successfulstudent-inquiry projects(31%of providers),and showingteach-
ers how to introduceGLOBEaccordingto their students'knowledge and
experience(31%of providers).Acrossthe 28 partners,they offered,on aver-
age, 20 hoursof professionaldevelopment,rangingfrom3 hoursto 48 hours.
The teachersin our sample also had diversebackgroundsand experi-
ences. Althoughthey all had in commonthe factthatthey participatedin ini-
tial GLOBEprofessionaldevelopment,they variedin the types and amounts
of additionalprofessionaldevelopment activities(in both traditionaland
reform-likeforms)in which they participated.Forexample,43%of teachers
participated in reform-like professional development activities, such as par-
ticipation in a teacher collaborative or network (14% of teachers); working
with a mentor, coach, lead teacher, or observer (9%of teachers); or working
936

in internshipsor immersionactivities(1%of teachers),whereas54%of teach-
ers participatedin traditionalformsof professionaldevelopment,such as in-
districtworkshops (22% of teachers), out-of-districtworkshops (19% of
teachers),or attendanceat a college course(4%of teachers).Not all teachers
implementedGLOBE,as measuredby protocoluse (48%of teachers)and
GLOBEdatareporting(27%of teachers).Furthermore, on average,teachers
took partin professionaldevelopmentthatwas spreadwithina 2-monthspan,
with a rangeof less than 1 monthto 2 years.
Procedures
GLOBEpartner surveydevelopmentand data collection.In spring2004,

a GLOBEpartnersurveywas developed that focused on currentgoals and
practicesof teacherprofessionaldevelopmentacrossGLOBEpartners.The
partnersurveyconsistedof 33 questions,focused on partners'goals, back-
groundsof trainers,the organizationof initialtrainingfor GLOBEteachers,
teachersupportstrategies,emphasisgiven to inquiryin the partnership,and
perspectiveson the GLOBEprogram'sleadership.The primarysources of
datafor this studywere informationon trainersand the organizationof ini-
tialteacherprotocoltraining.These questionsfocused on trainers'organiza-
tionalcontexts(a categoricalitem),durationof GLOBEtraining(open-ended
item, continuousvariable),content of protocoltraining(scale consistingof
multiple Likert-typescale items), and pedagogical strategiesthat trainers
employedas partof teacherprofessionaldevelopment(categoricalitem;sum
of strategiesused as variable).
Thissurveywas validatedusinga processof expertreview:Two partner
coordinators,one externalresearcher,and the GLOBEadministrator for part-
nershipseach revieweditemsforrelevance,appropriateness, andsignificance
for programimprovement.Itemsjudgedto be less appropriate,relevant,or
significantwere dropped;in addition,we made changes to items based on
specificfeedbackto improvethe likelycomprehensibility of the itemsto part-
nerleaders.We collecteddatafromthe GLOBEpartnersinJuly2004through
a mailedquestionnaireand online survey.Partnerscould eitherreturntheir
questionnaireusinga postage-paidenvelope or respondto the onlinesurvey.
The responseratefor the 28 partnersincludedin this studywas 100%.
GLOBE teachersurveydevelopmentand data collection.In winter2005,

a teacher survey was developed, adaptingthe questions that Garetet al.
(2001) used in theiroriginalanalysisof the featuresof effectiveprofessional
development.In addition,we reusedquestionsfrompastsurveysof GLOBE
teachersaboutaspectsof GLOBEimplementationand aboutcontextualbar-
riersand supportsto implementation.Because the questionswere all from
earlier instruments, we did not conduct extensive pilot testing of the items.
We did, however, conduct confirmatoryfactor analyses after the survey data
were collected to ensure that constructs were measured reliably as well as
937

Penuel et al.
to investigatethe discriminant validityof our measures.All scales functioned
as expected;we reportCronbach'salphastatisticsin the body of the text as
an index of scale reliability.Some of the latentconstructswere moderately
to highlycorrelated,as one mightinferfroman examinationof the scale cor-
relationsin Table 1. We note that a strong correlationbetween two latent
constructsis not by itselfsufficientjustificationfor combiningthem.Two cor-
relatedconstructsmay interactin importantways with otherpredictorsand
with the dependent variables.We retainedhighly correlatedconstructsas
separatepredictorswhen it was theoreticallyor empiricallyjustified.
To ensure that the orderof the questionswas sensible and that teach-
ers were interpretingquestionsthe way we intendedthem, we conducted
face-to-face cognitive interviews(Desimone & Le Floch, 2004) with five
teachersin the Midwestduringthe winterof 2005. Cognitiveinterviewsare
intendedto improvethe validityand reliabilityof surveysby findingout from
potentialrespondentswhat theirthoughtprocessesare when respondingto
particularitems.We did not conductfurtheranalyseswith the cognitiveinter-
view data, relyingon it exclusivelyto verifythat the questionson our sur-
vey were indeed revealingthe informationwe desired.
Afterthe pilot-testphase, the finalteachersurveywas mailedin March
2005 to a random sample of 1,467 teachers from the GLOBETeacher
Databasewho had been trainedin 2002-2003or 2003-2004.We initiallyran-
domly selected 1,800teachersfromthe GLOBEdatabase.However,because
the databaseis not updatedwith currentinformation,only 1,467 teachers
had valid addressesand were currentlyteachingscience. A largersample of
teachersthan needed for analysiswas selected because a low responserate
was expected.One reasonto expect a low responseratewas thatthe GLOBE
databaseof teacherstends to have manyinaccuraciesbecause of changesin
teachers' assignmentsfrom year to year and partners'differinglevels of
capacityfor managinginformationon the teachersthatthey serve. Second,
past surveys have shown that teacherswho are active in GLOBE,as mea-
suredby levels of datareporting,are much more likelyto respondthan are
inactive teachers. In follow-up telephone calls, teachers who have been
trainedto implementGLOBEsay that if they have not been implementing
the program,they feel embarrassedand are reluctantto complete surveys
sent to them.
The prediction of a low response rate was accurate:454 teachers
returnedtheirsurveys,for a responserateof 31%.A nonresponsebias analy-
sis was conducted to determinewhether there was systematicbias in the
school settingsof respondingteachers.The GLOBE-trained teacherdatabase
does not includedemographicinformation; therefore, mergedextantdata
we
from the National Center for Education Statistics (NCES). This process
resulted in having complete demographicinformationfor 1,467 GLOBE
teachers, 31% of whom completed the survey. For the survey results to be
deemed reliable, it was necessary to demonstrate that the respondents
reflected the larger population in terms of their distribution according to
demographic variables, such as school size and proportions of low-income
938

and minoritystudents. Logisticregression was performedto determine
whetherthe probabilityof surveycompletionwas relatedto bias based on
demographicfactors.In addition,we analyzedwhether there were differ-
ences in surveycompletionbased on GLOBEdatareporting.
The resultsshowed no significantdifferencesby school characteristics.
Thisfindingsuggeststhattherewas no systematicbias in respondentsbased
on school size or on race/ethnicityor povertylevels of students.However,
therewere significantdifferencesby GLOBEdatareporting(odds ratio[OR]=
2.03;confidenceinterval[CI]= 1.51,2.73).Of teacherswho did not complete
the survey,19%reportedsome type of GLOBEdata,whereas27%of teachers
who completedthe surveyreportedGLOBEdata.We ranfurtheranalysesto
determinewhether there were substantialdifferencesbetween the survey
respondentsand the surveypopulationof GLOBEteacherstrainedbetween
2002and 2004by the partnersstudiedto whom we sent surveys.Specifically,
we testedwhethertherewere differenteffectsof totaldatareportingbecause
these dataareavailablefor all teacherstrainedby particular
partners.We con-
ductedtwo hierarchical generalizedlinearmodelsof totaldatareporting,one
on the completeGLOBEteachersurveypool (n = 1,467)and one fromthe
GLOBEsample(n = 454). Both models producedsimilarresults,suggesting
thatrelationshipsfound in the teachersamplemay approximatethose in the
GLOBEpopulationof teacherstrainedby the partnersincludedas partof our
study. As expected, survey respondentsdo appear to be more actively
engaged in the GLOBEprogramthan nonrespondents,althoughthe vast
majorityof teachersin both the respondingand nonrespondinggroupshad
neverreportedGLOBEdata.
Ouranalysisof respondentand nonrespondentcharacteristics suggests
thatalthoughour respondentswere somewhatmore likelyto reportGLOBE
datathannonrespondents,the majorityof respondents,like nonrespondents,
did not implementthat aspect of the program.Therefore,resultsthat we
reporthere arestillapplicabletowarda broadrangeof GLOBE-trained teach-
ers and schools. Ourtests of models thatincludeprofessionaldevelopment
designdata,but no teachersurveydata,furthersuggestthatthe variablesthat
are significantpredictorsof teacheroutcomesare, in fact, likely to be simi-
lar for the broaderpopulationof GLOBE-trained teachersincluded in our
sampling frame.
GLOBE Data Archive.As described earlier,the GLOBEData Archive

containsinformationabout GLOBEdatareporting.In orderto fulfillthe sci-
entific goals of the program,teachers are expected to reportdata on the
GLOBEWeb site. Hence, the GLOBEDataArchiveprovidesobjectivedata
on teacherdatareporting,a key aspect of GLOBEProgramimplementation.
Morespecifically,the DataArchivestoresa recordforeach day in which stu-
dents collect data using any of the protocols.As teachersor studentsenter
data on the Web, the data are verified either by a program that checks the
plausibility of the data or directly by scientists (for less often implemented
protocols). Some protocols call for data reporting at daily intervals; others
939

Penuel et al.
requireteachersto reporton a monthlyor even a one-timebasis.We down-
loaded these datain summer2005,and to accountforvariabilityin expected
datareporting,we coded datareportingas a binaryvariable.
Measures
We relied on three sourcesof data:(a) informationabout GLOBEpro-
fessionaldevelopmentfromthe partnersurvey;(b) informationaboutteacher
knowledge, change, and continued professional development from the
teachersurvey;and (c) informationaboutteacherdatareporting,an indicator
for GLOBEimplementation, fromthe GLOBEDataArchive.
To determinewhether to analyze measuresat the teacher or partner
level, we based decisionsaboutwhat to includein Level1 (teacherlevel) of
our model and what to includein Level2 (partnerlevel) on where the vari-
able was most likely to vary,regardlessof whetherthe variablecame from
the partneror teachersurvey.We used data fromthe partnersurveyabout
the design and featuresof the initialprotocol training;however, we used
datafromthe teachersurveyabout the durationof theirprofessionaldevel-
opmentand perceivedcoherenceof the activitieswith theirschool'sand dis-
trict'sgoals for studentlearning,as durationvariedby teacherdependingon
how many follow-up activitiesthe teacherparticipatedin, and perceptions
of coherencecould be expectedto varyby teacher.Below,we describeeach
of the measuresthatwe used and the reasonsfor includingeach variablein
the level to which we assignedit for purposesof analysis.Table1 shows the
descriptivestatisticsfor the variablesin the analyses.
Dependentvariables(Level1). Using data fromthe teachersurveyand

the GLOBEData Archive,five outcomes were measured.Three outcomes
relatedto fidelityof programimplementation:data reporting,protocoluse,
andpreparationforstudentinquiry.Two outcomesrelatedto generalchanges
associatedwith teacherprofessionaldevelopment:knowledge of pedagogy
and teacherchange in practice.
Data reportingis a binary variable that measures whether teachers
reporteddata on the GLOBEWeb site. Fromthe GLOBEDataArchive,the
distributionranged from no data reportingto 70,963 reports;the average
numberof datareportswas 253.27.Because of the positiveskewness of the
distribution,a binaryvariablewas created(1 = reporteddata; 0 = did not
reportdata).
Protocoluse is a binaryvariable(1 = yes, 0 = no) showingwhetherteach-
ers implementedthe GLOBEprotocolsin theirclass.Teachersmarkedyes to
conductingthe protocolsforatmosphere,hydrology,biology,soil,GPS,and/or
phenology.Thesevariableswere summed,butbecauseof the skewnessof the
summedresponses(most teachersreportedconductingprotocolsfor atmos-
phere), a binaryvariablewas created that measured whether teachers followed
some type of GLOBEprotocols.
940

17
1.00 0.32***
16
1.00 0.16***
-0.09
15
1.00
-0.31***
-0.31*** -0.36***
14
1.00 0.07 0.03 0.12** 0.10**
13
1.00 0.26***
-0.01 -0.12***
-0.15*** -0.16***
12
1.00 0.10** 0.14***0.13*** 0.08
-0.12*** -0.07
11
1.000.26*** 0.05 0.11** 0.13*** 0.01
-0.01 -0.02
10
1.000.43***
0.27***0.01 0.10** 0.08 0.17*** 0.07
-0.01
Variables
1 9
1.000.49***
0.45***
0.27***0.00 0.07 0.13***0.18*** 0.06
-0.01
1 Level
8
for 1.000.19***
0.13***
0.18***0.09
0.18*** 0.08 0.06 0.07 0.18*** 0.01
Table
7
Matrix 1.00 0.42***
0.070.17***
0.14***0.08
0.13*** 0.12** 0.15***0.14***0.08 0.01
6
1.000.10** 0.000.15***
0.09 0.23***0.10** 0.12***0.09** 0.08
-0.05 -0.17**-0.03
5
Correlation 1.00 0.13***0.16***
0.26***
0.19***
0.17***
0.33***0.08 0.11** 0.02 0.01 0.01 0.03
-0.03
4
1.000.47*** 0.16***0.25***
0.27***
0.24***
0.24***
0.66*** 0.04 0.10** 0.13*** 0.02
-0.12** -0.05 -0.05
454.
3 =
1.000.68***
0.50*** 0.14***0.13***
0.23***
0.21***
0.22***
0.54*** 0.03 0.04 0.15*** 0.05 N
-0.04 -0.09 -0.07
2
1.000.28***
0.34***
0.10** 0.050.28***0.28***
0.15***
0.15***
0.16***
0.33***0.01 0.09 0.11**0.14***0.12** 0.07 Teacher
1
1.000.22***
0.010.00 0.21***0.23***
0.030.020.070.06 0.06 0.10**
-0.04 -0.15*** -0.01 -0.06 -0.01 -0.07
SD 0.450.500.990.990.99 4.480.50 0.420.890.812.730.99 0.48 0.49 0.50 0.45 0.38 0.40 development.
M
0.270.480.000.010.00 6.840.50 0.230.430.541.910.00 0.25 0.62 0.44 0.29 0.18 0.20
.01.
PD <
use professional
PD school
teacher
reporting span school = ***p
PD .05.
Protocol
Data Inquiry
Knowledge
Change
Barriers
EquipmentReform
Technology
Independent support TimeCoherence
Traditional Collective
participation
Graduate
degree
Elementary
school
Middle High
teacher Science-
teacher certified <
education
1. 2. 3. 4. 5. 6. 7. 8. 9. 10. 11. 12.
Dependent 13. 14. 15. 16. 17. 18. **p
Note.
cn

Penuel et al.
Preparationfor studentinquiryis a 16-itemfactor(alpha= 0.94), mea-
suring how much knowledge or confidence that teachershad in inquiry-
based activities,such as hands-on/laboratoryactivities,workingon projects
for a week or more, collecting environmentaldata in the field, and inter-
pretingmultiplerepresentationsof the same data.This is a continuous,nor-
mallydistributedvariable;the higherthe number,the moreteachersreported
being preparedto engage in inquiry-basedpractices.
Knowledgeof pedagogy is a six-itemfactor(alpha = 0.91), measuring
the extent to which teachers felt that their knowledge and skills had
enhancedtheircurriculum,instructional methods,approachesto assessment,
use of technology,teachingdiversestudents,and deepening knowledge of
science. This is a continuous,normallydistributedvariable;the higher the
number,the more teachersreportedthat they had enhanced their knowl-
edge. This variablewas adaptedfrom the knowledge and skill variableof
Garetet al. (2001).
Teacherchange in practiceis a six-itemfactor(alpha= 0.84) measuring
the extent to which teachershad made changes in theirteachingpractice.
Also adaptedfromthe Garetet al. (2001) study, changes includedtypes or
mix of assessmentsused, ways thattechnologyis integratedinto instruction,
and instructionalmethods.This is a continuous,normallydistributedvari-
able; the higher the number, the more the teachers indicated that they
changed theirpractice.
Independentvariables:Teacherlevel(Level1). We adaptedsome inde-

pendent variablesfrom prior work by Garet et al. (2001) and from past
researchon the GLOBEprogram.They includeboth measuresof characteris-
tics of the professionaldevelopment and factors that GLOBEevaluation
researchsuggestedmightmediatethe efficacyof the professionaldevelopment
activities.These variablesare (a) barriers,(b) equipment,(c) technologysup-
port, (d) reform-likeprofessionaldevelopment,(e) traditionalprofessional
development,(f) time span, (g) coherence,and (h) collectiveparticipation.
Teachercharacteristic variablesare (a) graduatedegree;(b) elementary,mid-
or
dle, high school teacher;and (c) science-educationcertification.
Someof these firstfourvariablesarearguablydesignfeaturesof the pro-
fessional developmentthat can be specified at the partnerlevel (Level 2).
Presumably,partnerscan set expectationsaboutcollectiveparticipation,and
they could and did offer a mix of reform-likeand traditionalprofessional
developmentwithina specifiedtime period.However,beyond initialproto-
col trainingprovidedby partners,teachers'experiencesvariedwidely as to
what follow-upactivitiesthey participatedin with partnersand otherteach-
ers. Forinstance,some partnersreportedofferingon-sitementoringto teach-
ers who requestedit, but only some GLOBEteacherstook advantageof that
mentoring. Therefore, ratings about the availability of mentoring were ana-
lyzed as a teacher-level variable. Similarly, the time span of professional
development experiences would have been longer for the teacher who
942

receivedmentoringthanfor the one who did not, which informedour deci-
sion to make time span a teacher-levelvariable.
In addition to adaptingthe Garet et al. (2001) general professional
developmentvariables,we includedGLOBE-specific variables.Forexample,
the variablethatwe termedbarriersis an index of 10 itemsthataskedabout
barriersin keeping teachersfromimplementingGLOBEwith theirstudents.
Barriersincludeddifficultyfindingtimeto prepare,lack of technologyaccess
or support,difficultyidentifyingappropriatesites for taking GLOBEmea-
surements,and changes in teachingassignments.This list was providedto
teachersbut was based on extensivecase-studyresearchconductedas part
of earlierevaluationsof the GLOBEcurriculum.This variablerangedfrom
0 to 20. Eachitem used a 3-pointscale:0 = not a barrier,1 = minorbarrier,
2 = majorbarrier.
Two variablesmeasuredprofessionaldevelopmentfollow-upand sup-
port. Once teachersreceivedtheir initialGLOBEprofessionaldevelopment
and protocol training,teachers indicatedwhether they received GLOBE-
specific support,such as being providedwith equipmentand technology
support.Equipmentis a single surveyitemthatasked teacherswhetherthey
receivedGLOBEequipmentfromtheirGLOBEpartner.Thisis a binaryvari-
able (0 = no, 1 = yes). Technologysupportis a single surveyitem thatasked
teacherswhetherthe GLOBEpartnerprovidedassistanceon technicalsetup
and equipmentuse. This is a binaryvariable(0 = no, 1 = yes).
In the Garetet al. (2001) study, the type of professionaldevelopment
was a binaryvariable-teachers participatedin eithertraditionalor reform
types of professionaldevelopment.We believe thatteacherscan participate
in multiple types of professionaldevelopment in the same school year.
Hence, we modifiedGaret'snotion by includingtwo continuousvariables
representingreform-likeand traditionalprofessionaldevelopment.Reform-
LikeProfessionalDevelopmentis an index of six types of professionaldevel-
opmentactivities,such as participation in a teachercollaborativeor network,
in
working internships or immersion activities,and workingwith a mentor,
coach, or lead teacher. The variable rangedfrom0 to 5. TraditionalProfes-
sional Developmentis an index of four types of professionaldevelopment
activities,such as participationin an in-districtworkshopand attendanceat a
college course, out-of-districtworkshop, and/or out-of-districtconference.
The variablerangedfrom0 to 4.
Similarto the Garetet al. (2001) study,Time Spanis an index, ranging
from0 to 24 months,which measureshow manymonthsthatteacherspartic-
ipatedin preliminaryor formalfollow-upprofessionaldevelopmentsessions.
Coherenceis a six-itemfactor(alpha= 0.86)thatmeasureshow well the
professionaldevelopmentmatchedthe teacher'sgoals for professionaldevel-
opment, the existingreformideas within the school, and whetherthe pro-
fessionaldevelopmentwas followedup with activitiesthatbuilton what had
already been learned. As a continuous, normally distributed variable, the
higher the number, the more teachers indicated that their professional devel-
opment was consistent with their own needs.
943

Penuel et al.
CollectiveParticipationis an index of two surveyitems:Participantsin
the professionaldevelopmentconsistedof all teachersin the departmentor
grade groupingsor of all teachersin the school or set of schools. This vari-
able was constructedin the sameway as in the Garetet al. (2001)study.The
variablerangedfrom0 to 2.
Teachercharacteristicsincluded whether they had a graduatedegree
(master'slevel or greater),the grade level that they taught (elementary,
middle, or high school), and whetherthey had a science educationteacher
certification.
Independentvariables:Partnerlevel (Level2). Six variableswere used

to measureprofessionaldevelopmentcontext:(a) planningfor implementa-
tion, (b) focus on studentinquiry,(c) GLOBEcontent,(d) totalhoursof ini-
tialprotocoltrainingoffered,(e) universitysponsored,and (f) school district
sponsored. Table 2 shows the descriptivestatisticsand correlationsof the
partner-levelvariablesdescribed.
Adaptedfromthe Garetet al. (2001)study,Planningfor Implementation
is an index of 12 instructionalapproaches,such as discussingalignmentof
GLOBEwith state,regional,or nationalstandards;engagingteachersin align-
ing GLOBEactivitieswith standardsand how they might integrateGLOBE
with their own curriculumand classroomactivities;and presentingtips on
ways to tailorGLOBEto students'needs. The planningvariableis a continu-
ous variablefrom0 to 12;the higherthe number,the more thatthe partner
helped teachersmake linkagesbetween the professionaldevelopmentand
theirlocal contexts,such as standardsand schedules.
Focus on StudentInquiryis an index of 13 instructionalapproachesto
supportingthe goal of studentinquiry,such as discussinghow data from
protocolscan be used to supportstudentinquiry,offeringexamplesof suc-
cessful studentinquiry,engagingteachersin an inquiryactivityduringtheir
initial protocol training,and modeling specific steps of using scientific
inquiryin the classroomduringthattraining.Focus on StudentInquiryis a
continuousvariablefrom0 to 13; the higherthe number,the more inquiry
based the initialprofessionaldevelopment.
Because this study focused on science, our measureof contentknowl-
edge, unlikethe Garetet al. (2001)generalvariableof contentknowledge,is
science and GLOBEspecific.GLOBEContentis a nine-itemfactor(alpha=
0.92), measuringhow muchtime teachersdevotedto GLOBEcontentduring
theirinitialprotocoltrainingin areassuchas atmosphere,hydrology,andbiol-
ogy protocolsand learningactivities.The factoris standardizedand is a con-
tinuous variable;the higher the number, the more GLOBEcontent was
offeredduringprofessionaldevelopment.
The variabletotal hours of initialprotocoltrainingoffered to teachers
is a compositeof two questionsand is adaptedfromthe Garetet al. (2001)
study. The first question asked whether a typical professional development
program was offered in a single session and, if so, for how many hours. The
second question asked whether a typical professional development program
944

Table2
DescriptiveStatistics for Partners(Level2 Variables)
M SD 1 2 3 4 5 6
1. Planningfor implementation 6.93 4.18 1.00

2. Focus on inquiry 5.79 4.65 0.53*** 1.00
3. GLOBEcontent 0.17 1.00 0.46** 0.36 1.00
4. Total hours training 21.61 13.97 0.41** 0.48*** 0.71*** 1.00
5. Universitysponsored 0.43 0.50 0.02 -0.07 0.16 0.26 1.00
6. Districtsponsored 0.11 0.31 -0.02 -0.03 0.13 -0.02 -0.30 1.00
7. Othersponsored 0.33 0.48 0.00 0.09 -0.24 -0.24 -0.81***-0.32
Note. PartnerN = 28.

**p < .05. ***p< .01.
was offeredin multiplesessionsand,if so, forhow manydaysand how many

hoursper day.
Similarto the Garetet al. (2001)study,we also includedthe professional
developmentsponsor.Specifically,a university,school district,or otherorga-
nizations,such as science museums,sponsoredeach partner.Dummyvari-
ables were createdto standfor universityor school districtsponsorship(as
opposed to othersponsors).
AnalysisPlan
Descriptivestatisticsand correlationswere conductedon all items,fac-
tors,and indicescreatedfor the analysis.Standardqualitycontrolof the data
was conducted,suchas checkingformulticollinearity, skewness,andvariance.
In the currentstudy,we have two levels of data:the teacherlevel, which
includes additional professional development activities, perceptions of
coherence, and barriers;and the partnerlevel, which includes information
regardingthe types of professionaldevelopmentteachersreceived.Because
of the nested structureof the dataset, we used HLMto examineteacherpro-
fessionaldevelopmentexperiences(Level1) and the professionaldevelop-
ment contextthatthe partnersoffered(Level2).
Foreach outcome variable,we firstfit an unconditionalmodel to deter-
minewhethertherewas adequatevarianceto explainat the teacherand part-
ner levels of the model. Therewas significancein the Level 2 variancefor
the outcome variables,rangingfrom 3%to 5%.We then proceeded to fit
conditionalmodels as follows:
Level 1: Teacher
Yij= o0j+ 3I,(BARRIERS)
+ 32j(EQUIPMENT)+ P3j(TECHNICAL
HELP)+
34,(REFORM PD) + Ps3(TRADITIONAL + f7(COHER-
PD) + G6(TIMESPAN)
ENCE) + Psj(COLLECTIVEPARTICIPATION) + 39J(GRADUATE DEGREE) +
+ l11j(MIDDLE)
+ P12j(SCIENCE
ED CERTIFICATION)
+ rii
oj10(ELEMENTARY)
945

Penuel et al.
Level 2: Partner
poj= 70oo
+ yo,(PLANNING)j ON STUDENT
+ y02(FOCUS +
INQUIRY)j
+ Yo4(TOTAL
CONTENT)) HOURS)j+
yo3(GLOBE
+ uoj
Yo6(DISTRICT)j
Yo5(UNIVERSITY)j
= 20
P2j
P21
= Yro
P3j= y7o
P4j= Y~o40
P5j= Yo50
P6j = 760
P7j= Y70
P8j8=Y0O
P9j= Y90
Ploj= Y710oo
P1ij=Y110
Pfi2j = 7120
For the binaryoutcome variables(Data Reportingand ProtocolUse),

a similarmodel was used, but for a Bernoullidistributionwith a log-link
function (Raudenbush,Bryk, Cheong, & Congdon, 2000; Raudenbush&
Bryk,2002).
Results
Impactof ProfessionalDevelopmenton ProgramImplementation
Data analyzed to address the first researchquestion (What kinds of
professionaldevelopmentactivitiesin GLOBEare associatedwith increased
levels of programimplementation?) show the resultsfor the kinds of pro-
fessional developmentactivitiesin GLOBEassociatedwith increasedlevels
of programimplementation,as measuredby total data reporting,protocol
use, and preparednessfor studentinquiry(see Table3). The kindsof initial
professionaldevelopment activitiesoffered by the GLOBEpartnershad a
significantimpacton GLOBEimplementation.First,we found that when
partnersprovidedprofessionaldevelopmentfocusedon GLOBEimplemen-
tation,such as discussingalignmentof GLOBE with state,regional,or national
standards,engagingteachersin aligningGLOBEactivitieswith standardsand
how they mightintegrateGLOBEwith theirown curriculumand classroom
946

activities,teachers were more likely to feel preparedfor student inquiry
(0 = 0.03;SE= 0.01; t ratio= 2.72) and have higherlevels of GLOBEproto-
col use (3 = 0.12; OR= 1.13;CI= 1.04, 1.22).This is similarto the Garetet
al. (2001) study,in which the researchersfound thatteacherperceptionsof
supportfor planninghad a positiveimpacton teacherlearning.
Second,total hours of professionaldevelopmentprovidedby partners
also matteredin whetherteachersused the GLOBEprotocolswith students
and felt preparedfor student inquiry. Interestingly,in both cases, there
was a negativerelationshipbetween totalhoursof professionaldevelopment
and teachers'preparednessfor studentinquiry(y = -0.01; SE= 0.00;t ratio=
3.01) and theiruse of GLOBEscientificprotocols(y = -0.05; OR= 0.96;CI=
0.93, 0.98).
Third,GLOBEcontentemphasizedduringthe initialprofessionaldevel-
opment had a significantpositiverelationshipto teachers'preparednessfor
studentinquiry(y = 0.17;SE= 0.05; t ratio= 3.06).
Fourth,partnerswho had a greaterfocus on studentinquiryand the sci-
entificprocesshad a significantpositiverelationshipto teachers'data-reporting
practices(y = 0.21; OR= 1.23;CI= 1.06, 1.42).
The kindsof additionalprofessionaldevelopmentthatteachershad after
the initialprofessionaldevelopmentwere also importantfor GLOBEimple-
mentation.First,the importanceof teachers'having meaningful,ongoing,
and coherent professionaldevelopment experiences that were consistent
with theirlocal school and districtgoals and other ongoing reformefforts
was significantforteachers'protocoluse (3 = 0.63;OR= 1.87;CI= 1.45,2.41)
and preparednessfor studentinquiry(P = 0.51; SE= 0.04; t ratio= 13.02).
Consistentwith the Garet et al. (2001) findings, perceived coherence of
teacherprofessionaldevelopmenthas a positiveimpacton GLOBEprogram
implementation.Second, teacherswho had more reform-likeprofessional
developmentwere morelikelyto reportfeelingpreparedfor studentinquiry
( = 0.10;SE= 0.04; t ratio= 2.80).
As an extensionof our basic model, we tested five potentialinteraction
effects aligned to our theoreticalframework.It is logical to consider that
ReformPD (professionaldevelopment),which frequentlyinvolvessite-based
assistance,would be relatedto providingequipment,technicalsupport,and
an extended time span. Frequently,equipmentand equipmentsupportare
relatedto activitieson site, andsite-basedassistancehappenssome timeafter
teachers' initial protocol training. The interaction of Reform PD and
Equipmentwas significantand negativefor both TotalDataReporting(I =
-.567, OR= .557, CI= .357, .901) and ProtocolUse ( = -.825, OR= .438,CI
= .232, .827).Giventhatequipmenthas a significantpositivemaineffect,and
ReformPD does not for these outcomes,this interactionmay be attenuating
the effectof equipmenton the outcomes.Thatis, for higherlevels of Reform
PD, the predictivepower of equipmentis reduced.
Second, we reasoned that coherence might be related to grade level of
schooling, because grade level is a good proxy for curricular alignment.
947

SE
for 0.034 0.010
0.012 0.054
0.004 0.0850.007
0.089 0.092
0.090
0.035
0.052
0.015
0.039
0.057 0.118
0.085 0.094
0.102
Inquiry
P
Student
Preparedness
0.032**
0.007
0.165***
-0.071 0.013*
0.129
0.016 0.002
0.098***
0.015
0.506****
-0.056 -0.012***
-0.228** -0.038
-0.075
-0.120
-0.166*
-0.011
CI 1.221
1.291 1.092
1.668 3.5941.085
2.652
0.980 3.008 1.404
4.524 1.425
1.130 1.969
2.412 1.789
5.230 2.471
3.528
95%
Use 1.041,
0.749,0.930,
0.881,1.033,
0.931, 1.157,
0.456,0.976,1.275,
0.723,
0.836,1.449,
0.975,0.811,
0.754,1.096,
1.628,0.817,
Implementation
Protocol
OR
Program 0.983 1.212
1.128***
1.008 1.2811.029
1.655**
0.955*** 1.866***
1.007
1.092
2.401*** 1.050
1.869****
1.264
1.162 1.966**
1.421
2.918**** interval.
on
3 (PD)
CI 0.325 1.422
1.188 1.146
1.028 3.278
3.2910.969
4.049 4.460 1.206
1.371 1.090 1.109
1.071 2.322
2.160 1.633confidence
1.057 =
Table
95% CI
1.060,
0.904,
0.123, 0.341,0.724,
0.924,0.321,0.829,1.346,
1.213,0.784,
0.774,
0.964,0.405,
0.702, 0.782,
0.980,0.496,
0.224,
Reporting ratio;
Development
Data odds
=
Total OR
OR
1.036
1.228***
0.200****0.625
0.975 1.994***
1.0270.896***
1.712 1.037
2.450*** 1.025
0.966 1.508*
0.670
0.867 1.299
0.724*
0.605
Professional
of
.001.
partnerships.
Impact P120 <
yo, 28
Ploo in ****p
702 y76
704 P80 P110
Yo5 s30 .01.
s6o
<
9soteacher, teachers
certification,
inquiry,
Yo3 i5o
340 ***p
training,
sponsored, PD,o70 teacher,454
implementation,
of support,
320 PD,PD, school .05.
sponsored, of participation, <
Yoo
forstudent 3o0 degree, were
content, education
school
on hoursdistrict span **p
2 1 There
.10.
Focus
GLOBE
Intercept,
Planning Barriers,
School
University
Total Equipment, Collective
Traditional
Time
Technology Graduate
Coherence,
Reform-like Science
Elementary
Middle <
Level Level *p
Note.

Althoughthe interactiontermsof coherenceand grade-levelindicatorswere
not significant,inclusion of interactionterms in the model did slightly
strengthenthe effect of being a middleschool teacheron preparednessfor
inquiry(fromP = -.166 to P = -.195, SE= .087, t ratio= -2.24).
Impactof ProfessionalDevelopmenton TeacherKnowledgeandChange

The second researchquestion(Whatkindsof professionaldevelopment
activitiesin GLOBEare associatedwith increasedteacher knowledge and
changesto science teachingpractice?)shows the resultsfor the kindsof pro-
fessionaldevelopmentactivitiesin GLOBEassociatedwith increasedteacher
knowledgeand changesto science-teachingpractice.Forteacherknowledge
and change,coefficientsrelatedto initialprofessionaldevelopmentfromthe
GLOBEpartnerswere not significant.
However,additionalprofessionaldevelopmentthatteachershadafterthe
initialsession(s)had a significantimpacton teacherknowledgeand change
(see Table4). First,consistentwith the Garetet al. (2001) study,coherence
was a significantpositive predictorfor teacherchange (3 = 0.28; SE= 0.05;
t ratio= 5.90)and knowledgeof pedagogy(3 = 0.59;SE=0.05;t ratio= 10.95).
Second,teachersreportedmore changewhen therewas collectiveparticipa-
tion in theirprofessionaldevelopment(P = 0.17; SE= 0.06; t ratio= 2.95).
Third,reform-like professionaldevelopmentexperienceshada positiveimpact
on teacherchange(P = 0.17;SE= 0.06;t ratio= 2.85).None of the interaction
termsin our extendedmodel were statistically significantpredictors.
Impactof SupportAfterProfessionalDevelopment
Pertainingto the thirdresearchquestion(How do supportandfollow-up
afterprofessionaldevelopmentinfluenceprogramimplementation andteacher
knowledgeandchangesto science-teaching GLOBE
practice?), equipmentand
technologysupporthad significantinfluenceson programimplementation
(GLOBEdatareportingand protocoluse), teacherknowledge,and changes
to science-teachingpractice.Not surprisingly,being providedwith GLOBE
equipmentwas importantfor totaldata reporting(3 = 0.69; OR= 1.99;CI =
1.21, 3.28) and protocoluse (P = 0.62; OR= 1.87;CI = 1.16, 3.01). Having
follow up technologysupportwas importantforteacherknowledge(3 = 0.24;
SE= 0.10;t ratio= 2.46),totaldatareporting(0 = 0.90;OR= 2.45;CI= 1.35,
4.46),andprotocoluse (3 = 0.88;OR= 2.40;CI= 1.28,4.52).None of the inter-
actiontermsin our extendedmodelwere statistically significantpredictors.
Discussion and Conclusions

Our resultsindicatethat the design elements of professionaldevelop-
ment that matteredmost for programimplementationin GLOBEvaried,
depending on the aspect of implementation being measured. To increase data
reporting, the most effective professional development strategy was to focus
949

Penuel et al.
Table4
Impactof Professional Development(PD)on TeacherChangeand Knowledge
TeacherChange Teacher Knowledge
Coef. SE Coef. SE
Level 2
Intercept,Yoo -0.081 0.054 0.045 0.054
Planningfor implementation,Yo, 0.014 0.016 0.024 0.014
Focus on student inquiry,702 0.001 0.013 -0.009 0.014
GLOBEcontent, 703 0.036 0.071 0.055 0.076
Total hours of training,704 -0.001 0.005 -0.007 0.006
Universitysponsored, 70s -0.177 0.136 0.027 0.112
School districtsponsored, y0, -0.308**** 0.074 -0.070 0.098
Level 1
Barriers,o10 0.001 0.009 -0.001 0.007
Equipment, 320 0.062 0.085 0.096 0.060
Technology support, P30 0.137 0.140 0.240** 0.098
Reform-likePD, 140 0.173*** 0.061 0.043 0.046
TraditionalPD, 350 0.017 0.058 0.044 0.045
Time span of PD, f,0 -0.001 0.015 0.014 0.016
Coherence, P70 0.281**** 0.048 0.593**** 0.054
Collective participation,s80o 0.165*** 0.056 0.089 0.118
Graduatedegree, o90 0.104 0.071 -0.042 0.063
Elementaryschool teacher, P100 -0.022 0.137 -0.010 0.088
Middle school teacher, P1110 -0.095 0.099 -0.031 0.091
Science education certification,P120 0.072 0.110 -0.117 0.108
Note. There were 454 teachers in 28 partnerships.Coef. = coefficient.

**p< .05. ***p< .01. ****p
< .001.
on promoting student inquiry in initial professional development sessions. In

other words, a unit increase in Focus on Student Inquiry increased the odds
of datareportingby 23%.Forboth protocoluse and preparednessfor student
inquiry,the opportunityto "localize"GLOBE-thatis, to plan for how to tai-
lor its implementation to local circumstances of teachers' classrooms-was a
significantpredictorof the extent to which teachers implementedthese
aspects of the program. In addition, a focus on the content of GLOBEwas a
significantpredictorof teachers'feelingmorepreparedto implementstudent
inquiryin GLOBE.We found conflictingresultsfor the effect of durationof
professionaldevelopmentand for the natureof the GLOBEpartnerprovid-
ing the professionaldevelopment:Morehoursof professionaldevelopment
supportedgreaterprotocoluse but seemed to undercuta focus on student
inquiry,and university-basedpartnerstended to do a betterjob supporting
protocol use, whereas reliance on school-based partnerswas associated with
less frequent use of student inquiry.
950

Someaspectsof the programdesign itselfare likelyto contributeto this
patternof findings.Forexample,to engage in studentinquiry,an initialhur-
dle thatteachersmustovercomeis the tendencyto collectbut not reportdata
because of time constraints.Reportingdata,however, makes it possible for
studentsand teachersto discussand analyzedatabecause the GLOBEWeb
site can quicklyproduce chartsand graphsof schools' data. Initialprofes-
sional developmentthat focuses on inquirymay have made teachersmuch
more likely to reportdata, even if they did not feel any more preparedto
implementstudentinquirywith GLOBEin theirclassrooms.The factthatgiv-
ing teacherstime to plan for implementationand a focus on GLOBEcontent
did help teacherswith feeling more preparedto facilitatestudentinquiryis
also not surprising;GLOBE's trainingof partnersin how to supportteachers
in inquiryemphasizesthe need to tailorthe contentto local standardsand to
local questionsof scientificinterestthatcan be investigatedby using GLOBE
protocols.Providingteacherswithtimeto considerthe linksbetweenGLOBE
contentand local questionsand learninggoals consequentlyappearsto have
helped thembe betterpreparedto facilitatestudentinquiry.
Anotherset of activitiesthat partnerstypicallyundertakeas partof the
programdesignbut thatare not partof the formalprofessionaldevelopment
also consistentlypredictedcurriculumimplementation.ProvidingGLOBE
equipmentand supportforsettingup thatequipmentpredictedbothprotocol
use and datareporting.Again,aspectsof the programdesignitselfcan help to
explainthese findings,which are consistentwith earlierfindingsabout sup-
ports necessaryfor programimplementationin GLOBEand otherprograms
(Blumenfeld,Fishman,Krajcik,Marx,& Soloway, 2000;Fishman& Krajcik,
2003; Penuel & Means,2004). At the same time, our findingssuggest that
althoughprovidingspecializedequipmentandsupportforusingit maynot be
a professionaldevelopmentactivityperse, theseactivitiesarelikelyto improve
the efficacyof those activitiesin fosteringcurriculumimplementation and are
thus importantto examinewhen tryingto determinethe impactof the pro-
fessionaldevelopment.
At the teacherlevel, perceivedcoherenceof the professionaldevelop-
mentactivitieswith teachers'own districts'goalsforstudentlearningandwith
theirgoalsforprofessionaldevelopmentwas a strongpredictorof two aspects
of curriculumimplementationin GLOBE:protocoluse and preparationfor
studentinquiry.These effectssuggestthatteachers'interpretations of profes-
sional developmentactivities,not justthe design of the activitiesthemselves,
are importantin shapingthe effectivenessof those activities,as both the ear-
lier Eisenhowerstudy(Garetet al., 2001) and otherscholarswho have stud-
ied teachers'interpretations of reformeffortshavefound(Spillane&Jennings,
1997). At the same time, significanceof thisfindingshouldbe interpreted
the
with caution,as it is impossiblefromourdatato know whetherteachers'judg-
ments of the professional development's coherence were made before they
had begun to implement the program; an alternative explanation would be
that teachers formed these judgments after an initial implementation of the
program. Interestingly,the second analysis of the Eisenhower study that relied
951

Penuel et al.
on longitudinaldata(Desimoneet al., 2002)did not findsuch a strongeffect
for coherence,suggestingthatalthoughit is accurateto say thatjudgmentsof
coherenceand programimplementation maybe related,the directionof influ-
ence may not so
be clear.
Interestingly,when we examinedcorrelationsamong our variablesand
predictorsof outcomevariablesthatthe earlierEisenhowerstudiesexamined
in theiranalysis-changes to teacherknowledgeand practice--ourpatternof
findingswas largelyconsistentwith these earlierresults.Likethe earlierstud-
ies, in our HLManalyses,we foundthatthe sponsorof the professionaldevel-
opment,the use of "reform-like" professionaldevelopmentactivities,perceived
coherence,andcollectiveparticipation predictedtheseoutcomes.Furthermore,
correlationsamong partner-levelvariablesshowed significantassociations
between a focus on contentand use of whatwe are calling"supportfor plan-
ning"but that Garetet al. (2001) referredto as "activelearningstrategies."
Amongvariablesat the teacherlevel, consistentwith the earlierstudies,we
found significantcorrelationsamong independentvariables:between profes-
sionaldevelopmenttype andtimespanandbetweentype and coherence.We
also found significantcorrelationsbetween type of activity,time span, and
coherence,on one hand,andchangesin knowledgeandpracticeon the other,
justas the earlierstudiesdid.
Takentogether,the patternof findingsfromour studysuggeststhatthe
emergingresearchon what makesfor effectiveprofessionaldevelopmentin
science educationconsideredbroadlydoes providea useful frameworkfor
examiningwhat makesprofessionaldevelopmenteffectivefor specificcurric-
ularprogramsbut thatthe particular aspectsof individualprogramscan make
some features more or less importantfor supportingimplementation.In
GLOBE,giving teacherstime to plan for implementationwas importantfor
helpingteachersintegratethe materialsintotheircurriculum anddevelopways
to promotestudentinquirythroughthe program'sactivitiesand materials,and
providingthe specializedequipmentneeded to collectdatawas criticalto the
eventualimpactof the professionaldevelopmenton practice,even if it was not
partof the professionaldevelopmentitself.In otherprograms,the configura-
tion of demandson teachersand adaptability to local contextmightbe differ-
ent, and we would expect that for professionaldevelopmentto be effective,it
would need to be better tailored both to the programand to the localcontext.
When considering how to "localize"their professionaldevelopment
activities,providersthus need to consider not only teachers'own contextsbut
on
also the program'sdemands teachers and how those demandscan be met
within their contexts. Essentially,as others have arguedregardingthe ele-
ments necessaryfor the scalingof educational innovations in science, there
mustbe a good "fit"between the curriculum and the local context,and our
findingssuggest that fit is shaped partlyby the effectiveness of the profes-
sional developmentactivities themselves and also by the abilityof providers
to meet other demands of teachers and by teachers' own judgments about
how coherent a program is with their personal professional goals and their
goals for their students' learning.
952

StudyLimitationsandFutureDirections
Becausewe did not use a random-assignment design or seek to control
any partners'approachto professionaldevelopment,our studycannotspeak
directlyto the impactof professionaldevelopmenton teacherpracticeor on
studentlearning.We agree with policy makerswho arguethatsuch studies
are needed, and we also believe thatthe ultimatemeasureof success for any
educationalreformor professionaldevelopmentprogramis whetherit leads
to improvementsin students'learning(Fishmanet al., 2003). However,cre-
atinga chainof evidence thatlinksstudentlearningto teacherlearning,pro-
fessional development, and policy is remarkablychallenging;and so far,
most studies,includingthis one, have elected to look at componentsof this
chain (Loucks-Horsley & Matsumoto,1999).
Ourstudy'sdatahave a furtherlimitation,in thatthe patternof responses
we receivedprecludesus fromsayingthatours is a representativesampleof
GLOBEteachers'experiences.Our missing-dataanalysisindicatedthat the
sampleconsistedof teachersfromschools thatwere much more likelyto be
implementingGLOBEthan the generalpopulationof GLOBEschools. Our
study, therefore,cannot give us precise estimatesneeded to determinethe
amountof time and professionaldevelopmentresourcesrequiredto achieve
a particularlevel of scalingwith all GLOBE-trained teachers.However,we do
believe that the models suggest the kinds of professionaldevelopment
resources that are essential to achieve scaling, including some kinds of
resources(especiallyequipmentand supportfor settingup equipment)that
are not always considered as part of professionaldevelopmentactivities.
Furthermore, some aspectsof professionaldevelopment,especiallyperceived
coherence,thatprovedsignificantsuggestthatfurtherresearchis needed into
how teachersmake such judgmentsand how they relateto influentialpoli-
cies and practicesin teachers'districtand school contexts.
We believe thatour studyfindingssuggestthatmultiplestudiesare nec-
essaryto determinewhatworksin professionaldevelopment,a view consis-
tentwithrecentpanelson scientificallybased researchin education(National
ResearchCouncil,2002).Ourfindingsare consistentwith the view thatstud-
ies of differentcurriculaare likely to yield overlappingbut distinctfindings
aboutwhat makesprofessionaldevelopmenteffective.In reviewingparticu-
lar studiesand synthesizingfindingsacrossstudies,the particularcurricular
and school contextsneed to be takencarefullyinto account,as do the limits
of generalizabilityof researchfindings.
We believethatat leastsome studiesof professionaldevelopmentshould
use randomassignmentof teachersand schools to condition,as such studies
have the fewestthreatsto internalvalidity.Somewill also need to analyzethe
impacton studentachievement,althoughsuch studiesare likelyto be expen-
sive, in partbecause large numbersof teachersare needed to detect small
effects. They are also likely to be expensive because measures of professional
development experiences and impacts on instruction are necessary in order
953

Penuel et al.
to establishthatthe logic modelsunderlyingprogramsare,in fact,sound.We
recommendthatscholarsdrawon existinginstrumentsandtheoryto develop
such instrumentsas a way to help reducecosts of experimentalstudiesthat
are focused on studentachievementoutcomes.As such, studieslike ours can
inform the instrumentationof experimentalstudies and contributeto the
knowledgebase foreffectivepractice,even if they do not answerthe ultimate
questionof whetherthe activitiesaffectstudentachievement.
Note
This work has been supportedby the NationalScience Foundation(NSF;GrantNo.
0223068). Any opinions, findings,and conclusions or recommendationsexpressed in this
materialare those of the authorsand do not necessarily reflect the views of the GLOBE
program,the NSF, the Universityof Michigan,SRIInternational,or Abt Associates. The
authorswould like to thank the following people for their advice and input in the design
of this researchand article:ChristineKorbak,Amy Lewis, HalJavitz,Willow Sussex, Craig
Blurton, Sandra Henderson, Teresa Kennedy, Kenny Nguyen, Michael Garet, Laura
Desimone, Heather Hill, Kwang Suk Yoon, Andy Porter,Jim Slotta,and the editors and
anonymous reviewers.The authorsalso wish to thank SRIresearcherswho were involved
in supporting data collection: Sharon Vazquez, ClaireMiddleton,Josh Cohen, Bowyee
Gong, KatieAnn Kaattari,Angeline Reyes, and Alexis Whaley.
References
Ball, D. L., & Cohen, D. K. (1996). Reformby the book: What is---or might be-the role
of curriculummaterialsin teacher learning and instructionalreform?Educational
Researcher,26, 6-8, 14.
Blumenfeld, P., Fishman,B. J., Krajcik,J. S., Marx,R. W., & Soloway, E. (2000). Creating
usable innovations in systemic reform:Scaling up technology-embedded project-
based science in urban schools. Educational Psychologist,35(3), 149-164.
Blumenfeld, P., Soloway, E., Marx,R. W., Guzdial, M., & Palincsar,A. (1991). Motivating
project-based learning:Sustainingthe doing, supporting the learning.Educational
Psychologist,26(3/4), 369-398.
Boone, W. J., & Kahle, K. B. (1998). Student perceptions of instruction, peer interest,
and adult supportfor middle school science: Differencesby race and gender.
Journal of Women and Minorities in Science and Engineering, 4, 333-340.
Borko, H. (2004). Professional development and teacher learning:Mapping the terrain.
Educational Researcher,33(8), 3-15.
Brooks, D. R., & Mims, F. M., III. (2001). Development of an inexpensive handheld
LED-basedSun photometerfor the GLOBEProgram.Journal of Geophysical
Research, 106(D5), 4733-4740.
Brown, A. L., & Campione, J. (1996). Psychological theory and the design of inno-
vative learning environments: On procedures, principles and systems. In
L. Schauble & R. Glaser (Eds.), Innovations in learning.-New environments for
education (pp. 289-325). Hillsdale, NJ:Erlbaum.
Brown, J. L. (2004). Making the most of Understanding by Design. Washington, DC:
Association for Supervision and CurriculumDevelopment.
Bryk, A. S., & Schneider, B. (2002). Trustin schools: A core resourcefor improvement.
New York: Russell Sage.
Bybee, R. (1993). Reforming science education: Social perspectives and personal
reflections. New York: Teachers College Press.
Carlsen, W. S. (1993). Teacher knowledge and discourse control: Quantitative evi-
dence from novice biology teachers' classroom. Journal of Research in Science
Teaching, 30(5), 417-481.
954

Coburn,C. E. (2001). Collectivesensemakingaboutreading:How teachersmediate
readingpolicy in theirprofessionalcommunities.EducationalEvaluationand
PolicyAnalysis,23(2), 145-170.
Coburn,C. E. (2004). Beyond decoupling:Rethinkingthe relationshipbetween the
institutionalenvironmentand the classroom. Sociologyof Education, 77(3),
211-244.
Cohen, D. K., & Hill, H. C. (2001). Learningpolicy: Whenstate education reform
works.New Haven,CT:Yale UniversityPress.
Cohen, D. K., Raudenbush,S. W., & Ball, D. L. (2003). Resources,instruction,and
research.EducationalEvaluationand PolicyAnalysis,25(2), 1-24.
Coleman,E. B., & Penuel,W. R. (2000).Web-basedstudentassessmentfor program
evaluation.Journalof ScienceEducationand Technology,9(4), 327-342.
Crawford,B. (2000). Embracingthe essence of inquiry:New roles for science teach-
ers.Journalof Researchin ScienceTeaching,37(9), 916-937.
Cronin-Jones, L.L.(1991).Scienceteachers'beliefsand theirinfluenceon curriculum
implementations:Two case studies.Journal of Researchin Science Teaching,
28(3), 235-250.
Cuban,L. (1986). Teachersand machines:Theclassroomuse of technologysince
1920. New York:TeachersCollegePress.
Cuban,L.,Kirkpatrick, H., & Peck, C. (2001). High access and low use of technolo-
gies in high school classrooms:Explainingan apparentparadox.American
EducationalResearchJournal,38(4), 813-834.
Darling-Hammond, L.,&McLaughlin, M.W. (1995).Policiesthatsupportprofessional
developmentin an era of reform.Phi DeltaKappan,76(8), 597-604.
Davis,E.A., & Krajcik, J. S. (2005).Designingeducativecurriculummaterialsto pro-
mote teacherlearning.EducationalResearcher,34(3), 2-14.
Desimone,L.(2002).How can comprehensiveschool reformmodelsbe successfully
implemented? ReviewofEducationalResearch,72(1), 433-479.
Desimone,L.,& LeFloch,K.C.(2004).Arewe askingthe rightquestions?Usingcog-
nitive interviews to improve surveys in education research. Educational
Evaluationand PolicyAnalysis,26(1), 1-22.
Desimone,L.,Porter,A. C., Garet,M.S., Yoon, K. S., & Birman,B. F. (2002).Effects
of professionaldevelopmenton teachers'instruction:Resultsfroma three-year
longitudinalstudy.EducationalEvaluationand PolicyAnalysis,24(2), 81-112.
Fishman,B.J., & Krajcik, J. S. (2003).Whatdoes it meanto createsustainablescience
curriculuminnovations?ScienceEducation,87(4), 564-573.
Fishman,B. J., Marx,R. W., Best, S., & Tal, R. (2003). Linkingteacherand student
learningto improveprofessionaldevelopmentin systemicreform.Teachingand
TeacherEducation,19(6), 643-658.
Frank,K.A., Zhao,Y., & Borman,K. (2004).Socialcapitaland the diffusionof inno-
vations within organizations:Applicationto the implementationof computer
technologyin schools. Sociologyof Education,77(2), 148-171.
Frechtling,J. A., Sharp,L.,Carey,N., & Vaden-Kiernan, N. (1995). Teacherenhance-
mentprograms:A perspectiveon the lastfour decades. RetrievedOctober23,
2001,fromhttp://www.ehr.nsf.gov/ehr/rec/pubs/eval/tep/tep.htm
Garet,M. S., Porter,A. C., Desimone,L.,Birman,B. F., & Yoon, K. S. (2001).What
makes professionaldevelopmenteffective?Resultsfrom a nationalsample of
teachers.AmericanEducationalResearchJournal,38(4), 915-945.
Gess-Newsome,J. (1999). Secondaryteachers'knowledgeand beliefs about subject
matterand theirimpacton instruction.In J. Gess-Newsome& L. M. Lederman
(Eds.),Pedagogicalcontentknowledgeand scienceeducation(pp. 51-94).Boston:
Kluwer.
955

Penuel et al.
Gess-Newsome,J., & Lederman,N. G. (Eds.).(1999). Examiningpedagogicalcon-
tent knowledge.Boston:Kluwer.
The GLOBEProgram.(2005).The"NextGeneration" GLOBE (NGG):Developingstrate-
gicpartnershipsto improvescienceeducation,enhanceenvironmental awareness,
and increaseunderstandingof theearthas a system.RetrievedDecember5, 2005,
fromhttps://globel.globe.gov/webarchs/all/NGG_WhitePaper_9_291.pdf
Hawley,W. D., &Valli,L.(1999).The essentialsof effectiveprofessionaldevelopment.
In L. Darling-Hammond & G. Sykes(Eds.),Teachingas the learningprofession:
Handbookofpolicyand practice(pp. 127-150).SanFrancisco: Jossey-Bass.
Hill,H. C.,Rowan,B., &Ball,D. L.(2005).Effectsof teachers'mathematical knowledge
for teachingon studentachievement.AmericanEducationalResearchJournal,
42(2), 371-406.
Hollon, R. E., Roth,K.J., & Anderson,C. W. (1991). Scienceteachers'conceptions
of teachingand learning.In J. Brophy(Ed.),Advancesin researchon teaching
(Vol. 2, pp. 145-186).Greenwich,CT:JAI.
Hollweg, K. S., & Hill, D. (2003). What is the influence of the National Science
EducationStandards? Reviewingtheevidence,a workshop summary.Washington,
DC:NationalAcademiesPress.
Kennedy,M. (1999). Formand substancein mathematicsand scienceprofessional
development(NISEBriefNo. 3[2]).Madison,WI:NationalCenterfor Improving
ScienceEducation.
Kubitskey, B. (2006). Extendedprofessional developmentfor systemic reform.
Unpublisheddoctoraldissertation,Universityof Michigan,AnnArbor.
Kubitskey,B., & Fishman,B. J. (2006). A role for professionaldevelopment in
sustainability:Linkingthe writtencurriculumto enactment.In S. A. Barab,K. E.
Hay, & D. T. Hickey(Eds.),Proceedingsof the 7thInternationalConferenceof
theLearningSciences(Vol. 1, pp. 363-369).Mahwah,NJ:LawrenceErlbaum.
Lieberman,A., & McLaughlin,M. W. (1992). Networks for educationalchange:
Powerfuland problematic.Phi DeltaKappan,73(9), 673-677.
Lieberman,A., & Miller,L. C. (2001). Teacherscaught in the action: Professional
developmentthat matters.New York:TeachersCollegePress.
Little,J. W. (1993). Teachers'professionaldevelopmentin a climateof educational
reform.EducationalEvaluationand PolicyAnalysis,15(2), 129-151.
Loucks-Horsley,S., Hewson, P. W., Love,N., & Stiles,K. E. (1998).Designingpro-
fessional developmentforteachersof scienceand mathematics.ThousandOaks,
CA:CorwinPress.
Loucks-Horsley, S., & Matsumoto,C. (1999). Researchon professionaldevelopment
for teachersof mathematicsand science:The stateof the scene. SchoolScience
and Mathematics,99(5), 258-271.
Loucks-Horsley,S., & Stiles, K. E. (2001). Professionaldevelopmentdesigned to
change teaching and learning.In J. Rhoton& P. Bowers (Eds.), Professional
development planning and design(pp. 13-24). Arlington,VA:NSTAPress.
Lumpe,A., Haney,J., & Czerniak,C. (2000). Assessingteachers'beliefs about their
science teachingcontext.JournalofResearchin ScienceTeaching,37, 275-292.
Marek,E. A., & Methven,S. B. (1991).Effectsof the learningcycle upon studentand
classroomteacherperformance. Journalof Researchin ScienceTeaching,28(1),
41-53.
Mayer,D. (1999). Measuringinstructionalpractice:Can policymakerstrustsurvey
data?EducationalEvaluationand PolicyAnalysis,21(1), 29-45.
Means, B., Coleman,E., Baisden, K., Haertel,G. D., Korbak,C., Lewis,A., et al.
(1999). GLOBEYear4 evaluation:Evolvingimplementationpractices. Menlo
Park,CA:SRIInternational.
956

Means, B., Coleman, E., & Lewis, A. (1998). GLOBEYear 3 evaluation:
Implementationand progress.MenloPark,CA:SRIInternational.
Means,B., & Penuel,W. R. (2005).Researchto supportscalingup technology-based
educationalinnovations.In C. Dede, J. P. Honan,& L. C. Peters(Eds.),Scal-
ing Up Success: Lessonsfrom technology-basededucational improvement
(pp. 176-197).SanFrancisco:Jossey-Bass.
Means,B., Penuel, W. R., Crawford,V. M., Korbak,C., Lewis,A., Murphy,R. F.,
et al. (2001).GLOBE Year6 evaluation:Explainingvariationin implementation.
MenloPark,CA:SRIInternational.
Means,B., Penuel,W. R.,& Padilla,C. (2001). Theconnectedschool:Technologyand
learningin highschool.SanFrancisco:Jossey-Bass.
Mims, F. M., III. (1999). An internationalhaze-monitoringnetwork for students.
Bulletinof theAmericanMeteorological Society,80(7), 1421-1431.
National ResearchCouncil. (1999). How people learn: Brain, mind, experience.
Washington,DC:NationalAcademiesPress.
NationalResearchCouncil. (2000). Inquiry and the National Science Education
Standards.Washington,DC:NationalAcademiesPress.
NationalResearchCouncil.(2002).Scientificresearchin education.Washington,DC:
NationalAcademiesPress.
Penuel,W. R., Frank,K. A., & Krause,A. (2006). The distributionof resourcesand
expertiseandthe implementation of schoolwidereforminitiatives.In S. A. Barab,
K. E.Hay,&D. T. Hickey(Eds.),Proceedingsof the 7thInternationalConference
of theLearningSciences(Vol. 1, pp. 522-528).Mahwah,NJ:LawrenceErlbaum.
Penuel, W. R., Korbak,C., Lewis,A., Shear,L., Toyama,Y., & Yarnall,L. (2002).
GLOBEYear7 evaluation:Exploringstudent researchand inquiryin GLOBE.
MenloPark,CA:SRIInternational.
Penuel,W.R.,&Means,B. (2004).Implementation variationandfidelityin an inquiry
science program:An analysis of GLOBEdata reportingpatterns.Journal of
Researchin ScienceTeaching,41(3), 294-315.
Penuel,W. R.,Riel,M.,Frank,K.A., & Krause,A. (in press).Analyzingteachers'pro-
fessionalinteractionsin a school as social capital:A social networkapproach.
Teachers College Record.
Penuel,W. R.,Shear,L.,Korbak,C.,& Sparrow,E. (2005).The rolesof regionalpart-
ners in supportingan internationalearthscience educationprogram.Science
Education,89(6), 956-979.
Porter,A. C., Garet,M.S., Desimone,L.M.,& Birman,B. F. (2003).Providingeffec-
tive professionaldevelopment:Lessonsfromthe EisenhowerProgram.Science
Educator,12(1), 23-40.
Porter,A. C., Kirst,M. W., Osthoff,E. J., Smithson,J. L.,& Schneider,S. A. (1993).
Reformup close:An analysisof highschoolmathematicsand scienceclassroom.
Madison:WisconsinCenterfor EducationalResearch.
Putnam,R. T., & Borko,H. (2000).Whatdo new views of knowledge and thinking
have to say aboutresearchon teacherlearning?EducationalResearcher,29(1),
4-15.
Raudenbush,S. W., & Bryk,A. S. (2002). Hierarchicallinear models:Applications
and data analysis methods(2nd ed.). ThousandOaks,CA:Sage.
Raudenbush,S. W., Bryk, A. S., Cheong, Y., & Congdon, R. T. (2000). HLM5:
Hierarchical linear and nonlinear modeling. Chicago: Scientific Software
International.
Richardson,V., & Placier, P. (2001). Teacher change. In V. Richardson(Ed.),
Handbookof researchon teaching (4th ed., pp. 905-947). Washington,DC:
AmericanEducationalResearchAssociation.
957

Penuel et al.
Rivet,A. (2006).Usingtransformative researchto explorecongruenciesbetween sci-
ence reformandurbanschools.InProceedingsof the 7thInternationalConference
of theLearningSciences(pp. 578-584).Mahwah,NJ:LawrenceErlbaum.
Roseman,J. E.,Kulm,G., & Shuttleworth, S. (2001).Puttingtextbooksto the test.ENC
Focus,8(3), 56-59.
Shulman,L. S. (1986). Those who understand:Knowledge growth in teaching.
EducationalResearcher,15(2), 4-14.
Singer,J. E., Krajcik,J. S., Marx,R. W., & Clay-Chambers, J. (2000). Constructing
extended inquiryprojects:Curriculummaterialsfor science educationreform.
EducationalPsychologist,35(3), 165-179.
Slotta,J. (2000,April).A WISEmodelfor teacherprofessional development.Paperpre-
sented at the AnnualMeetingof the NationalAssociationof Researchin Science
Teaching,New Orleans,LA.
Smith,M. J., Southard,J. B., & Mably,C. (2001). Investigatingearth systems:An
inquiryearthscienceprogram(water).Armonk,NY:It'sAboutTime.
Smylie, M. A. (1996). From bureaucraticcontrol to building human capital:The
importanceof teacherlearningin educationreform.EducationalResearcher,
25(9), 9-11.
Spillane,J. P. (1999). Externalreforminitiativesand teachers'effortsto reconstruct
their practice:The mediatingrole of teachers'zones of enactment.Journal of
CurriculumStudies,31, 143-175.
Spillane,J. P. (2004). Standardsdeviation:How schoolsmisunderstandeducation
policy. Cambridge, MA:HarvardUniversityPress.
Spillane,J. P., & Jennings,N. E. (1997). Alignedinstructionalpolicy and ambitious
pedagogy: Exploringinstructionalreform from the classroom perspective.
TeachersCollegeRecord,98, 449-481.
Spillane,J. P., & Thompson,C. L.(1997).Reconstructing conceptionsof local capac-
ity: The local educationagency's capacityfor ambitiousinstructionalreform.
EducationalEvaluationand PolicyAnalysis,19(2), 185-203.
Stein,M. K., & Kim,G. (2006, April).Therole of mathematicscurriculumin large-
scale urbanreform:An analysisof demandsand opportunities for teacherlearn-
ing. Paperpresentedat the AnnualMeetingof the AmericanEducational Research
Association,SanFrancisco.
Supovitz,J. A., & Turner,H. M. (2000).The effects of professionaldevelopmenton
science teachingpracticesand classroomculture.JournalofResearchin Science
Teaching,3 7(2), 963-980.
Tyack,D., & Cuban,L. (1995). Tinkeringtowardutopia:A centuryofpublic school
reform.Cambridge,MA:HarvardUniversityPress.
Veenman,S., & Denessen, E. (2001).The coachingof teachers:Resultsof five train-
ing studies.EducationalResearchand Evaluation,7(4), 385-417.
Wiggins, G., & McTighe,J. (1998). Understandingby design. Alexandria,VA:
Associationfor Supervisionand Curriculum Development.
ManuscriptreceivedMarch13, 2006
RevisionreceivedDecember17, 2006
AcceptedJanuary29, 2007
958


What Makes Professional Development Effective Strategies That Foster Curriculum Implementation

Uploaded by

Document Information

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

What Makes Professional Development Effective Strategies That Foster Curriculum Implementation

Uploaded by

Copyright:

Available Formats

What Makes Professional Development Effective?

Strategies That Foster Curriculum

This content downloaded from 91.238.114.210 on Sat, 28 Jun 2014 12:09:51 PM

Thisstudy uses a sample of 454 teachers engaged in an inquiry science pro-

Policymakers,school and districtleaders,and researchersare all increas-

This content downloaded from 91.238.114.210 on Sat, 28 Jun 2014 12:09:51 PM

WILLIAM R.PENUEL is Directorof EvaluationResearchat the CenterforTechnology

This content downloaded from 91.238.114.210 on Sat, 28 Jun 2014 12:09:51 PM

Backgroundto the Study

This content downloaded from 91.238.114.210 on Sat, 28 Jun 2014 12:09:51 PM

This content downloaded from 91.238.114.210 on Sat, 28 Jun 2014 12:09:51 PM

This content downloaded from 91.238.114.210 on Sat, 28 Jun 2014 12:09:51 PM

Requirements for Studies ExaminingAspects of Professional Development

This content downloaded from 91.238.114.210 on Sat, 28 Jun 2014 12:09:51 PM

Our Theoretical Framework: The Importance of General

This content downloaded from 91.238.114.210 on Sat, 28 Jun 2014 12:09:51 PM

This content downloaded from 91.238.114.210 on Sat, 28 Jun 2014 12:09:51 PM

The Roleof Colleagues

This content downloaded from 91.238.114.210 on Sat, 28 Jun 2014 12:09:51 PM

This content downloaded from 91.238.114.210 on Sat, 28 Jun 2014 12:09:51 PM

This content downloaded from 91.238.114.210 on Sat, 28 Jun 2014 12:09:51 PM

Context for the Current Study

This content downloaded from 91.238.114.210 on Sat, 28 Jun 2014 12:09:51 PM

This content downloaded from 91.238.114.210 on Sat, 28 Jun 2014 12:09:51 PM

This content downloaded from 91.238.114.210 on Sat, 28 Jun 2014 12:09:51 PM

This content downloaded from 91.238.114.210 on Sat, 28 Jun 2014 12:09:51 PM

This content downloaded from 91.238.114.210 on Sat, 28 Jun 2014 12:09:51 PM

GLOBEpartner surveydevelopmentand data collection.In spring2004,

GLOBE teachersurveydevelopmentand data collection.In winter2005,

This content downloaded from 91.238.114.210 on Sat, 28 Jun 2014 12:09:51 PM

This content downloaded from 91.238.114.210 on Sat, 28 Jun 2014 12:09:51 PM

GLOBE Data Archive.As described earlier,the GLOBEData Archive

This content downloaded from 91.238.114.210 on Sat, 28 Jun 2014 12:09:51 PM

Dependentvariables(Level1). Using data fromthe teachersurveyand

This content downloaded from 91.238.114.210 on Sat, 28 Jun 2014 12:09:51 PM

This content downloaded from 91.238.114.210 on Sat, 28 Jun 2014 12:09:51 PM

Independentvariables:Teacherlevel(Level1). We adaptedsome inde-

This content downloaded from 91.238.114.210 on Sat, 28 Jun 2014 12:09:51 PM

This content downloaded from 91.238.114.210 on Sat, 28 Jun 2014 12:09:51 PM

Independentvariables:Partnerlevel (Level2). Six variableswere used

This content downloaded from 91.238.114.210 on Sat, 28 Jun 2014 12:09:51 PM

1. Planningfor implementation 6.93 4.18 1.00

Note. PartnerN = 28.

was offeredin multiplesessionsand,if so, forhow manydaysand how many

This content downloaded from 91.238.114.210 on Sat, 28 Jun 2014 12:09:51 PM

For the binaryoutcome variables(Data Reportingand ProtocolUse),

This content downloaded from 91.238.114.210 on Sat, 28 Jun 2014 12:09:51 PM

This content downloaded from 91.238.114.210 on Sat, 28 Jun 2014 12:09:51 PM

This content downloaded from 91.238.114.210 on Sat, 28 Jun 2014 12:09:51 PM

Impactof ProfessionalDevelopmenton TeacherKnowledgeandChange

Discussion and Conclusions

This content downloaded from 91.238.114.210 on Sat, 28 Jun 2014 12:09:51 PM

TeacherChange Teacher Knowledge

Note. There were 454 teachers in 28 partnerships.Coef. = coefficient.

on promoting student inquiry in initial professional development sessions. In

This content downloaded from 91.238.114.210 on Sat, 28 Jun 2014 12:09:51 PM

This content downloaded from 91.238.114.210 on Sat, 28 Jun 2014 12:09:51 PM

This content downloaded from 91.238.114.210 on Sat, 28 Jun 2014 12:09:51 PM

This content downloaded from 91.238.114.210 on Sat, 28 Jun 2014 12:09:51 PM

This content downloaded from 91.238.114.210 on Sat, 28 Jun 2014 12:09:51 PM

This content downloaded from 91.238.114.210 on Sat, 28 Jun 2014 12:09:51 PM

This content downloaded from 91.238.114.210 on Sat, 28 Jun 2014 12:09:51 PM

This content downloaded from 91.238.114.210 on Sat, 28 Jun 2014 12:09:51 PM