EXAMINING THE IMPACT OF CHANGES IN

DATA-DRIVEN TEACHING AND LEADING ON COLLECTIVE EFFICACY

A Dissertation

Presented to

The Morgridge College of Education

W
University of Denver
IE
In Partial Fulfillment
EV
Of the Requirements for the Degree

Doctor of Philosophy
PR

Presented by

Janet E. Filbin

August, 2008

Advisor: Dr. Elinor Katz

 3320581

W
IE
EV
PR

3320581
2008
 ©Copyright by Janet E. Filbin 2008
All Rights Reserved

W
IE
EV
PR
Author: Janet E. Filbin
Title: THE EFFECTS OF DATA-DRIVEN TEACHING AND LEADING ON
COLLECTIVE EFFICACY
Advisor: Dr. Elinor Katz
Degree Date: August, 2008

Abstract

The sanctions are mounting for those schools who are failing to meet federal,

state, and local accountability requirements. States and districts continue to spotlight

schools that are considered underperforming for scrutiny. As a result, staff within these

schools often struggle with feelings of hopelessness and dissatisfaction. Collective

W
efficacy, the belief that an individual has about their own and their colleague’s

competencies to take on difficult tasks, has been found to be a significant predictor of

IE
student achievement. Prior student achievement has also been shown to be highly
EV
correlated with the individual and collective efficacy of a school staff. However,

enhancing or sustaining the efficacy of educators in schools targeted for improvement

year to year is challenging. Research in this field is emerging and little is known about
PR

the factors that directly influence the collective efficacy of a school staff.

This research examines the relationships that teacher pedagogical precision, the

use and application of data to inform teaching for learning, and the principal’s use of data

with staff has on collective efficacy. The mediating effects of changes in student

achievement on the relationship between teacher and principal pedagogical precision and

collective efficacy were also investigated. Educators from 50 elementary, middle, and

high schools in a large urban/suburban district participated in this cross-year study.

ii
Survey and achievement results were collected three times throughout the school year

and analyzed using structural equation modeling.

The findings of this study indicate that there is a direct and significant impact on

collective efficacy when teachers report higher uses of data to change instruction with an

effect size of .908 (p<.01) by spring. Teachers’ perception of the principal’s use of data

had a positive effect size ranging from .795 in fall (p<.001) to .409 in spring (p<.001).

Results show that, as a school risk factor, socio-economic status had a significant,

negative impact on individual teacher’s perception of collective efficacy throughout the

W
year (p<001) but a small, significant, and positive impact on achievement in winter and

spring (p<.001). IE
EV
PR

iii
 Acknowledgements

I would like to extend my deepest appreciation to Dr. Elinor Katz for her

unwavering support and gentle, calm spirit in stewarding me through this journey. Ellie,

you are an inspiration to us all. I would also like to thank Dr. Martin Tombari for his

encouraging push toward the methodology used in this study. Marty, not only are the

results more robust, but my own sense of efficacy has been greatly enhanced because you

thought I had the abilities to undertake such a complicated approach. For my other

W
committee members, Dr. Brian McNulty and Dr. Susan Korach, your participation and

expertise meant a great deal to me. Brian, you will forever been my role model of a

transformational leader.
IE
EV
I must also give credit to my colleague, Norman Alerta, for his patient guidance

and tutoring in helping me to understand how to conduct structural equation modeling.

Normie, it goes without saying, I could not have done this without you! To my BFF,
PR

Toni Mote-Ponce, for letting me process excessively about this work and for lending me

your editing expertise. To my mother and siblings, thanks for your understanding and for

letting me make this a priority for a while. Finally and most importantly, to my greatest

contribution to this world, Michael and Megan, for it was only when I looked into your

eyes that I believed I was worthy of pursuing this goal. I will never catch up with you

both but know that I am right behind!

iv
 TABLE OF CONTENTS

Chapter 1 INTRODUCTION............................................................................................. 1

The Birth of the Urban Myth................................................................................... 1

The Call to Dispel the Myth .................................................................................... 2
Problem Statement .......................................................................................................... 4
Purpose of the Study ....................................................................................................... 7
Research Questions....................................................................................................... 10
Methodology ................................................................................................................. 11
Significance of the Study .............................................................................................. 12
Exclusions ..................................................................................................................... 13

W
Definition of Terms....................................................................................................... 13
Organization of the Dissertation ................................................................................... 14
Summary ....................................................................................................................... 15
IE
Chapter 2 REVIEW OF THE LITERATURE................................................................. 16
EV

Introduction....................................................................................................................... 16
An Overview of the Theoretical Underpinnings of Collective Efficacy .................. 16
The Role of Social Cognitive Theory in Efficacy Development............................ 18
PR

Mastery Experiences............................................................................................. 19
Increasing the Likelihood of Mastery Experience through Goal Setting ............. 20
Evaluating the Precision of Practice .................................................................... 25
Data-Driven Decision-Making and Pedagogical Precision................................. 26
Teacher Perception, Teacher Efficacy, and Collective Efficacy .............................. 27
Impact of Teacher Perception on Student Achievement ....................................... 27
Teacher Efficacy and Student Achievement.......................................................... 30
The Link between Individual Efficacy, Collective Efficacy, and Achievement ..... 31
Measuring Collective Efficacy.............................................................................. 33
Pedagogical Precision and Collective Efficacy ........................................................ 35
The Influence of Pedagogical Precision on Efficacy............................................ 35
Creating a Culture of Collective Efficacy through Pedagogical Precision ......... 41
The Role of Leadership in the Development of Pedagogical Precision ................... 42
The Principal’s Contribution to Pedagogical Precision ...................................... 42
Defining the Vision, Mission, and Goals of the School ............................................ 44
Managing Curriculum, Instruction, and Assessment................................................ 45
Supervising and Supporting Teaching .................................................................. 47

v
 Monitoring Student Progress................................................................................ 48
Promoting an Instructional Climate ..................................................................... 49
Leadership in Action: Cultivating Collective Efficacy through Pedagogical Precision
................................................................................................................................... 50
Measuring Pedagogical Precision of Teachers and Data Use by Principals ...... 51
The Impact of Data-Driven Changes in Teaching and Leading on Collective
Efficacy ..................................................................................................................... 52
Summary ....................................................................................................................... 54

Chapter 3 METHODOLOGY.......................................................................................... 56

Introduction................................................................................................................... 56
Approach....................................................................................................................... 57

W
Research Questions....................................................................................................... 60
Setting of the Study....................................................................................................... 60
Population ..................................................................................................................... 61
IE
Survey Sample....................................................................................................... 65
Instrumentation ............................................................................................................. 67
Collective Efficacy Survey .................................................................................... 67
EV
Data Use Survey ................................................................................................... 70
Validity Considerations ............................................................................................ 72
Content Validity .................................................................................................... 72
Construct Validity ................................................................................................. 73
Operational Survey ............................................................................................... 77
PR

Establishing Concurrent Validity and Discriminate Validity............................... 80

Achievement Measures.......................................................................................... 83
Data Analysis ............................................................................................................ 86
Summary ....................................................................................................................... 91

Chapter 4 RESULTS........................................................................................................ 93

Introduction................................................................................................................... 93
Description of the Sample......................................................................................... 94
Correlations .......................................................................................................... 99
Missing Data....................................................................................................... 100
Validation of the Measurement Model ................................................................... 101
Factor Analysis ................................................................................................... 101
Verification of the Structural Model....................................................................... 107
Preparation of the Data ...................................................................................... 107
vi
 Modeling ............................................................................................................. 112
Variables in the Model............................................................................................ 112
Latent Variables.................................................................................................. 112
Observed Variables............................................................................................. 113
Null Hypothesis ...................................................................................................... 114
Verification of the Measurement Model................................................................. 115
Analysis of the variables ..................................................................................... 115
The Structural Model .............................................................................................. 118
Parsimonious Models.......................................................................................... 137
Analysis at the School Level................................................................................... 142
Criterion Validity Check..................................................................................... 143
Summary ..................................................................................................................... 145

Chapter 5 SUMMARY, DISCUSSION, CONCLUSIONS AND

W
RECOMMENDATIONS................................................................................................ 148

Summary ..................................................................................................................... 148

IE
Discussion ................................................................................................................... 152
Question 1 Discussion......................................................................................... 158
Question 2 Discussion......................................................................................... 164
EV
Question 3 Discussion......................................................................................... 168
Conclusion .................................................................................................................. 169
Limitations .................................................................................................................. 172
PR

Recommendations....................................................................................................... 175
District Leadership.................................................................................................. 175
Building Principals.................................................................................................. 177
Teacher Leaders ...................................................................................................... 179
Considerations for Future Research............................................................................ 181
Conclusions................................................................................................................. 183

REFLECTIONS.............................................................................................................. 184

REFERENCES ............................................................................................................... 185

vii
Appendix A -Collective Efficacy Scale and Instructional Data Decision-Making Survey

......................................................................................................................................... 193

Appendix B-Item Means................................................................................................. 201

Mean and Standard Deviations for Survey Responses across the School Year ............. 202

Appendix C -Spring Correlations between Latent Variables and Demographics .......... 207

Appendix D -Fall Factor Analysis Loadings .................................................................. 210

Appendix E -Winter Factor Analysis Loadings.............................................................. 212

W
Appendix F –Spring Factor Analysis Loadings............................................................. 214
IE
EV
PR

viii
 LIST OF TABLES

TABLE 1. DESCRIPTION OF SCHOOLS PARTICIPATING IN THE DISTRICT INITIATIVE .......... 63

TABLE 2. COLLECTIVE EFFICACY SCALE ITEMS ................................................................ 67

TABLE 3. FACTOR MATRIX FOR THE 12 ITEM COLLECTIVE EFFICACY SCALE REPORTED BY
GODDARD (2002)....................................................................................................... 69

TABLE 4. INSTRUCTIONAL DATA-DECISION MAKING SURVEY ITEMS ............................... 70

TABLE 5. PILOT TEACHER DATA USE AND TEACHER PERCEPTION OF PRINCIPAL DATA
USE TOTAL SCORE ..................................................................................................... 73

TABLE 6. FACTOR LOADINGS AFTER VARIMAX ROTATION............................................... 74

TABLE 7. FACTOR ANALYSIS OF THE PILOT DATA-DECISION MAKING SURVEY .............. 75

W
TABLE 8. SURVEY QUESTIONS FACTORED TOGETHER ...................................................... 75
IE
TABLE 9. SUBSCALE LABELS ............................................................................................ 75

TABLE 10. SUBSCALE ALPHA COEFFICIENTS .................................................................... 76

EV
TABLE 11. ITEMS REPRESENTED IN THE OPERATIONAL FORM COMBINING THE COLLECTIVE
EFFICACY SCALE AND INSTRUCTIONAL DATA-DECISION MAKING SURVEY .................... 78

TABLE 12. COACH SURVEY ITEMS USED TO VALIDATE TEACHER DATA USE .................. 81
PR

TABLE 13. DIBELS INDICATORS USED IN THE STUDY DESIGN ........................................ 84

TABLE 14. MODELS INVESTIGATED THROUGH SEM......................................................... 90

TABLE 15. SURVEY RESPONSE RATE BY LEVEL FOR EACH COLLECTION PERIOD ............ 95

TABLE 16. YEARS TEACHING IN THE SCHOOL BY DATA COLLECTION PERIOD ................ 96

TABLE 17. TOTAL YEARS TEACHING BY DATA COLLECTION PERIOD .............................. 96

TABLE 18. POSITION IN SCHOOL BY DATA COLLECTION PERIOD ..................................... 96

TABLE 19. DESCRIPTIVE FOR THE FALL TEACHER DATA USE, TEACHER PERCEPTION OF
PRINCIPAL DATA USE, AND TEACHER COLLECTIVE EFFICACY BELIEFS .................... 97

ix
TABLE 20. DESCRIPTIVE FOR THE WINTER TEACHER DATA USE, TEACHER PERCEPTION OF
PRINCIPAL DATA USE, AND TEACHER COLLECTIVE EFFICACY BELIEFS .................... 98

TABLE 21. DESCRIPTIVE FOR THE SPRING TEACHER DATA USE, TEACHER PERCEPTION OF
PRINCIPAL DATA USE, AND TEACHER COLLECTIVE EFFICACY BELIEFS .................... 99

TABLE 22. FALL SURVEY EIGENVALUES AFTER VARIMAX ROTATION ............................ 102

TABLE 23. WINTER EIGENVALUES AFTER VARIMAX ROTATION ..................................... 102

TABLE 24. SPRING SURVEY EIGENVALUES AFTER VARIMAX ROTATION ......................... 102

TABLE 25. INITIAL EXPLANATORY FACTOR ANALYSIS: QUESTIONS FACTORING TOGETHER

................................................................................................................................. 103

TABLE 26. SECONDARY EFA EIGENVALUES FOR THE FALL SURVEY ............................. 104

W
TABLE 27. SECONDARY EFA EIGENVALUES FOR THE WINTER SURVEY ........................ 104

TABLE 28. SECONDARY EFA EIGENVALUES FOR THE SPRING SURVEY .......................... 105
IE
TABLE 29. CHRONBACH’S ALPHA COEFFICIENTS ESTIMATIONS FOR FACTORS ............... 106
EV
TABLE 30 EFA LOADINGS OF OBSERVED VARIABLES ON LATENT VARIABLES ............ 107

TABLE 31. NUMBER OF CASES BY SCHOOL AFTER DELETION OF RESPONSE SETS WITH
MISSING ITEMS ........................................................................................................ 109
PR

TABLE 32. PARAMETER ESTIMATES AND FIT STATISTICS FOR MODEL 1 TDU TO CE .. 121

TABLE 33. PARAMETER ESTIMATES AND FIT STATISTICS FOR MODEL 2 TPPDU TO CE
................................................................................................................................. 124

TABLE 34. PARAMETER ESTIMATES AND FIT STATISTICS FOR MODEL 3 TPPDU TO TDU
................................................................................................................................. 127

TABLE 35. PARAMETER ESTIMATES AND FIT STATISTICS FOR MODEL 4 TDU TO AC ... 130

TABLE 36. MODEL 5 TPPDU TO AC .............................................................................. 133

TABLE 37. MODEL 6 AC MEDIATING CE ....................................................................... 136

TABLE 38. PARSIMONIOUS MODEL FOR FALL, WINTER, AND SPRING ........................... 141

x
TABLE 39. MEAN DIFFERENCES FOR CE, TDU, AND TPPDU ACROSS FALL, WINTER, AND
SPRING ..................................................................................................................... 142

TABLE 40. MEAN SCORES FOR TEACHERS BY HIGH OR LOW DATA USE ....................... 144

TABLE 41. MEAN SCORES FOR TEACHERS’ PERCEPTION OF PRINCIPALS BY HIGH OR LOW
DATA USE ................................................................................................................ 145

W
IE
EV
PR

xi
 LIST OF FIGURES

FIGURE 1. THEORETICAL MODEL OF COLLECTIVE EFFICACY USED BY GODDARD (2002)... 34

FIGURE 2. CONCEPTUAL MODEL TO EXAMINE THE RELATIONSHIP BETWEEN TEACHER

DATA USE, TEACHER PERCEPTION OF PRINCIPAL DATA USE, COLLECTIVE EFFICACY,
AND STUDENT ACHIEVEMENT .................................................................................... 89

FIGURE 3. PARAMETER ESTIMATES FOR THE MEASUREMENT MODEL ............................ 118

FIGURE 4. MODEL 1 TDU TO CE.................................................................................... 119

FIGURE 5. MODEL 2 TPPDU TO CE ............................................................................... 122

FIGURE 6. MODEL 3 TPPDU TO TDU ............................................................................ 125

FIGURE 7. MODEL 4 TDU TO AC ................................................................................... 128

W
FIGURE 8. MODEL 5: TPPDU TO AC ............................................................................. 131
IE
FIGURE 9. MODEL 6 AC TO CE ...................................................................................... 134

FIGURE 10. PARSIMONIOUS MODEL FOR THE FALL DATA ............................................. 138
EV
FIGURE 11. PARSIMONIOUS MODEL FOR THE WINTER DATA ......................................... 139

FIGURE 12. PARSIMONIOUS MODEL FOR THE SPRING DATA ........................................... 140
PR

FIGURE 13. CHANGES IN CE TOTAL MEAN SCORES ACROSS THE SCHOOL YEAR........... 154

FIGURE 14. TOTAL MEAN OF SUMMED COLLECTIVE EFFICACY SCORES BY SCHOOL SES
STATUS .................................................................................................................... 160

FIGURE 15. TOTAL MEAN OF SUMMED CE, TDU, AND TPPDU SCORES BY ACHIEVEMENT
GAINS AND SES STATUS .......................................................................................... 167

xii
 Chapter 1

INTRODUCTION

“We pass through this world but once. Few tragedies can be more
extensive than the stunting of life, few injustices deeper than the denial of
an opportunity to strive or even to hope, by a limit imposed from without,
but falsely identified as lying within.”
Gould, 1996 (p. 50).

W
The Birth of the Urban Myth

Twenty-seven middle school faculty members from a large urban district huddle
IE
together around two long tables in the school library for a mandatory in-service. Theirs is

a school identified as “in need of improvement”; the achievement scores of the 7th and 8th
EV
grade students were far below the expected level of proficiency for their grade. It is

February and just three weeks away from the next administration of the state assessment.
PR

The professional development activity is designed to help staff use grade-level

benchmark assessment results to identify the language arts concepts and skills with which

their students may be struggling. “This won’t make a difference. Our kids can’t read,

they’re low in everything. They can’t do this test. This doesn’t give me any information I

don’t already know,” states one teacher. “This isn’t fair to test them on grade-level. I

can’t even teach the stuff on the test because my kids just aren’t ready for it,” disputes

another. “No one really understands what these kids are dealing with. With their home

life, we are just happy they come to school,” adds yet another educator. “Okay, okay I

know. We all have to realize we have a much harder population to deal with and we have
1
a long way to go. I agree it’s not fair that we have to have the same results as other

schools in the better neighborhoods. I know you all are trying your hardest,” sighs the

school principal. “We just have to face the fact that ours is the urban challenge.”

The Call to Dispel the Myth

Since the No Child Left Behind (NCLB) legislation took root eight years ago, a

substantial amount of attention and energy has been directed toward increasing student

achievement within public schools as measured by a single point-in-time state test.

W
Although this joint legislative commitment has provided the national, state, and local

impetus to seriously wrestle with the inequities inherent to the public education system,
IE
the achievement gaps continue to persist into the 21st century. Despite some promising

practices that were launched in the 1960’s, such as Head Start and the Elementary and
EV
Secondary Education Act, progress in closing these gaps stopped around 1990 (Ferguson

& Mehta, 2004). Hence, there is a push for a potent accountability system designed to
PR

hold schools, districts, and states responsible for ensuring that every learner achieves

proficiency in reading and math by the year 2014.

NCLB requires that all students be included in state assessments and their results

be reported based on their grade, language, ethnic, gender, and economic identifiers as

well as specialized learning needs. Therefore, the “not so hidden” agenda that the NCLB

mandate brings to the forefront is accountability at a school level for every learner sitting

within a classroom, regardless of their preparation to access the test. As expected, the

pressure to include each student in the state assessment system and rapidly raise scores

2
continues to mount. Yet, despite the significant sanctions attached to federal and state

mandates for failure in meeting these requirements, there continues to be a number of

children who still do not read or apply mathematical reasoning skills at grade level.

Students of color, lower socio-economic status, those with language differences, and

students with special education support needs continue to be the majority of youth in

today’s classrooms that are not achieving at a proficient level as defined by the state

developed measurement of academic standards.

With punitive consequences looming, school administrators and teachers must

W
continue to problem-solve ways to effectively impact the learning of students who

continue to be at a disadvantage in the educational system. Yet, many educational

IE
professionals feel overwhelmed and hopeless by the increasingly diverse and complex
EV
learning challenges of their changing student populations, particularly in schools where

achievement has remained static or steadily decreased over time. In fact, many of these

educators arrive at the school doorsteps less than prepared to undertake such a task.
PR

Indeed, teacher retention has been found to be problematic in these schools; and this in

turn translates to a less than qualified staff for students who have the greatest need for

highly-trained teachers (Darling-Hammond & Sykes, 2003). These teachers are often not

adequately prepared to support youth who require someone with extensive content

knowledge to be able to scaffold information, create inductive learning experiences, and

who also have the instructional finesse to motivate learners.

Research has shown that teacher satisfaction is related to how much influence

they feel they can make on the student population with whom they are working

3
(Coladarci, 1992). As a result, efficacious teachers are able to produce environments

where learning occurs. There is also evidence that teacher efficacy translated to the

school level provides a picture of how staff perceives their ability as a group to impact

changes. (Bandura, 1997; Goddard, Hoy, & Hoy, 2000; Goddard, 2001). Group pressure,

or press, has been shown to significantly influence individual teacher perceptions.

Therefore, if they believe as a group that they can accomplish the task at hand then

individual teachers are more likely to believe that they can do so as well.

Problem Statement

W
The notion of collective efficacy, the belief of a group that they have the ability

perform and succeed at responsibilities that may be challenging, has been the subject of
IE
educational research in the past decade. Higher levels of collective efficacy in school
EV
staff have been associated with increased student achievement (Bandura, 1997; Goddard,

2001; Tschannen-Moran, 2004). Goddard, Hoy, and Hoy (2000) and Goddard (2001)

found that when controlling for demographic factors that have been highly associated
PR

with lower performing schools, collective efficacy had a substantial positive effect on

student achievement. Collective efficacy holds great promise as a factor associated with

schools that are making differences for students who otherwise might be at risk.

However, the body of research available on collective efficacy is fairly recent. Although

the findings of Bandura (1997), Goddard, Hoy, and Hoy (2000), and Goddard (2001)

have been replicated, exploration of the direct and indirect influences on collective

efficacy is only beginning to emerge in the literature. Teacher collaboration, shared

decision-making, and prior student achievement are variables that have been found to be

4
correlated with collective efficacy within schools (Bandura, 1993; Goddard, 2001; Ross,

Hogaboam-Gray, & Gray, 2004).

Higher achievement results, then, can introduce a self-fulfilling, recursive cycle in

a school system. When provided with evidence that their students are making gains,

teachers feel increasingly competent to create effective learning environments, which in

turn, manifest in higher levels of student achievement. As this cycle becomes more

integrated into the organization, teachers become acculturated in this belief system.

However, there are schools plagued with repeated poor performance on state and national

W
assessments.

IE
The low levels of efficacy at the group level may influence teachers’ perceptions

that their students are incapable of learning skills defined as proficient for their grade
EV
level regardless of what they do in their classroom. As a result, this problem may become

endemic in schools identified by state and local educational agencies as “failing”,

PR

producing a cycle of chronic underperformance by students that is reinforced by the

educational staff. Tournaki and Podell (2005) found that teachers with lower levels of

efficacy predicted less chance of success for students who were reading two grade levels

below while teachers with stronger efficacy were more positive about student outcomes.

Although these are the schools that require the most competent staff, teachers may

be subject to pervasively low levels of efficacy. Fernández-Ballesteros, Diez-Nicholás,

and Bandura (2002) suggest that “unless people believe they can produce desired

outcomes and forestall undesired ones through their actions, they have little incentive to

5
act or to persevere in the face of difficulties” (p.108). Creating an atmosphere of

optimism is a difficult task under these conditions. Once the culture of low expectation

for student performance has been established and embedded, the effort required to shift

teacher perception is considerable (Tschannen-Moran & Barr, 2004).

A key to turning around schools that struggle to support student learning may lie

in the ability of building and district administrators to cultivate the collective efficacy of

the staff. However, the theoretical underpinnings of collective efficacy have only recently

been established and related studies are sparse. Ross, Hogaboam-Gray, and Gray (2004)

W
note that currently there are no studies that have examined the interaction of the principal

in developing the collective efficacy of the school staff, however, Bandura (1993) and
IE
Goddard (2001) have provided evidence that a supportive principal can contribute to the
EV
development of both individual and group efficacy. Creating an organization of shared

inquiry and decision-making using student achievement data has been suggested as a

positive administrative action that can impact collective efficacy (Goddard, 2002; Ross,
PR

Hogaboam-Gray, and Gray, 2004).

There have also been a number of studies that validate the link between strong

instructional leadership, data use, and student achievement. Marzano, Waters, and

McNulty (2005), Reeves (2002), and Youngs and King ( 2002) found that shared

decision-making in using data to identify and evaluate the progress of school goals,

understand and monitor curriculum and instruction, and to assess student progress with

staff contributed to student achievement. Teacher collaboration and shared decision-

making have also been found to be highly predictive of collective efficacy (Bandura,

6
1993; Goddard, 2001; Ross, Hogaboam-Gray, & Gray, 2004). Therefore, the use and

application of data for shared-instructional decision-making has the potential to leverage

changes in achievement as well as the collective efficacy of a staff.

Purpose of the Study

With the call for proficiency for all students by 2014, as mandated by NCLB,

schools are placing greater emphasis on achievement in math and reading. Schools

identified as “lower performing” are uniquely challenged to make the progress necessary

to meet the performance targets, a task that may seem ominous and unattainable to the

W
staff involved. However, there are a number of promising school characteristics that

have been found to be associated with schools that are making strides forward even with
IE
variables traditionally associated with the lower performing schools, such as greater
EV
ethnic diversity and students from lower socio-economic backgrounds. The

characteristics of these schools include collective efficacy of the staff, shared decision-

making, and the use of data for collaborative inquiry (Bandura, 1993; Goddard, 2001;
PR

Marzano, Waters, & McNulty, 2005; Reeves, 2002; Ross, Hogaboam-Gray, & Gray,

2004; Youngs & King, 2002).

Although results from standardized assessment in combination with classroom

formative measures can provide powerful information for building leaders and teachers

(Robinson, Phillips, & Timperley, 2002; Shepard, 2002; Stiggins, 2005), the information

must be used to shift teacher pedagogy to affect student learning. Using assessment data

can help provide more precision to inform teachers about what students have learned as

7
well as the best way to adjust their pedagogical approach (Shepard, Hammerness,

Darling-Hammond, & Rust, 2005).

The purpose of this study is to examine the effect that “pedagogical precision”, or

the analysis and application of data to adjust instructional approaches, has on student

achievement and on the collective efficacy beliefs of teachers. This research also

investigates the relationship between teachers’ perception of the principal’s use of data to

drive instructional conversations and the impact this has on their own use of pedagogical

precision, and their individual collective efficacy beliefs. Therefore, the unit of analysis

W
of this study is at the individual teacher. In addition, since collective efficacy is a group

trait, school-level data will also be examined.

IE
A sample of over 1,000 kindergarten through twelfth grade teachers representing
EV
50 elementary, middle, and high schools within a large urban/suburban district identified

as under-performing participated in this study. The population of interest was schools that
PR

were provided with direct district support in the implementation of an improvement

initiative that included the administration and use of district benchmark assessments three

times throughout the year to inform school and classroom curriculum alignment and

responsive instruction during the 2007/2008 school year. Data from the entire population

of 31 elementary schools, 10 middle schools, and 9 high schools were used in the

analysis.

The district of interest designed a comprehensive approach to support schools in

the initiative. Building leadership teams that included principals, building instructional

8
coaches, grade or content area teacher leaders, and technology specialists were identified

and charged with overseeing that all teachers in the building followed the improvement

practices. To carry out this charge, the building leadership team members were provided

with training and technical assistance to:

1. Implement the district-developed curriculum, including suggested

pacing guides;

2. Administer benchmark assessments at the trimester;

3. Conduct embedded professional development with staff to analyze the

W
data, interpret results to inform the coverage and depth of curriculum

coverage, and to set instructional targets;

IE
4. Implement research-based strategies related to literacy gaps; including
EV
adjusting instruction to include higher order thinking tasks; and,

5. Progress monitor individual students on a weekly basis using

curriculum-based probes to determine instructional effectiveness and to

PR

adjust strategies as needed.

Principals were asked to share results of the benchmark assessments with staff

immediately after the assessment window and to provide time for staff to have

conversations about the results. Principals were also engaged in guided inquiry during

their monthly meetings to exchange ways in which they were supporting shared decision-

making and collaborative conversations within their buildings.

9
 A comprehensive program evaluation was utilized to monitor the fidelity to

implementation of the curriculum and the use of the information provided through the

benchmark assessments to shift instructional practices. The district also gathered teacher

perception of the efficacy of the staff to impact changes. A set of surveys to examine

changes throughout the school year in teachers’ use of data-driven instructional decision-

making, their perception of the principal’s use of data for instructional conversations, and

their collective efficacy was administered three times from September 2007 through May

2008 to evaluate the initiative impact. In addition to the survey data, changes in student

achievement throughout the school year was investigated to determine if shifts in

W
achievement mediate the relationship between the pedagogical precision of teachers and
IE
their collective efficacy, and between teacher perception of principals’ use of data and the

teachers’ collective efficacy beliefs.

EV

Research Questions
PR

The following three research questions were used to structure the study and are

used to organize the findings:

1. Do higher levels of pedagogical precision, as defined by teacher analysis and

application of data to change their instructional practice, result in increases in

teacher collective efficacy?

2. Do higher levels of principal use of data to inform instructional conversations, as

reported by the teacher, result in higher levels of teacher pedagogical precision

and collective efficacy?

10