Teaching Educators Habits of Mind for

Using Data Wisely

CANDICE BOCALA
Harvard Graduate School of Education

KATHRYN PARKER BOUDETT

Harvard Graduate School of Education

Background: Institutions of higher education, specifically schools of education, should

play a pivotal role in supporting educators’ development of data literacy for teaching.
While novice teachers are often prepared to use test-based assessment data, they learn these
experiences in isolated courses that do not connect to instruction or school improvement.
Moreover, once these novice teachers begin working in schools, they are increasingly expected
to work with colleagues to apply data literacy skills, yet few preparation programs provide
sustained support with using data collaboratively for whole-school improvement.
Purpose: This essay describes the habits of mind, or ways of thinking and being, that un-
derlie data literacy courses offered by the Data Wise Project at the Harvard Graduate School
of Education. The habits include: shared commitment to action, assessment, and adjust-
ment; intentional collaboration; and relentless focus on evidence. Adding an emphasis on
habits of mind expands building data literacy beyond accumulating discrete knowledge
and skills or learning a process that becomes routine.
Research Design: The authors provide suggestions for instructional design than can be in-
corporated both in degree-program courses and in ongoing professional development. These
suggestions provide opportunities for participants to actively cultivate the three habits of
mind.
Conclusions: In order to support all educators while learning data literacy for teaching,
there is a need to bridge the resources of an institution of higher education with the in-
structional capacity of professional development providers and the authentic experiences of
school-based practitioners.

Educators1 are increasingly responsible for using multiple sources of

data in collaborative conversations about student learning and school
improvement decisions (Coburn & Turner, 2011, 2012; Marsh, 2012).
Learning to be an educator no longer just encompasses the acquisi-
tion of deep content knowledge and flexible application of pedagogy; it
also means learning to work with colleagues to use evidence of student

Teachers College Record Volume 117, 040304, April 2015, 20 pages

Copyright © by Teachers College, Columbia University
0161-4681
1
Teachers College Record, 117, 040304 (2015)

learning to make instructional decisions. The phenomenon of data in-

quiry—referred to variously as data-informed decision making and using
data for improvement—has become pervasive in schools (Means, Chen,
DeBarger, & Padilla, 2011; Young & Kim, 2010). We define data inquiry
as educators working in teams to analyze student progress using data,
make recommendations about curricular and instructional next steps,
and follow up on the results of these actions (Hamilton et al., 2009;
Kekahio & Baker, 2013; National Forum on Education Statistics, 2012).
There is much attention paid to the need for educators to gain timely
access to multiple forms of data and learn how to use them for decision-
making and instructional conversations (Supovitz, Foley, & Mishook,
2012; Wayman, Jimerson, & Cho, 2012). If there is a growing consensus
that strong educator practice now involves having professional conversa-
tions about student learning and instruction, then it would make sense
for preservice preparation programs and ongoing professional develop-
ment to build educators’ capacity to participate in such practices. Across
the field of education, we do not have clarity about the role that insti-
tutions of higher education—especially schools of education that train
novice teachers and school leaders—might play, particularly in contrast
to the role of professional developers who work with practicing educa-
tors (Mandinach & Gummer, 2013). However, it is becoming clearer that
schools of education can play an important part to ensure that teachers
enter into schools well prepared to engage in data inquiry (Mandinach,
Friedman, & Gummer, 2015, this issue).
This article offers the authors’ perspective as teachers and learners at
the Data Wise Project at the Harvard Graduate School of Education,2
which has worked closely with educators for over a decade to develop
teaching strategies to support data inquiry in schools. It begins with an
overview of the need to build data literacy as part of preservice and on-
going professional learning for educators. It then describes the three
habits of mind around data use that educators need the most support in
developing. It describes how Data Wise courses are designed to give par-
ticipants hands-on opportunities to cultivate these habits. The focus is
on illuminating the authors’ rationale behind the instructional choices
in hopes of sparking a conversation about teaching educators how to
engage in data inquiry. The article concludes by pointing out the need
to bridge the resources of an institution of higher education with the
instructional capacity of professional development providers and the au-
thentic experiences of school-based practitioners.

2
 TCR, 117, 040304 Teaching Educators Habits of Mind for Using Data Wisely

PREPARING EDUCATORS TO BE DATA LITERATE

Currently there is interest in the field around supporting all educators—

both those in preservice training and those already working in schools—
to become more “data literate,” or able to understand, analyze, and act
on multiple forms of data about student learning (Turner & Coburn,
2012). Learning to use data to inform instruction requires sophisticat-
ed content, pedagogical, and professional knowledge (Young & Kim,
2010). Collectively, these capacities have been called “data literacy” skills
(Mandinach & Gummer, 2013). Many experts agree that data literacy
includes problem-focused skills—knowing how to frame questions, identify
problems, and make decisions; data-focused skills—knowing how to access,
generate, and interpret data; and process-focused skills—knowing how to en-
gage in collaborative inquiry and evaluate cause and effect (Mandinach
& Gummer, 2013). Other data literacy skills include understanding the
strengths and limitations in data collection and reporting tools (National
Forum on Educational Statistics, 2012), as well as responding to concerns
about equity by applying culturally responsive interventions and research-
based instructional strategies to address achievement gaps identified
in the data (Love, Stiles, Mundry, & DiRanna, 2008). After examining
schools of education, data literacy experts, and state licensure require-
ments, Mandinach and Gummer (2015, this issue) propose the framework
of data literacy for teaching as a comprehensive definition of data literacy,
which combines traditional data analysis skills with content and pedagogi-
cal knowledge as educators use data to inform instruction.
Moreover, teachers are increasingly expected to use data literacy for
teaching in collaborative inquiry as they work with other educators. Nelson
and Slavit (2008) defined collaborative inquiry as a way of “co-investi-
gating a commonly agreed-upon element of teaching and learning” (p.
103). Through teamwork, educators are better able to build data literacy
for teaching: In a study of over 200 teachers, researchers discovered that
educators were more comfortable and adept at interpreting data when
working with groups of colleagues (Means et al., 2011). This emphasis on
teamwork represents a departure from the way teachers have historically
worked (Little, 1990; Lortie, 1975), but it is strongly supported by scholar-
ship that argues that learning is built socially through dialogue and reflec-
tion with colleagues (Lave, 1996; Wenger, 1998; Vygotsky, 1978). In this
perspective, teachers learn while interacting within communities that en-
gage in critical and reflective discussions about instruction (Little, 1982,
2002; Putnam & Borko, 2000).
Collaborative work is also important because it makes teachers’ think-
ing the subject of discussion so that teachers can learn from one another.

3
Teachers College Record, 117, 040304 (2015)

Hiebert, Gallimore, and Stigler (2002) described this as creating a “knowl-

edge base” for teaching: “Professional knowledge must be public, it must
be represented in a form that enables it to be accumulated and shared
with other members of the profession, and it must be continually veri-
fied and improved” (p. 4). That is, professional knowledge must be based
on context and concrete examples, integrated into teachers’ experienc-
es, and then stored and shared with others. Effective collaborative work
helps teachers engage in inquiry, or the stance of asking questions to
shape conversation (McLaughlin & Talbert, 2002). Educators who engage
in recursive cycles of collaborative inquiry are better able to understand
the causal connections between the instructional practices they are us-
ing and student outcomes, ultimately resulting in higher student achieve-
ment (Gallimore, Ermeling, Saunders, & Goldenberg, 2009; Saunders,
Goldenberg, & Gallimore, 2009).
New teachers must be better prepared to engage in collaborative data
inquiry, but schools of education tend to include instruction about data
in stand-alone courses rather than as an integrated approach to teaching,
and they tend to focus mostly on test-based assessment data (Mandinach
et al., 2015, this issue). Even within the limited category of “assessment
preparation,” teacher candidates have the strongest preparation in find-
ing different measures that track student learning, but teacher candidates
are poorly prepared in responsibly analyzing that data to select instruc-
tional strategies within specific content or subject areas (Greenberg &
Walsh, 2012). Teacher candidates are rarely taught to use data for whole-
school improvement, as opposed to improving the performance of their
own students, and they are rarely exposed to collaborative, team-based
activities centered on data (Greenberg & Walsh, 2012). Overall, although
schools of education might be building teachers’ assessment literacy, they
are not preparing teachers to engage in data literacy for teaching, as defined
earlier (Mandinach et al., 2015, this issue).
Other research on teachers practicing in schools further suggests that
some veteran teachers do not have a solid foundation in data literacy or
collaborative improvement skills. Indeed, many professional development
experiences about data use in education focus on teaching data literacy
in isolation rather than in support of instructional actions for school im-
provement (Mandinach & Gummer, 2013). There is evidence that edu-
cators go through a developmental progression in their understanding
of using data for improvement, beginning with using data to identify
students in need for specialized programs or services; then using data to
make decisions about curriculum and whether to review material; and fi-
nally, looking at data as a way to design their pedagogy based on evidence
of student learning (Means, Padilla, & Gallagher, 2010). For example, in

4
 TCR, 117, 040304 Teaching Educators Habits of Mind for Using Data Wisely

a study of nine schools in two districts, teachers were more likely to use
the results of interim assessment data primarily to decide what content
to review with students, how to place students into instructional groups,
and how to identify which students needed extra support, rather than re-
thinking how they would teach differently (Goertz, Oláh, & Riggan, 2009).
Clearly, more needs to be done to support both novice and veteran educa-
tors in using data effectively. But what principles should guide the design
of that support? In hopes of generating a discussion around this impor-
tant question, the approach the Data Wise Project takes to working with
practitioners and graduate students is described next.

THE DATA WISE IMPROVEMENT PROCESS

For over a decade, the Data Wise Project at the Harvard Graduate
School of Education (HGSE) has been honing and teaching a model
that provides a systematic approach to improving classroom practice.
The work began when education economist Richard Murnane convened
a group comprising researchers from HGSE and school leaders from
three Boston public schools to work together to articulate what school
leaders need to know and do in order to use data to improve instruc-
tion. The findings were captured in Data Wise: A Step-by-Step Guide to
Using Assessment Results to Improve Teaching and Learning (Boudett, City, &
Murnane, 2013). Contributions from Murnane, statistician John Willett,
and psychometrician Daniel Koretz grounded the work in theory; con-
tributions from practitioners ensured that it would be accessible and rel-
evant to principals and teachers.
Figure 1 shows the Data Wise Improvement Process, which breaks the
work of improvement into three phases. The “prepare” phase involves
creating and maintaining a culture in which staff members can collabo-
rate effectively and use data responsibly. In the “inquire” phase, educa-
tors use a wide range of data sources, including student work and class-
room observations, to articulate a very specific problem of practice that
they are committed to solving. In the “act” phase, teams articulate how
they will learn about and employ instructional strategies to address this
problem and how they will assess the extent to which the plan improved
student learning. The model is characterized by an arrow that curves
back on itself because after educators assess the effectiveness of their ac-
tions, they are well positioned to determine the focus for the next cycle
of collaborative inquiry.
As they work their way through the process, teachers use field-tested
protocols to examine a wide range of data sources. They then develop
action plans that contain focused strategies for improving their teaching

5
Teachers College Record, 117, 040304 (2015)

practice. These plans almost inevitably involve giving students a more cen-
tral role in driving their own learning. Because teachers develop the plans
themselves, their commitment to enacting them is much stronger than it
would have been if central office or even school leaders handed down the
plans. Because Data Wise explicitly supports and empowers teachers, it
can serve as a useful counterbalance to the anxiety produced by increas-
ing accountability pressures.

Figure 1. The Data Wise Improvement Process

Source: Boudett, K. P., City, E. A., & Murnane, R. J. (2013). Data Wise: A step-by-step
guide to using assessment results to improve learning and teaching (revised and expanded
ed.). Cambridge, MA: Harvard Education Press.

The Data Wise Project was established to support educators in using this
process to organize the core work of schools around evidence of learning.
Over the past 10 years, the Project has taught more than 2,500 educators
worldwide who have enrolled in courses designed to either introduce the

6
 TCR, 117, 040304 Teaching Educators Habits of Mind for Using Data Wisely

process or coach teams of educators in working their way through it. Most
courses are designed for school- and district-based teams, although one
is geared specifically to students in graduate programs. Delivery models
range from programs that are entirely face-to-face to those that are com-
pletely online, including a course that employs a hybrid strategy. There
are two different types of online courses: one that provides live custom
coaching to each team and another that involves asynchronous commu-
nication between coaches and participating teams, with a particular focus
on encouraging feedback among teams. Table 1 shows the purpose, audi-
ence, and delivery models for Data Wise courses.

Table 1. Summary of Data Wise Courses

Introductory Introductory Integration
Course Institute Courses
Graduate students in School-level and School-level and
Participants
degree programs district-level teams district-level teams
Prepare participants to Prepare participants to Provide participants
get ready to lead col- get ready to lead col- with step-by-step coach-
Purpose laborative inquiry laborative inquiry and ing as their collab-
provide guidance as orative inquiry journey
the journey unfolds unfolds
On Campus Only: Hybrid: Online Only:
One intensive week on One intensive week One option involves
campus at Harvard on campus at Harvard live custom coach-
Delivery followed by three live ing over a period of
Model online sessions 6 months; another
involves asynchronous
coach and peer sup-
port over 3 months

THREE HABITS OF MIND FOR USING DATA WISELY

Although all these courses use the Data Wise Improvement Process at the
lead framework, participants are told from the start that simply checking
off the steps of an improvement process will not by itself be enough to
bring about real changes in learning and teaching. For meaningful change
to occur, educators must bring a distinctive approach to their work. The
Data Wise Project captured this disciplined way of thinking in the “ACE
Habits of Mind,” in which each letter represents a habit:
A: Shared commitment to Action, Assessment, and Adjustment
C: Intentional Collaboration
E: Relentless focus on Evidence.3

7
Teachers College Record, 117, 040304 (2015)

Costa and Kallick (2008) defined habits of mind as what successful people
do “when they are confronted with problems to solve, decisions to make,
creative ideas to generate, and ambiguities to clarify” (p. 1). The ACE Habits
of Mind are the stances that educators take when they approach data inqui-
ry and improvement because they are ways of thinking that will help them
achieve more productive outcomes. Adding an emphasis on habits of mind
expands building data literacy beyond merely accumulating discrete knowl-
edge and skills or learning a process that becomes routine—it makes explic-
it that there are certain ways of thinking about data that separate emerging
forms of data literacy from sophisticated, well-developed approaches.
Costa and Kallick (2008) noted that to activate habits of mind, one must
be able to know the context in which such habits are useful and also have
the capability and commitment to enact those habits. Because the ACE
Habits of Mind do not come naturally for many educators, the instruc-
tional design of Data Wise courses provides opportunities that allow edu-
cators to experience the habits firsthand and incorporate them into their
professional practice. Table 2 summarizes the teaching strategies that sup-
port the cultivation of each habit; however, it is important to note that
using any one or even a few strategies will not help educators acquire the
habits of mind automatically. The key is to explicitly teach the habits and
then create learning experiences that incorporate the habits holistically,
such that participants have practice activating the habits over time. The
following sections discuss each set of strategies in turn. These strategies
can be incorporated into any course about improvement, not just those
that use the Data Wise Improvement Process specifically. Taken together,
these strategies contribute to a robust instructional design regardless of
the particular inquiry model used.

Table 2. Strategies for Supporting Educators to Cultivate Habits of Mind

Habit of Mind Pedagogical Strategy
A Organizing syllabus around actions that result from inquiry process
Shared commit- steps
ment to Action, Teaching and modeling tools for gathering immediate feedback
Assessment, & Supporting thorough documentation of accomplishments and
Adjustment reflection
Requiring students to participate in the course as a team
C
Teaching and modeling tools for designing and facilitating effective
Intentional
meetings
Collaboration
Teaching and modeling tools for working in groups
Using case studies to expose participants to a broad range of data
E
Teaching and modeling tools for sticking to evidence
Relentless focus
Providing structured opportunities for participants to discuss one
on Evidence
another’s evidence

8
 TCR, 117, 040304 Teaching Educators Habits of Mind for Using Data Wisely

A: SHARED COMMITMENT TO ACTION, ASSESSMENT, AND

ADJUSTMENT

The first of habit of mind that educators need to develop as they learn
how to engage in collaborative data inquiry is an orientation toward pur-
poseful, reflective action. When practiced broadly, this habit leads to or-
ganizational learning by creating a continuous feedback loop that allows
educators to improve how they improve. The importance of feedback loops
has been documented by organizational theorists, who argue that organi-
zational learning occurs when cycles of action and reflection are iterative
and recursive and build on one another over time (Argyris & Schön, 1996;
Edmondson, 2002). Effective teams engage in reflection, or discussing
new insights and learning, as well as action, or testing and implementing
new ideas, applying insight, and producing change (Edmondson, 2002).
Unfortunately, educators have several “bad” habits that work against this
habit of mind. Educators do not seek to practice bad habits, but the frag-
mented and rushed nature of much data work done in schools leads to
bad habits that are created to manage the work. One such habit is the ten-
dency to create action plans without following up on the results of those
actions, without making “midcourse corrections,” and without reflecting
on progress and learning over time (Boudett & City, 2013, p. 1). Educators
are frequently asked to develop action plans for various reasons; for ex-
ample, leadership teams create action plans focused on improving their
schools’ performance on state assessments, and many educator evaluation
systems require individual teachers to create action plans detailing how
they will seek new professional development. However, they rarely have
time to thoughtfully reflect on the results of those action plans in a way
that produces learning and informs the next plan.
To be effective with data inquiry, educators need to complete the feed-
back loop and frequently gather evidence of progress and reflect on what
happened as a result of their actions. There are three pedagogical strate-
gies that institutions of higher education can use to help educators learn
to choose this path.

Organizing the Syllabus Around Actions That Result From Inquiry Process Steps

Every class is explicitly tied to a step of the Data Wise Improvement

Process, and course participants are often nervous that “act” is the very
last phase. It is a misconception that the prior work done in the “prepare”
and “inquire” phases does not involve action, because inquiry teams gen-
erate outcomes all along the inquiry cycle. The beginning of each class
explains how the featured step of the improvement process connects to

9
Teachers College Record, 117, 040304 (2015)

the ones before it and clarifies what the outcome of the step is that par-
ticipants are working toward. This emphasis on intermediate outcomes
helps participants see that action is required at every step. For example,
when organizing for collaborative work (Step 1), the objective is to put
into place the teams and structures that will support inquiry; when digging
into student data (Step 4), the objective is to articulate a statement of the
learner-centered problem that is supported by the data.

Teaching and Modeling Tools for Gathering Immediate Feedback

Courses place a heavy emphasis on training educators to pause and take

stock of what they have learned. Discussion protocols, or rules for structur-
ing conversations that help ensure a clear focus and broad engagement by
everyone involved, often help teachers synthesize and reflect on learning
(McDonald, Mohr, Dichter, & McDonald, 2007). Teams that use protocols
or other structures to explicitly guide conversations tend to open up more
opportunities for teachers to learn and participate (Levine & Marcus,
2010). One protocol4 that targets gathering immediate feedback is called
“plus/delta.” It helps a group develop a shared sense of responsibility for
improving by engaging everyone in assessing what worked well about a
meeting, event, or plan, and what they would have liked to change. For
example, by capturing plus/delta reflections within a meeting, it offers
facilitators—and participants—immediate feedback on how to improve
subsequent meetings. Another useful protocol called “success analysis”
gives participants an opportunity to reflect deeply on a year’s work and
articulate the specific things they did that contributed to improvement.
The third protocol, called “SUMI,” allows participants to take a close look
at the impact of protocols themselves. This protocol encourages partici-
pants to consider what Surprised them about a particular protocol, how
they might Use or Modify it, and what Impact they think it might have.
By engaging participants in a SUMI reflection, educators can understand
that the primary reason for practicing a protocol during a class or session
is to get them ready to use it in their professional work.

Requiring Thorough Documentation of Accomplishments and Reflection

To help educator teams reflect on their efforts with an improvement pro-

cess, teams document their progress and learning over time in a “journey
presentation.” For each step of the improvement process, teams create a
summary that explains their process, evidence generated, and reflections
on what was learned. By telling not just the story of what they did to pre-
pare, inquire, and act, but also the story of any changes to teaching and
learning, participants position themselves well to begin the cycle again

10
 TCR, 117, 040304 Teaching Educators Habits of Mind for Using Data Wisely

with a new guiding question or a new way of thinking about the current
issue. This written record can also serve as a tool for showing new team
members what has happened so far and for demonstrating to other audi-
ences at or beyond their school what the team has done.

C: INTENTIONAL COLLABORATION

This second habit of mind requires making deliberate decisions about

how to engage colleagues in working together. Data inquiry relies on col-
laborative process: a team of educators sitting together discussing some
form of data, whether it is aggregated student assessment scores or a class-
room sample of completed student work. Educators who are able to enact
this habit are thoughtful about which colleagues need to be part of the
team, and they design and facilitate meetings so that each team mem-
ber has opportunities to contribute to the data inquiry. The potential bad
habit is assuming that just because a group of educators has been asked
to work as a team, they will be collaborative and productive. Many have
argued that team formation is not obvious or easy—instead, it requires
leadership, routine processes, meaningful and interdependent tasks, and
norms like trust, safety, and mutual accountability for completing work
(Edmondson, 1999; Hackman, 2002; Katzenbach & Smith, 2003; Troen &
Boles, 2012).
Collaboration is complex work, and many educators lack meeting fa-
cilitation and team leadership skills. Effective team facilitation is crucial
to develop an inquiry-based culture; facilitators help to implement tools
such as meeting agendas and discussion protocols that give routine and
structure to collaboration (Gallimore et al., 2009; Nelson & Slavit, 2008;
Park & Datnow, 2009; Talbert, Mileva, Chen, Cor, & McLaughlin, 2010).
In a meeting where educators are engaged in data inquiry, facilitation
enables all team members to contribute to the knowledge that is being
created. Unfortunately, teacher preparation programs rarely teach educa-
tors how to be purposeful team facilitators, and thus new teachers enter
the profession with little experience or knowledge about how to lead or
support collaborative dialogue.
To help educators become more fluent collaborators, the following
strategies are useful.

Requiring Students to Participate in the Course as a Team

In most of our courses, this means that everyone enrolls as a member of a

school-level team (comprising the principal, a teacher, and usually at least
one other faculty member) or a district-level team (comprising the people
who are responsible for supporting schools in using data effectively). In

11
Teachers College Record, 117, 040304 (2015)

the degree-program course, individual enrollees are placed on teams for

the duration of the course and given frequent opportunities to practice
their collaboration skills by working on a group project and reflecting on
the experience. Having all enrollees engage as members of a team instead
of as individuals provides them with a safe space to practice how to both
facilitate and participate in collaborative meetings.

Teaching and Modeling Tools for Designing and Facilitating Effective Meetings

To address the lack of facilitation skills among educators, courses offer

the Meeting Wise Checklist as a tool for planning and evaluating meetings
(Boudett & City, 2014). The checklist encourages participants to think
about the purpose, process, preparation, and pacing of their meetings.
It places particular emphasis on articulating meeting objectives, assign-
ing roles to participants (such as facilitator, timekeeper, and note taker),
and planning and following up on next steps. Instructors also build par-
ticipants’ capacity by practicing “transparent facilitation” (the equivalent
of the instructional “think aloud”) while working with school teams. For
example, facilitators might explain why they decided to shorten a protocol
because of time constraints or why they chose to write suggestions on chart
paper to keep a visual record of what was said in conversation. By making
the decision-making process that facilitators use explicit, this practice en-
courages participants to notice this often invisible skill.

Teaching and Modeling Tools for Working in Groups

It is important to set the expectation early on that every participant can

grow as a facilitator, and to use protocols and agendas to take the mystery
out of facilitation by providing a sequence of steps and time limits for con-
versations. These are valuable tools for supporting the habit of intentional
collaboration because they make it clear when each person should con-
tribute ideas—and when he or she should to listen to the ideas of others.
At the same time, there is a need to clarify that each individual will bring
his or her own style and preferences to the team’s experience. To help par-
ticipants to understand this, courses often begin by engaging participants
in the Compass Points protocol, which helps them understand how team
members best learn and participate in group work.5 For example, some
team members may want to move quickly to action or creating products,
whereas other team members would prefer to discuss and ensure that ev-
eryone has a say in the process. Once team members have discussed these
preferences, they are often more comfortable setting norms or expecta-
tions for how their team will work together. When informed by conversa-
tions about group preferences, the norms are more authentic, rather than

12
 TCR, 117, 040304 Teaching Educators Habits of Mind for Using Data Wisely

simply rules for behavior that have little grounding in an understanding of

how a particular team prefers to operate. For example, although common
norms specify that the meetings should “assume positive intentions,” this
norm becomes more important after team members have discussed how
people who seek the big picture—as well as those who are determined to
get right down to details—are acting from their sincere instincts about
what will help the team make progress toward its goals rather than trying
to be oppositional.

E: RELENTLESS FOCUS ON EVIDENCE

This habit is closely linked to the habit of intentional collaboration, and

educators frequently report that the single most important thing that
builds trust in collaboration is making sure they ground statements in evi-
dence. This habit supports a culture in which people make decisions based
on specific, objective, and descriptive statements about what they see. But
drawing conclusions from data that are unfounded or unwarranted is a
deeply ingrained bad habit for most people, so it takes a determined effort
to break it. It is easy to fall into this bad habit because human beings are
naturally predisposed to making meaning and creating interpretations us-
ing data from the world around us (Senge et al., 2000). However, these
inferences or judgments are not helpful at the early stages of data inquiry
because they move educators away from looking closely at the data and
honing the practice of observation. As an example, a statement such as this
student doesn’t care about school is more likely to close down possibilities for
inquiry and prevent conversations about alternative explanations than a
factual statement like this student has not turned in assignments for three days.
Another bad habit that might arise is when educators have a narrow
definition of what constitutes useful data. Many people begin by assuming
that data refers to numeric representations of student performance on
assessments, such as how many students in a grade obtained scores below,
at, and above levels of proficiency on a high-stakes standardized test. The
accountability system in the United States, which allocates punishments
or rewards based on aggregates of student performance, exerts pressures
on educators to use data in certain ways (Jennings, 2012). At times, the
accountability pressures are so strong that they might dominate educa-
tors’ initial assumptions about what forms of data are most important to
examine, and school teams arrive at Data Wise expecting to learn how to
increase student performance on standardized tests. Fortunately, educa-
tion as a field has become wiser about the need to triangulate, or examine
multiple forms of data that provide evidence of student learning (Love
et al., 2008; Patton, 2002). Further, assessment experts have reinforced

13
Teachers College Record, 117, 040304 (2015)

the idea that consequential decisions about student learning should not
be based on the results of a single test (Koretz, 2009). Researchers docu-
ment instances where data teams meet to examine not only test scores but
student work produced in class and for homework, and they argue that
the most helpful sources of instructional information are assessments that
demonstrate student thinking, developmental pathways or approaches, or
misconceptions (Supovitz, 2012).
Practitioners often overlook data collected from classroom observa-
tions about student learning and teaching practice. Yet, being able to
accurately describe what is currently happening in classrooms is essen-
tial to figuring out what improvements to instruction are needed (City,
Elmore, Fiarman, & Teitel, 2009). When the data in question are obser-
vations about practice, it is particularly important to cultivate the habit
of maintaining a relentless focus on evidence. When an educator reports
that teachers in multiple classrooms failed to engage students in rigorous
work, a skillful facilitator will ask that person to provide the evidence sup-
porting his or her claim. A more useful data statement about classroom
practice might be: Most students answered teachers’ questions with one- or two-
word statements.
Strategies for helping educators develop a relentless focus on evidence
follow.

Using Case Studies to Expose Participants to a Broad Range of Data

Research supports the idea that children learn best when abstract ideas
are put into a context that is meaningful to them (National Research
Council, 2000); the same is true for adults (Kolb & Kolb, 2005). Efforts
to teach educators how to read score reports or develop action plans can
fall flat if not embedded in case studies of real schools. Teaching educa-
tors data literacy through examples of student work or videos of classroom
instruction from real schools helps them learn new skills while putting
themselves in the shoes of the educators in the cases. It also increases
engagement by making it easier for participants to see how to transfer
their learning to their own situations. Finally, case studies demonstrate
how real educators gather a broad range of data to address one focus of
inquiry. By following a case study school throughout the entire inquiry
cycle, participants see firsthand how a school or educator team might start
by examining student performance data, then move to looking at student
work, and finally to observing classroom instruction. Each next source of
data adds another layer of understanding about students’ and teachers’
strengths and challenges.

14
 TCR, 117, 040304 Teaching Educators Habits of Mind for Using Data Wisely

Teaching and Modeling Tools for Sticking to Evidence

Once again, protocols can be quite useful for helping educators try out
habits that might at first feel unfamiliar. For example, many educators
think that when they observe a classroom, it is their job to identify what the
teacher is doing “right” and “wrong.” So when they observe a video of in-
struction, participants should take very descriptive and specific notes about
what they see and hear students and teachers doing. The “affinity” protocol
helps educator teams analyze the classroom observation data. This protocol
allows educators to work with colleagues to sort, categorize, and label this
observational evidence and come to a shared understanding of what is hap-
pening—and not happening—in classrooms. In addition, educators might
use the mental model of “the ladder of inference,” an idea developed by
Chris Argyris, Peter Senge, and others (Argyris & Schön, 1996; Senge et al.,
2000). The bottom rung of the ladder contains descriptive statements, but
as one climbs the ladder conceptually, the higher rungs lead to inferences,
conclusions, and actions. Participants learn how to classify their statements
and how to go up the ladder deliberately, making sure they have enough
evidence to support the climb. To avoid making judgmental statements, par-
ticipants might ask one another, “What evidence do you see that makes you
say that?” The metaphor of being “high on the ladder” and “coming down
the ladder” can make the process of developing the habit of evidence more
playful and easier to talk about and do.

Providing Structured Opportunities for Participants to Discuss One Another’s Evidence

Perhaps the most effective strategy to maintain a relentless focus on evi-

dence is having educators share the evolving evidence of their learning
with their classmates. Students in the week-long degree program course
engage in peer consultancies in which they present and get feedback on
their group projects. Teams enrolled in the professional education cours-
es, which extend over a period of months, have even richer opportunities
for discussing ongoing work. After learning the process during an intense
week on Harvard’s campus, teams return to their settings, where they inte-
grate the improvement process into their daily work and document what
happens. During virtual coaching calls and live online sessions with their
institute peers, teams practice marshaling evidence of their improvement
work, describing it to others, and listening to their peers use evidence
to describe their successes and challenges. In addition to building par-
ticipants’ muscles around using evidence to support their claims, this in-
structional approach helps sustain motivation as participants share work,
receive feedback, make revisions, and hear from colleagues.

15
Teachers College Record, 117, 040304 (2015)

TRANSLATING LESSONS FROM THE FIELD

On the last day of a Data Wise course, it is typical for educators to report
that the experience was not what they expected. One year, a student sum-
marized her learning by writing, “I used to think that it was all about the
data: its accuracy, validity, the amount we have. And now I think that to
achieve success in using data to affect change, the attitudes and skills of
the people implementing the change are more important.” Interestingly,
in the early years, our courses placed more emphasis on helping educators
to retrieve assessment results from databases and create formal presenta-
tions of their data. However, feedback from participants revealed that they
needed more than a toolkit of discrete skills, protocols, and forms—they
needed a way to understand the process of inquiry in its entirety and the
habits of mind that they could integrate across all their work with data.
Reciprocal relationships between institutions of higher education, pro-
fessional developers, and K–12 educators are essential to supporting edu-
cators to become data literate. As noted in prior research, very few teacher
candidates learn to engage deeply in data literacy for teaching in their
coursework. Further, even when teacher candidates are placed in schools
with a mentor teacher who can model and support the development of
good instructional practices, there is no guarantee that the novice teacher
will be exposed to a team that engages in data inquiry. Novice teachers are
socialized into the professional act of collaborative improvement vicari-
ously, if at all. As a result, we arrive at our current situation: Most educators
have not learned data literacy skills in preservice preparation, such that
schools and districts must provide ongoing professional development to
make up for this gap.
Institutions of higher education and professional development provid-
ers must encourage learning through clinical practice, meaning courses
that combine practitioners and students occurring in real school settings
and using authentic examples. It is usually said that institutions of higher
education have a responsibility to bridge what is known from research
studies with the everyday work of educators. The Data Wise Project sees
its responsibility as both bringing research to practitioners and translat-
ing knowledge from practitioners into frameworks, tools, and supports
for other educators. This involves creating instructional designs and peda-
gogy that are responsive to participants’ needs. Practitioners are therefore
involved in coauthoring materials, codesigning educational experiences,
and cofacilitating our courses. Their reflections and journey presenta-
tions demonstrate how educators apply the Data Wise Improvement
Process at their schools, including how they have integrated the process
with their other work. Practitioners’ questions inform the next iteration of

16
 TCR, 117, 040304 Teaching Educators Habits of Mind for Using Data Wisely

instructional design; their frustrations lead to the creation of new practi-

cal tools and problem-solving suggestions, and their successes are docu-
mented in ways that can be shared for the benefit of others’ learning.

CONCLUDING THOUGHTS

Education has been assailed for lacking ways to socialize educators into
the profession, leading many to observe that teaching practice in schools
appears haphazard and largely attributable to each individual teacher’s
dispositions. In his call for defining and developing signature pedagogies
in education, Shulman (2005) described a vision that education might
have a set of practices to induct novices into ways of “thinking like an edu-
cator,” and all institutions and programs that train and support educators
would use these methods. These signature pedagogies would encourage
the development of “habits of the mind, habits of the heart, and habits
of the hand” common to all members in the profession (p. 59). As the
field continues in search of a signature pedagogy that might socialize new
teachers into a collaborative, data-literate profession, there must be less
emphasis on teaching data literacy as a set of isolated skills and practices,
and instead a focus on teaching data inquiry as a holistic process ground-
ed in habits of mind necessary for collaborative improvement. The goal
is not just getting teachers to be comfortable with data, but allowing the
profession to evolve to a place where understanding of data is thoroughly
integrated with the work of learning and teaching.

NOTES
1. In this article, we refer to “educators” instead of just teachers to include ad-
ministrators, specialists, and coaches.
2. For more information about the Data Wise Project, please visit http://www.
gse.harvard.edu/datawise.
3. For further discussion of the ACE Habits of Mind, see Boudett & City, 2013.
4. Instructions for all the protocols mentioned in this article can be found on-
line: Plus/Delta, Affinity, SUMI: http://www.gse.harvard.edu/datawise; Success
Analysis: http://www.tcpress.com/pdfs/mcdonaldprot.pdf; Compass Points:
http://www.schoolreforminitiative.org/.
5. For instructions, please see http://www.schoolreforminitiative.org/.

17
Teachers College Record, 117, 040304 (2015)

REFERENCES
Argyris, C., & Schön, D. A. (1996). Organizational learning II: Theory, methods and practice.
Reading, MA: Addison-Wesley.
Boudett, K. P., & City, E. A. (2013). Lessons from the Data Wise Project: Three habits of mind
for building a collaborative culture. Harvard Education Letter, 29(3), 1–2. Retrieved from
http://hepg.org/hel/article/567
Boudett, K. P., & City, E. A. (2014). Meeting wise: Making the most of collaborative time for educators.
Cambridge, MA: Harvard Education Press.
Boudett. K. P., City, E. A., & Murnane, R. J. (Eds.). (2013). Data Wise, revised and expanded
edition: A step-by-step guide to using assessment results to improve teaching and learning.
Cambridge, MA: Harvard Education Press.
City, E. A., Elmore, R. F., Fiarman, S. E., & Teitel, L. (2009). Instructional rounds in education: A
network approach to improving teaching and learning. Cambridge, MA: Harvard Education Press.
Coburn, C. E., & Turner, E. O. (2011). Research on data use: A framework and analysis.
Measurement, 9, 173–206.
Coburn, C. E., & Turner, E. O. (2012). The practice of data use: An introduction. American
Journal of Education, 118(2), 99–111.
Costa, A. L., & Kallick, B. (Eds.). (2008). Learning and leading with habits of mind: 16 essential
characteristics for success. Alexandria, VA: Association for Supervision and Curriculum
Development.
Edmondson, A. (1999). Psychological safety and learning behavior in work teams.
Administrative Science Quarterly, 44(1999), 350–383.
Edmondson, A. (2002). The local and variegated nature of learning in organizations: A
group-level perspective. Organization Science, 13(2), 128–146.
Gallimore, R., Ermeling, B., Saunders, W., & Goldenberg, C. (2009). Moving the learning
of teaching closer to practice: Teacher education implications of school-based inquiry
teams. Elementary School Journal, 109(5), 537–553.
Goertz, M., Oláh, L., & Riggan, M. (2009, December). Can interim assessments by used for
instructional change? (CPRE Policy Briefs RB-51). Philadelphia, PA: Consortium for Policy
Research in Education.
Greenberg, J., & Walsh, K. (2012). What teacher preparation programs teach about K-12 assessment:
A review. Washington, DC: National Council on Teacher Quality. Retrieved from http://
www.nctq.org/p/publications/docs/assessment_report.pdf
Hackman, R. (2002). Leading teams: Setting the stage for great performances. Cambridge, MA:
Harvard Business School Press.
Hamilton, L., Halverson, R., Jackson, S., Mandinach, E., Supovitz, J., & Wayman, J. (2009).
Using student achievement data to support instructional decision making (NCEE 2009-4067).
Washington, DC: National Center for Education Evaluation and Regional Assistance,
Institute of Education Sciences, U.S. Department of Education. Retrieved from http://
ies.ed.gov/ncee/wwc/publications/practiceguides/
Hiebert, J., Gallimore, R., & Stigler, J. (2002). A knowledge base for the teaching profession:
What would it look like and how would we get one? Educational Researcher, 31(5), 3–15.
Jennings, J. L. (2012). The effects of accountability system design on teachers’ use of test
score data. Teachers College Record, 114(11), 1–23.
Katzenbach, J. R., & Smith, D. K. (2003). The wisdom of teams: Creating the high performance
organization. Cambridge, MA: Harvard Business School Press.
Kekahio, W., & Baker, M. (2013). Five steps for structuring data-informed conversations and action
in education (REL 2013–001). Washington, DC: U.S. Department of Education, Institute of
Education Sciences, National Center for Education Evaluation and Regional Assistance,
Regional Educational Laboratory Pacific. Retrieved from http://ies.ed.gov/ncee/edlabs

18
 TCR, 117, 040304 Teaching Educators Habits of Mind for Using Data Wisely

Kolb, A. Y., & Kolb, D.A. (2005). Learning styles and learning spaces: Enhancing experiential
learning in higher education. Academy of Management Learning and Education, 4(2), 193–212.
Koretz, D. (2009). Measuring up: What educational testing really tells us. Cambridge, MA:
Harvard University Press.
Lave, J. (1996). The practice of learning. In S. Chaiklin & J. Lave (Eds.), Understanding practice:
Perspectives on activity and context (pp. 3–32). Cambridge, England: Cambridge University Press.
Levine, T. H., & Marcus, A. S. (2010) How the structure and focus of teachers’ collaborative activities
facilitate and constrain teacher learning. Teaching and Teacher Education, 26(3), 389–398.
Little, J. W. (1982). Norms of collegiality and experimentation: Workplace conditions of
school success. American Educational Research Journal, 19, 325–340.
Little, J. W. (1990). The persistence of privacy: Autonomy and initiative in teachers’
professional relations. Teachers College Record, 91(4), 509–536.
Little, J. W. (2002). Locating learning in teachers’ communities of practice: Opening up problems
of analysis in records of everyday work. Teaching and Teacher Education, 18(7), 917–946.
Lortie, D. C. (1975). Schoolteacher: A sociological study. Chicago, IL: University of Chicago.
Love, N., Stiles, K. E., Mundry, S., & DiRanna, K. (2008). The data coach’s guide to improving learning
for all students: Unleashing the power of collaborative inquiry. Thousand Oaks, CA: Corwin Press.
Mandinach, E. B., Friedman, J. M., & Gummer, E. S. (2015). How can schools of education help to
build educators’ capacity to use data? A systemic view of the issue. Teachers College Record, 117(4).
Mandinach, E. B., & Gummer, E. S. (2013). Defining data literacy: A report on a convening
of experts. Journal of Educational Research and Policy Studies, 13(2), 6–28.
Marsh, J. A. (2012). Interventions promoting educators’ use of data: Research insights and
gaps. Teachers College Record, 114(11), 1–48.
McDonald, J. P., Mohr, N., Dichter, A., & McDonald, E. C. (2007). The power of protocols (2nd
ed.). New York, NY: Teachers College Press.
McLaughlin, M., & Talbert, J. E. (2002). Reforming districts: How districts support school reform.
Seattle: Center for the Study of Teaching and Policy, University of Washington.
Means, B., Chen, E., DeBarger, A., & Padilla, C. (2011). Teachers’ ability to use data to inform
instruction: Challenges and supports. Washington, DC: U.S. Department of Education
Office of Planning, Evaluation and Policy Development.
Means, B., Padilla, C., & Gallagher, L. (2010). Use of education data at the local level: From
accountability to instructional improvement. Washington, DC: U.S. Department of Education
Office of Planning, Evaluation and Policy Development.
National Forum on Education Statistics. (2012). Forum guide to taking action with education data
(NFES 2013-801). U.S. Department of Education. Washington, DC: National Center for
Education Statistics.
National Research Council. (2000). How people learn: Brain, mind, experience, and school
(expanded ed.). Washington, DC: National Academies Press.
Nelson, T., & Slavit, D. (2008). Supported teacher collaborative inquiry. Teacher Education
Quarterly, 35(1), 99–116.
Park, V., & Datnow, A. (2009): Co-constructing distributed leadership: district and school
connections in data-driven decision-making. School Leadership & Management, 29(5), 477–494.
Patton, M. Q. (2002). Qualitative research and evaluation methods. Thousand Oaks, CA: Sage.
Putnam, R. T., & Borko, H. (2000). What do new views of knowledge and thinking have to say
about research on teacher learning? Educational Researcher, 29(1), 4–15.
Saunders, W. M., Goldenberg, C. N., & Gallimore, R. (2009). Increasing achievement by focusing
grade-level teams on improving classroom learning: A prospective, quasi-experimental
study of Title I Schools. American Educational Research Journal, 46(4), 1006–1033.
Senge, P., Cambron-McCabe, N., Lucas, T., Smith, B., Dutton, J., & Kleiner, A. (2000).
Schools that learn: A Fifth Discipline fieldbook for educators, parents, and everyone who cares about
education. New York, NY: Crown Business.

19
Teachers College Record, 117, 040304 (2015)

Shulman, L. S. (2005). Signature pedagogies in the professions. Daedalus, 134(3), 52–59.

Supovitz, J. (2012). Getting at student understanding—The key to teachers’ use of test data.
Teachers College Record, 114(11), 1–29.
Supovitz, J., Foley, E., & Mishook, J. (2012) In search of leading indicators in education. Education
Policy Analysis Archives, 20(19) Retrieved from http://epaa.asu.edu/ojs/article/view/952
Talbert, J. E., Mileva, L., Chen, P., Cor, M. K., & McLaughlin, M. (2010). Developing school
capacity for inquirybased improvement: Progress, challenges, and resources. Stanford, CA:
Center for Research on the Context of Teaching.
Troen, V., & Boles, K. C. (2012). The power of teacher teams: With cases, analyses, and strategies for
success. Thousand Oakes, CA: Corwin.
Vygotsky, L. S. (1978). Mind in society: The development of higher psychological processes (M. Cole, V.
John-Steiner, S. Scribner, & E. Souberman, Eds.). Cambridge, MA: Harvard University Press.
Wayman , J. C., Jimerson, J. B., & Cho, V. (2012). Organizational considerations in establishing
the Data-Informed District. School Effectiveness and School Improvement: An International
Journal of Research, Policy and Practice, 23(2), 159–178.
Wenger, E. (1998). Communities of practice: Learning, meaning, and identity. Cambridge,
England: Cambridge University Press.
Young, V. M., & Kim, D. H. (2010). Using assessments for instructional improvement: A
literature review. Education Policy Analysis Archives, 18(19), 1–40.

CANDICE BOCALA is the co-chair of the Data Wise Summer Institute and
a senior team member for the Data Wise Project. Since 2009, she has been
working with educators to use the Data Wise Improvement Process. She
also conducts research and program evaluation for WestEd, where her
research interests include data use, professional development, and school
improvement. Previously, she taught elementary school in Washington,
DC. Candice holds a BA in government from Cornell University, an MA
in policy analysis and evaluation from Stanford University, an MAT in
elementary education from American University, and an EdD from the
Harvard Graduate School of Education

KATHRYN PARKER BOUDETT is director of the Data Wise Project, co-

chair of the Data Wise Summer Institute, and co-chair of the Data Wise
Coach Certification Program. She is a lecturer at the Harvard Graduate
School of Education (HGSE). Her publications include: Data Wise: A Step-
By-Step Guide to Using Assessment Results to Improve Teaching and Learning,
Data Wise in Action: Stories of Schools Using Data to Improve Teaching and
Learning, Key Elements of Observing Practice: A Data Wise Facilitator’s Guide and
DVD and Meeting Wise: Making the Most of Collaborative Time for Educators.
Kathy holds a BA in economics from Yale University, an MPP from the
Harvard Kennedy School and a PhD in public policy from the Harvard
Graduate School of Arts and Sciences.

20