Studies in Educational Evaluation 64 (2020) 100831

Contents lists available at ScienceDirect

Studies in Educational Evaluation

journal homepage: www.elsevier.com/locate/stueduc

Using data on school strengths and weaknesses for school improvement T

a, ,1 b
Tali Aderet-German * , Miriam Ben-Peretz
a
Department of Education, Ben-Gurion University of the Negev, P.O.B. 653, Beer-Sheva 84105, Israel
b
Faculty of Education, University of Haifa, Haifa, Israel

A R T I C LE I N FO A B S T R A C T

Keywords: This paper examines evaluation data use practices of a network of schools implementing an internal, in-
School self-evaluation dependent, school self-evaluation process for more than a decade. This network currently uses data on its
School improvement strengths alongside data indicative of its weaknesses, collecting and utilizing both positive and negative data for
Data use improvement and accountability purposes. We conducted multiple formal and informal interviews with 24
School change
school management members and teachers, and gathered and analyzed 50 school documents in order to un-
School evaluation
derstand how the network used evaluation findings for school improvement. We identified types of data use
described in the literature (instrumental, conceptual, and symbolic), and propose a complementary type of use
we termed “reinforcement data use”. Our findings suggest that identifying strengths is one of the valid goals and
outcomes of evaluation, shedding light on its potential to enhance school ethos, and to promote a positive
attitude toward evaluation processes and their subsequent effects.

1. Introduction more than a decade. The school network conducted evaluations on a

Over the past decade, the use of evaluation data in schools has been
one of the more increasingly researched topics in education (see for
example: Coburn & Turner, 2012b; Schildkamp et al., 2012b, 2013;
Datnow & Hubbard, 2015a). The use of data refers to decision making
in schools based on various types of data, such as assessment data,
student demographics, and parent and student surveys. Data can be
used for both accountability and improvement purposes (Schildkamp &
Kuiper, 2010; Schildkamp, Earl et al., 2013). School improvement is
commonly associated with educational change (Hopkins, Stringfield,
Harris, Stoll, & Mackay, 2014), thus many of the ways in which schools
use data entail change.
School evaluation data conveys strong possibilities for school im-
provement, yet schools face barriers to using evaluation findings due to
teachers fearing evaluation processes (Conley & Glasman, 2008;
Terhart, 2013), their limited capacity regrading data use (Datnow &
Hubbard, 2015a), and the common misuse, and even nonuse of data
(Alkin and King, 2016). In this article we discuss the different ways a
network of schools uses data indicative of its strengths, as well as its
weaknesses.
We present a case study of “Rishonim’’ (all names in this paper are
pseudonyms), an autonomous network of schools which has been
conducting an internal, independent school self-evaluation (SSE) for
wide variety of topics, and unlike many school evaluations (Marsh,
2012) was not limited to student performance data, providing a po-
tentially lucrative platform to explore different types of use of evalua-
tion findings. In light of the common barriers to data use (fear of eva-
luations, limited capacity, misuse and nonuse) Rishonim’s ongoing
efforts to use data they collect stand out, leading us to the research
question at the center of this study: How does an autonomous school
network use data collected through the school’s self-evaluation pro-
cess?.
The first part of this paper provides theoretical background for the
study, detailing the terminology used throughout the paper, and ela-
borating on existing types of data use. This part examines the purpose
each type of data use has to enable a more complete understanding of
how schools use data in practice. The second part describes the context
of the study, the SSE enacted at the school network at the center of this
study. Following this background, we detail the methods we employed
and present our analysis of the network’s data use practices which
identified types of use described in the literature. In this section we
suggest an additional, complementary type of data use that we termed
“reinforcement data use”. The fifth and final part of the paper discusses
these findings, as well as the possible implications reinforcement data
use has for school improvement.

⁎
Corresponding author.
E-mail addresses: tali100@gmail.com (T. Aderet-German), mperetz@edu.haifa.ac.il (M. Ben-Peretz).
1
The study was conducted at University of Haifa, Haifa, Israel.

https://doi.org/10.1016/j.stueduc.2019.100831
Received 26 February 2019; Received in revised form 3 November 2019; Accepted 5 December 2019
0191-491X/ © 2019 Elsevier Ltd. All rights reserved.
T. Aderet-German and M. Ben-Peretz
2. Evaluation data use
One of the questions that must be addressed by school leadership at
the end of an evaluation process is “How do we use the data?”. This
question is widely dealt with in the literature, referring mainly to the
changes in schools and other organizations lead by evaluation use
(Alkin & Taut, 2002; Cousins & Leithwood, 1986; Johnson et al., 2009;
Contandriopoulos & Brousselle, 2012). While many studies in this area
deal with how teachers use data to improve their classroom instruction
(for instance Assessment Reform Group, 2002; Bertrand & Marsh, 2015;
Birenbaum, Kimron, & Shilton, 2011), this study follows Schildkamp,
Earl et al. (2013) data use framework which expands this focus on
classroom use, to explore the use of data by the school for informing
policy decisions in addition to improving instructional practices.
Accordingly, in this paper we define data as “systematically col-
lected information on students, schools, school leaders, and teachers,
which can be obtained from qualitative (e.g., classroom observations)
and quantitative (e.g., assessment results) methods” (Schildkamp, Lai
et al., 2013, p. 137). In the school at the center of this study, the data
was obtained through SSE, therefore we use the terms ‘data’ and ‘eva-
luation data’ intermittently. Furthermore, we define data use (some-
times termed ‘data-based decision-making’, e.g. Schildkamp, Earl et al.,
2013; Staman, Timmermans, & Visscher, 2017, or ‘data-driven decision
making’, e.g. Mandinach, 2012; Reeves & Chiang, 2018) as the ways
school leaders and teachers shape local policy and practice toward
school improvement based on these data.
2.1. Main types of evaluation data use
There are several differentiations when examining the use of eva-
luation findings. The traditional three types are instrumental, con-
ceptual, and symbolic uses (Alkin & King, 2016; King & Pechman, 1984;
Leviton & Hughes, 1981).
Instrumental use refers to cases when the findings are used to di-
rectly influence action (King & Pechman, 1984; Leviton & Hughes,
1981). For example, when a teacher decides to emphasize a certain
topic which seems to have been misunderstood by the students, or
when a principal decides to host a lecture for the teachers on a peda-
gogical subject that is lacking according to a recent evaluation. Con-
ceptual use refers to the influence of evaluation processes or their re-
sults over time on changing the thinking of users (King & Pechman,
1984; Leviton & Hughes, 1981). For instance, when teachers are in-
volved in evaluating a program they are exposed to the details of the
program, its advantages and limits, and consequently take a position as
to its ability to enrich students. Symbolic use refers to the instances in
which users apply the evaluation process for their own agenda, such as
attaining political support or confirming a policy (King & Pechman,
1984; Leviton & Hughes, 1981). An example is a principal using student
achievements on standardized tests to persuade stakeholders to support
a newly implemented program which the principal has been promoting.
King and Pechman (1984) introduced two types of uses in their
analysis of a case study of a research and evaluation department of a
large city: signaling use and charged use. Their discussion of the two
types emphasized the interrelations and overlaps among the types they
identified and the previous typology. Signaling use is viewed as a
routine accountability signal to outside agencies, regarding the fulfill-
ment of expectations and requirements of the system (King & Pechman,
1984). Charged use is a term from the field of chemistry, where eva-
luation results are viewed as “positive” or “negative” and are used ac-
cordingly to initiate change. Negative data point to school weaknesses
and areas in need of change. The way in which schools use positive data
that highlight strengths is less developed in the theoretical literature on
data use.
2
Studies in Educational Evaluation 64 (2020) 100831
2.2. Using data in practice: purposes of evaluation data
A comprehensive review by Cousins and Leithwood (1986) of 65
empirical studies on evaluation data use found that users of evaluations
should be involved in their design and implementation to ensure the
reliability and relevance of the findings, creating a greater commitment
to the use of the findings in practice. Johnson et al. (2009) corroborated
these findings, adding the importance of stakeholder involvement in
facilitating evaluation use.
Schildkamp and Kuiper (2010) reviewed studies regarding the
purposes of data use by schools. Some of these purposes were intended
to lead to improvement and change, for instance: instructional pur-
poses; policy development and planning; shaping professional devel-
opment; meeting accountability demands; legitimizing existing pro-
grams; motivating students and staff; supporting personnel decisions.
The researchers also identified other types of data use with different
purposes, such as accountability demands and legitimizing actions.
Combining the two purposes of improvement and accountability is
one of Nevo’s (1995, 2006) arguments for a dialog between external
and internal evaluation in schools. Nevo (2006) claims this dialog can
enable schools to use various kinds of evaluation data from a wide
range of variables that are relevant and meaningful to the specific
school’s context for both purposes.
Much of the recent work done on data use in schools refers to the
way teachers use data to improve their own instruction and their stu-
dents’ learning, focusing on data generated through tests and other
evidence of students learning (see for example: Coburn & Turner,
2012b; Schildkamp et al., 2012b; Datnow & Hubbard, 2015b; Van Geel,
Keuning, Visscher, & Fox, 2016). Similarly, assessment for learning is a
type of data use which refers to a form of formative assessment which
incorporates teachers’ continuous feedback to students as part of on-
going classroom practices and that focuses on the quality of the learning
process (Assessment Reform Group (ARG), 2002; Birenbaum et al.,
2011; Wiliam, 2011; Heitink, Van der Kleij, Veldkamp, Schildkamp, &
Kippers, 2016).
These types of data use are more local, and usually do not involve
managerial decision-making or suggest policy change. Using data for
whole-school development is a complex task that has many enablers
and barriers. Schildkamp, Karbautzki, and Vanhoof (2014) found that
the main barriers to data use were limited accessibility to high quality
data, lack of professionalism in using data, and staff collaboration issues
around the use of data. In their examination of schools’ data use in five
European countries they noted that a positive attitude toward data use
was mentioned by respondents in all countries as an enabling influence
on data use” (p. 21). These findings concur with previous findings on
the importance of principals’ and teachers’ positive attitude towards
self-evaluation as a pre-condition for its successful enactment. They
emphasize the wariness of school staff from the process, explaining that
“Self-evaluation is not a popular activity among many teachers because
it is seen as a form of social control” (p. 27) and as a threat to their
autonomy.
2.3. Evaluation data use and school improvement
As presented above, a considerable amount of literature has been
published on evaluation data use. Fig. 1 presents a classification of the
types of data use we reviewed, according to their purposes (i.e., ac-
countability and improvement purposes), and the level of data sources
used in the school (i.e., school-level and teacher-level).
These studies have typically examined the way in which schools use
data on areas in which they are lacking in order to initiate change,
improve practices, and support decision-making (Coburn & Turner,
2012a; Hargreaves, Lieberman, Fullan, & Hopkins, 2010; Hopkins &
West, 2002; Schildkamp, Ehren, & Lai, 2012; Schildkamp & Kuiper,
2010; Chapman and Sammons, 2013). Scholars inherently link school
improvement with educational change (de Clercq, 2007; Fullan, 2006;
T. Aderet-German and M. Ben-Peretz
Fig. 1. Classification and examples of types of data use, according to their purposes and the level of generation of data used.
Hopkins & West, 2002; Hopkins, Stringfield, Harris, Stoll, & Mackay,
2014). Does school improvement implicitly entail change? When the
data gathered implies that the issue requires the school’s attention and
action, the answer to this question is yes. However, this raises a follow-
up question: how do schools use data that identifies and confirms school
strengths?
2.4. Identifying school strengths
MacBeath, Schratz, Meuret, and Jakobsen (2000) presented an in-
strument consisting of a set of twelve areas of school life as the starting
point of a school evaluation process. One of the aims of this instrument
was to help the schools “identify and prioritize areas for deeper in-
quiries” (p. 97). This instrument served as the starting point for sys-
tematic investigation and evidence collection. MacBeath et al. (2000)
found that the areas chosen for further inquiry included issues in which
there was a wide divergence of opinion, areas with apparent absence of
evidence, areas with an obvious weakness, or a practice the school
wished to enhance. The Scottish HM Inspectorate of Education (HMIE)
called this approach a “proportionate approach… [since it] enables you
to focus on areas of priority rather than routinely covering all aspects of
the school’s work in turn” (HMIE, 2007, p. 15). School evaluation fra-
meworks by both the HMIE (2007) as well as MacBeath et al. (2000)
included indicators concerning the stronger school practices, however
the focus in both frameworks was on uncovering problematic areas.
An example of a study which concentrated mainly on school
strengths is Lawrence-Lightfoot’s (1983) study, in which she con-
structed portraits of high schools, selecting “good high schools” as
identified by faculty, students, parents, and communities. These por-
traits explored the “goodness” of schools “…that refers to what social
scientists describe as school’s ‘ethos’, not discrete additive elements. It
refers to the mixture of parts that produce a whole.” (p. 23). Similarly,
research approaches such as the “success case method” (SCM;
Brinkerhoff, 2003) and Marzano’s review of What works in schools
(2003), highlight the positive impact which research and evaluation
can have in practice.
2.5. How could schools use evaluation data on school strengths?
While many researchers recognize the utility and advantages of
learning from school strengths (for example Brinkerhoff, 2003;
MacBeath et al., 2000; Marzano, 2003; Reynolds, 2010), they are not
3
Studies in Educational Evaluation 64 (2020) 100831
always clear as to how the schools themselves should use this data. The
orientation of these studies are in contrast with the claim of Bubb,
Earley, Ahtaridou, Jones, and Taylor (2007) that one of the aims of SSE
is “shining a light into dark corners” (p.35), requiring schools to pay
attention to neglected areas in order to inform the school improvement
plan, together with data gathered on school strengths. Similarly,
Reynolds (2010) views identifying “the excellent departments” (p. 605)
in school as an important part of school improvement, but neither study
details how information on school strengths could be later used.
In a project involving 101 schools from eighteen European coun-
tries, one of the SSE benefits noted by MacBeath and colleagues
(MacBeath et al., 2000) was the affirmation of “the good thing the
schools were doing, recognizing and celebrating aspects of school life
which had previously gone unnoticed” (p. 184). Yet, while acknowl-
edging the importance and value of identifying school strengths, Mac-
Beath and colleagues do not explain how to use the school strength
which has been identified.
The conclusions from these studies (Bubb et al., 2007; MacBeath
et al., 2000; Reynolds, 2010) emphasize the significance of under-
standing the ways in which schools can benefit from positive SSE
findings, calling for further analysis of positive data use in schools. The
current study follows this call, by attending to the ways schools can use
data that confirm school strengths.
3. Study context: “Rishonim” school network
This paper presents data from a case-study of a network of schools
called “Rishonim”, a semi-private Israeli school. Semi-private schools in
Israel are partially funded by the Ministry of Education, and are also
allowed to collect tuition from the parents of the students. Students
enrolled in the school come from a wide geographical area, and it is a
highly reputed and in-demand school, doing well consistently in ex-
ternal and internal evaluations.
The network employs approximately 340 teachers, who teach 4000
students between the ages of five and eighteen, at six different branches
located in four campuses. The school network is headed by a Chief
Executive Officer (CEO) and has a pedagogical department headed by a
Chief Pedagogical Officer (CPO).
Rishonim has been engaged in independent SSE for the past decade,
and for the past six years has employed a full-time skilled evaluation
coordinator who leads and implements the evaluation processes un-
dertaken by school management. The SSE at Rishonim addresses
T. Aderet-German and M. Ben-Peretz
various subjects such as effectiveness of homework, student learning
characteristics and preferences, school climate from the perspective of
both teachers and students, and many other aspects of school life. The
evaluation coordinator employs qualitative and quantitative methods,
such as interviews, focus groups, and questionnaires, as suitable to the
objectives of the evaluations. Usually, the evaluation method is de-
signed by the coordinator, adapting existing tested methods to the
particular school context, or constructing new instruments to address
specific issues. Thus, the SSE enacted at Rishonim is not based on a
predetermined evaluation model, and is aligned to the research ques-
tions the school management decides upon.
4. Methods
The findings presented in this paper are based on data from a case-
study that utilized several qualitative methods in order to investigate
the SSE process at the “Rishonim” network of schools.
4.1. Participants and sources
We analyzed school documents and conducted semi-structured in-
terviews with school management and key figures in the school eva-
luation process, as well as with teachers from two of the network’s
branches: a junior high-school (i.e., grades seven to nine) and an ele-
mentary school (i.e., grades one to six). The interviews were conducted
with 24 participants: five teachers from each branch, and 14 manage-
ment-level administrators (i.e., principals in different levels of school
hierarchy and other non-teaching staff members who were key figures
in the school network).
Participants were purposively sampled (Patton, 1990) for variation
in role in school, and involvement in the SSE. As Rubin and Rubin
(2012) recommended, the participants in this case study were not se-
lected to be representative of the whole school staff population, but
rather on the basis of their ability to illuminate issues. Participants were
suggested by the school management and by other participants, as well
as chosen to participate in the study because of their role in the school.
The semi-structured interviews’ length varied, from 45 min (the
shortest interview) to two hours (the longest interview), the average
interview was approximately an hour long. These formal interviews
consisted of questions focused on understanding the participants’ per-
spectives on the SSE processes in all their phases, for instance, what
they thought of the topics which were evaluated, how the process was
actually implemented, how much were they a part of the design of the
process, and how the findings were communicated. The participants
were asked to elaborate on SSEs in which they were involved, how they
viewed the process, and how the findings were useful in the classroom
or school context, if at all. Interview transcripts were based on notes
taken during the interview, completed in detail by the researcher on the
same day of each interview.
Many of the participants were interviewed, formally and informally,
several times, in order to follow-up and elaborate on underdeveloped
issues (see Table 1 for precise details regarding the number and type of
interviews). These interviews and informal conversations were oppor-
tunities for the participants to expand ideas from their previous inter-
view, and for the researcher to ascertain the interpretations made from
the previous interview. These follow-up interviews had another major
benefit, as they facilitated the formation of a trusting and listening-
Table 1
Number and type of interviews with teachers and management members.
One semi-structured Two semi -structured
interview interviews
Management members (n = 14) 2 8
Teachers (n = 10) 2 8
4
Studies in Educational Evaluation 64 (2020) 100831
oriented “research relationship” (Josselson, 2013, p. 24) between the
researcher and the participants.
An additional method used for understanding the way in which the
school utilizes data was through gathering and analyzing approxi-
mately 50 school documents. The documents included presentations,
reports and other material on the SSE, reports to the school board, as
well as booklets produced by the school with regard to school programs
and activities, and school documentation (e.g., such as the school
newsletter) available online about school activities and school life in
general. These documents shed light on the context, background, and
outcomes of the evaluation process enacted at Rishonim.
4.2. Data analysis
We analyzed the data using thematic analysis, identifying recurrent
themes in the texts transcribed from the interviews with the partici-
pants, as well as those in the first author’s field notes on the interviews
(Ryan & Bernard, 2003). The analysis was carried out manually, using
iterative readings to search for evidence of the three main types of data
use presented above: instrumental, conceptual, and symbolic, allowing
for the addition of emerging new types. This integration between de-
ductive and inductive analysis (Patton, 1990) enabled us to expand the
existing typology of data use found in the literature.
Ryan and Bernard (2003) suggest reading the text over and over in
order to identify themes through topics that reoccur throughout the
text; “the more the same concept occurs in a text the more likely it is a
theme” (Ryan & Bernard, 2003, p. 89). On the first reading, major
themes (i.e. data use types) can be quickly marked; in the next stage less
obvious themes can be uncovered. In qualitative research the process of
analyzing data is an ongoing endeavour, commencing at the data col-
lection phase and proceeding throughout the remainder of the study.
During the course of the data collection and the analysis process the
first and second author discussed the ideas and themes that were con-
structed, facilitating both better understanding of the data, and trian-
gulation of our interpretations.
In addition, we implemented another triangulation strategy by
collecting data from different sources in the school. We analyzed the
school documents we collected as to the context they gave to under-
standing the SSE enacted at Rishonim, and specifically the network’s
data use practices. These documents provided information on school
policy and infrastructure, allowing us to triangulate our thematic in-
terview analysis and form a comprehensive picture of how the network
used data. We matched each data use example our participants de-
scribed to the documents that could complement it. This data aided us
in developing questions for the follow up interview with the partici-
pant. Thus, our findings are presented in this paper through excerpts
from interview transcripts which represented the types of data use we
identified, while document data was used at the analysis stage.
Special attention has been given in this study to the researchers’
position in the school. The first author had no formal status in the
school, developed relationships with the school staff through regular
interviews and visits throughout the two-year data collection, spending
many hours in the school getting to know the staff and the school
context. The school management and staff cooperated fully and will-
ingly with the research and data collection. This allowed for the first
author to develop a unique “insider-outsider” point of view.
The second author has a long-time close acquaintance with the
Three semi -structured interviews and Documented informal conversations
more
4 10
0 8
T. Aderet-German and M. Ben-Peretz
school, in various roles, thus adding her own “insider-outsider” point of
view from a different, longitudinal, perspective. This involvement in
the school’s life provided the study with an insider’s point of view on
the schools’ vision and history. The combination of this insider’s point
of view and the first author’s insider-outsider knowledge of the school
aided in the analysis process and in ensuring the validity of the findings
(Surra & Ridley, 1991).
5. Findings: types of data use at Rishonim
In order to answer the question “‘How does Rishonim use self-eva-
luation data?”, data from this case study was categorized according to
the three traditional types of data use: instrumental, conceptual, and
symbolic use. These terms have been repeatedly examined in the lit-
erature since their initial introduction (Alkin & King, 2016; Alkin &
Taut, 2002; Cousins & Leithwood, 1986), and expanded and evolved by
leading scholars in the field (Johnson et al., 2009; Shulha & Cousins,
1997). Accordingly, in addition to identifying these three traditional
data-use types, the analysis process described above identified an ad-
ditional type of data use, namely, “reinforcement data use”. In this
section we present each of the four types of data-use, and demonstrate
the various ways in which they were enacted following the SSE process.
5.1. Instrumental data use: action-oriented SSE
Instrumental data use occurs when the evaluation findings lead di-
rectly to specific changes in practice. In this case study we found in-
strumental data use both by the teachers as well as by the management
of Rishonim.
One of the recurring issues described by teachers in their interviews
was their use of SSE data to actively change specific classroom and
school conduct. For instance, one of the teachers said, “I use evaluation
findings [on school violence] to set specific goals for my class, in order
to change recess activities” (F2). Another example for teachers using
data to change and improve the school in a concrete way was given by a
teacher who described a problem the school had with the cafeteria
lines, where they were constantly overcrowded and caused many stu-
dents to be late to classes. The teacher explained how following an
evaluation process, the problem and possible solutions were deliberated
with the students and a practical course of action was found: the ca-
feteria added another cashier, and the student council organized
timetables for priority in the lines in order to control the flow of stu-
dents.
The management emphasized the importance of using the knowl-
edge created from SSEs: the CPO stated, “when you ask questions, you
must use the answers; asking in order to know is not enough”. In line
with this, the CEO said, “The challenge is not only to conduct a survey,
but to do something with the findings”. This perception of the man-
agement is mirrored in one of the teachers’ interviews:
The surveys aren’t done in vain, they are used for improvement, and
that is very important to us – there is serious consideration [of the
findings]. That is how it should be done, and if you give feedback
then there must be a reference to this feedback – it is supposed to
help growth (S).
The school administration aimed to generate evaluation findings
that could be used in the short term by the teachers, for instance, by
considering the options available for acting upon evaluation findings
when selecting questionnaires items. The CPO emphasized, “I can’t ask
about something that can’t be dealt with this year”, referring to not
asking teachers if they were happy with the availability of computers
for their use in the teacher’s lounge, since there were no resources for
2
These unique codes are used to allow the reader to differentiate between the
participants (teachers and management members).
5
Studies in Educational Evaluation 64 (2020) 100831
adding computers if the answers pointed to such a need.
Another strategy for generating findings usable by the teachers in
the short term is to provide them with specific data on their practices.
For example, one of the school’s recent pedagogical changes involved
teachers implementing a mentorship program for every student. This
program is accompanied by continuous periodical evaluations, in which
the students are asked about their relationship with their mentor-tea-
cher. Each mentor receives their own mentees’ confidential evaluation,
to which only they have access, so that they can determine the best way
to proceed with their role as mentors.
5.2. Conceptual data use: using accumulative knowledge to change
conceptions
Conceptual data use addresses instances when the process of the
evaluation, as well as its results, are used to change the users’ thinking
about the issues examined, although evaluation data is not used directly
in order to change specific practices. Teachers referred to the way in
which the school management used SSE data to initiate discussion: “We
always learn about the findings in our staff meetings, so it feels we
know what is done with the questionnaires we and the students an-
swer.” (L). The school staff has routine weekly meetings in each of the
branches in order to provide a consistent venue for these discussions. In
addition, once every few months the pedagogical department organizes
whole-school staff meetings in which various issues are presented and
debated.
In recent years Rishonim has been gradually implementing a school-
wide pedagogical and organizational reform. When asked about the
origins of the reform, the management interviewed could not point to
any one evaluation process which had pointed to an immediate action
the school needs to take toward change, or which had provided evi-
dence of its direct necessity. It may be that the accumulated knowledge
gained over the years of SSE, as well as the revelations from some of the
findings, raised the need for profound change which was answered by
the school-wide reform.
One of the processes which could exemplify the way SSE has in-
cremental power on the school’s decision-making was described by
several management-level study participants. At the center of a specific
SSE process was an evaluation of students’ ways of learning, which
showed the diversity of the students’ preferences, and the limited
variety of opportunities of expression these preferences were given. The
participants claimed this process led to the understanding that the
school needs to find ways to enable teachers to allow for diverse
learning tendencies. This conclusion may have been one of the SSE
findings which initiated the discussion with regard to profound changes
in pedagogic and organizational aspects of the school. The finding from
these, and other, SSE processes, were viewed as part of an accumulative
sequence which led to the school-wide reform, as one of the manage-
ment members said: “The reform is based on the continuum of eva-
luations we executed” (D).
An evaluation of homework practices as perceived by teachers,
parents, and students, serves as an example of an evaluation with
findings that caught the management by surprise, and led to a process
of discussing the school’s educational objectives. This evaluation re-
vealed that most of the high school students were struggling with
homework overload and were aided by private tutors. One of the school
leaders stated: “I was rattled by these findings [of the evaluation on
homework practices] it required a deep thinking, I had no idea how
many students were assisted by private tutoring lessons” (N). Another
school leader explained that “these disturbing findings …led to a pro-
fessional symposium on this subject, involving the school community:
teachers, parents, and alumni, using the presentation of the findings as
a motor to start a discussion on how to change the situation, and to
build the awareness of teachers.” (E). A central school leader concluded
that “the product [of this evaluation process] is the discussion and the
awareness of staff and parents [with regard to this issue]” (N).
T. Aderet-German and M. Ben-Peretz
5.3. Symbolic data use: validating prior decisions
Symbolic data use refers to data used to retroactively support de-
cisions already made, as well as political and personal use of findings.
Referring to the school-wide reform initiated by the management, a
school leader explained how the school management used surveys on
the new programs as a legitimizing tool in order to “ignite” (in his
words) the innovative reform, and to support the new initiative by
providing information on the process to stakeholders such as parents
and the school board.
A member of the school’s management described one of the ways
the findings could be used politically and in order to support existing
processes:
The school has implemented significant procedures…and this is the
role of the surveys, to support the direction the school wants to go,
or suggest new paths. If you want to start a communication major in
school…a survey can support or reject this direction because you
check with the teachers, the parents and the students if there is
interest and value in this initiative, and according to the outcome
decide if you implement. It gives corroboration… the survey is not
the only source leading to new directions, but it has significance, it
can support or confirm… [the school-wide reform] was not initiated
because of surveys, but rather they started [the reform] and learned
from the evaluation how it is done, and then fine-tuned it. (K).
This school leader also emphasized the importance of the back-
ground for the evaluation process, and how it was initiated after an
unstable era in the school’s history, both economically and organiza-
tionally. The school needed a tool to verify that it was on the right path,
and the school’s stakeholders wanted an accountability tool.
Another example for symbolic data use refers to the use of an eva-
luation process on teachers’ perceptions of the school as a way to un-
derscore the management’s commitment to the teachers in the school. It
was important for a new CTO of the school to highlight how important
and central teachers are to him specifically, and to the school in gen-
eral, and conducting a large comprehensive SSE on their perceptions
was a means for not only understanding their needs and views, but also
a way to underscore their significance to him.
5.4. Reinforcement data use: the value of identifying strengths
Reinforcement data use is the fourth type of data use that we are
introducing, having emerged from inductive thematic analysis. This
type is similar to conceptual data use as it refers to a change in people’s
thoughts and beliefs and is not directly linked to an explicit action.
Whereas conceptual data use involves using the process, or its results,
for gradual conceptual change, which is often inadvertent, reinforce-
ment data use involves a conscious, intentional act of utilizing eva-
luation findings on school strengths, rather than a gradual change in
perceptions (as in conceptual data use).
5.4.1. Unofficial shared language recognizing strengths alongside
weaknesses
During data collection in this case study, it became clear that
Rishonim had an unofficial shared language. Many of the participants,
independently, and spontaneously, used the same terms to refer to ac-
tivities related to the SSE process. One of these terms (in Hebrew - the
language in which the interviews were conducted) was “Leshamer ve
Leshaper”, an expression that means sustaining and improving. This
expression is commonly used in schools when assessing strengths and
weaknesses of student achievements, but usually not to describe school
processes. Both teachers and management members used this expres-
sion to describe SSE as a process of sustaining and improving school
practices. One of the new teachers used it when describing the weekly
staff meeting in which the teachers learned about SSE findings:
6
Studies in Educational Evaluation 64 (2020) 100831
I answered the survey, and I saw the results at the teachers meeting
– the results were projected on the big screen, and the management
didn’t only present them – ‘come see the results’, but also – ‘let’s see,
where were we wrong, where were we right, what should we sustain
and what improve.’ (L)
An experienced teacher described her approach to the SSEs. And
what is learned from them, using the same expression:
I am all for the surveys etc., it is very convenient that we have a way
of knowing what to improve and what to sustain….I know that lessons
are learned, and I benefit from it. There is feedback to the teachers,
for instance there was a whole staff meeting [on one of the SSE
findings]… we can see that someone has thought this through –
anyway, it is a way to ventilate your feelings, it doesn’t have to
imply significant change all the time (R)
Teachers and management members used this expression to explain
how the SSE was used not only for changing and improving bad pro-
cesses and practices, but also to intentionally recognize, understand,
and celebrate what is working and what the school should keep doing.
5.4.2. Three ways to use data on school strengths
Our findings suggest three ways in which Rishonim school staff
members used positive data: 1. recognizing and sustaining good prac-
tices; 2. enhancing a sense of pride; and 3. promoting a positive attitude
toward evaluation. As presented in the previous section, Rishonim staff
viewed the evaluation process as a process aimed at improvement, but
not necessarily entailing change. Many participants explained that
evaluation findings included both good and bad aspects, and that both
positive and negative findings should be taken into consideration when
setting the school agenda. When discussing the use of evaluations in-
cluding positive data, one of the principals described his goals for
evaluation, stressing the need for reinforcing existing practices by sys-
tematic evidence: “Evaluation is multidimensional and should answer
the questions: Am I happy and satisfied with the situation? What am I
happy with? Is it what we aimed for? [The last survey] gave me cor-
roboration to the climate I am leading.” (A). This principal obtained
positive and negative evaluation findings and used both to decide how
to continue leading the school while aiming for continual school im-
provement. This example demonstrates how positive evaluation data
could be used to better the school: By understanding which practices
worked, the principal was able to reinforce them.
One of the management members explained the importance of re-
ceiving data on both the good and bad aspects of the school as being
accountable to stakeholders in terms of areas in need of improvement,
while also publicizing and celebrating its strengths:
We want to really understand, not from informal conversations – to
understand where we need to improve and what we are good at.
What we do well, what less so and needs improving. It is essential
for us, also because of the school’s special situation in the education
system [as a semi-private school]. (O)
Several other administration members expressed the value of SSE
outcomes that were not straightforward change: “The outcome is dis-
cussion and awareness…a sense of pride in what we are” (D). This view
was also evident in the process of choosing the questions for an eva-
luation of teachers’ perceptions of the school. The administration added
survey items that concerned strengths: school ethos, goals, and vision as
well as perceptions on collegial work at the school. These areas were
known to be school strengths, as opposed to areas the management felt
were in need of change such as the issue of trust and communication
with the management, as well as professional development options for
teachers.
One of the principals, who was previously a teacher in the school
network, explained the importance of emphasizing strengths when
presenting SSE findings:
T. Aderet-German and M. Ben-Peretz
We showed the staff what our strengths were, what the good things
were. If you want the staff to cooperate, and to see the benefits of
cooperation, you need to tell them when they are good at co-
operating. I think it strengthens the solidarity, ‘esprit de corps’(team
spirit), gives a sense of fulfillment and worth when you see the good
things that come up. It also puts the bad in perspective – this is a
good school and good things happen here, it is important to have
this feedback. (Y)
In addition to explaining how SSE can strengthen team spirit and
school staff solidarity, this principal adds another benefit to recognizing
school strengths through SSE: Giving the staff the opportunity to see a
more comprehensive picture of school activity. In his opinion, a more
positive attitude towards evaluations is facilitated by the acknowl-
edgement through evaluations’ findings that “good things happen”
alongside that which needs fixing.
It is important to note that the school network management re-
ceived periodic updates of the evaluation processes implemented at the
school, and that the reports explicitly stated both negative and positive
findings. The management was proud of the school’s progress and the
satisfactory findings (see Study Context), but at the same time they
were careful to question the methodology employed in order to ascer-
tain the reliability of these positive findings.
6. Discussion
This case study examined how SSE data on school-wide issues be-
yond individual instructional practices and student achievements are
used for different purposes. The findings we presented in the previous
section answer the question “How does Rishonim use self-evaluation
data?” by elaborating on the various ways in which the school enacted
the three traditional types of data use – instrumental, conceptual and
symbolic – and by proposing a fourth type, reinforcement data use. This
section discusses the implications for research and practice of the fourth
type of data use we identified.
6.1. Four types of data use based on data on school strengths and
weaknesses
Our findings regarding Rishonim’s data use suggest that data col-
lected by and for schools could be differentiated between two cate-
gories (see Fig. 2): “positive data” which point to school strengths and
“negative data” which point to school weaknesses. This is similar to
King and Pechman’s (1984) reference to the term “charged data use”,
which views evaluation data as positive or negative, using both to in-
itiate change. Fig. 2 shows an overview of how these two categories of
data could be used internally and externally, and how this relates to the
four types of data use we presented in the findings.
Instrumental data use is usually based on negative data (data in-
dicative of school weaknesses) in order to change existing practices at
either the teacher/classroom level or at the school level. Positive data
(data indicative of school strengths) does not have a part in instru-
mental data use, as it does not point to areas in need of improvement
and does not relate directly to an action. Conceptual data use also fo-
cuses on negative data in order to change beliefs held on specific issues,
but could use positive data for this goal. Symbolic data use occurs when
the school or teacher uses evidence retrospectively in order to justify
actions.
Reinforcement data use, a fourth type of data use that was identified
in this case study, is the use of positive data for reinforcing existing
school strengths. This kind of use and its meaning for schools was in-
frequently discussed in the literature, yet it could be used as an im-
portant tool with which schools can enhance their ethos and team
spirit. One of the barriers to school evaluations is the staff’s resistance
to them, associating evaluations with ways to control and inspect them
rather than as tools for improvement (Bubb et al., 2007; Vanhoof et al.,
7
Studies in Educational Evaluation 64 (2020) 100831
2009). It seems that reinforcement data use has the potential to dissolve
some of this resistance and enable ongoing non-intimidating evalua-
tions that are not perceived as inspections.
It is important to note that we do not attest that reinforcement data
use is opposed to current typologies. Rather, our findings suggest that
this type of use could be viewed as complementary to existing types,
facilitating a more comprehensive understanding of data use both
theoretically and in practice. In a similar way that Zwart et al.’s (2015)
conceptual framework on professional learning explicitly addresses the
value of confirming existing conceptions, reinforcement data use
highlights this aspect in data use frameworks, as well as the affordances
of intentionally using data on school strengths.
6.2. Using data on school strengths to enhance school ethos
When discussing the goals of evaluation, the literature notes that
evaluation can reveal strengths as well as weaknesses (Bubb et al.,
2007; HMIE, 2007; MacBeath et al., 2000; Reynolds, 2010). One of
Leithwood et al.s’ (1998) findings when examining conditions that
foster organizational learning in schools refers to using positive data:
“Teachers’ commitments to their own learning appeared to be re-
inforced by shared celebrations of successes by staff…” (p. 263).
Nonetheless, most of the research in the field argues for the need to
change weaknesses and strive for improvement, without explaining
what could be done when strengths are identified. This study’s findings
show that the management at Rishonim deliberately collected data that
was meant to uncover positive aspects about the school, in order to
confirm them and celebrate them. Thus, along with conducting eva-
luations that were meant to improve the school and change areas that
identified as weaknesses through the collection of negative data, the
school network systematically and intentionally collected positive data
and communicated it to the school community.
Teachers, administration, and other key figures in the school’s self-
evaluation process emphasized the importance of knowing what they
should sustain and what they should change, providing multiple ex-
amples for both. Some of the examples for change were of actions
limited to specific contexts, such as adding teachers to yard duty during
recess to prevent violence, while some examples were of broad edu-
cational policy decisions, such as the network-wide reform. The ex-
amples of how data was used to sustain practices focused on the col-
lection and use of positive data which expressed the school leadership’s
agenda in enhancing school ethos and team spirit. The school ethos at
Rishonim commemorated their high academic achievements, but also
the emphasis the school gave to character education and values,
striving for educational excellence and innovation in all aspects of
school-life. The school network emphasized the importance of cher-
ishing strengths, and the school evaluation process identifying positive
data enabled this ethos to be uncovered and celebrated. This approach
to using positive data coincides with and expands upon arguments
made by previous researchers with regard to the importance of iden-
tifying the good aspects of schools in order to build on these aspects for
improvement (Brinkerhoff, 2003; Lawrence-Lightfoot, 1983; Marzano,
2003).
6.3. Limitations and implications for further research
This study followed the need to expand the research on data use
practices (Coburn & Turner, 2012a; Johnson et al., 2009) by gaining
insights into the various ways data was used by a network of schools
conducting independent evaluation processes. However, the methodo-
logical choices that were made entail limitations that should be ad-
dressed by future research. This case study relies on qualitative data
collected in a unique network of schools. The specific characteristics of
this network, as well as the methods of the study, limit the causal
generalizability of the findings. However, Maxwell (2012) emphasized
that “causal generalizations are necessarily secondary to particular
T. Aderet-German and M. Ben-Peretz
Fig. 2. Classification of types of data use, according to the kind of data (“positive data” and “negative data”), and the kind of data use (internal and external).
causal understanding” (p. 657). Thus, this case study adopts Maxwell’s
approach in exploring the specific conditions and context in which a
network of schools successfully implemented an ongoing systematic
self-evaluation process, to understand how the network used data for
school improvement. Future research could explore the concept of re-
inforcement data use in other organizations and contexts.
In addition to the study’s setting, another limitation of the study
concerns the interview transcription process. Experts in the field ex-
plicitly recommend not taking notes during an interview in order to
promote concentration and rapport, viewing audio recording as an
obvious and necessary method (Josselson, 2013). However, following
several participants’ refusals to be recorded, the current study was
compelled to carry out most of the interviews without audio-recording
them. Interview transcriptions were based on handwritten notes taken
during the interview and completed in detail later that day by the re-
searcher. Despite this drawback, and partly due to the multiple, lengthy
meetings conducted with many of the participants, the majority of the
interviews were aligned with Josselson’s (2013) main characteristic of
“good” interviews – creating an empathic and dynamic “research re-
lationship” (p. 24).
In this paper we examined data use mainly from the perspectives of
the leaders of Rishonim, as the data we collected emphasized this
perspective. Further research could examine the impact of reinforce-
ment data use on teachers’ ownership of evaluation processes, espe-
cially in light of the current study’s findings on the affordance of this
type of use on the acceptance of evaluation processes, and in light of
emerging participatory models of evaluation.
6.4. Implications for Policy and Practice
Fig. 2 demonstrates possible ways schools can use different types of
data for different types of purposes: Accountability and improvement.
Both purposes can be based on data that highlights school strengths
(i.e., positive data) alongside the traditional use of data pointing to
school weaknesses (i.e., negative data). Using internal and external data
on school weaknesses is widely discussed in the literature on school
evaluation and school improvement (for instance Coburn & Turner,
2012a; Hargreaves, Lieberman, Fullan, & Hopkins, 2010; Hopkins &
8
Studies in Educational Evaluation 64 (2020) 100831
West, 2002; Schildkamp, Ehren & Lai, 2012; Schildkamp & Kuiper,
2010). This type of data is usually used conceptually and instrumentally
in order to change areas in which the school is lacking. External data
use of positive data, aimed at accountability, is typically used for po-
litical symbolic purposes, either promoting the school to various sta-
keholders, or using the data to legitimize actions after they are carried
out. As discussed previously, internal data use of positive data, aimed at
improvement, could enhance school ethos, support educational in-
novations, and has meaningful practical implications for organizations.
An important practical implication of reinforcement data use is the
potential positive data has in enabling ongoing evaluation processes.
The findings of this study suggest that by using positive data in con-
junction with negative data, school evaluations were perceived as a
process oriented to learning and improvement rather than an ac-
countability process with an inspection orientation. Acknowledging and
sustaining strengths encouraged teachers to accept the evaluation
process, and enabled educational innovations building on the strong
aspects of the school, reducing the threating facet of evaluations pre-
vious research noted (for example Conley & Glasman, 2008; Terhart,
2013; Vanhoof et al., 2009).
The concept of reinforcement data use has implications for practice
in schools. It raises questions regarding the role of evaluators in pre-
senting and communicating positive data together with negative data
rather than focusing exclusively on school weaknesses. This study sheds
light on the importance of the school management’s awareness of the
potential in utilizing positive data as a constructive force in school.
These findings may help us understand better ways to use data in
schools and other organizations for accountability and improvement
purposes, recognizing that identifying strengths is one of the valid goals
and outcomes of evaluation aimed at improvement. Yet, it is important
to keep in mind that in order to fully benefit from the use of data on
school strengths, schools need to have access to high quality data on
school weaknesses that point to areas in need of improvement (as
emphasized by Schildkamp & Kuiper, 2010). Using data only for re-
inforcement limits the purposes of data use. Both practice and further
research may benefit by examining how data indicative of school
strengths could be used for school improvement and for facilitating
continuous non-threatening evaluations.
T. Aderet-German and M. Ben-Peretz
References
Alkin, M. C., & King, J. A. (2016). The historical development of evaluation use. The
American Journal of Evaluation, 37(4), 568–579. https://doi.org/10.1177/
1098214016665164.
Alkin, M. C., & Taut, S. M. (2002). Unbundling evaluation use. Studies in Educational
Evaluation, 29(1), 1–12. https://doi.org/10.1016/S0191-491X(03)90001-0.
Assessment Reform Group (ARG) (2002). Assessment for learning: 10 principles. Cambridge,
UK: University of Cambridge School of Education.
Bertrand, M., & Marsh, J. A. (2015). Teachers’ sensemaking of data and implications for
equity. American Educational Research Journal, 52(5), 861–893. https://doi.org/10.
3102/0002831215599251.
Birenbaum, M., Kimron, H., & Shilton, H. (2011). Nested contexts that shape Assessment
for Learning: School-based professional learning community and classroom culture.
Studies in Educational Evaluation, 37, 35–48. https://doi.org/10.1016/j.stueduc.2011.
04.001.
Brinkerhoff, R. (2003). The success case method: Find out quickly what’s working and what’s
not. Berrett-Koehler Publishers.
Bubb, S., Earley, P., Ahtaridou, E., Jones, J., & Taylor, C. (2007). The self-evaluation
form: Is the SEF aiding school improvement? Management in Education, 21(3), 32–37.
https://doi.org/10.1177/0892020607079991.
Chapman, C., & Sammons, P. (2013). School self-evaluation for school improvement: What
works and why? England: CfBT Education Trust.
de Clercq, F. (2007). School monitoring and change: A critical examination of whole
school-evaluation. Education As Change, 11(2), 97–113.
Coburn, C. E., & Turner, E. O. (2012a). The practice of data use: An introduction.
American Journal of Education, 118(2), 99–111. https://doi.org/10.1086/663272.
Coburn, C. E., & Turner, E. O. (2012b). The practice of data use [Special issue]. American
Journal of Education, 118(2).
Conley, S., & Glasman, N. S. (2008). Fear, the school organization, and teacher evalua-
tion. Educational Policy, 22(1), 63–85. https://doi.org/10.1177/0895904807311297.
Contandriopoulos, D., & Brousselle, A. (2012). Evaluation models and evaluation use.
Evaluation, 18(1), 61–77.
Cousins, J. B., & Leithwood, K. A. (1986). Current empirical research on evaluation uti-
lization. Review of Educational Research, 56(3), 331–364. https://doi.org/10.3102/
00346543056003331.
Datnow, A., & Hubbard, L. (2015a). Teacher capacity for and beliefs about data-driven
decision making: A literature review of international research. Journal of Educational
Change, 17(1), 7–28.
Datnow, A., & Hubbard, L. (2015b). Teachers’ use of assessment data to inform instruc-
tion: Lessons from the past and prospects for the future. Teachers College Record,
117(4), n4.
Fullan, M. (2006). Change theory: A force for school improvement. Seminar series no. 157.
Centre for strategic education. Retrieved from http://www.michaelfullan.ca/Articles_
06/06_change_theory.pdf.
Hargreaves, A., Lieberman, A., Fullan, M., & Hopkins, D. (Eds.). (2010). (Vol. 23).
Springer Science & Business Media.
Heitink, M. C., Van der Kleij, F. M., Veldkamp, B. P., Schildkamp, K., & Kippers, W. B.
(2016). A systematic review of prerequisites for implementing assessment for
learning in classroom practice. Educational Research Review, 17, 50–62.
Hopkins, D., Stringfield, S., Harris, A., Stoll, L., & Mackay, T. (2014). School and system
improvement: a narrative state-of-the-art review. School Effectiveness and School
Improvement. An International Journal of Research, Policy and Practice, 25(2),
257–281. https://doi.org/10.1080/09243453.2014.885452.
Hopkins, D., & West, M. (2002). Evaluation as school improvement: A developmental
perspective from England. In D. Nevo (Ed.). School-Based Evaluation: An International
Perspective (pp. 89–112). Oxford, UK: Elsevier Science.
HM Inspectorate of Education (HMIE) (2007). How Good is our School ? – The Journey to
Excellence, Part 3. https://www.educationscotland.gov.uk/Images/
HowgoodisourschoolJtEpart3_tcm4-684258.pdf (accessed 30 September 2014).
Johnson, K., Greenseid, L. O., Toal, S. A., King, J. A., Lawrenz, F., & Volkov, B. (2009).
Research on evaluation use: A review of the empirical literature from 1986 to 2005.
The American Journal of Evaluation, 30(3), 377–410. https://doi.org/10.1177/
1098214009341660.
Josselson, R. (2013). Interviewing for qualitative inquiry: A relational approach. Guilford
Press.
King, J. A., & Pechman, E. M. (1984). Pinning a wave to the shore: Conceptualizing
evaluation use in school systems. Educational Evaluation and Policy Analysis, 6(3),
241–251.
Leithwood, K., Leonard, L., & Sharratt, L. (1998). Conditions fostering organizational
learning in schools. Educational Administration Quarterly, 34(2), 243–276.
Leviton, L. C., & Hughes, E. F. X. (1981). Research on the utilization of evaluations: A
Studies in Educational Evaluation 64 (2020) 100831
review and synthesis. Evaluation Review, 5, 525–549. https://doi.org/10.1177/
0193841X8100500405.
Lawrence-Lightfoot, S. (1983). The good high school: Portraits of character and culture.
Jossey-Bass Incorporated Pub.
MacBeath, J., Schratz, M., Meuret, D., & Jakobsen, L. (2000). Self-evaluation in European
schools: A story of change. London: RoutledgeFalmer.
Mandinach, E. B. (2012). A perfect time for data use: Using data-driven decision making
to inform practice. Educational Psychologist, 47(2), 71–85. https://doi.org/10.1080/
00461520.2012.667064.
Marsh, J. (2012). Interventions promoting educators’ use of data: Research insights and
gaps. Teachers College Record, 14(11).
Marzano, R. J. (2003). What works in schools: Translating research into action. Alexandria,
VA: ASCD.
Maxwell, J. A. (2012). The importance of qualitative research for causal explanation in
education. Qualitative Inquiry, 18(8), 655–661.
Nevo, D. (2006). Evaluation in education. In I. F. Shaw, J. C. Greene, & M. M. Mark (Eds.).
The sage handbook of evaluation. Sage.
Nevo, D. (1995). School-based evaluation: A dialogue for school improvement. Oxford:
Pergamon.
Patton, M. Q. (1990). Qualitative evaluation and research methods. SAGE Publications.
Reeves, T. D., & Chiang, J. L. (2018). Online interventions to promote teacher data-driven
decision making: Optimizing design to maximize impact. Studies in Educational
Evaluation, 59, 256–269.
Reynolds, D. (2010). Smart school improvement: Towards schools learning from their
best. In A. Hargreaves, A. Lieberman, M. Fullan, & D. Hopkins (Eds.). Second
International handbook of educational change (pp. 595–610). Netherlands: Springer.
Rubin, H. J., & Rubin, I. S. (2012). Qualitative interviewing: The art of hearing data. Sage
Publications.
Ryan, G. W., & Bernard, H. R. (2003). Techniques to identify themes. Field Methods, 15(1),
85–109. https://doi.org/10.1177/1525822X02239569.
Schildkamp, K., Ehren, M., & Lai, M. K. (2012a). Editorial article for the special issue on
data-based decision making around the world: From policy to practice to results.
School Effectiveness and School Improvement, 23(2), 123–131. https://doi.org/10.
1080/09243453.2011.652122.
Schildkamp, K., Ehren, M., & Lai, M. K. (2012b). Data-based decision making around the
world: From policy to practice to results [Special issue]. School Effectiveness and
School Improvement, 23(2).
Schildkamp, K., Karbautzki, L., & Vanhoof, J. (2014). Exploring data use practices around
Europe: Identifying enablers and barriers. Studies in Educational Evaluation, 42, 15–24.
https://doi.org/10.1016/j.stueduc.2013.10.007.
Schildkamp, K., & Kuiper, W. (2010). Data-informed curriculum reform: Which data,
what purposes, and promoting and hindering factors. Teaching and Teacher Education,
26(3), 482–496. https://doi.org/10.1016/j.tate.2009.06.007.
Data-based decision making in education. In K. Schildkamp, M. Lai, & L. Earl (Vol. Eds.),
Studies in educational leadership: vol 17Dordrecht: Springer. https://doi.org/10.1007/
978-94-007-4816-3_10.
Schildkamp, K., & Lai, M. (2013). Conclusions and data-use framework. In K. Schildkamp,
M. Lai, & L. Earl (Vol. Eds.), Data-based decision making in education. Studies in edu-
cational leadership: vol 17, (pp. 137–191). Dordrecht: Springer. https://doi.org/10.
1007/978-94-007-4816-3_10.
Shulha, L. M., & Cousins, J. B. (1997). Evaluation use: Theory, research, and practice
since 1986. Evaluation Practice, 18, 195–208.
Staman, L. L., Timmermans, A. A., & Visscher, A. A. (2017). Effects of a data-based de-
cision making intervention on student achievement. Studies in Educational Evaluation,
55, 58–67.
Surra, C. A., & Ridley, C. A. (1991). Multiple perspectives on interaction: Participants,
peers, and observers. In B. M. Montgomery, & S. Duck (Eds.). Studying interpersonal
interaction (pp. 25–55). New York: Guilford Press.
Terhart, E. (2013). Teacher resistance against school reform: Reflecting an inconvenient
truth. School Leadership and Management, 33(5), 486–500. https://doi.org/10.1080/
13632434.2013.793494.
Van Geel, M., Keuning, T., Visscher, A. J., & Fox, J. P. (2016). Assessing the effects of a
school-wide data-based decision-making intervention on student achievement growth
in primary schools. American Educational Research Journal, 53(2), 360–394.
Vanhoof, J., Van Petegem, P., & De Maeyer, S. (2009). Attitudes towards school self-
evaluation. Studies in Educational Evaluation, 35(1), 21–28.
Wiliam, D. (2011). What is assessment for learning? Studies in Educational Evaluation,
37(1), 3–14.
Zwart, R. C., Korthagen, F. A., & Attema-Noordewier, S. (2015). A strength-based ap-
proach to teacher professional development. Professional development in education,
41(3), 579–596.