Children and Youth Services Review 61 (2016) 165–175

Contents lists available at ScienceDirect

Children and Youth Services Review

journal homepage: www.elsevier.com/locate/childyouth

Effectiveness of anti-bullying school programs: A meta-analysis

José Antonio Jiménez-Barbero a,⁎, José Antonio Ruiz-Hernández a, Laura Llor-Zaragoza b,
María Pérez-García c, Bartolomé Llor-Esteban a
a
University of Murcia, Faculty of Psychology, Espinardo, Murcia CP: 30100, Spain
b
Catholic University San Antonio, Faculty of Social Sciences and Communication, Guadalupe, Murcia CP: 30107, Spain
c
Mental Health Center of Cieza, Murcia CP: 30530, Spain

a r t i c l e i n f o a b s t r a c t

Article history: The large number of studies published in recent years aimed at evaluating the effectiveness of school-based anti-
Received 9 November 2015 bullying programs recommends research to focus on synthesizing the evidence found in this regard.
Received in revised form 15 December 2015 This study presents a meta-analysis of random clinical trials that assess the efficacy of 14 anti-bullying school
Accepted 15 December 2015
programs. Sample size was of 30,934 adolescents aged between 7 and 16 years of whom 16,243 made up the In-
Available online 28 December 2015
tervention Groups and 14,691 made up the Control Groups. Meta-analysis was conducted for each outcome mea-
Keywords:
sure, as well as heterogeneity analysis. Analysis of subgroups was performed when necessary, as well as analysis
Bullying of publication bias.
Violence Results show moderate effect sizes for the outcome measures Bullying Frequency and Victimization Frequency,
Adolescents Attitudes and School Climate. Greater impact was observed in interventions of less than one school year duration,
School program as well as those targeting children younger than 10 years. Subgroup analysis confirmed greater heterogeneity in
Effectiveness studies evaluating complex interventions.
Meta-analysis In general, our results indicate that bullying and violence prevention programs in school settings are obtaining
beneficial, albeit discrete, results in the outcome measures evaluated.
© 2015 Elsevier Ltd. All rights reserved.

1. Introduction choice of deviant friends (Olalla-Cutrín, Gómez-Fraguela, & Luengo,

The first studies of school bullying were performed by Heinemann
(1972) and Olweus (1978, 1993, 1996, 2005). These authors refer
to bullying, as a repeated behavior of persecution and physical, psycho-
logical, or moral aggression carried out by one student or group of stu-
dents against another, with an inequality of power. School violence
includes behaviors than can cause physical or emotional harm, ranging
from verbal aggression to humiliation, social exclusion, physical harm,
and the destruction of property (Allen, 2009; Benbenishty & Astor,
2005), including various categories, such as disruption in the classroom,
disciplinary problems, and maltreatment among classmates (Olweus,
1993; Walker, 1995). The causes of this phenomenon are complex
and multifactorial, involving psychological aspects (Jiménez-Barbero,
Ruiz-Hernández, Llor-Esteban, & Waschgler, 2014), as well as variables
related to family functioning, either directly or indirectly, through the
⁎ Corresponding author at: Plaza Vistabella, 17, 2L, CP: 30820 Alcantarilla, Murcia,
Spain.
E-mail addresses: barbero49@hotmail.com (J.A. Jiménez-Barbero), jaruiz@um.es
(J.A. Ruiz-Hernández), llorzaragoza@ucam.edu (L. Llor-Zaragoza),
mariapg78@hotmail.com (M. Pérez-García), bllor@um.es (B. Llor-Esteban).
2015).
We are facing a phenomenon that, although it has been the object
of attention and social alarm in the past few decades, has probably
always been present in our schools. Diverse studies have analyzed
its prevalence in countries like the USA, Spain, Australia, the UK, or
Germany, indicating that between 20 and 30% of the students were
involved in episodes of violence ranging from simple verbal intimi-
dation to severe forms of physical or sexual aggression (Currie
et al., 2008; Defensor del Pueblo [Ombudsman], 2007; Department
of Health and Human Services & Center for Disease Control and
Prevention, 2006), which can foment the development of early be-
havior problems and social alienation among adolescents (Rudolph
et al., 2013), causing important relational problems during adult-
hood (Ruiz-Hernández, García-Jiménez, Llor-Esteban, & Godoy-
Fernández, 2015).
In recent years, the awareness of this problem has increased, leading
to the proliferation of school prevention programs (Jiménez-Barbero,
Ruiz-Hernández, Llor-Esteban, & Pérez-García, 2012; Ttofi & Farrington,
2011) as well as studies aimed at assessing their efficacy. In this sense,
although several systematic reviews and meta-analyses in this
regard have been published (Merrel, Gueldner, Ross, & Isava, 2008;
Mytton, DiGuiseppi, Gough, Taylor, & Logan, 2006; Park-Higgerson,

http://dx.doi.org/10.1016/j.childyouth.2015.12.015
0190-7409/© 2015 Elsevier Ltd. All rights reserved.
166 J.A. Jiménez-Barbero et al. / Children and Youth Services Review 61 (2016) 165–175
Perumean-Chaney, Bartolucci, Grimley, & Singh, 2008; Vreeman &
Carroll, 2007), the capacity to generalize the results has been
limited due to the inclusion criteria: for example, due to the date con-
straints imposed on the bibliographic searches, the use of evaluations
that combine primary and secondary prevention interventions, or be-
cause the authors used random clinical trials with non-randomized
methodologies.
The goal of our study was to assess the efficacy of anti-bullying
school programs. For this purpose, we conducted a meta-analysis of
random clinical trials (RCTs), contributing the following improve-
ments with respect to the meta-analyses already carried out on this
topic. Firstly, the meta-analyzed studies had to present an experi-
mental design in order to ensure the internal validity of the effect
size estimations. Secondly, the range of inclusion years of the studies
was updated, extending it to June 2015, taking into account that the
most recent meta-analyses of the efficacy of school programs for the
prevention of bullying included up to May 2009 (Ttofi & Farrington,
2011). However, statistical random effects models were applied,
which are currently considered more appropriate for the integration
of the results of empirical studies, due to the variability they usually
present (Rosa-Alcázar, Sánchez-Meca, Gómez-Conesa, & Marín-
Martínez, 2008).
2. Method
2.1. Inclusion criteria of the studies
The studies were included in the review if they met the following
criteria: (a) their goal should be the evaluation of the effectiveness of
an intervention program to prevent violence or bullying in the school
setting; (b) the assessed interventions had to target the study popula-
tion directly (students of Primary or Secondary schools), not teachers
or parents; (c) the studies should use an experimental design; (d) the
studies had to provide the necessary statistical information from the
control group (CG) and the intervention group (IG), at least at posttest
(means and standard deviation); and (e) the studies should be pub-
lished between 2000 and 2015.
Only comparisons between outcome measures assessed through
standardized tests were included. The outcome measures included in
the study were:
1. Bullying or school violence frequency. Including direct (physical
or verbal) and indirect aggression.
2. School victimization frequency. Direct or indirect aggression
would be taken into account.
3. Favorable attitudes toward bullying or school violence.
4. Attitudes against bullying or school violence.
5. School climate. Students' perceptions of school climate (trust,
cooperation, and willingness to help…).
2.2. Search strategy
A systematic search in the following electronic databases was con-
ducted: Medline, Tripdatabase, Cochrane, Academy Search Premier,
PsycINFO, ERIC, and PsycARTICLES. The keywords and terms used
were: “bullying” OR “school violence” AND “attitudes toward violence
AND “adolescents” AND “intervention” OR “prevention program” AND
“self-esteem” AND “empathy” AND “school climate” AND “victimization”.
The search was restricted to works published between 01/01/2000
and 31/05/2015. In all cases, the titles and abstracts were examined,
rejecting the works that did not meet the previously defined inclusion
criteria. The complete texts of the accepted articles were carefully
read, and their lists of bibliographic references were examined in
order to identify possible relevant articles that had not been located in
initial search.
2.3. Rating criteria of the methodological quality
The studies were selected by two independent reviewers who
assessed the methodological quality of the selected studies, observing
a high interjudge reliability in the application of the assessment criteria,
according to the Pearson correlational analysis of the obtained scores
(r = .86). The criteria used by the reviewers to carry the assessment
were the following:
(A) Score equal to or higher than 6 on a scale of methodological qual-
ity elaborated ad hoc from the guide published by the York University in
2001 for the elaboration of systematic reviews (NHS Centre for Reviews
and Dissemination, 2001). This scale comprised 10 items: (1) operation-
al definition of the constructs and terms used in the study; (2) adequate
sample selection method; (3) adequate sample size; (4) prior distinc-
tion of the subgroups or use of appropriate clustering techniques;
(5) validity of the assessment (information collected directly by the re-
searchers); (6) reliability of the assessment (use of a validated instru-
ment and/or with a high level of internal consistency to assess the
intervention); (7) follow-up of the results; (8) use of outcome mea-
sures matching the object of study; (9) adequate statistical analyses;
and (10) adequate presentation of the results by means of tables,
figures, or similar.
(B) Score equal to or higher than 3 on the scale elaborated by Jadad
et al. (1996) for randomized clinical trials. Studies that did not meet
either of these two conditions were excluded.
2.4. Tabulation and data analysis
The eligible studies were directly coded on an Excel database
data by the first author. The coding was reviewed by the second
and third authors, and doubts were resolved through discussion
among all the authors. Subsequently, a summary table was created
in which the data of each selected study was recorded according
to the following categories: date and country of study, goal of the
investigation, name of the prevention or intervention program
assessed, size and age of the sample used, and significant findings
(Table 1).
2.5. Statistical analysis
The effects of the characteristics of 14 school programs to prevent
or reduce violence or bullying in the school setting were assessed by
means of comparison of the results of the IG and those of the CG at
posttest, due to the fact that various studies included in the meta-
analysis did not report the pretest results. The variable randomiza-
tion (individual, class, or school) was not taken into account due
to the small number of available studies for the comparisons. The
total size of the effect was calculated for each comparison by
means of RevMan 5.2 (Cochrane Centre). Due to the variability fore-
seen in the individual effect sizes, a random effects model was used.
The differences between the IG and the IC for each comparison were
grouped to obtain the estimation of the total effect. The grouped re-
sults were expressed as standardized mean differences (SMD) with a
95% confidence interval. As the studies included in the meta-analysis
used different instruments to assess the outcome measures, we used
SMD to standardize all results to a common scale, measured in units
of standard deviation. As meta-analyses summarize the results
through studies with different methods, topics, and results, it is im-
portant to include a measure of heterogeneity to determine whether
the variation observed among the results in the studies is greater
than that which could be expected by chance, so we used chi-
square test with a significance level of .05 and calculated the I 2
index. For cases with significant heterogeneity, we conducted an
analysis of subgroups according to the following moderating vari-
ables: (a) year of publication of the study (studies published between
2000 and 2007 vs. studies published after 2007); (b) Sample size
Table 1
Overview of coding of studies included in the meta-analysis.

Author/country Objective Sample size Age Program/intervention Duration/ Outcome measures Significant results
follow-up

Stevens, Van Oost, Assess the effect of an intervention program of 13 PS 10–16 “The curriculum-based intervention 7 months. –Students' attitudes toward Significant changes in the variables measured at posttest 1,
and De anti-bullying in the attitudes of adolescents. 11 SS among peers”. Follow-up: bullying. which disappear in the follow-up, except in PS, where they
Bourdeaudhuij N = 2193 1 year. are maintained.
(2000)/Belgium
Teglasi and Rothman Assess the effectiveness of a school program to 1 PS 9–10 “The Structure/Themes/Open 15 weeks. –Social behavior. Significant reduction of antisocial and externalizing
(2001)/USA reduce aggressive behavior in class. N = 59 Communication/Reflection/Individuality/ Without –Externalizing behavior. behaviors for non-aggressive children. Among the
–IG = 8 Experiential Learning/Social follow-up. –Aggression-related beliefs. aggressive children, significant improvements in IG.
aggressive Problem-Solving (STORIES) Program”
–CG = 8

J.A. Jiménez-Barbero et al. / Children and Youth Services Review 61 (2016) 165–175
non-aggressive
Baldry and Farrington Assessment of an intervention program to reduce 2 MS & 1 SS 10–15 Ad hoc intervention: develop social skills 3 days. –Frequency of bullying and Reduction of some types of victimization, but only in the
(2004)/Italy bullying and victimization N = 239 to understand the negative consequences Without victimization. older students (ages 14–15).
–IG = 131 of bullying. follow-up. –Types of bullying and Younger students (ages 11–13) show an increase of
–CG = 106 victimization. bullying and victimization.
DeRosier and Marcus Test the effectiveness of a school group 11 PS 8 “Social Skills GRoup Intervention 6 months. –Self-perception./Self-efficacy. In the IG, an increase of sympathy among peers,
(2005)/USA intervention based on social skills, for children N = 381: (S.S.GRIN)”. Follow-up: –Social anxiety. self-esteem, and self-efficacy were found, and a decrease of
who experience social anxiety or bullying. –IG = 187 1 year. –Self-esteem. social anxiety and anti-social affiliations.
–CG = 194 –Depressive symptoms
–Bullying/Victimization.
Frey et al. Assess the impact of a school program aimed at: 6 PS 8–10 “Steps to Respect: to Bullying Prevention 5 months. –Interaction skills. In the IG, a reduction in the acceptance of bullying and
(2005)/USA –Reducing bullying and destructive ‘spectator’ N = 1126: Program”. Follow-up: –Bullying-related beliefs. greater involvement of spectators when their friends were
behavior. –IG = 549 1 year –Bullying-related behaviors. bullied was observed.
–Promoting pro-social beliefs. –CG = 577 –Behavior outside the Behavior in the schoolyard: physical and verbal bullying
–Increasing socio-emotional skills of children classroom (recess) were significantly reduced.
involved in bullying.
Fekkes, Pijpers, and Assess the effects of an anti-bullying school 47 PS 9–12 Program based on the “Olweus Bullying 2 school –Bullying/victimization Reduction of % of bullied children, victimization and
Verloove-Vanhorick program in primary schools. N = 3816 Prevention Program”. years. –Depression bullying behavior.
(2006)/Holland –IG = 1214 Follow-up: –Psychosomatic complaints No significant differences in the follow-up
–CG 1 = 1552 yes. –Criminal offenses
–CG 2 = 1050 –Satisfaction with school life
and relationship with
classmates
Hunt (2007)/Australia Assess the educational impact of an anti-bullying 6 SS 12–15 Ad hoc intervention. 1 school –Frequency and type of No significant differences in the majority of the measures.
intervention in secondary schools in Australia. N = 444 year. bullying.
–IG = 152 Follow-up: –Attitude toward the victim.
–CG = 258 1 year –Attitude toward the bully

Author/country Objective Sample size Age Program/intervention Duration/follow-up Outcome measures Significant results

Beets et al. (2009)/USA To assess the efficacy of a school program designed 20 PS Pre-test: “The Positive Action 4 years. –Violent behaviors Reduction of drug abuse.
(Hawaii) to prevent violent behavior and risk among N = 1714: 10–11 Program” Without follow-up –Drug abuse Decrease of risky sexual behavior.
Primary Education students –IG = 976 Post-test: –Voluntary sexual
–CG = 738 15–16 activity
Berry and Hunt Evaluate the efficacy of a cognitive–behavioral 7 SS 12–14 “Confident Kids 8 weeks. –Experiences of Reduction of experiences of bullying, symptoms of anxiety, and
(2009)/Australia intervention focused on factors that increase N = 46 (males): Program” Follow-up: 3 bullying depressive symptoms.
vulnerability for being bullied in adolescence. –IG = 22 months. –Depressive symptoms No significant differences in self-esteem or maladaptive
–CG = 24 –Symptoms of anxiety responses.
–Self-perception
–Maladaptive
responses

(continued on next page)

167
 168
Table 1 (continued)

Author/country Objective Sample size Age Program/intervention Duration/follow-up Outcome measures Significant results

Fonagy et al. (2009)/USA To compare the effectiveness of psychiatric school 9 PS 8–10 –Psychiatric school 24 months. –Aggression The intervention program (CAPSLE) moderated the tendency to
consultation (SPC) versus a school program N = 1345: consultation (SPC) Follow-up: 36 –Victimization increased victimization and aggression and reduced the number
(CAPSLE) versus usual treatment (TAU), to reduce –CAPSLE = 563 –“Creating a Peaceful months. –Awareness of aggressive spectators, compared with the other interventions
aggression in the PS. –SPC = 422 School Learning” –Behavior in class (SPC and TAU).

J.A. Jiménez-Barbero et al. / Children and Youth Services Review 61 (2016) 165–175
–TAU = 360 (CAPSLE)
–Usual treatment
(TAU)
Brown, Low, Smith, and Extend previous results obtained by the Program 34 PS 7–11 “Steps to Respect” 6 months –Student behavior Positive effects of the program on a series of results (for
Haggerty (2011)/USA Steps to Respect in the reduction of victimization N = 2940 –Environment/school example, improvement of school climate, lower levels of
and bullying –IG = 1440 climate perpetration of physical bullying, and reduction of school
–CG = 1500 –Attitudes toward problems related to bullying).
bullying
–Intervention of
students against
bullying
–Positive behavior of
the spectators
–Victimization,
perpetration of
bullying
Kärnä et al. (2011)/Finland Evaluate the effect of an intervention program 78 PS 10–12 “KiVa Antibullying 1 year –Bullying/victimization Significant reductions in 7 of the 11 dependent variables,
based on empathy, self-efficacy, attitudes of Grades 4–6 Program” –Roles of participants including self-reported victimization and self-reported bullying.
spectators to reduce bullying and victimization N = 8237 –Anti-bullying
IG = 4207 attitudes
CG = 4030 –Empathy with victims
–Self-efficacy
–Well-being at school
Waasdorp, Bradshaw, and Evaluate the effectiveness of the global approach 37 PS 8–11 “School-Wide 4 years –Bullying In the IG, decreased rates of bullying and rejection reported by
Leaf (2012)/USA school program to reduce physical and relational N = 12,344 Positive Behavioral –Rejection by peers teachers.
bullying IG = 6614 Interventions and
CG = 5124 Supports”
Jiménez-Barbero, Evaluate the efficacy of a school violence 1 SS 12–15 “Count on Me” 4 months –Attitudes toward Reductions of violence were observed in the schoolyard,
Ruiz-Hernández, prevention program based on attitudes N = 180 violence following the intervention in the IG.
Llor-Esteban, IG = 96 –Perceived violence No differences were obtained in the variable attitudes toward
Llor-Zaragoza, and Pérez CG = 84 –Impulsivity violence.
García (2013)/Spain –Externalizing
–Parental styles

Abbreviations: N = sample size; IG = intervention group; CG = control group; PS = primary schools; MS = middle schools; SS = secondary school.
 J.A. Jiménez-Barbero et al. / Children and Youth Services Review 61 (2016) 165–175
(studies that employed N N 400 participants vs. studies with sample
sizes N ≤ 400); (c) average age of the sample (age ≤ 10 years vs.
age N 10 years); (d) features of the intervention (interventions based
on a global or multidisciplinary approach vs. interventions based on
more specific models); and (e) duration of the intervention, (continuous
interventions or with a duration equal to or greater than a school
course, integrated in the study plan vs. brief interventions or less than
a school course). In order to examine the influence of each of the studies
in the results obtained, a sensitivity analysis was performed, replicating
the results of the meta-analysis, excluding in each step one of the stud-
ies included in the review. Finally, we included in the analysis a study of
publication bias to determine whether this could be a threat to the
validity of the results of the meta-analysis.
3. Results
The electronic search initially located 12,262 publications. After
reviewing the titles, abstracts, and reference lists, 315 potentially
eligible articles were identified. Of these, 280 were excluded for
not presenting experimental designs. Of the 35 resulting RCTs,
14 were excluded for failing to meet the indicated criteria of method-
ological quality. Five other studies had to be excluded for not
Fig. 1. Search strategy.
169
providing sufficient statistical data to carry out the analysis (they
did not provide the means and standard deviations) (Appendix).
We finally selected 14 studies that were included in the meta-
analysis (see Fig. 1).
Although the established time interval of the search strategy was up
to May 2015, the most recent study included herein is from 2013 be-
cause the few studies carried out subsequently had to be excluded for
the above-mentioned reasons.
The study population included children and adolescents
aged from 7 to 16 years of Primary and Secondary schools. The
total size of the sample of the meta-analysis included 30,934 individ-
uals at post-test, of whom 16,243 subjects formed the IG (mean
sample size N = 1160.21 subjects, SD = 1917.57) and the remain-
ing 14,691 subjects comprised the CG (mean sample size N =
1049.35 subjects, SD = 1590.44). Meta-analyses for each outcome
measure were conducted in order to obtain an estimate of the
mean effect size and a 95% confidence interval. For the analysis of
heterogeneity, we estimated the values of χ2 and the I 2 index
(see Table 2).
Although the programs were aimed at the prevention or reduction of
violence or intimidation at school, the outcome measures varied among
the different studies. The analysis therefore focused on the outcome
170 J.A. Jiménez-Barbero et al. / Children and Youth Services Review 61 (2016) 165–175

Table 2 this variable. Immediately after intervention, the frequency of bully-

Analysis of heterogeneity for each outcome measure.* ing and school violence was reduced, albeit discreetly, in the IG in
Outcome measures K d (95% CI) Q I2 comparison with the CG: SMD, IC (95%) = −.12 [−.17, −.06], how-
Bullying or school violence frequency 14 −.12[−.17, −.06] 36.76⁎⁎ 65%
ever, observing significant heterogeneity among the studies com-
Victimization frequency 8 −.09 [−.18, .01] 26.49⁎⁎ 74% pared (χ2 = 36.76, p = .0005). Therefore, we conducted analysis
Attitudes favoring bullying or school 3 −.18 [−.30, −.06] 2.09 4% of subgroups based on the moderating variables (year of publication,
violence sample size, age of the sample, duration and characteristics of inter-
Attitudes against bullying or school 4 .06 [.03, .10] 1.5 0%
vention), which revealed a higher mean effect size (d+) for interven-
violence
School climate 3 −.03 [−.08, .02] 1.36 0% tions lasting less than one year, (d+ = −.24 vs. d+ = −.10), as well
as the interventions targeting children of age ≤ 10 years, (d+ = −.17
Note. The sign of the value of d is interpreted according to the meaning of the variable.
K = number of studies; d = average effect size; 95% CI = 95% of the confidence interval of
vs. d+ = −.08). In addition, there was greater homogeneity among
the effect size; Q = Q statistic of heterogeneity; I2 = heterogeneity index. the studies evaluating the effectiveness of interventions that did
⁎ p b .05. not employ global or multi-disciplinary approaches (χ2 = 11.45,
⁎⁎ p b .001. p = .08), those that lasted less than one school year (χ2 = 7.61,
p = .11), and those using smaller sample sizes (χ2 = 11.45, p =
.08). See Table 3.
measures most frequently employed by the studies included in the
meta-analysis, and which were previously described.
3.2. Frequency of victimization
3.1. Bullying or school violence frequency
The analysis of this outcome measure included 8 studies (N =
The data of 30,934 adolescents were available (IG = 16,243, 14,293; IG = 7683, CG = 6610). In this case, the intervention had re-
CG = 14,691), included in the 14 complete trials that measured duced effects on the frequency of victimization by bullying, SMD, IC

Table 3
Subgroup analysis for the frequency of bullying or school violence outcome measure.

Moderator variable K d+ 95% CI Heterogeneity Differences between subgroups

dl du

Year of publication
≤2007 7 −.05 −.14 .04 χ2 = 10.09, p = .12, I2 = 41% χ2 = 36.76, p = .02, I2 = 59%
N2007 7 −.15 −.20 −.10 χ2 = 14.58, p = .02, I2 = 59%
Sample size
N N 400 7 −.12 −.18 −.07 χ2 = 23.79, p = .0006, I2 = 75% χ2 = 0.06, p = .81, I2 = 0%
N ≤ 400 7 −.10 −.26 .05 χ2 = 11.45, p = .08, I2 = 48%
Age of the sample
N10 years 8 −.08 −.16 0 χ2 = 36.76, p = .005, I2 = 65% χ2 = 4.13, p = .04, I2 = 75.8%
≤10 years 6 −.17 −.20 −.14 χ2 = 5.05, p = .41, I2 = 1%
Duration of the intervention
≥1 year 9 −.10 −.15 −.05 χ2 = 26.62, p = .0008, I2 = 70% χ2 = 2.02, p = .16, I2 = 50.4%
≤1 year 5 −.24 −.42 −.05 χ2 = 7.61, p = .11, I2 = 47%
Characteristics of the intervention
– Global focus 7 −.12 −.18 −.07 χ2 = 23.79, p = .0006, I2 = 75% χ2 = 1.26, p = .26, I2 = 20.8%
– Other models 7 −.10 −.26 .05 χ2 = 11.45, p = .08, I2 = 48%

K = number of studies; d+ = mean effect size; 95% CI = 95% confidence interval; dl and du = lower and upper confidence limits; χ2 = chi-square statistic of the analysis of heterogeneity;
p = level of significance; I2 = heterogeneity index.

Table 4
Subgroup analysis for the frequency of victimization outcome measure.

Moderating variables K d+ 95% CI Heterogeneity Differences between subgroups

dl du

Year of publication
≤2007 4 −.02 −.09 .04 χ2 = .56, p = .9, I2 = 0% χ2 = 2.32, p = .13, I2 = 56.8%
N2007 4 −.13 −.27 0 χ2 = 15.61, p = .001, I2 = 81%
Sample size
N N 400 4 −.09 −.19 .01 χ2 = 21.22, p b .0001, I2 = 86% χ2 = 0, p = .97, I2 = 0%
N ≤ 400 4 −.10 −.30 .11 χ2 = 6.62, p = .09, I2 = 55%
Age of the sample
≤10 years 3 −.07 −.13 −.01 χ2 = .05, p = .98, I2 = 0% χ2 = .17, p = .68, I2 = 0%
N10 years 5 −.10 −.24 .04 χ2 = 24.31, p b .0001, I2 = 84%
Duration of the intervention
≥1 year 6 −.08 −.17 0 χ2 = 22.64, p = .0004, I2 = 78% χ2 = .36, p = .55, I2 = 0%
≤1 year 2 −.33 −1.15 .48 χ2 = 6.23, p = .01, I2 = 84%
Characteristics of the intervention
Global focus 4 −.09 −.19 .01 χ2 = 21.22, p b .0001, I2 = 86% χ2 = 0, p = .97, I2 = 0%
Other models 4 −.10 −.30 .11 χ2 = 6.62, p = .09, I2 = 55%

K = number of studies; d+ = mean effect size; 95% CI = 95% confidence interval; dl and du = lower and upper confidence limits;
χ2 = chi-square statistic of the analysis of heterogeneity; p = level of significance; I2 = heterogeneity index.
 J.A. Jiménez-Barbero et al. / Children and Youth Services Review 61 (2016) 165–175 171

Fig. 2. Effect size for each outcome measure.

(95%) = −.09 [−.17, .00], observing heterogeneity among the studies,
(χ2 = 28.95, p = .0001).
The analysis of subgroups as a function of the moderating
variables (see Table 4) indicates better results in studies after 2007,
(d+ = −.13 vs. d+ = −.02), as well as in those in which the interven-
tion had lasted less than one school year (d+ = −.33 vs. d+ = −.08).
On the other hand, we observed homogeneity among the studies
on interventions targeting children aged ≤ 10 years (χ2 = .05, p =
.98), those that did not use global or multidisciplinary approaches
(χ2 = 6.62, p = .09), or that used smaller sample sizes (χ2 = 6.62,
p = .09). We also observed greater homogeneity among the effect
sizes of the studies that were conducted in or before the year 2007
(χ2 = .56, p = .9).
3.3. Attitudes favoring school violence and bullying
Only 3 studies included this outcome measure (N = 1336; IG =
660, CG = 676). In this case, the meta-analysis reported the
existence of moderate beneficial effects in this measure, ([SMD, IC
(95%) = −.18 [−.30, −.06]), observing homogeneity among the
studies (χ2 = 2.09, p = .35; I2 = 4%).
172 J.A. Jiménez-Barbero et al. / Children and Youth Services Review 61 (2016) 165–175

Fig. 3. Analysis of publication bias: Funnel-plot for each outcome measure.

 J.A. Jiménez-Barbero et al. / Children and Youth Services Review 61 (2016) 165–175
3.4. Attitudes against bullying or school violence
Attitudes against bullying or school violence were used as outcome
measure in 4 studies (N = 11,900; IG = 5929, CG = 5971). Although
they revealed homogeneous results (χ2 = 1.50, p = .68; I2 = 0%), the
mean effect size of the intervention was small, (SMD, IC (95%) = .06
[.03, .10]).
3.5. School climate
Improving school climate or school well-being as an outcome
measure was used in 3 studies (N = 6106; IG = 2806, CG = 3300).
The studies are comparable according to the analysis of hetero-
geneity (χ2 = 1.36, p = .51; I2 = 0%). However, the meta-analysis
hardly showed any significant differences after the intervention,
finding small reductions of the values of this measure (SMD, IC
(95%) = −.03 [−.08, .02]). See Fig. 2.
3.6. Analysis of publication bias
In order to evaluate the potential of publication bias (i.e., the
impact of the studies that are missing from the meta-analysis), we
conducted a sensitivity analysis using funnel-plot, which represents
the magnitude of the effect of each trial compared to a measure of its
size, such as the standard error (Sutton, Duval, Tweedie, Abrams, &
Jones, 2000), so that asymmetry in the funnel plot may indicate pub-
lication bias in the meta-analysis. For this study, we used a funnel
plot for each analyzed outcome measure. There was clear asymmetry
in the figures of victimization and school climate (Fig. 3). For the
other variables, the distribution was symmetrical, so there could be
no publication bias that would compromise the interpretation of
effect sizes found. However, in some cases, the number of compari-
sons was reduced, so this type of figures may not be suitable to detect
this bias.
4. Discussion
In this study, we presented the results of a meta-analytic investi-
gation on the effectiveness of school-based programs to prevent or
reduce bullying or violence at school. For this purpose, we selected
a total of 14 studies that met the selection criteria, creating a total
of 24 comparisons.
In general, our results indicate that the programs of prevention of
bullying and violence in school settings are obtaining beneficial,
albeit discrete, results in the outcome measures evaluated. In this
sense, they coincide with prior meta-analyses like that of Park-
Higgerson et al. (2008), which reported that no significant effects
in 4 of the 5 characteristics of the examined programs had been
found. In the same vein, Merrel et al. (2008) only observed signifi-
cant positive effects for one-third of the variables (knowledge,
attitudes and self-perceptions of abusive conduct). For the rest
of measures, among which were some of those analyzed herein
(bullying and victimization), these authors did not obtain significant
changes.
The outcome measures most frequently used by the studies in-
cluded in this work were Frequency of bullying or school violence
and Frequency of school victimization, in which our analysis shows
modest reductions. These results follow the line observed by recent
meta-analyses such as that of Ttofi and Farrington (2011), who
only observed significant reductions in the variable victimization in
3 of 14 meta-analyzed experimental studies. On the other hand,
Vreeman and Carroll (2007) did report reductions in these variables
for interventions based on a multidisciplinary or global approach,
but their conclusions should be taken with caution as theirs is a sys-
tematic review in which a comparison of the individual effect sizes
was not made.
173
In any case, our results about these measures (bullying/victimization)
should be considered under the perspective of the heterogeneity
observed among different studies. The subgroup analysis showed
that, in these cases, such heterogeneity existed in the comparisons
that included studies of greater complexity, that is, with larger sam-
ple sizes, that evaluated multidisciplinary or global interventions, or
that involved a longer time-interval. In this sense, some studies
have mentioned the difficulty of solving this variability when pro-
grams involving many variables are studied because they should
be considered as complex systems (Burton, 2012). This author
stated that heterogeneity in such studies should be considered as
the natural variation of the effects observed. In the case of
the outcome measure Frequency of victimization, heterogeneity
among studies may also be related to a possible publication bias,
as reflected by the asymmetry observed in the funnel-plot.
Regarding attitudes toward school violence, the authors have
used two outcome measures: favorable attitudes and attitudes
against school violence or bullying. Our analysis shows a moderate
impact of interventions on attitudes favoring school violence, with
significant reductions in the scores on this measure, which confirms
the results obtained by previous studies, suggesting that attitudes
toward violence gain the most benefit from such programs (Fox,
Elder, Gater, & Johnson, 2010; Jiménez-Barbero et al., 2012; Mytton
et al., 2006). However, although in our work we observed homoge-
neity among the different studies, the small number of comparisons
carried out (only 3 studies used this outcome measure) does not
allow us to assess the existence of publication bias, so these results
should be taken with prudence.
We also included in the meta-analysis the impact of the interven-
tions on attitudes against bullying. In this case, we observed homogene-
ity among the four studies that are compared, but the mean effect size,
although positive, is weak and insignificant.
Finally, only three studies considered school climate as an out-
come measure, and no significant improvement was observed after
the intervention. As there are no meta-analytic studies that have
considered this variable, it is complex to assess such a measure, al-
though the figure of the funnel-plot could be indicating a possible
publication bias.
5. Conclusion
These results lead us to conclude, provisionally, that there is some
evidence of the effectiveness of interventions aimed at preventing
school violence and bullying, reducing the frequency of victimization
and bullying, and improving attitudes toward school violence. This pre-
caution in the interpretation of the results is due to the fact that most of
the mean effect sizes are too weak to be considered significant, which
also occurs with most of the individual effect sizes. Likewise, some mea-
sures, such as improving school climate, have not yet been sufficiently
studied.
However, this study presents some limitations that should be
taken into account when assessing its results. On the one hand,
we have only included high-quality RCTs in the analysis, which
has led to excluding 293 potentially eligible studies (279 for not
presenting experimental designs and 14 RCTs, for not presenting
sufficient quality). Although it guarantees higher quality analysis,
this process has the disadvantage of reducing the number of studies
for each comparison, so there may be few studies for some mea-
sures. A less rigorous selection of the studies would probably have
provided more power in the statistical analysis, which in turn
would explain the heterogeneity in some variables. On the other
hand, this heterogeneity, predictable by the very nature of the de-
sign of the studies, has been reduced to some extent through the
use of statistical random effects models to estimate the effect sizes
and analyze the subgroups.
 174
Appendix A. Overview of coding of studies excluded in the meta-analysis for not providing sufficient statistical data

Author/country Objective Sample size Age Program/intervention Duration/follow-up Outcome measures Significant results

Farrell, Meyer, and To assess the impact in 6th-grade pupils of an in- 3 SS 10–15 “Responding in Peaceful 7 months/6- and –Violent behaviors. Reduction of disciplinary violations in the IG. At
White (2001) tervention program on knowledge, attitudes and N = 626 and Positive Ways 12-month –Drug consumption. post-test, the effects of the intervention were almost

J.A. Jiménez-Barbero et al. / Children and Youth Services Review 61 (2016) 165–175
USA behaviors concerning violence –IG = 305 (RIPP)” follow-ups –Attitude toward violence. significant, especially in boys.
–CG = 321 –Attitude toward non-violent methods
of solving problems.
Cross et al. (2012) To assess the efficacy of a socio-ecological inter- 29 PS “Friendly Schools 2 years/1–2, 24-, and –Psychological variables. Significant reductions observed in the GI in observed
Australia vention aimed at reducing bullying. N = 1968 8–9 Project” 36-month –Observed and experienced bullying. and experienced bullying, as well as in reports of epi-
–IG = 1046 follow-ups –Attitudes toward el bullying. sodes of harassment.
–CG = 922 –Perception of social support.
–Knowledge about bullying.
–School adaptation.
Espelage, Low, To assess the impact of a program on reducing 36 MS 11–13 “Second Step: Student First year of 3. –Verbal/relational bullying Multilevel analyses revealed significant intervention
Polanin, and youth violence N = 3616 Success Through perpetration. effects with regard to physical
Brown (2013) among middle school 6th-grade students. –IG = 1940 Prevention –Peer victimization. aggression.
USA –CG = 1676 (SS-SSTP)” –Physical aggression. No significant intervention effects for verbal/relational
–Homophobic name-calling perpetra- bully
tion and victimization. perpetration, peer victimization, homophobic teasing,
–Sexual harassment/violence perpetra- and sexual violence.
tion and victimization.
To assess the effects of a school-based 14 Schools, 7–13 “Positive Action 6 years –Normative beliefs supporting Positive Action mitigated increases over time in
Lewis et al. (2013) socio-emotional learning and health Grades 3 to 8. Program” aggression. (1) youth reports of normative beliefs supporting
USA promotion program on problem behaviors and re- N = 1170 –Bullying. aggressive behaviors and of engaging in disruptive
lated attitudes among low-income urban youth. –Disruptive behaviors. behavior and bullying (girls only); and (2) parent
–Violence. reports of youth bullying behaviors (boys only). At
study end-point, students in Positive Action schools
also reported a lower rate of violence-related behavior
than students in control schools.
Espelage, Low, To assess the impact of the SS-SSTP program in the 36 MS 11–13 “Second Step: Student Second year of –Verbal/relational bullying Multilevel analyses revealed significant intervention
Polanin, and second year of follow-up. N = 3658 Success Through follow-up perpetration. effects for two of the seven outcomes. Students in
Brown (2015) –IG = 1961 Prevention –Peer victimization. intervention schools were 56% less likely to self-report
USA –CG = 1697 (SS-SSTP)” –Physical aggression. homophobic name-calling victimization, and 39% less
–Homophobic name-calling perpetra- likely to report sexual violence perpetration than stu-
tion and victimization. dents in control schools in one state. SS-SSTP holds
–Sexual harassment/violence perpetra- promise as an efficacious program to reduce homo-
tion and victimization. phobic name-calling and sexual violence in adolescent
youth.
 J.A. Jiménez-Barbero et al. / Children and Youth Services Review 61 (2016) 165–175
References
Allen, S. F. (2009). A study of a violence prevention program in prekindergarten class-
rooms. Children and Schools, 31, 177–187. http://dx.doi.org/10.1093/cs/31.3.177.
Baldry, A. C., & Farrington, D. P. (2004). Evaluation of an intervention program for the
reduction of bullying and victimization in schools. Agressive Behavior, 30, 1–15.
Beets, M. W., Flay, B. R., Vuchinich, S., Snyder, F. J., Acock, A., Li, K., et al. (2009). Use of
a social and character development program to prevent substance use, violent behav-
iors, and sexual activity among elementary-school students in Hawaii. American
Journal of Public Health, 99, 1438–1445. http://dx.doi.org/10.2105/AJPH.2008.142919.
Benbenishty, R., & Astor, R. A. (2005). School violence in context: Culture, neighborhood,
family, school, and gender. New York: Oxford University Press.
Berry, K., & Hunt, C. J. (2009). Evaluation of an intervention program for anxious
adolescent boys who are bullied at school. Journal of Adolescent Health, 45, 376–382.
http://dx.doi.org/10.1016/j.jadohealth.2009.04.023.
Brown, E. C., Low, S., Smith, B. H., & Haggerty, K. P. (2011). Outcomes from a school-
randomized controlled trial of steps to respect: A bullying prevention program.
School Psychology Review, 40, 423–443.
Burton, C. (2012). Heavy tailed distributions of effect sizes in systematic reviews of com-
plex interventions. PloS One, 7, 1–7. http://dx.doi.org/10.1371/journal.pone.0034222.
Cross, D., Waters, S., Pearce, N., Shaw, T., Hall, M., Erceg, E., et al. (2012). The Friendly
Schools Friendly Families programme: Three-year bullying behaviour outcomes in
primary school children. International Journal of Educational Research, 53, 394–406.
http://dx.doi.org/10.1016/j.ijer.2012.05.004.
Currie, C., Gabhainn, S. N., Godeau, E., Roberts, C., Smith, R., Currie, D., ... Barnekow, V.
(2008). Inequalities in young people's health. Health behavior in school-aged children:
International report from the 2005–2006 survey (Rep. No. 5). Edinburgh: Health Policy
for Children and Adolescents.
Defensor del Pueblo (2007). Violencia escolar: el maltrato entre iguales en la Educación
Secundaria Obligatoria 1999–2006 (Nuevo estudio y actualización del informe 2000
[[School violence: Peer maltreatment in Compulsory Secondary Education 1999–
2006 (new study and update of 2000 report)])]. (Rep. No. 22). Madrid: Publicaciones
de la Oficina del Defensor del Pueblo.
Department of Health and Human Services & Center for Disease Control and Prevention
(2006s). Youth risk behavior surveillance-United States. Morbidity and Mortality
Weekly Report, 55, 6–7.
DeRosier, M. E., & Marcus, S. R. (2005). Building friendships and combating bullying:
Effectiveness of S.S.GRIN at one-year follow-up. Journal of Clinical Child & Adolescent
Psychology, 34, 140–150. http://dx.doi.org/10.1207/s15374424jccp3401_13.
Espelage, D. L., Low, S., Polanin, J. R., & Brown, E. C. (2013). The impact of a middle school
program to reduce aggression, victimization, and sexual violence. Journal of
Adolescent Health, 53, 180–186. http://dx.doi.org/10.1016/j.jadohealth.2013.02.021.
Espelage, D. L., Low, S., Polanin, J. R., & Brown, E. C. (2015). Clinical trial of Second Step©
middle-school program: Impact on aggression & victimization. Journal of Applied
Developmental Psychology, 37, 52–63. http://dx.doi.org/10.1016/j.appdev.2014.11.
007.
Farrell, A. D., Meyer, A. L., & White, K. S. (2001). Evaluation of Responding in Peaceful
and Positive Ways (RIPP): A school-based prevention program for reducing vio-
lence among urban adolescents. Journal of Clinical Child Psychology, 30, 451–463.
http://dx.doi.org/10.1207/S15374424JCCP3004_02.
Fekkes, M., Pijpers, F. I., & Verloove-Vanhorick, S. P. (2006). Effects of antibullying school
program on bullying and health complaints. Archives of Pediatrics & Adolescent
Medicine, 160, 638–644.
Fonagy, P., Twemlow, S. W., Vernberg, E. M., Nelson, J. M., Dill, E. J., Little, T. D., et al.
(2009). A cluster randomized controlled trial of child-focused psychiatric consulta-
tion and a school systems-focused intervention to reduce aggression. The Journal of
Child Psychology & Psychiatry, 50, 607–616. http://dx.doi.org/10.1111/j.1469-7610.
2008.02025.x.
Fox, C. L., Elder, T., Gater, J., & Johnson, E. (2010). The association between adolescents'
beliefs in a just world and their attitudes to victims of bullying. British Journal of
Educational Psychology, 80, 183–198.
Frey, K. S., Hirschstein, M. K., Snell, J. L., Edstrom, L. V., MacKenzie, E. P., & Broderick, C. J.
(2005). Reducing playground bullying and supporting beliefs: An experimental
trial of the Steps to Respect Program. Developmental Psychology, 41, 479–490.
http://dx.doi.org/10.1037/0012-1649.41.3.479.
Heinemann, P. (1972). Mobbing-group violence by children and adults. Stockholm,
Sweden: Natur and Kultur.
Hunt, C. (2007). The effect of an education program on attitudes and beliefs about bully-
ing and bullying behaviour in junior secondary school students. Child and Adolescent
Mental Health, 12, 21–26. http://dx.doi.org/10.1111/j.1475-3588.2006.00417.x.
Jadad, A. R., Moore, R. A., Carroll, D., Jenkinson, C., Reynolds, D. J., Gavaghan, D. J., &
McQuay, H. J. (1996). Assessing the quality of reports of randomized clinical trials:
is blinding necessary? Controlled Clinical Trials, 17, 1–12.
Jiménez-Barbero, J. A., Ruiz-Hernández, J. A., Llor-Esteban, B., Llor-Zaragoza, L., & Pérez
García, M. (2013). Efficacy of a brief intervention on attitudes to reduce school
175
violence: A randomized clinical trial. Children and Youth Services Review, 35,
1313–1318. http://dx.doi.org/10.1016/j.childyouth.2013.05.010.
Jiménez-Barbero, J. A., Ruiz-Hernández, J. A., Llor-Esteban, B., & Pérez-García, M. (2012).
Effectiveness of antibullying school programmes: A systematic review by evidence
levels. Children and Youth Services Review, 34, 1646–1658. http://dx.doi.org/10.
1016/j.childyouth.2012.04.025.
Jiménez-Barbero, J. A., Ruiz-Hernández, J. A., Llor-Esteban, B., & Waschgler, K. (2014).
Influence of attitudes, impulsivity, and parental styles in adolescents' externalizing
behavior. Journal of Health Psychology. http://dx.doi.org/10.1177/1359105314523303.
Kärnä, A., Voeten, M., Little, T. D., Poskiparta, E., Kaljonen, A., & Salmivalli, C. (2011).
A large-scale evaluation of the KiVa antibullying program: Grades 4–6. Child
Development, 82, 311–330. http://dx.doi.org/10.1111/j.1467-8624.2010.01557.x.
Lewis, K. M., Schure, M. B., Bavarian, N., DuBois, D. L., Day, J., Ji, P., et al. (2013). Problem
behavior and urban, low-income youth: A randomized controlled trial of positive ac-
tion in Chicago. American Journal of Preventive Medicine, 44, 622–630. http://dx.doi.
org/10.1016/j.amepre.2013.01.030.
Merrel, K. W., Gueldner, B. A., Ross, S. W., & Isava, D. M. (2008). How effective are school
bullying intervention programs? A meta-analysis of intervention research. School
Psychology Quarterly, 23, 26–42. http://dx.doi.org/10.1037/1045-3830.23.1.26.
Mytton, J., DiGuiseppi, C., Gough, D., Taylor, R., & Logan, S. (2006). School-based secondary
prevention programmes for preventing violence. Cochrane Database of Systematic
Reviews, 19. http://dx.doi.org/10.1002/14651858.CD004606.pub2.
NHS Centre for Reviews and Dissemination (2001). Undertaking systematic reviews of re-
search on effectiveness. CRD's guidance for those carrying out or commissioning reviews
(rep. No. 4). York, UK: University of York.
Olalla-Cutrín, O., Gómez-Fraguela, J. A., & Luengo, M. A. (2015). Peer-group mediation
in the relationship between family and juvenile antisocial behavior. The European
Journal of Psychology Applied to Legal Context, 7, 59–65. http://dx.doi.org/10.1016/j.
ejpal.2014.11.005.
Olweus, D. (1978). Aggression in the school: Bullies and whipping boys. Washington, DC:
Hemisphere Press (Wiley).
Olweus, D. (1993). Bullying at school: What we know and what we can do. Cambridge, MA:
Blackwell.
Olweus, D. (1996). Bullying at school: Knowledge base and an effective intervention
program. Annals of the New York Academy of Sciences, 794, 265–276.
Olweus, D. (2005). A useful evaluation design, and effects of the Olweus Bullying
Prevention Program. Psychology, Crime & Law, 11, 389–402. http://dx.doi.org/10.
1080/10683160500255471.
Park-Higgerson, H. K., Perumean-Chaney, S. E., Bartolucci, A. A., Grimley, D. M., & Singh, A. P.
(2008). The evaluation of school-based violence prevention programs: A meta-
analysis. The Journal of School Health, 78, 465–479. http://dx.doi.org/10.1111/j.1746-
1561.2008.00332.x.
Rosa-Alcázar, A. I., Sánchez-Meca, J., Gómez-Conesa, A., & Marín-Martínez, F. (2008). Psy-
chological treatment of obsessive–compulsive disorder: A meta-analysis. Clinical
Psychology Review, 28, 1310–1325. http://dx.doi.org/10.1016/j.cpr.2008.07.001.
Rudolph, K. D., Lansford, J. E., Agoston, A. M., Sugimura, N., Schwartz, D., Dodge, K. A., ...
Bates, J. E. (2013). Peer victimization and social alienation: Predicting deviant peer
affiliation in middle school. Child Development, 85, 124–139. http://dx.doi.org/10.
1111/cdev.12112.
Ruiz-Hernández, J. A., García-Jiménez, J., Llor-Esteban, B., & Godoy-Fernández, C. (2015).
Risk factors for intimate partner violence in prison inmates. The European Journal of
Psychology Applied to Legal Context, 7, 41–49. http://dx.doi.org/10.1016/j.ejpal.2014.
11.003.
Stevens, V., Van Oost, P., & De Bourdeaudhuij, I. (2000). The effects of an anti-bullying in-
tervention programme on peers' attitudes and behaviour. Journal of Adolescence, 23,
21–34. http://dx.doi.org/10.1006/jado.1999.0296.
Sutton, A. J., Duval, S. J., Tweedie, R. L., Abrams, K. R., & Jones, D. R. (2000). Empirical as-
sessment of effect of publication bias on meta-analyses. British Medical Journal, 320,
1574–1577.
Teglasi, H., & Rothman, L. (2001). STORIES: A classroom-based program to reduce
aggressive behavior. Journal of School Psychology, 39, 71–94. http://dx.doi.org/10.
1016/S0022-4405(00)00060-1.
Ttofi, M. M., & Farrington, F. D. (2011). Effectiveness of school-based programs to reduce
bullying: A systematic and meta-analytic review. Journal of Experimental Criminology,
7, 27–56. http://dx.doi.org/10.1007/s11292-010-9109-1.
Vreeman, R. C., & Carroll, A. E. (2007). A systematic review of school-based inter-
ventions to prevent bullying. Archives of Pediatrics & Adolescent Medicine, 161,
78–88. http://dx.doi.org/10.1001/archpedi.161.1.78.
Waasdorp, T. E., Bradshaw, C. P., & Leaf, P. J. (2012). The impact of schoolwide positive
behavioral interventions and supports on bullying and peer rejection: A randomized
controlled effectiveness trial. Archives of Pediatrics & Adolescent Medicine, 166,
149–156. http://dx.doi.org/10.1001/archpediatrics.2011.755.
Walker, H. M. (1995). The acting-out child: Coping with classroom disruption (2nd ed.).
Longmont, CO: Sopris West.