See discussions, stats, and author profiles for this publication at: https://www.researchgate.

net/publication/303886869

Disentangling functions of online aggression: The Cyber-Aggression Typology

Questionnaire (CATQ)

Article in Aggressive Behavior · June 2016

DOI: 10.1002/ab.21663

CITATIONS READS

60 6,704

3 authors, including:

Kevin Runions
Telethon Kids Institute
80 PUBLICATIONS 3,102 CITATIONS

SEE PROFILE

All content following this page was uploaded by Kevin Runions on 03 April 2018.

The user has requested enhancement of the downloaded file.

 AGGRESSIVE BEHAVIOR
Volume 9999, pages 1–11 (2016)

Disentangling Functions of Online Aggression: The

Cyber-Aggression Typology Questionnaire (CATQ)

re
Kevin C. Runions1*, Michal Bak2, and The se Shaw1
1
Telethon Kids Institute, Faculty of Medicine, Telethon Kids Institute, Dentistry and Health Sciences, University of Western
Australia, Perth, Western Australia, Australia
2
University of Western Ontario, London, Ontario, Canada
.. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. ..
Aggression in online contexts has received much attention over the last decade, yet there is a need for measures identifying the
proximal psychological drivers of cyber-aggressive behavior. The purpose of this study was to present data on the newly developed
Cyber-Aggression Typology Questionnaire (CATQ) designed to distinguish between four distinct types of cyber-aggression on
dimensions of motivational valence and self-control. A sample 314 undergraduate students participated in the study. The results
confirmed the predicted four-factor structure providing evidence for distinct and independent impulsive-aversive, controlled-
aversive, impulsive-appetitive, and controlled-appetitive cyber-aggression types. Further analyses with the Berlin Cyberbullying
Questionnaire, Reactive Proactive Aggression Questionnaire, and the Behavior Inhibition and Activation Systems Scale provide
support for convergent and divergent validity. Understanding the motivations facilitating cyber-aggressive behavior could aid
researchers in the development of new prevention and intervention strategies that focus on individual differences in maladaptive
proximal drivers of aggression. Aggr. Behav. 9999:1–11, 2016. © 2016 Wiley Periodicals, Inc.
.. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. ..

Keywords: cyber-aggression; reactive aggression; proactive aggression; motivation; scale development

INTRODUCTION been applied to cyber-aggression. To address this gap,

As a new field of study, cyberbullying research
provides an important foundation for intervention and
prevention development. Heterogeneity in why online
aggression (commonly referred to as cyberbullying)
arises has the potential to obstruct intervention efforts.
For example, prevention of deliberate, goal-driven
cyberbullying may fail to deter those who are aggressing
in response to provocation; prevention of revenge-
driven cyber-aggression may not reach someone
spontaneously “trolling,” just for fun. But little research
attention has been paid to diversity in the proximal,
functional, drivers of cyber-aggression. As with aggres-
sion more broadly, cyber-aggression may arise in
different ways, reflecting different functions. However,
instrumentation developed to measure cyberbullying
and cyber-aggression has failed to capture heterogeneity
in why cyber-aggression arises, instead focusing on the
range of modes or manifestations (e.g., flaming, cyber-
stalking, masquerading; Willard, 2007). In aggression
research, this would be considered a focus on the form at
the expense of the function. Indeed, even basic
distinctions in human aggression functions—between
proactive and reactive aggression—have only recently
our study tested the validity and reliability of a newly
developed measure derived from the quadripartite
typology proposed for general violent aggression by
Howard (2011). This measure aims to assess heteroge-
neity in the function of cyber-aggression, and thereby
provide more targeted research to guide intervention
development.
WHAT DRIVES CYBER-AGGRESSION?
Relatively little research has examined motivations
for cyberbullying or cyber-aggression more broadly.
Raskauskas and Stolz (2007) examined open-ended
responses about why the adolescents’ surveyed thought


Correspondence to: Kevin Runions, Faculty of Medicine, Telethon Kids
Institute, Dentistry and Health Sciences, University of Western
Australia, Perth, P.O. Box 855, Western Australia 6872, Australia.
E-mail: kevin.runions@telethonkids.org.au
Received 25 September 2015; Revised 28 April 2016; Accepted 9 May
2016
DOI: 10.1002/ab.21663
Published online XX Month Year in Wiley Online Library
(wileyonlinelibrary.com).

© 2016 Wiley Periodicals, Inc.

2 Runions et al.
young people engaged in cyberbullying. The most
common responses were for recreational purposes
and for revenge. Recently, Compton, Campbell, and
Mergler (2014) interviewed students, parents, and
teachers about their perceptions of motivations for
cyberbullying, after being provided with a definition of
bullying. Amongst the key themes for cyberbullying
was the pursuit of power and status, fun-seeking or relief
of boredom, and use of cyber-aggression as a safe means
of retaliating. A qualitative study of college students
focused on the use of online aggression to achieve social
control, to obtain valued social resources, and for
entertainment, as exemplified by trolling (Rafferty &
Vander Ven, 2014). Consistent to these few studies, and
much less commonly noted in research in offline
aggression, are the notions of fun-seeking, revenge
and retaliation.
Revenge and fun-seeking motivations for aggression
are notable in their poor fit with the reactive-proactive
aggression paradigm that dominates developmental
psychological research on the function of aggression
(Dodge & Coie, 1987; Dodge, 1991). Reactive aggres-
sion is conceived as an impulsive aggressive response in
the context of retaliation for some (perceived or real)
insult or attack. Unlike revenge, reactive aggression is
commonly conceived as non-deliberated. Proactive
aggression is generally conceived as unprovoked acts
of aggression enacted with deliberation. In distinction
from the fun-seeking discussed in research on cyber-
aggression motives, proactive aggression is deliberated,
not impulsive.
Following Dooley, Pyzalski, and Cross (2009), who
first explicitly discussed cyberbullying in terms of
reactive and proactive aggression, a small body of
pioneering research has examined cyber-aggression
through the reactive-proactive lens. Calvete, Orue,
Est
evez, Villard
on, and Padilla (2010) published the
first empirical investigation of proactive and reactive
functions in cyber-aggression. Using a measure of
cyberbullying and the Reactive-Proactive Aggression
Questionnaire (RPQ; Raine et al., 2006), they found
that only self-reported proactive aggression predicted
cyber-aggression when controlling for reactive
aggression in a sample of 12–17-year-old Spanish
adolescents. In a similar study with American and
Singaporean youth, proactive aggression was uniquely
associated with cyberbullying, after partialling out the
influence of reactive aggression (Ang, Huan, & Florell,
2014). These studies replicate findings for face-to-face
bullying (Fontaine, 2007; Salmivalli, Lagerspetz,
Bj€ €
orkqvist, Osterman, & Kaukiainen, 1996), which
could lead to the conclusion that cyberbullying is an
expression of proactive aggression, regardless of how
it is measured.
Aggr. Behav.
More sensitive measures directly examining the
function of cyber-aggressive behavior, and not simply
offline reactive or proactive aggression, indicate a more
complex picture. In Canada, Law and her colleagues
(Law, Shapka, Domene, & Gagne, 2012) modified the
RPQ to reflect online aggression, but found no factor
analytic support for a proactive-reactive distinction. A
single-factor solution was found, leading Law et al.
(2012) to conclude that “participants did not differenti-
ate between proactive and reactive reasons for being
aggressive online” (p. 667). It is possible, however, that
this reflects Type 2 error, and the modified instrument
failed to capture diversity in reasons for cyber-
aggression. In a later study using the same modified
online RPQ measure (Shapka & Law, 2013), two-factors
were found, and reactive motives were stronger
predictors of general cyber-aggression than were
proactive motivations, contrasting with the finding of
Calvete et al. (2010).
Studying the motives behind online aggression can be
valuable for intervening and preventing acts of aggres-
sion and bullying. The pioneering research on reactive
and proactive functions of aggression, however, has left
an ambiguous picture that does not lend itself to clear
application in prevention or intervention efforts. We
suggest that a re-conceptualization of the motives or
functions of aggression in general is required to make
further progress in this area. Specifically, we argue that
the reactive-proactive distinction bears a core concep-
tual confound that hinders its further utility, including its
utility in understanding cyber-aggression.
A QUADRIPARTITE MODEL OF THE FUNCTION
OF CYBER-AGGRESSION
Howard (2011) has argued that the reactive-proactive
distinction is limited by its confounding of regulatory
processes and motivational goals. Certainly many—
perhaps even most—aggressive retaliatory responses are
enacted impulsively, and many unprovoked aggressive
acts may be committed after deliberation. But deliber-
ated aggressive acts may also be taken in retaliation to
provocation—that is to say, revenge as commonly
conceived. Similarly, impulsive acts of aggression that
are motivated by appetitive motivational goals, such as
the immediate elicitation of positive affect (or “fun”), are
poorly captured in the traditional reactive-proactive
distinction. Traditional measures of reactive and proac-
tive aggression conflate these functional drivers by
assuming all retaliatory aggression is impulsively
conducted (and thus subsumed under the reactive
aggression banner), and that all appetitive aggression
is deliberately conducted (and thus subsumed under the
proactive aggression category).

As a response to this conceptual and methodological
limitation, Howard (2011) has proposed a quadripartite
model of aggression based on two orthogonal dimen-
sions—motivational goals (appetitive vs. aversive) and
regulatory control (impulsive vs. deliberated). As
Howard (2009, 2011) has reviewed, impulsive acts
arise in the context of minimal or subconscious cognitive
appraisal of an environmental cue prompting a strong
affective response and a strong action tendency or urge
to act (Baumeister, Vohs, DeWall, & Zhang, 2007). In
the production of impulsive behavior, that action
impulse is acted upon without heed of consequences,
especially in the long-term. Rather, the behavior is
engaged in for the immediate emotional pay-off. In
contrast, deliberated acts of aggression are conducted via
the recruitment of self-control capacities in service of
some goal.
The second dimension in the quadripartite model
reflects the nature of this emotional pay-off. Appetitive
motives are aimed at obtaining a positive emotional state
for the aggressor. Aversive motives are aimed at the
removal of negative emotional states, including embar-
rassment, shame or guilt. We believe this includes the
painful experiences of having been victimized or
harassed by others (or perceiving victimization). In
this sense, aversive motives may often be seen as
retaliatory, as they may often arise in the context of
provocation.
In postulating two orthogonal dimensions (appetitive
vs. aversive; impulsive vs. controlled), this motive and
self-control model produces four distinct quadrants (see
Fig. 1). The impulsive-aversive quadrant reflects acts,
often in reaction to perceived provocation, shame
elicitation, and other aversive emotional states, that
are spontaneously produced or arise with little or no
Fig. 1. Quadripartite typology of aggression, from Howard’s (2010)
Quadripartite violence typology, distinguishing motivational valence
of aggression and recruitment of self-control capacities.
Cyber-Aggression Typology Questionnaire 3
recruitment of capacities for self-control. This quadrant
maps most clearly to traditional conceptualization and
measurement of reactive aggression. The controlled-
aversive quadrant, by contrast, also arises as a means to
exorcise uncomfortable emotions (e.g., shame, embar-
rassment) arising from perceived provocation. But here,
capacities for self-control are exercised, and the
production of aggression is more likely to take the
form of calculated payback, or revenge. This distin-
guishes it from controlled-appetitive aggression, where
self-control is also exerted, but in the interest of
generating positive affect via deliberated efforts. This
quadrant most closely maps onto traditional conceptu-
alizations of “proactive” aggression, with its focus on
effortful control in the pursuit of some environmental
reward, such as attaining social status through bullying
(e.g., Vaillancourt, Hymel, & McDougall, 2003). The
impulsive-appetitive quadrant reflects spontaneously
produced behavior, conducted without heed of long-
term consequences, arising from the immediate positive
affect generated by the aggressive act. These impulsive-
appetitive motives are commonly noted in the self-
generated accounts of violent offenders (Howard, 2011).
Bjørnebekk and Howard (2012) developed a measure
(the Angry Aggression Scales, AAS) assessing the four
types of aggression suggested by this typology,
specifically for use with youth who have a diagnosed
conduct disorder. Using a sample of 101 young people
with serious conduct problems and 101 non-aggressive
control youth, their study found evidence supporting the
four-factor solution mapping on to the four types of
aggression.
To assess how this quadripartite typology mapped onto
the traditional reactive-proactive distinction, Bjørnebekk
and Howard (2012) correlated factor scores for each
quadrant and a teacher-rating of reactive and proactive
aggression amongst the deviant youth group (Roland &
Idsøe, 2001). As expected, the impulsive-aversive scale
was significantly correlated with reactive aggression
scores. But the controlled-aversive scale was also
significantly associated with reactive aggression. The
appetitive subscales were not significantly correlated with
reactive aggression, providing evidence of convergent
and divergent validity of scores on the AAS. Teacher-
rated proactive aggression was correlated with both
impulsive and controlled appetitive aggression, as might
be expected, but also with impulsive-aversive aggression.
In the same study, Bjørnebekk and Howard (2012)
also examined correlations with self-rated behavioral
inhibition and behavioral activation systems (BIS/BAS).
This distinction between behavioral inhibition and
activation systems was proposed by Gray (1987) to
account for hypothesized neural systems involved in
regulating attention to novel environmental cues
Aggr. Behav.
4 Runions et al.
potentially signaling danger and/or punishment (BIS),
and neural systems regulating approach behavior to
rewarding behavior (BAS). Carver and White’s (1994)
operationalization of these systems involves three sub-
scales for BAS, namely Fun Seeking, Drive, and Reward
responsiveness. BAS Fun Seeking includes items
emphasizing the pursuit of potentially rewarding
conditions in a spontaneous manner. The BAS Drive
subscale measures a willingness to pursue desired goals
in a persistent manner. The BAS Reward Responsive-
ness subscale focuses on positive affective responses to
realized or anticipated reward. Bjørnebekk and Howard
(2012) found that the impulsive-appetitive subscale was
positively correlated only with BAS Fun Seeking.
Contrary to expectations, impulsive-aversive aggression
also correlated with Fun Seeking, but also showed a
significant positive correlation with BIS or behavioral
inhibition. The two controlled types of aggression were
both positively correlated with BAS Drive. Controlled-
aversive aggression was also correlated with behavioral
inhibition, whereas controlled-appetitive aggression was
correlated with BAS Reward Responsiveness. This
network of associations provides evidence of convergent
and divergent validity for the quadripartite model.
Following Howard, Runions (2013) examined the
relevance for a quadripartite model of cyber-aggression
and cyberbullying and presented novel hypotheses about
the co-action of personal characteristics and aspects of
the online context as drivers of distinct types of cyber-
aggression. Those hypotheses, however, presume that
the quadripartite model is relevant in an online context,
but no measures have yet been developed to examine
these distinct types of cyber-aggression.
THE PRESENT STUDY
Currently, research on cyber-aggression is without
tools for measuring diverse functions of cyber-aggres-
sion and cyberbullying. As such, our understanding of
why cyber-aggression arises is constrained, placing
limits on intervention planning and evaluation. The aim
of the present study was to present data on a new Cyber-
Aggression Typology Questionnaire (CATQ). This
conceptually derived scale was designed to capture
Aversive versus Appetitive motives and Impulsive
versus Controlled modes of expressing those motives.
Our goal was to present evidence for the predicted four-
factor structure, and test for convergent and divergent
validity via the Reactive-Proactive Aggression Ques-
tionnaire (RPQ; Raine et al., 2006), a measure of
cybervictimization, and the BIS/BAS Scales (Carver &
White, 1994). We were also interested in examining how
the four types of cyber-aggression were related to
reported experiences of cyber-victimization.
Aggr. Behav.
As evidence of convergent validity, we first predicted
that impulsive-aversive cyber-aggression would be
associated with established measures of reactive aggres-
sion as reactive aggression is traditionally operationalized
as aggression in response to aversive stimuli and
conducted in an uncontrolled manner. We also expected
it to be associated a relative lack of behavioral inhibition.
Second, as proactive aggression is traditionally
operationalized as a premeditated form of aggression
without provocation, we predicted that controlled-
appetitive cyber-aggression would be most strongly
correlated with traditional measures of proactive
aggression generally. Due to its calculated, deliberate,
reward-seeking nature, we also predicted it would be
associated with the BAS Drive and BAS Reward-
Responsiveness scales.
Our third hypothesis posited that as provocation can be
responded to via planned retaliatory actions, controlled-
aversive cyber-aggression was likely to be associated
with both proactive and reactive aggression as tradition-
ally operationalized. We also expected it to be associated
with victimization experiences and as controlled actions
with BAS Drive scores.
Finally, as aggression may take place without advance
planning and in the service of immediate positive
affective consequences for the aggressor, our fourth
hypothesis predicted that impulsive-appetitive cyber-
aggression would be associated with both proactive (due
to its reward-driven nature) and reactive (due to its
impulsive nature) aggression. We also predicted that it
would be associated with fun seeking behavior, and not
related to victimization experiences.
METHODS
Participants
University students at a small Canadian university
were invited to participate in the study in an e-mail,
which also provided them with a link to access the web
survey. A wide range of the university’s departments
were involved in the distribution of study invitation e-
mails. These e-mail invitations were sent to an
undisclosed total number of students enrolled in first
year courses through the university departments and
faculties. As a result, we are unable to present response
rates for this convenience sample. In total, 314 students
(63.4% women) participated. The final sample com-
prised students from a range of disciplines, including arts
(11%), science and mathematics (12.2%), engineering
and computer sciences (23.2%), business and commerce
(13.1%), psychology (17.1%), other social sciences
(20.4%), with a small set still undecided (3.0%). Almost
half were 18 years of age (46.5%, n ¼ 146), with 29.9%
(n ¼ 94) 19 year olds, 7.0% (n ¼ 22) 17 year olds, and the

remaining 17.5% (n ¼ 55) were 20 years or over. We did
not collect data on ethnicity or socio-economic status. As
to their experience with online communication, 97.3%
indicated that they were users of social networks, 82.4%
reported they have been victims of some online harm at
some point in their past, and 73.8% indicated that they
had harmed someone else online, even if accidental.
Measures
Cyber-agression and cyber-victimization.
The Berlin Cyberbullying-Cybervictimisation Question-
naire (BCyQ; Schultze-Krumbholz & Scheithauer,
2009) is a 17-item index assessing distinct categories
developed in light of Willard’s (2007) proposed seven
categories of cyber-aggressive behavior. Cronbach’s
alpha for the cyber-aggression subscale in this study
sample was .87, and for the cyber-victimization scale
was .89.
Reactive and proactive aggression. The 23
item Reactive-Proactive Questionnaire (Raine et al.,
2006) consists of 11 items assessing reactive aggression
(e.g., “reacted angrily when provoked by others”) and 12
items assessing proactive aggression (e.g., “used force to
obtain money or things from others”). Its psychometric
properties have been well established in the original and
translated versions (e.g., Cima, Raine, Meesters, &
Popma, 2013). In the current study, both reactive and
proactive scales showed acceptable internal consisten-
cies, a ¼ .77 and .76, respectively.
Behavioral inhibition and behavioral activa-
tion. The 20-item BIS/BAS scales (Carver & White,
1994) were used to provide an operationalization of
Gray’s (1987) BIS/BAS theory. BIS items reflect
contexts that contain signals of potential punishment
(8 items; e.g., “if I think something unpleasant is going to
happen I usually get pretty “worked up”). Behavioral
Activation is assessed via three subscales: BAS Reward-
Responsiveness (five items; e.g., “When I get something
I want, I feel excited and energized), BAS Drive (four
items; e.g., “When I want something, I usually go all-out
to get it”), and BAS Fun-Seeking (four items; e.g., “I will
often do things for no other reason but that they might be
fun”). All items are rated on a 4-point Likert scale
(1 ¼ not at all true of me; 4 ¼ very true of me). Validity
and reliability have been well-established for these
scales (Carver & White, 1994). For our sample, internal
consistency was good, with a Cronbach alpha coefficient
of .81, .72, .77 for BIS, Reward-Responsiveness, Drive,
respectively. Alpha for Fun-Seeking was suboptimal
(a ¼ .65). This did not appear to be resulting from any
single item, and all items were retained for analyses.
Given the importance of this construct for our hypothe-
ses, however, we have used it in our study, and advise
caution in interpreting relationships with this construct.
Cyber-Aggression Typology Questionnaire 5
Procedure
Items for the Cyber-Aggression Typology Question-
naire (CATQ) were generated by adapting items from
the Angry Aggression Scale (Bjørnebekk & Howard,
2012) and applying them to the online context, and by
generation of items inspired by the conceptual frame-
work proposed by Runions (2013). As noted above,
recruitment was conducted via email distributed through
university departments, with a link to the secure online
survey service FluidSurvey. The notices indicated that
the study was testing a new instrument about internet and
social network use. There were no restrictions or
exclusion criteria deployed. Where possible, research
assistants addressed students in key courses prior to
lectures to encourage participation. A script was
prepared for these addresses. Participants were entered
into a draw for one of three $100 iTunes vouchers. All
aspects of the study were approved by the university’s
institutional human ethics review board.
A formative version of the scale was developed and
piloted with a sample of 37 university undergraduate
students, aged 18–19. In this version, impulsive-
appetitive, controlled-appetitive, impulsive-aversive,
and controlled-aversive forms of cyber-aggression
were assessed with 21, 13, 17, and 13 items,
respectively. With this preliminary scale data, we
conducted two exploratory principal components anal-
yses with varimax and equimax rotations, on the items
that were predicted to form distinct subscales. In light of
the very small sample size for PCA analyses, we adopted
a very conservative cutoff of .70 for factor loadings, and
retained only those items that loaded above that
threshold on a factor and below .40 on all other factors
identified when subjected to both varimax and equimax
rotation. Only those items that met this criteria were used
in a second phase, described below.
The retained items were then presented via online
survey to a new sample of participants, along with the
other measures and items assessing basic demographics.
(Following collection of demographic data, we counter-
balanced the order of the CATQ scale, the BIS/BAS
scales, and the Reactive-Proactive Questionnaire.) These
items were then examined via Confirmatory Factor
Analysis in MPlus 6 to test construct validity. We
compared our four-factor model against two competing
two-factor models reflecting controlled versus impulsive
cyber-aggression and appetitive versus aversive cyber-
aggression. Scores on the four subscales were calculated
as mean values of the final items. Gender invariance was
assessed for these categorical items through simultaneous
testing of the invariance of the factor loadings and
thresholds across gender groups as recommended
(Muthen & Muthen, 1998–2009). The Satorra–Bentler
Aggr. Behav.
6 Runions et al.
scaled chi-square difference test was utilized to compare
the fit of the models assuming and not assuming
invariance (Muthen & Muthen, 1998–2009, 2012). These
residualized mean subscale scores were examined in
relation to the other study variables to assess convergent
and divergent validity, as described in the hypothesis.
RESULTS
Testing of CATQ Construct Validity
The development of the CATQ aimed to produce a
measure of cyber-aggression that was sensitive to not
only a reactive-proactive distinction in function of
aggression, but a conceptually clearer distinction
between cyber-aggression driven by appetitive versus
aversive motives, wherein self-control is or is not
recruited. As such, construct validity was assessed via a
four-factor confirmatory factor analysis.
CFA was conducted based on the four motivationally
distinct categories of cyber-aggression, which included
12 impulsive-aversive items, six controlled-aversive
items, six controlled-appetitive items, and five impul-
sive-appetitive items. As with most measures assessing
aggression in a non-clinical, non-antisocial population,
the distribution of the ordinal cyber-aggression items
showed substantial positive skew. Given this, we used
the weighted least squares mean variance estimator
(WLSMV), recommended for ordinal skewed data
(Muth
en & Muth
en, 1998–2009). Fit statistics values
for adequate model fit were RMSEA <.06, and TLI and
CFI both >.95 (Hu & Bentler, 2009), as well as the
weighted root mean residual (WRMR) <.90 (Schreiber,
Nora, Stage, Barlow, & King, 2006). We also examined
the standardized model results for each item to ensure
factor loadings loaded highly on the scale. All items
showed loadings well above our criterion of .4 or above
(see Table I). We examined the modification indices, and
as none were substantively larger than the rest (all <.20),
there was little evidence to drop items or modify the
model. All fit indices indicated good fit for the four-
factor model, and the subscale factor loading values
were in line with the predicted model. As would be
expected, the four factors were inter-correlated
(Table II.). The two types of aversive cyber-aggression
were correlated at .91, and the two types of appetitive
cyber-aggression at .82. Both impulsive forms of cyber-
aggression were correlated at .66, and controlled forms
of cyber-aggression at .89. Finally, the cross-diagonals
show that impulsive-aversive was correlated with
controlled-appetitive at .83, and impulsive-appetitive
was correlated with controlled-aversive at .66.
For a more stringent test of construct validity, we also
compared the four-factor model with two competing
two-factor model interpretations. The first combined all
Aggr. Behav.
retaliatory items and appetitive items. The second
comparator model combined all impulsive and con-
trolled items. Neither two-factor models fit the data
adequately, and the four-factor model showed best fit
across both the chi-square and the approximate fit
indices (see Table III) and significantly improved fit.
Difference testing via MPlus indicated that the four-
factor model showed significantly better fit than both the
appetitive versus aversive two-factor model (x2 ¼ 91.7,
df ¼ 5, P < .001), and the impulsive versus controlled
two-factor model, (x2 ¼ 119.7, df ¼ 5, P < .001). This
added evidence for a four-factor structure to the data.
The factor structure was invariant across gender groups,
Satorra–Bentler scaled chi-square difference test TRd
¼ 43.7, df ¼ 50, P ¼ .722.
Table II reports mean values and standard deviations
for the CATQ variables. Notable positive skew was
observed for all subscales (range 1.29–2.22), and inverse
transformations (Tabachnick & Fidell, 2013) were
conducted for each, which reduced skew values (range
0.44–1.03). To control for the shared variation between
the subscale scores, standardized residual scores were
calculated for each subscale by regressing the target
subscale (inverse-transformed) scores onto the other
three subscales (also inverse transformed).
Convergent and Divergent Validity
To examine convergent and divergent validity we
examined correlations of the standardized residual
CATQ scores with the BCyQ (perpetration and
victimization), the RPQ, and the BIS/BAS scales.
Examination of the correlations between the CATQ
subscales and our covariates provide support for
convergent and divergent validity. To further investigate
these forms of validity, we examined the residualized
score associations with cyber-victimization as measured
by the BCyQ, although we present raw score associa-
tions as well (see Table IV). Although we did not
hypothesize the magnitude and directions of associa-
tions between the CATQ subscales and cyberbullying
(as measured by the BCyQ), we were interested to
investigate these further. Raw correlations between
CATQ subscales and cyberbullying as indexed by the
BCyQ ranged from .40–.60 (see Table IV).
Examination of these residualized correlations indi-
cate that Impulsive-aversive cyber-aggression was
significantly correlated with reactive aggression
(r ¼ .20, P < .001) and with cyber-victimization
(r ¼ .16, P < .05), as predicted. However, the correlation
with the BIS or behavioral inhibition scores was not
significant. Additional non-predicted, small but signifi-
cant associations with BAS Drive (r ¼ .12, P < .05) and
BAS Reward-Responsiveness (r ¼ .12, P < .05) were
noted.
 Cyber-Aggression Typology Questionnaire 7

TABLE I. Items, Item Descriptives and Item Loadings for CATQ Factor 1 (Impulsive-Aversive), Factor 2 (Controlled-Aversive),
Factor 3 (Controlled-Appetitive), and Factor 4 (Impulsive-Appetitive)
Item F1 F2 F3 F4

If someone tries to hurt me, I will use an ICT device to immediately get back at them 0.86
If I get teased or threatened, I get angry easily and strike back online right away 0.86
I use ICT to get back at someone as soon as they post a hurtful message about me 0.85
If someone makes me angry online I quickly post mean texts and messages online 0.82
If someone makes fun of me on the internet, I get frustrated and respond angrily online right away 0.81
I overreact before I have a chance to think about the consequences when someone says something mean online 0.81
If I see a message online that gets me angry, I react too quickly and then regret the way I responde 0.79
If someone tries to cyberbully me, I quickly lash back with something online 0.79
If someone says something online to hurt me, I post something back right away to get back at them 0.75
If somebody criticizes me online or in a text, I often react aggressively without thinking of the consequences 0.71
I hastily respond to something written online and regret it later 0.67
I respond very quickly to a message or post that is disrespectful to me 0.57
If someone does something to hurt me, I would get back at them in my own time by using my ICT device(s) 0.90
If someone tries to hurt me, I will use my ICT device(s) to get back at them in my own time 0.89
I get back at people who make fun of me on the internet because their posts hurt more the more I think about them 0.84
I like using my ICT device(s) to plan my revenge when I feel angry at someone 0.76
If I need to get revenge on someone, I would rather strike back using my ICT device(s) where I can plan out how to do it 0.72
If I see a mean message about me on my ICT, it bothers me more and more when I think about it, and I try to get even 0.62
If I don’t like someone, I use the internet to turn others against them 0.92
Sometimes I’ll team up with my friends to bring someone down online 0.84
Sometimes I can be mean to people online to get what I want 0.82
When I don’t like a person, I use the internet to make them feel like they do not belong in my group 0.79
I pretend to be someone else online to ruin somebody else’s friendships 0.76
I have at times used the internet to make someone look like bad 0.75
I get carried away having fun online and others think I’m being a cyberbully or a troll 0.90
I make fun of people I don’t know on the internet without thinking about whether they will see it or not 0.83
If I’m having fun and joking online, I don’t care if someone’s feelings get hurt 0.81
I repeatedly annoy people online because I think it’s funny 0.77
Joking online is so much fun that I don’t worry about whether someone might be bothered by what I say 0.69

Controlled-appetitive cyber-aggression correlated sig-
nificantly with proactive aggression (r ¼ .18, P < .01)
and BAS Drive (r ¼ .14, P < .05), as predicted. A
significant negative correlation with BAS Reward-
Responsiveness is counter to the expected positive
association. This form of cyber-aggression was moder-
ately correlated with both cyberbullying (r ¼ .28,
P < .001) and cybervictimization (r ¼ .24, P < .001).
Unexpectedly, the residualized scores for controlled-
aversive cyber-aggression were not significantly
correlated with either proactive or reactive aggression
or any of the measures.
The impulsive-appetitive subscale behaved largely as
predicted. It was correlated with proactive aggression
(r ¼ .17, P < .01), BAS Fun-Seeking (r ¼ .22, P < .001),
and not with cybervictimization (r ¼ .04, ns). However,
the anticipated association with reactive aggression was
not found (r ¼ .06, ns). Also noted were a small negative
correlation with the BIS and a positive correlation with
cyberbullying (r ¼ .14, P < .05).

DISCUSSION
TABLE II. Descriptives and Correlations Between CATQ To date, cyber-aggression and cyberbullying have
Subscales as Factors and as Final Variables (n ¼ 317) predominantly been assessed with checklists and scales
Variables 1 2 3 4 M (SD) that enumerate the different forms or means of engaging
in aggressive behavior via new information and
Impulsive-aversive — 0.91 0.83 0.66 1.35 (0.48) communication technologies. The present study pro-
Controlled-aversive 0.67 — 0.89 0.66 1.34 (0.51)
Controlled-appetitive 0.66 0.63 — 0.82 1.28 (0.45)
vided a new conceptually driven direction for the
Impulsive-appetitive 0.47 0.45 0.62 — 1.46 (0.57) assessment of cyber-aggression. Based on Howard’s
(2011) quadripartite model of aggression, and Runions’
Note. Values above the diagonal reflect correlations between factors in
CFA analyses (MPlus); values below the diagonal reflect correlations
(2013) extension of that model to cyber-aggression, this
between final analysis variable (mean of items with inverse transforma- new measure aimed to disentangle the motive for
tion; SPSS). aggressing online (appetitive vs. aversive) from the

Aggr. Behav.
8 Runions et al.
TABLE III. Goodness of fit statistics for CFA models
x2
4 factor 178.58
2 factor (controlled and impulsive) 280.19
2 factor (appetitive and aversive) 231.1
CFI, comparative fit index; TLI, Tucker–Lewis index; RMSEA, root mean squared error of approximation; WRMR, weighted root mean square residual.
degree of impulsivity or self-control recruited in the
production of aggressive behavior. The present study
was the first to provide data indicating a valid factor
structure and nomological network for the CATQ.
Results of this first study involving undergraduate
university students support the validity of the CATQ.
Confirmatory factor analysis provided support for the
hypothesized factor structure, with distinct and indepen-
dent scales assessing impulsive-aversive, controlled-
aversive, controlled-appetitive, and impulsive-appetitive
subscales. Runions (2013) had proposed that cyber-
aggression may arise differentially based on motives and
capacities for self-control, and our data provide support for
the hypothesis that the four subscales of the CATQ reflect
distinct types of cyber-aggression. We precited that
aversive modes of cyber-aggression would arise in the
context and be associated with cyber-victimization. This
was the case for impulsive-aversive cyber-aggression, but
not controlled-aversive. Controlled-appetitive cyber-
aggression, however, showed the strongest correlation
with cyber-victimization, suggesting that this mode of
aggression may be part of a vicious cycle of perpetration
and (from the target) retaliation. Finally, as predicted,
impulsive-appetitive cyber-aggression was not related to
cyber-victimization.
The present results provide support, building on that of
Bjornnebekk and Howard (2012), for the hypothesis that
the reactive-proactive aggression distinction is limited
by its conflation of motive and self-control. The
quadripartite model clarifies that not all retaliatory (or
“reactive”) aggression is impulsive and not all appetitive
(or “proactive”) aggression is controlled. Aversive
aggression may be enacted impulsively or in a more
deliberated manner. When enacted impulsively, it is
conceptually most akin to reactive aggression. Appeti-
tive-aggression may be enacted in a controlled,
deliberated manner, or impulsively, in a spontaneous
pursuit of immediate thrill and fun. As predicted,
impulsive-aversive cyber-aggression was uniquely
related to reactive aggression and controlled-appetitive
cyber-aggression was uniquely related to proactive
aggression. In disentangling the conflation, we were
particularly interested in relationships of the other two
subscales to reactive and proactive aggression. Impul-
sive-appetitive aggression, was associated only with
Aggr. Behav.
df CFI TLI RMSEA WRMR
81 0.95 0.99 0.061 0.905
68 0.89 0.97 0.097 1.277
68 0.92 0.97 0.085 1.161
proactive, not reactive, aggression. The impulsivity
assessed by this subscale does not reflect uncontrolled
anger, but rather “getting carried away” and going too
far, which may diverge from the type of self-control
failure assessed by traditional reactive aggression items.
The controlled-aversive cyber-aggression was not
associated with either reactive or proactive aggression,
once the other CATQ subscales were controlled for.
Cyber-aggression based on a deliberated response to
perceived provocations does not fit neatly with
traditional conceptualizations of reactive or proactive
aggression, and consequently, the more closely aligned
CATQ subscales may dominate the shared variance
between reactive and proactive measures. This raises a
broader question of whether reactive and proactive
scales have been systematically missing out on
potentially important type of aggression motivated by
the alleviation of averse emotions, fueling wrath and
driving revenge. Frey and colleagues (Frey, Pearson, &
Cohen, 2015) have explored the importance of consid-
ering retaliation motives for bullying. We suggest that
the CATQ may provide a means to address this research
priority for cyber-aggression more broadly. Indeed,
given that the residualized controlled-aversive subscale
scores did not significantly correlate with any of the
other study variables, this mode of cyber-aggression is
especially in need of further study.
The analyses with the BIS/BAS scales provided
limited support for the CATQ and its conceptual
grounding. The impulsive appetitive mode of aggression
was positively related with BAS Fun-Seeking. This
relationship was based on the idea that for some young
people, cyber-aggression may arise out of a desire for
novel, potentially (emotionally) rewarding experiences
without thinking about the consequences.
The BAS Drive subscale correlated with controlled-
appetitive cyber-aggression, as expected in reflecting the
“persistent pursuit of goals” (Carver & White, 1994,
p.322) of those who score highly on the BAS scale.
However, the lack of correlation with the other form of
controlled aggression, namely controlled-aversive, may
indicate that the pursuit of goals does not extend to goals
aimed at eliminating averse emotional states or
achieving revenge. Perhaps this is medium-specific,
and people who choose to use ICT devices.
 Cyber-Aggression Typology Questionnaire 9

Residual

.26

.22



.17


Impulsive-aversive cyber-aggression was related to

.14

.04

.06

.04
.02
Impulsive-appetitive
the BAS dimensions of Drive and Reward-responsive-
ness, indicating a potential role of approach motivation
dysregulation in the production of retaliatory responses
to perceived provocation that are enacted without the
.49


recruitment of self-control capacities. Impulsive-appeti-
.36



.45



.30



.28

.27



.15


Raw

tive cyber-aggression was the only subscale to uniquely

-.03
TABLE IV. Raw and Residualized Correlations Between CATQ Factors, Cyberbullying, Reactive/Proactive Aggression, and BIS-BAS (n ¼ 317)

relate to low BIS activity. This suggests that this mode of

cyber-aggression is particularly a function of low levels
of responsivity to punishment, and may more generally
reflect low levels of fear and/or anxiety. Although
Residual

further research is required to better test this notion, it

.28



.24



.18

-.14

.12

BCyQ, Berlin cyberbullying questionnaire; RPQ, reactive proactive aggression questionnaire; BIS/BAS, behavioral inhibition and activation systems scale.
.06
.03

-.04
Controlled-appetitive

may be that some degree of fear/anxiety—reflected here

in BIS scores—may deter young people from engaging
in impulsive-appetitive cyber-aggression.
Limitations
.60



.51



.49



.34

.21

Our research has several notable limitations. First, the

Raw

.14

.12

.13

sample size for our preliminary analyses was very small

for exploratory factor analysis or principal components
analysis. Some statisticians have argued that smaller
sample sizes can be acceptable if communalities are
high, model error low, and only a small number of
Residual

.01
.02
.02
.04
.05
.05
.11
.12

factors are expected for the final solution (Preacher &

Controlled-aversive

MacCallum, 2002). Our efforts to impose conservative

criteria for factor loadings was ad hoc, and may have
resulted in an invalid solution. We believe this limitation
is mitigated by the fact that we were adapting a scale that
.40



.35



.36



.27

had good established validity and by the results of the

Raw

.04
.08
.08
.03

second phase, that providing evidence via CFA that the

quadripartite factor structure did indeed fit the data well.
Our sample size provided a good estimate of CFA, but
was somewhat overpowered for our correlational analyses.
Consequently some of our significant correlations are
Residual

.28

potentially trivial in magnitude, with Pearson r values as low

.13

.13

.12

.12

.02

.06

.10
Impulsive-aversive

as 0.12 statistically significant, but arguably of very limited

importance. For current purposes, we were primarily
concerned that our four factors operated as they would be
expected based on our predicted nomological network.
In recruiting a convenience sample of university
.46



.48



.41



.42

.20



Raw

.16

students, our study may have introduced bias that we

.01
.02

cannot control. For example, we cannot establish a

response rate, as we do not know exactly how many
students received the invitation e-mails. Further research
should aim for a representative, or at least well-
BIS/BAS reward-responsiveness

characterized sample. Another limitation of this sampling

BIS/BAS behavioral inhibition

strategy is that the severity of cyber-aggression is likely to

BCyQ cyber-victimization
RPQ proactive aggression
RPQ reactive aggression

be low overall: future research should aim to examine

BIS/BAS fun-seeking
BCyQ cyber-bullying

cyber-aggressive motives in samples with greater overall

levels of cyber-aggression, possibly younger adolescents.
BIS/BAS drive

P < .001.

Future Directions
P < .01.
P < .05.

The present study provides promising support for a

questionnaire that assesses a new typology of cyber-

Aggr. Behav.
10 Runions et al.
aggression that is developed from a theoretically
driven framework and thereby holds potential to
further our understanding of the diversity of types of
online aggression, any or all of which may be
mislabeled as cyberbullying. Additional research is
required with larger sample sizes that is representative
of the population in question, drawn from a wider age
range of participants that is better reflective of the
population of young people engaging online to test
whether the factor structure confirmed here is consis-
tent across samples. Given that capacities for behav-
ioral and emotional self-regulation continue to develop
through adolescence into adulthood (Steinberg et al.,
2008), it may be that this factor structure is not
consistent with young adolescents, for example.
Alternately, the structure may hold, but mean differ-
ences in subscale scores from those of young adults
may be observed.
Further testing is also required to better establish the
validity and reliability of the CATQ. In particular, test-
retest reliability needs to be established to determine the
extent to which these items assess consistent behavioral
tendencies. Further tests of convergent and divergent
validity are also possible. For example, a direct measure
of impulsivity or self-regulation capacity may help
elaborate the nomological network of these subscales,
and in theory would predict the two impulsive modes.
Controlled-aversive cyber-aggression may be predicted
by anger rumination tendencies and/or self-efficacy for
written communication over face-to-face interaction
(Runions, 2013). Impulsive-aversive cyber-aggression
may be predicted by hostile attribution tendencies,
particularly those assessed via real-time methodologies
(Yaros, Lochman, Rosenbaum, & Jimenez-Camargo,
2014), and/or trait measures of anger. Other hypotheses
to test may focus on the online context, and test the
differential sensitivity of impulsive-appetitive cyber-
aggression to online social network contexts, in which a
peer audience is implicit, versus private, text-message
delivery of cyber-aggression (Runions, 2013). Con-
trolled-appetitive cyber-aggression may be sensitive to
online contexts in which the lack of nonverbal cues may
result in a failure to activate affective empathy, and place
excessive burdens on less efficient cognitive empathic
processes, which may be less mature in young
adolescents.
The establishment of a reliable and valid scale that
captures the diversity of function of cyber-aggression
provides the means of gaining a deeper understanding of
the distinct individual differences and situational cues
that facilitate cyber-aggression. The CATQ may provide
important insight into cyberbullying intervention
research, as different health promotion messages and
intervention strategies may differentially influence
Aggr. Behav.
different types of cyber-aggression. Such investigations
would provide important evidence for researchers
testing cyberbullying intervention and prevention strat-
egies. Such program evaluation would benefit from a
tool that recognizes that “cyberbullying” is not a single
monolithic phenomenon and that the motives and
capacities for self-control behind cyber-aggression
matter.
REFERENCES
Ang, R. P., Huan, V. S., & Florell, D. (2014). Understanding the
relationship between proactive and reactive aggression, and cyber-
bullying across United States and Singapore adolescent samples.
Journal of Interpersonal Violence, 29, 237–254. doi: 10.1177/
0886260513505149
Baumeister, R. F., Vohs, K. D., DeWall, C. N., & Zhang, L. (2007). How
emotion shapes behavior: Feedback, anticipation, and reflection,
rather than direct causation. Personality and Social Psychology
Review, 11, 167–203. doi: 10.1177/1088868307301033
Bjørnebekk, G., & Howard, R. (2012). Validation of a motivation-based
typology of angry aggression among antisocial youths in Norway.
Behavioral Science and the Law, 30, 167–180. doi: 10.1002/bsl.2007
Calvete, E., Orue, I., Est
evez, A., Villard
on, L., & Padilla, P. (2010).
Cyberbullying in adolescents: Modalities and aggressors’ profile.
Computers in Human Behavior, 26, 1128–1135. doi: 10.1016/j.
chb.2010.03.017
Carver, C. S., & White, T. L. (1994). Behavioral inhibition, behavioral
activation, and affective responses to impending reward and
punishment: The BIS/BAS scales. Journal of Personality and Social
Psychology, 67, 319–333. doi: 10.1037/0022-3514.67.2.319
Cima, M., Raine, A., Meesters, C., & Popma, A. (2013). Validation of the
Dutch Reactive Proactive Questionnaire (RPQ): Differential corre-
lates of reactive and proactive aggression from childhood to
adulthood. Aggressive Behavior, 39, 99–113. doi: 10.1002/ab.21458
Compton, L., Campbell, M., & Mergler, A. (2014). Teacher, parent and
student perceptions of the motives of cyberbullies. Social Psychology
of Education, 17, 383–400. doi: 10.1007/s11218-014-9254-x
Dodge, K. A. (1991). The structure and function of reactive and proactive
aggression. In D. J. Pepler, & K. H. Rubin (Eds.), The development and
treatment of childhood aggression. (pp. 201–218). Hillside, NJ:
Erlbaum.
Dodge, K. A., & Coie, J. D. (1987). Social-information-process factors in
reactive and proactive aggression in children’s peer groups. Journal of
Personality and Social Psychology, 53, 1146–1158. doi: 10.1037/
0022-3514.53.6.1146
Dooley, J. J., Py_zalski, J., & Cross, D. (2009). Cyberbullying versus face-
to-face bullying: A theoretical and conceptual review. Zeitschrift f€ ur
Psychologie, 217, 182–188. doi: 10.1027/0044-3409.217.4.182
Fontaine, R. G. (2007). Toward a conceptual framework of instrumental
antisocial decision making and behavior in youth. Clinical Psychology
Review, 27, 655–675. doi: 10.1016/j.cpr.2007.01.012
Frey, K. S., Pearson, C. R., & Cohen, D. (2015). Revenge is seductive, if
not sweet: Why friends matter for prevention efforts. Journal of
Applied Developmental Psychology, 37, 25–35. doi: 10.1016/j.
appdev.2014.08.002
Gray, J. A. (1987). The neuropsychology of anxiety. Oxford: Oxford
University Press.
Howard, R. C. (2009). The neurobiology of affective dyscontrol:
Implications for understanding &lsquo dangerous M. severe person-
ality disorder. In M. McMurran, & R. Howard (Eds.), Personality,
personality disorder and violence. (pp. 157–174). Chichester: Wiley-
Blackwell.

Howard, R. C. (2011). The quest for excitement: A missing link between
personality disorder and violence? Journal of Forensic Psychiatry and
Psychology, 22, 692–705. doi: 10.1080/14789949.2011.617540
Hu, L-T., & Bentler, P. M. (2009). Cutoff criteria for fit indexes in
covariance structure analysis: Conventional criteria versus new
alternatives. Structural Equation Modeling: A Multidisciplinary
Journal, 6, 1–55. doi: 10.1080/10705519909540118
Law, D. M., Shapka, J. D., Domene, J. F., & Gagn
e, M. H. (2012). Are
cyberbullies really bullies? An investigation of reactive and proactive
online aggression. Computers in Human Behavior, 28, 664–672. doi:
10.1016/j.chb.2011.11.013
Muth
en, L. K., & Muth
en, B. O. (1998–2009). MPlus user’s guide
(5th ed.). Los Angeles, CA: Muth
en & Muth
en.
Muthen, B. O., & Muthen, L. K. (2012). Chi-square difference testing
using the Satorra-Bentler scaled chi-square. Retrieved January 2012,
from www.statmodel.com/chidiff.shtml
Preacher, K. J., & MacCallum, R. C. (2002). Exploratory factor analysis in
behavior genetics research: Factor recovery with small sample sizes.
Behavior Genetics, 32, 153–161.
Rafferty, R., & Vander Ven, T. (2014). “I hate everything about you”: A
qualitative examination of cyberbullying and on-line aggression in a
college sample. Deviant Behavior, 35, 364–377. doi: 10.1080/
01639625.2013.849171
Raine, A., Dodge, K. A., Loeber, R., Gatzke-Kopp, L., Lynam, D.,
Reynolds, C., & Liu, J. (2006). The reactive-proactive aggression
questionnaire: Differential correlates of reactive and proactive
aggression in adolescent boys. Aggressive Behavior, 32, 159–171.
doi: 10.1002/ab.20115
Roland, E., & Idsøe, T. (2001). Aggression and bullying. Aggressive
Behavior, 27, 446–462. doi: 10.1002/ab.1029
Runions, K. C. (2013). Toward a conceptual model of motive and self-
control in cyber-aggression: Rage, revenge, reward and recreation.
Journal of Adolescence, 42, 751–771. doi: 10.1007/s10964-013-9936-2
View publication stats
Cyber-Aggression Typology Questionnaire 11
Salmivalli, C., Lagerspetz, K., Bj€ €
orkqvist, K., Osterman, K., &
Kaukiainen, A. (1996). Bullying as a group process: Participant roles
and their relations to social status within the group. Aggressive
Behavior, 22, 1–15. doi: 10.1002/(sici)1098-2337(1996)22:1<1::aid-
ab1>3.0.co;2-t
Schreiber, J. B., Nora, A., Stage, F. K., Barlow, E. A., & King, J. (2006).
Reporting structural equation modeling and confirmatory factor
analysis results: A review. Journal of Educational Research, 99,
323–338. doi: 10.3200/JOER.99.6.323-338
Schultze-Krumbholz, A., & Scheithauer, H. (2009). Social-behavioral
correlates of cyberbullying in an German student sample. Zeitschrift
f€
ur Psychologie, 217, 224–226. doi: 10.1027/0044-3409
Shapka, J. D., & Law, D. M. (2013). Does one size fit all? Ethnic
differences in parenting behavior and motivations for adolescent
engagement in cyberbullying. Journal of Youth and Adolescence, 42,
723–738. doi: 10.1007/s10964-013-9928-2
Steinberg, L., Albert, D., Cauffman, E., Banich, E., Graham, S., &
Woolard, J. (2008). Age differences in sensation seeking and
impulsivity as indexed by behavior and self-report: Evidence for a
dual systems model. Developmental Psychology, 44, 1764–1778. doi:
10.1037/a0012955
Tabachnick, B. G., & Fidell, L. S. (2013). Using multivariate statistics
(6th ed.). Boston: Pearson.
Vaillancourt, T., Hymel, S., & McDougall, P. (2003). Bullying is
power: Implications for school-based intervention strategies.
Journal of Applied School Psychology, 19, 157–176. doi: 10.1300/
J008v19n02_10
Willard, N. E. (2007). Cyberbullying and cyberthreats: Responding to the
challenge of online social aggression, threats, and distress.
Champaign, IL: Research Press.
Yaros, A., Lochman, J. E., Rosenbaum, J., & Jimenez-Camargo, L. A.
(2014). Real-time hostile attribution measurement and aggression in
children. Aggressive Behavior, 40, 409–420. doi: 10.1002/ab.21532
Aggr. Behav.