Developmental Psychology Copyright 1997 by the American Psychological Association, Inc.

1997, Vol. 33, No. 6, 906-9 0012-1649/97/$3.00

Emotion Regulation Among School-Age Children:

The Development and Validation of a New Criterion Q-Sort Scale
Ann Shields and Dante Cicchetti
Mt. Hope Family Center, University of Rochester

To foster the study of emotion regulation beyond infancy and toddlerhood, a new criterion Q-sort
was constructed. In Study 1, Q-scales for emotion regulation and autonomy were developed, and
analyses supported their discriminant validity. Study 2 further explored the construct validity of the
Emotion Regulation Q-Scale within a sample of 143 maltreated and 80 impoverished children, aged
6 to 12 years. A multitrait—multimethod matrix and confirmatory factor analyses indicated impressive
This article is intended solely for the personal use of the individual user and is not to be disseminated broadly.

convergence among the Emotion Regulation Q-Scale and established measures of affect regulation.
This document is copyrighted by the American Psychological Association or one of its allied publishers.

This new scale also was discriminable from measures of related constructs, including Q-sort assess-
ments of ego resiliency. The use of this new measure was further supported by its ability to distinguish
between maltreated and comparison children and between groups of well-regulated versus dysregu-
lated children.

Early research and theory on emotional development have
highlighted the first years of life as fundamental to affect regula-
tion (Cicchetti & Schneider-Rosen, 1986; Eisenberg & Fabes,
1992; Gianino & Tronick, 1988; Kopp, 1989). Recent research
has expanded on this work by demonstrating that emotion regu-
lation continues to play a key role in adaptation during middle
childhood. During these years, regulatory capacities become
increasingly sophisticated and predict children's competence in
a number of important domains, including behavioral regulation
and social competence (Lewis & Haviland, 1993). To foster
the continued study of emotion regulation beyond infancy and
toddlerhood, a new criterion Q-scale for older children was
constructed.
This process was undertaken from a developmental psychopa-
thology perspective, which asserts that an understanding of de-
velopment is best informed by studying pathways leading to
both adaptation and maladaptation (Cicchetti, 1990, 1993;
Sroufe, 1990). One important implication of this perspective
Ann Shields and Dante Cicchetti, Mt. Hope Family Center and the
Department of Clinical and Social Sciences in Psychology, University
of Rochester. Ann Shields is now at the Department of Psychiatry and
Human Behavior, Brown University School of Medicine.
This work was supported by grants from the William T. Grant Founda-
tion, the National Center on Child Abuse and Neglect, the Spunk Fund,
and a National Research Service Award granted to the Human Motivation
Program at the University of Rochester by the National Institute of
Mental Health (MH18922). We are especially grateful to the criterion
raters whose generous donations of time and expertise contributed so
much to this project. We also wish to thank Michael Lynch, Fred Ro-
gosch, and Sheree Toth for their helpful comments on a draft of this
article.
Correspondence concerning this article should be addressed to Ann
Shields, Bradley Hospital, 1011 Veterans Memorial Parkway, East Provi-
dence, Rhode Island 02915, or Dante Cicchetti, Mt. Hope Family Center,
University of Rochester, 187 Edinburgh Street, Rochester, New 'fork
14608. Electronic mail may be sent via Internet to AnnShields®
Brown.edu.
for measurement construction is that assessments must capture
both pathological and nonpathological processes. Furthermore,
measures ideally would be sensitive to heterotypic continuity
(Caspi & Moffitt, 1995; Kagan, 1971) and appropriate across
ages and research contexts. The Q-sort methodology effectively
addresses these concerns by virtue of its capacity to span a wide
age range and to target a variety of adaptive and maladaptive
characteristics within an individual's personality organization
(Block, 1961; Block & Block, 1980a).
Emotion Regulation in Middle Childhood
Consistent with an organizational perspective, which concep-
tualizes development as the increasing differentiation and hierar-
chical integration of biological and psychological systems (Cic-
chetti, 1993; Cicchetti & Sroufe, 1978), recent research has
shown that children's capacities for emotion regulation become
more complex and integrated during the preschool and grade-
school years (Fox, 1994; Saarni, 1990). These gains are fostered
by increased sophistication in representational thought and in-
formation processing abilities (Harris, 1989; Saarni & Harris,
1989), which promote emotional understanding and adaptive
coping (Harter & Buddin, 1987; McCoy & Masters, 1985; Win-
tre & Vallence, 1994). Specifically, school-age children are more
cognizant of display rules and of potential discrepancies be-
tween emotional expressions and internal experiences (Gnepp &
Hess, 1986; Saarni, 1979). School-age children also are more
adept at integrating complex cues regarding others' emotions
and at managing their own affective expressions in accord with
situational demands (Gnepp, 1989; Saarni, 1989; Underwood,
Coie, & Herbsman, 1992). Nevertheless, the study of children's
emotional development still is in its formative stages, such
that measurement development is an important step in fur-
thering understanding of this complex developmental process
(Shields & Cicchetti, 1995).
Q-Sort Methodology
A number of protocols for assessing emotion regulation have
been developed, drawing upon physiological, observational, in-

906
 CRITERION Q-SCALES
terview, and questionnaire methodologies. The Q-sort is distin-
guished from these protocols in that it allows researchers to
evaluate the role of a particular developmental process (such
as emotion regulation) within an individual's comprehensive
personality profile (Block & Block, 1980a). Other advantages
include the Q-sort's ability to keep observers naive of target
constructs and to reduce response biases by requiring raters to
sort items into a fixed distribution (Waters, Noyes, Vaughn, &
Ricks, 1985). Because the Q-sort is applicable across a wide
age range, it is an effective tool for longitudinal research. Finally,
a criterion Q-sort also would allow investigators to explore
questions regarding children's emotional development in extant
data sets that include the California Child Q-Set (CCQ).
Q-sort scales also may be computed from criterion sorts. This
This article is intended solely for the personal use of the individual user and is not to be disseminated broadly.
procedure enhances the specificity of criterion sorts by the use
This document is copyrighted by the American Psychological Association or one of its allied publishers.
of only those items identified as either most or least descriptive
of a particular construct (Eisenberg et al., 1996; John, Caspi,
Robins, Moffit, & Stouthamer-Loeber, 1994). This highly salient
subset, or Q-scale, may include those items falling in the ex-
tremely characteristic and extremely uncharacteristic piles. By
disregarding less salient items lacking in discriminant validity,
the specificity and homogeneity of Q-sort assessments are en-
hanced. Nevertheless, raters should continue to use the full 100-
item CCQ deck when evaluating children to preserve the unique
benefits of the Q-sort methodology.
Study 1
Method
Participants
To compile the Emotion Regulation Q-Sort, from which the Emotion
Regulation Q-Scale was derived, 17 doctoral-level psychologists with
expertise in emotional development generated individual criterion Q-
sorts on a prototypically well-regulated child.1 For the purposes of this
study, emotion regulation was denned in terms of lability, flexibility,
and situational responsivity and conceptualized as the capacity to mod-
ulate one's emotional arousal such that an optimal level of engagement
with one's environment is fostered (Cicchetti, Ganiban, & Barnett,
1991; Thompson, 1994).
A criterion Autonomy Q-Sort also was compiled, then used in analyses
evaluating the discriminant validity of the new emotion Q-scale. Auton-
omy was explored in conjunction with emotion regulation, as both are
early stage-salient issues that have been associated with adaptation in
middle childhood and adolescence (Ryan, 1993; Ryan, Deci. & Grolnick,
1995). If the new emotion regulation sort were discriminable from the
Q-sort assessment of autonomy, then this would suggest that the emotion
scale assessed a specific construct rather than general competence associ-
ated with adaptive early development. To develop this autonomy sort,
18 doctoral-level psychologists specializing in motivation and self-devel-
opment conducted ratings of an optimally autonomous child, defined
along Deci and Ryan's (1985) criteria as one who displayed initiative,
agency, and self-determination.2
Procedure
All 35 raters used the 100-item CCQ deck, following Block and
Block's (1980a) methodology. Because the CCQ's emotion items cap-
tured affective homeostasis, flexibility, situational responsivity, and em-
pathic attunement to others, these descriptors were judged to be broad
enough to conceptualize regulation across a wide age range. Accordingly,
907
consistent with the method used by Block and Block (1980a, 1980b),
raters were asked to portray the personality characteristics of a hypotheti-
cal emotionally well-regulated or optimally autonomous child, without
regard for a specific target age. Raters were instructed to sort the 100
CCQ items into nine categories, ranging from extremely characteristic
to extremely uncharacteristic. These nine categories approximated a
normal distribution in that fewer items were placed in the extreme piles
than in the middle categories.
Composite Q-sort definitions then were created by averaging individ-
ual raters' placements of all 100 descriptors (in Piles 1 through 9 ) .
Emotion Regulation and Autonomy Q-scales then were constructed from
the 10 items (i.e., the 5 highest and 5 lowest items) rated as most salient
for each construct. As a first step toward examining discriminant validity,
Wilcoxon pairwise comparisons then were conducted across the auton-
omy and emotion scales. Significant differences between the mean place-
ments of descriptors in each scale would suggest that they captured
distinct constructs.
Results
Emotion Regulation Q-Sort
For this Q-sort, correlations between individual ratings and
the composite Q-sort ranged from .76 to .93. Internal consis-
tency, as assessed through Cronbach's alpha, was .98. The Emo-
tion Regulation Q-Scale then was created with the five items
with the highest mean rating and the five items with the lowest
mean score across criterion raters. The Cronbach's alpha for
this Q-scale was .85.
The items in this Q-scale are presented in Table 1, along with
pairwise Wilcoxon comparisons examining differences between
their mean placements (in Piles 1 through 9) in the emotion
regulation versus autonomy sorts. Wilcoxon tests showed that
the ratings for all but 1 of the items comprising the emotion Q-
scale were significantly different from those in the autonomy
scale. However, 2 items reflecting a lack of resiliency in the
face of stress were placed in both scales: CCQ Item 46, '"fends
to go to pieces under stress," and CCQ Item 39, "Tends to be
rigidly repetitive in stressful situations."
Autonomy Q-Sort
For the Autonomy Q-Sort, correlations between individual
criterion ratings and the composite sort ranged from .82 to .91.
The Cronbach's alpha for the criterion sort was .98. An Auton-
omy Q-Scale then was created, and the internal consistency of
this scale, assessed through Cronbach's alpha, was .79. This Q-
1
We are indebted to the following raters for their generous contribu-
tions of time and expertise toward the construction of the Emotion
Regulation Q-Sort: Karen C. Barrett, Jude Cassidy, Jeffrey Cohn, Pamela
Cole, Patricia Crittenden, Kenneth A. Dodge, Byron Egeland, Nancy
Eisenberg, Judy Garber, John Gottman, Carroll Izard, Carol Malatesta-
Magai, John Richters, Carolyn Saarni, Judith Solomon, Ross A. Thomp-
son, and Carolyn Zahn-Waxler,
2
We are grateful to the following Autonomy Q-Sort criterion raters,
whose generous contributions of time and expertise made this work
possible: Jay Belsky, Edward L. Deci, Ann Easterbrooks, Wendy S.
Grolnick, Daniel Hart, R. Rogers Kobak, Grazyna Kochanska, Richard
Koestner, Michael Lewis, Alicia Lieberman, Sandra Pipp-Siegel, Robert
Plant, Johnmarshall Reeve, Richard M. Ryan, Ellen Skinner, Susan
Spieker, Jaine Strauss, and Joan Vondra.
 908 SHIELDS AND CICCHETTI

Table 1
Emotion Regulation Q-Scale: Mean-Item Comparisons With the Autonomy Q-Scale

CCQ no. CCQ item Emotion Regulation (Af) Autonomy (Af)

Positively weighted
43 Can recover from stress 8.9 7.6*
81 Can admit negative feelings 8.4 7.3*
3 Is warm/responsive 8.1 7.3*
31 Is empathic 8.1 6.9*
9 Has genuine/close relationships 8.1 6.7*

Negatively weighted
91 Emotional reactions are inappropriate 1.2 3.2*
39 Is rigid/repetitive when stressed 1.4 2.3*
This article is intended solely for the personal use of the individual user and is not to be disseminated broadly.

46 Goes to pieces under stress 1.5 1.9

This document is copyrighted by the American Psychological Association or one of its allied publishers.

54 Is emotionally labile 1.7 4.0*

95 Is easily irritated 1.8 2.9*

Note. Reproduced by special permission of the distributor, MIND GARDEN, Inc., P.O. Box 60669, Palo
Alto, CA 94306 from the California Child Q-Sort items by Jack Block. Copyright 1980 by Consulting
Psychologists Press. All rights reserved. Further reproduction is prohibited without the distributor's written
consent. CCQ = California Child Q-Set.
*p< .05.

scale was correlated significantly with the emotion regulation
scale (/• = .56, p < .001).
Table 2 lists the items that comprise this scale, along with
pairwise Wilcoxon tests comparing these items' mean place-
ments in the autonomy and emotion regulation sorts. The place-
ment of 8 of the 10 autonomy scale items differed significantly
from their placement in the emotion regulation sort. The 2 items
on which no differences were found were CCQ Item 46, "Tends
to go to pieces under stress," and CCQ Item 23, "Is fearful
and anxious."
Discussion
Initial analyses support the promise of the new Emotion Regu-
lation Q-Scale in furthering the study of this early stage-salient
issue across development. First, substantial interrater reliability
was found among criterion raters, suggesting consistency and
clarity in the experts' use of Q-sort descriptors to define af-
fective regulation. This high concordance is especially compel-
ling, given the raters' level of expertise.
Item-by-item comparisons of scale items also suggested
substantial discriminant validity between the Emotion Regula-
tion and Autonomy Q-scales. The rankings of 9 of the 10 items
in the Emotion Regulation Q-Scale were significantly different
from their placement in the autonomy sort. Similarly, the rat-
ings for 8 of the 10 autonomy scale items were significantly
different from their placement in the emotion regulation sort.
This discriminability is consistent with the suggestion that the
emotion Q-scale assesses a specific affective construct rather

Table 2
Autonomy Q-Scale: Mean-Item Comparisons With the Emotion Regulation Q-Scale

CCQ no. CCQ item Autonomy (Af) Emotion regulation (Af)

Positively weighted
88 Is self-reliant and confident 8.6 7.4*
40 Is curious and exploring 8.6 7.1*
36 Is resourceful/takes initiative 8.6 7.0*
83 Seeks to be autonomous 8.5 6.4*
82 Is self-assertive 8.2 6.4*

Negatively weighted
77 Feels unworthy/bad 1.8 2.8*
46 Goes to pieces under stress 1.9 1.5
53 Is indecisive/vacillating 2.2 3.9*
39 Is rigid/repetitive when stressed 2.3 1.4*
23 Is fearful and anxious 2.3 2.6

Note. Reproduced by special permission of the distributor, MIND GARDEN, Inc., P.O. Box 60669, Palo
Alto, CA 94306 from the California Child Q-Sort items by Jack Block. Copyright 1980 by Consulting
Psychologists Press. All rights reserved. Further reproduction is prohibited without the distributor's written
consent. CCQ = California Child Q-Set.
* p < .05.
 CRITERION Q-SCALES
than general competence associated with adaptive early
development.
Furthermore, the descriptors comprising the Emotion Regula-
tion Q-Scale reflected fundamental aspects of the construct
definition presented to criterion raters. For example, 2 items
reflected emotional homeostasis versus lability: CCQ Item 54,
"Has rapid shifts in mood, is emotionally labile," and CCQ
Item 95, "Overreacts to minor frustrations, is easily irritated."
Flexibility versus rigidity also was evident in 2 items—CCQ
Item 39, "Tends to become rigidly repetitive or immobilized
under stress," and CCQ Item 43, "Can recoup or recover after
stressful experiences." Situational responsivity was captured by
Item 91 ("Emotional reactions are inappropriate.").
Affective engagement with the social surround also was sug-
This article is intended solely for the personal use of the individual user and is not to be disseminated broadly.
gested by CCQ Item 3, "Is warm and responsive," CCQ Item
This document is copyrighted by the American Psychological Association or one of its allied publishers.
31, "Shows a recognition of others' feelings, is empathic,"
and CCQ Item 9, "Develops genuine and close relationships."
Although not specified in the criterion definition, emotional self-
awareness was represented by CCQ Item 81, "Can acknowl-
edge unpleasant experiences and admit negative feelings." Items
held in common between the emotion regulation and autonomy
scales suggest that the criterion raters regarded both poorly
regulated and non-autonomous children to be vulnerable to dis-
organization when stressed.
Study 2
To evaluate the construct validity of the Emotion Regulation
Q-Scale, both convergent and discriminant validities were exam-
ined in Study 2. First, it was necessary to demonstrate that
this Q-scale was related to established measures of emotion
regulation. Next, its discriminability from other assessments of
early developmental competence (e.g., autonomy) was ex-
plored. Finally, the extent of overlap between the Q-scale and
other Q-sort assessments of putatively related constructs also
was evaluated.
Three statistical procedures were used: First, a multitrait-
multimethod matrix (MTMM) was constructed to examine rela-
tionships among assessments of emotion regulation, autonomy,
and ego resiliency; second, confirmatory factor analyses were
conducted to further evaluate this matrix; and, finally, t tests
were conducted to determine whether the Q-scale could distin-
guish between well-regulated and dysregulated groups. To en-
sure the broadest applicability of this new measure, populations
likely to evidence a wide range of functioning were targeted.
Participants were maltreated children, who are at enhanced risk
for deficits in self-regulation (Cicchetti et al., 1991), and non-
maltreated comparison children. Because comparison partici-
pants were matched on a number of important demographic
indexes, including low socioeconomic status, the entire sample
experienced variable degrees of risk for negative developmental
outcomes.
MTMMs allow researchers to evaluate construct validity by
examining a measure's convergent and discriminant validity
among related and divergent methods and constructs (Camp-
bell & Fiske, 1959). This matrix is comprised of two or more
traits assessed by two or more methods. Convergent validity is
supported when strong correlations are found between a new
measure and established assessments of the same construct. Dis-
909
criminant validity is demonstrated whenever lower correlations
between the target measure and assessments of divergent con-
structs are obtained.
Discriminant validity also would be supported if the emotion
Q-scale were a stronger predictor of criterion emotion measures
than were related Q-sorts. One plausible hypothesis would be
that the emotion sort merely reflected a higher order construct,
such as ego resiliency, already captured by the Q-sort methodol-
ogy. Indeed, ego resiliency has been suggested as an index of
optimal regulation (Eisenberg et al., 1996), as it is defined in
terms of key regulatory processes, including flexibility, adapt-
ability, and an ability to maintain homeostasis when faced with
changing situational demands (Block & Block, 1980b). This
overlap is further suggested by the fact that 3 of the 10 Emotion
Regulation Q-Scale items also were rated as most salient in the
ego resiliency criterion sort: CCQ Item 39, "Tends to become
rigidly repetitive or immobilized under stress," CCQ Item 46,
"Tends to go to pieces under stress," and CCQ Item 91, "Emo-
tional reactions are inappropriate."
A confirmatory factor analysis (CFA) of the MTMM data
also was conducted (Marsh, 1989; Schmitt & Stults, 1986).
This procedure is recommended because the MTMM design
lacks clear, quantifiable criteria for determining appropriate lev-
els of convergence and divergence among variables (Jackson,
1969) and because it relies on observed rather than latent vari-
ables (Greenbaum, Dedrick, Prange, & Friedman, 1994). Al-
though the complete CFA model, which contains separate latent
variables for trait and method effects, is the most widely used, it
tends to generate ill-defined solutions and unreliable estimations
(Greenbaum et al., 1994; Kenny & Kashy, 1992). Therefore, the
correlated uniqueness model, which does not include separate
method factors, was used in this study (Kenny & Kashy, 1992;
Marsh & Bailey, 1991).
Finally, the capacity of the Emotion Regulation Q-Scale to
discriminate between well-regulated versus dysregulated chil-
dren was examined. An ability to differentiate between mal-
treated and nonmaltreated children also would support construct
validity, for abused children evidence chronic patterns of af-
fective negativity and lability from toddlerhood through middle
childhood (Alessandri, 1991; Erickson, Egeland, & Pianta,
1989; Shields & Cicchetti, 1995). Specifically, maltreated chil-
dren evidence heightened sensitivity to anger (Cummings, Hen-
nessy, Rabideau, & Cicchetti, 1994; Hennessy, Rabideau, Cic-
chetti, & Cummings, 1994; Pollak, Cicchetti, Klorman, & Bru-
maghim, in press), rigid and situationally inappropriate
affective displays (Rogosch, Cicchetti, & Aber, 1995; Shields
et al,, 1994), attenuated empathy (Howes & Eldredge, 1985;
Main & George, 1985), and a vulnerability to dysphoria and
internalizing symptoms (Hart, Gunnar, & Cicchetti, 1996; Kauf-
man, 1991; Kazdin, Moser, Colbus, & Bell, 1985; Toth,
Manly, & Cicchetti, 1992).
Method
Participants
The participants in the study were 143 maltreated children (52 girls
and 91 boys) and 80 nonmaltreated comparison children (29 girls and
51 boys), aged 6 to 12 years (Af = 9 years 11 months). Groups were
equivalent on age, sex, and minority status: Sixty-nine percent of the
 910 SHIELDS AND CICCHETTI
maltreated children and 75% of the nonmaltreated children were His-
panic, African American, or biracial; the remainder of each group was
Caucasian. The two groups also were matched on socioeconomic status
in that both maltreating and nonmaltreating families were impoverished:
Total yearly income, which included public assistance payments, aver-
aged $18,200.00 for maltreating families, whereas nonmaltreating fami-
lies' incomes averaged $ 15,700.00. The per capita annual incomes for the
maltreating and nonmaltreating families were $3,866.00 and $3,956.00,
respectively (see Table 3).
Although efforts were taken to ensure that the groups were compara-
ble, some differences were evident in family size and configuration.
More adults tended to be present in maltreating families (t = —2.75,
p < .001), and maltreating families tended to include more children
(M = 3.4) than nonmaltreating families (M = 2.6), ?(220) = -4.05,
p < .001. However, no associations were found between these factors
This article is intended solely for the personal use of the individual user and is not to be disseminated broadly.
and the outcome variables in Study 2.
This document is copyrighted by the American Psychological Association or one of its allied publishers.
The maltreating families were identified by the Monroe County De-
partment of Social Services (DSS) as having documented reports of
child abuse or maltreatment in accordance with New %rk state laws.
After parental consent to do so was obtained, these families' Child
Protective Services records were examined and coded according to the
Barnett, Manly, and Cicchetti (1993) maltreatment classification system.
The presence or absence of each subtype of maltreatment was ascer-
tained for the families, and the children were classified in the following
ways: sexually abused (13.3%, n = 19); physically abused (41.3%, n =
59); physically neglected (37.8%, n = 54), and emotionally maltreated
(7.6%, n — 11), It should be noted that, consistent with the literature,
the majority of the children (i.e., 53.8%) experienced multiple subtypes
of maltreatment (Cicchetti & Rizley, 1981). Because the focus of this
article is on the development and validation of a new criterion Q-sort
scale, research questions and statistical analyses center on maltreated
versus nonmaltreated comparisons, and analyses by subtype were not
conducted.
The nonmaltreated comparison families were recruited through post-
ers in welfare offices, housing projects, and neighborhoods similar to
those in which the maltreating families resided. An initial telephone
screening ensured that the families met the criteria for the study. Nonmal-
treated status in the comparison group was determined by interviews
conducted with the parents by trained research assistants and, on receipt
of parental consent to examine DSS records, by a careful check of CPS
records to verify the absence of maltreatment reports.
Procedure
Data were collected during the eighth annual summer day camp for
at-risk inner-city children sponsored by Mt. Hope Family Center at the
University of Rochester. Maltreated and comparison children attended
camp for week-long sessions, participating in both recreational activities
and research (see Cicchetti & Manly, 1990, for detailed information
about the camp setting). Before camp, parents gave informed consent
for their children's participation in research activities during the week.
Children then were given a choice about participating in research inter-
views, for which they earned points to be exchanged for small prizes.
Six 1-week sessions were held during the summer. Bach week, chil-
dren were assigned to groups of 8 to 10 children of the same sex
and age; groups also included a balance of maltreated and comparison
children. Each group was led by three counselors, who were unaware
of the experimental hypotheses and of the children's maltreatment status.
Counselors' intensive contact with the children (approximately 35 hours
across the week) enabled them to assess children's functioning effec-
tively across a number of domains. Independent observers, similarly
unaware of hypotheses and participant status, also conducted observa-
tions and rated children's functioning on 2 days during each camp
Measures, Multitrait—Multimethod Analysis
California Child Q-Set. The CCQ (Block & Block, 1980a) method-
ology requires raters to sort 100 descriptors of children's common be-
haviors and personality characteristics into ipsative personality profiles
(Block & Block, 1980a). At least two counselors completed these sorts
on each child after camp sessions, and means were computed across
reliable raters. From these Q-sorts, individual scores on the Emotion
Regulation and Autonomy Q-scales were generated. In addition, an Ego
Resiliency Q-Scale was computed for inclusion in the MTMM analysis,
along the lines of the Q-scale methodology reported by John et al.
(1994) and Eisenberg et al. (1996). As few as 7 and as many as 24 items
have been used to compile these Q-scales, all of which are constructed to
produce purer assessments of target constructs. For this study, the Ego
Resiliency Q-Scale was comprised of the 10 most extreme items to be
in keeping with the broader literature on Q-sort assessments (specifying
the two most extreme piles as most salient in defining a target construct)
and to be consistent with the construction of the Emotion Regulation
and Autonomy Q-scales.
Emotion Regulation Checklist. The Emotion Regulation Checklist
(ERC; Shields & Cicchetti, 1995), a 24-item other-report measure of
children's self-regulation, also was completed by counselors.3 This
checklist was selected as a criterion emotion measure, as its conceptual-
ization of regulation is consistent with the construct definition presented
to expert raters in Study 1. That is, this measure targets such processes
as affective lability, intensity, valence, flexibility, and situational appro-
priateness. The checklist includes both positively and negatively
weighted items to be rated on a 4-point Likert scale.
To evaluate the factor structure of the ERC, a principal-components
factor analysis with a varimax rotation was conducted. This analysis
yielded two factors representing Lability/Negativity and Emotion Regu-
lation. The Lability/Negativity subscale is comprised of items represent-
ing a lack of iexibility, mood lability, and dysregulated negative affect;
sample items include "Exhibits wide mood swings," and "Is prone
to angry outbursts." The Emotion Regulation subscale includes items
describing situationally appropriate affective displays, empathy, and
emotional self-awareness; sample items include "Is empathic toward
others," and "Can say when s/he is feeling sad, angry or mad, fearful
or afraid." Internal consistencies, assessed through Cronbach's alpha,
were .96 for Lability/Negativity and .83 for Emotion Regulation. In
addition, the two subscales were significantly correlated (r = -.50,
p < .001). A composite ERC score also was generated, as a single
emotion regulation criterion measure was needed for confirmatory factor
analyses, and it was important that this variable capture processes related
to both regulation and dysregulation. The internal consistency of this
composite ERC score, which was computed by reverse scoring all nega-
tively weighted items and then averaging across all 24 items, was .89.
Emotion regulation observations. Emotion regulation observations
(Shields et al., 1994) took place on 2 days during each camp session,
when independent observers, not camp counselors, coded children's
emotion expressions during semistructured play periods. These periods
were chosen because they provided an opportunity to observe partici-
pants' naturalistic interactions in a social context that was both familiar
and important to grade-school children. During these play periods, chil-
dren participated in recreational activities with both peers and authority
figures; often, counselors gave children a choice of several group activi-
ties or games. No effort was made by experimenters to direct these
interactions or to present specific social and emotional challenges to
3
Copies of the Emotion Regulation Checklist, along with instructions
for administration and scoring, may be obtained from Dante Cicchetti,
Mt. Hope Family Center, University of Rochester, 187 Edinburgh Street,
Rochester, New York 14608.
 CRITERION Q-SCALES
Table 3
Contrasts of Maltreated and Nonmaltreated Groups on Demographic Variables
Maltreated
Demographic variable Af
Child's age (yrs) 9.07
Total family income ($)" 18,200
Number of children 3.4
Number of adults 1.9
Per capita annual income ($) 3,866
a
Amount includes public assistance.
***p < .001.
This article is intended solely for the personal use of the individual user and is not to be disseminated broadly.
This document is copyrighted by the American Psychological Association or one of its allied publishers.
children, and the observers made every effort to be as unobtrusive as
possible.
Three 10-min observations of each child's affective expressions were
collected, during which emotions were identified by facial expressions
(Ekman & Friesen, 1975), voice tone, and body posture (see Izard,
1982; Scherer, 1979) and evaluated for valence, intensity, flexibility, and
situational responsivity. Emotion expressions were judged to be adaptive
or appropriate if they were situationally responsive to the context in
which the child was interacting. For example, expressions of anger to-
ward an intrusive or aggressive peer were coded as appropriate, for
these served important self-protective goals. However, if this anger were
directed toward an inappropriate target (e.g., a nonaggressive child),
then this expression would be coded as maladaptive/inappropriate. Sim-
ilarly, if this anger were so intense as to undermine an adaptive response,
or so persistent as to outlive its usefulness, then it would be coded as
maladaptive/inappropriate as well.
Coders noted the presence versus the absence of seven emotion catego-
ries during each 10-min observation period: appropriate positive with
peers or adults, inappropriate positive with peers or adults, appropriate
negative with peers, and inappropriate negative with peers or adults.
(Appropriate negative with adults was not coded because of a failure
to achieve reliability on this variable, perhaps because of its low base
rate.) Interobserver agreement on the emotion scales ranged from a
kappa of .54 for inappropriate positive with peers to a kappa of .74 for
inappropriate negative with peers.
A composite emotion regulation observation score then was computed
as follows: First, mean scores for each of the seven emotion expression
categories were computed across the three observation periods; then,
mean scores of all inappropriate affective displays were summed, as
were the scores of all emotion expressions judged to be appropriate. The
summary maladaptive/inappropriate score then was subtracted from the
summary adaptive/appropriate score to create the emotion regulation
observation composite. Higher scores reflected more adaptive self-
regulation.
Minnesota Behavior Ratings. The Minnesota Behavior Ratings
(Sroufe, 1983), an other-report measure, asks adults familiar with indi-
vidual children to generate molar ratings of adaptive functioning in a
number of key domains. Subscales assessing dispositional tendencies
toward Positive Mood and Negative Mood were included as approxima-
tions of children's affective modulation. Higher scores on Positive Mood
reflect a tendency toward warmth and equanimity, whereas lower scores
reflect characteristically flat or negative expressions. Similarly, higher
scores on Negative Mood reflect chronic anger and hostility, whereas
lower scores reflect a tendency toward positive emotion expressions.
Also included as a criterion autonomy measure was the Agency sub-
scale, on which higher scores reflect a tendency toward self-assertion,
mastery, and intrinsic motivation, whereas lower scores suggest a relative
passivity and a lack of initiative. The Dependency subscale also was
911
Nonmaltreated
SD M SD t
1.95 8.77 1.78 -1.15
11,000 15,700 9,000 -1.71
1.6 2.6 1.2 -4.05***
0.9 1.6 0.5 -2.75***
2,524 3,956 2,306 0.26
included as an approximation of low levels of agency and autonomy,
for higher scores reflect an extreme reliance on others, whereas lower
scores suggest that the child demonstrates a high degree of self-direction.
Correlations and standard deviations among counselors' ratings for
individual children were computed. If these analyses indicated that a
rater was unreliable across measures, then the data provided by that
counselor were dropped from analyses. Means were computed across
reliable raters to yield one score each for the Agency, Dependency,
Positive Mood, and Negative Mood scales.
Student-Teacher Relationship Scale. The Student-Teacher Rela-
tionship Scale (STRS; Pianta, 1992; Pianta & Steinberg, 1992) is a 5-
point Likert scale, completed by adults familiar with a child, that targets
the quality of children's relationships with important adults. Five rela-
tional characteristics are assessed by the STRS: dependent, conflicted,
warm, open, and troubled. The Dependency subscale again was selected
as another approximation of a child's lack of relative agency and auton-
omy. The internal consistency of this subscale, assessed through Cron-
bach's alpha, was .81.
Measures, Confirmatory Factor Analysis
Big Five scales of the California Child Q-Set. The Big Five scales
of the California Child Q-Set (John et al., 1994) reflect dimensions of the
Five-Factor Model of personality (McCrae & John, 1992): extroversion,
conscientiousness, openness/intellect, agreeableness, and neuroticism.
Two of these scales, Neuroticism and Agreeableness, were included in
confirmatory factor analyses, as overlaps between these two constructs
and emotion regulation were thought to be likely. The Big Five scales
were derived by expert raters who selected Q-sort items most salient
for each target construct. Items lacking in discriminant validity then
were dropped, resulting in a 10-item scale for Neuroticism and a 13-
item scale for Agreeableness.
That neuroticism has a significant affective component is suggested
by the fact that individuals high on this personality characteristic are
seen as anxious, tense, and emotionally labile. Similarly, agreeable indi-
viduals are described as warm and empathic. Furthermore, these two
scales were the only ones of the Big Five to hold items in common with
the Emotion Regulation Q-Scale. For neuroticism, three shared items
were found: CCQ Item 39, "Tends to become rigidly repetitive or immo-
bilized under stress," CCQ Item 43, "Can recoup after stressful experi-
ences," and CCQ Item 46, "Tends to go to pieces under stress." For
Agreeableness, two items were held in common: CCQ Item 3, "Is
warm and responsive," and CCQ Item 9, "Develops genuine and close
relationships." Therefore, testing whether the new Emotion Regulation
scale could be distinguished statistically from the Neuroticism and
Agreeableness scales was seen as a rigorous test of discriminant validity.
Child Behavior Checklist—Teacher's Report Form. The Child Be-
havior Checklist—Teacher's Report Form (CBCL-TRF; Achenbach,
 912 SHIELDS AND CICCHETTI

1991), one of the most widely used and well-validated other-report
measures of children's adjustment, is composed of 118 items designed
to assess children's emotional and behavioral difficulties. This measure
yields two broad-band factors representing internalizing and externaliz-
ing difficulties as well as nine subscales assessing more specific symp-
tom patterns (Achenbach, 1991). Of these nine scales, the Anxiety/
Depression subscale was paired with Neuroticism in confirmatory factor
analyses, for both target symptoms of anxiety and dysphoria.
Counselors completed the CBCL-TRF on all children in their groups,
and correlations and standard deviations among ratings then were com-
puted. If these analyses indicated that a rater was unreliable across
measures for a child, then these data were dropped from analyses. Means
were computed across reliable raters to yield one score for Anxiety/
Depression.
Pupil Evaluation Inventory. The Pupil Evaluation Inventory (PEI;
Pekarik, Prinz, Liebert, Weintraub, & Neale, 1976) is an assessment
This article is intended solely for the personal use of the individual user and is not to be disseminated broadly.
correlations were found between this Q-scale and the criterion
measures of emotion regulation. These correlation coefficients
ranged from r = .44, p < .001, for the emotion regulation
observations to r = - . 7 9 , p < .001, for Lability/Negativity.
In general, these correlations were of greater magnitude than
were those between the Emotion Regulation Q-Scale and the
criterion measures of autonomy. That is, correlations between
the emotion scale and the criterion autonomy measures ranged
from r = - . 1 3 , p < .05, for the STRS rating of dependency to
r= -.24, p< .001, for the Minnesota dependency rating. These
findings suggest that the Emotion Regulation Q-Sort captures
regulation, in particular, rather than developmental competence
associated with adaptive early development, which also presum-
ably would be captured by autonomy measures.

Next, the Emotion Regulation Q-Scale's discriminant validity

This document is copyrighted by the American Psychological Association or one of its allied publishers.

of social behavior, completed by camp counselors, that targets three

interpersonal dimensions: withdrawal, aggression, and likability. The
Likability scale was paired with the John et al. (1994) Agreeableness
scale in confirmatory factor analyses, as both assess the degree to which
children arouse liking in others. The internal consistency of this subscale
was .87.
Results
MTMM
An MTMM was generated to evaluate relationships among
Q-scales assessing emotion regulation, autonomy, and ego resil-
iency. Included also were the five criterion measures of emotion
regulation and the three criterion measures of autonomy; the
pattern of results then was examined for evidence of convergent
and discriminant validity (see Table 4). Convergent validity is
supported when a new assessment is highly correlated with
criterion measures of the same construct. Discriminant validity
is indicated when these correlations with criterion measures are
of a greater magnitude than are correlations between the new
measure and a lesser related construct (in this case, autonomy).
In addition, discriminant validity would be supported if the
Emotion Regulation Q-Scale were a more powerful predictor
of criterion emotion measures than Ego Resiliency.
As can be seen in Table 4, results supported the convergent
validity of the Emotion Regulation Q-Scale in that significant
with regard to ego resiliency was examined. The Emotion Regu-
lation and Ego Resiliency Q-scales were significantly correlated
(r = .69, p < .001), raising questions of a substantial overlap
between these constructs. Therefore, an important next step
would be to investigate whether the emotion Q-scale was more
highly correlated with criterion measures of emotion regulation
than was the Ego Resiliency Q-Scale. To test these differences
among correlations, t tests were conducted in accord with the
methodology suggested by Cohen and Cohen (1983) for depen-
dent correlations. These statisticians recommended these t tests
(rather than Fischer's r t o Z ) as most appropriate for testing
differences between correlations that have been generated from
data collected from a single sample.
These analyses further supported the discriminant validity of
the new Q-scale. For example, the Emotion Regulation Q-Scale
was more strongly associated with Lability/Negativity (r =
-.79, p < .001) than was the resiliency Q-scale (r = - . 3 7 ,
p < .001; t = 14.06, p < .001). Similarly, the emotion Q-scale
was a stronger predictor of negative mood (r = - .74, p < .001)
than was the ego resiliency scale (r = —.49, p < .001; t =
6.99, p < .001). Finally, the Emotion Regulation Observations,
which are sensitive to situationally inappropriate and inflexible
affective displays, were correlated more strongly with the emo-
tion scale (r = .44, p < .001) than with the resiliency scale
(r = .18, p < .01; t = 5.54, p < .01). However, no substantial

Table 4
Multitrait-Multimethod Matrix: Convergent and Discriminant Validity of the Emotion Regulation Q-Scale
Subscale 1 2 3 4 5 6 7 8 9 10

1. E Reg Q-Scale
2. Lability/Negativity -.79*** —
3. Emotion Regulation .68*** -.50*** —
4. Emotion Observations .44*** _49*** .23***
5. Positive Mood 72*** -.58*** .77*** .32***
6. Negative Mood _.74*** .74*** -.55*** _ 40*** -.74*** —
7. Autonomy Q-Scale .57*** -.30*** .55*** .13* .50*** -.38*** —
8. Agency .20** .07 .43*** .01 .35*** -.04 .69*** —
9. Dependency 1 -.24*** .23*** -.08 -.18** -.17* .26*** -.45*** -.22**
10. Dependency 2 -.13* .13 .02 .06 -.03 .07 -.35*** -.16** .63*** —
11. Ego Resiliency .69*** -.37*** .67*** .18** .64*** -.49*** .90*** .62*** -.37*** -.26***

Note. E Reg Q-Scale = Emotion Regulation Q-Scale.

*p < .05. **p < .01. ***/>< .001.
 CRITERION Q-SCALES
differences were evident in the two Q-scales' abilities to predict
relative equanimity and positive mood, as assessed by the Min-
nesota Positive Mood and ERC Emotion Regulation subscales.
These findings suggest that, whereas the Q-sort assessment of
children's ego resiliency may capture a tendency toward equa-
nimity and positive mood, it is less sensitive to indexes of emo-
tion dysregulation.
A different pattern of findings was obtained for the Autonomy
Q-Scale. Its discriminant validity initially was supported by the
fact that it was more highly correlated with the criterion auton-
omy measures than with the emotion measures. However, the
Autonomy Q-Scale was correlated with ego resiliency to such
a degree as to suggest that the constructs (as assessed by Q-
sorts) were not discriminable (r = .90, p < .001). As can be
This article is intended solely for the personal use of the individual user and is not to be disseminated broadly.
seen, the ego resiliency scale was as highly correlated with
This document is copyrighted by the American Psychological Association or one of its allied publishers.
criterion autonomy measures as was the autonomy scale. For
example, Agency was correlated significantly with both the au-
tonomy scale (r = .69, p < .001) and with the ego resiliency
scale (r = .62, p < .001). Therefore, although the Autonomy
Q-Scale appears to capture a construct distinct from emotion
regulation, it appears to be subsumed by existing Q-sort mea-
sures of ego resiliency.
Confirmatory Factor Analysis
A confirmatory factor analysis with the correlated uniqueness
model (Kenny & Kashy, 1992; Marsh & Bailey, 1991) was
conducted to further explore the discriminant validity of the
Emotion Regulation Q-Scale. This model included two methods
(Q-sorts and questionnaires) assessing four traits (emotion reg-
ulation, autonomy, agreeableness, and neuroticism). That is,
the four Q-scales assessing neuroticism, agreeableness, emotion
regulation, and autonomy (which also may be seen as an ap-
proximation of ego resiliency) were paired with counselor-re-
port measures tapping each of these four constructs: the CBCL-
TRF subscale assessing Anxiety/Depression (for neuroticism),
the PEI subscale for Likability (for agreeableness), the ERC
composite score (for emotion regulation), and the Minnesota
Agency rating (for autonomy). The overall fit of the model was
evaluated by Bentler's (1990,1992) normed comparative fit and
nonnormed indexes, and values greater than .90 were obtained
on each of these indexes, reflecting an acceptable fit.
The chi-square difference test evaluated whether the four mea-
surement dyads assessed constructs that were discriminable
from one another. This test compares pairs of variables that are
either constrained or allowed to covary through chi-square tests
on the significance levels of the models for each pair (Ander-
son & Gerbing, 1988). All of the chi-square analyses were
significant (p < .001), indicating that the standard measurement
model (in which the variables were allowed to covary) provided
a better fit than did the unidimensional model (in which the
variables were constrained). In other words, the standard mea-
surement model, in which the factors were viewed as distinct
but correlated constructs, provided a fit that was significantly
better than the fit provided by the unidimensional model. This
indicates that the factors assessed by each measurement dyad
could be statistically differentiated from one another (see Ta-
ble 5).
The statistic for the fully covaried model was x 2 (14, N =
913
Table 5
Chi-Square Difference Tests: Evaluating the Discriminant
Validity Between Unidimensional Sets of Factors
Incorporated in the Confirmatory Factor Analysis
Factor model X2(15, N = 207) Difference*
Factor 1/Factor 2 350.19*** 269.07***
Factor 2/Factor 3 180.15*** 99.03***
Factor 1/Factor 3 97.72*** 16.60***
Factor 1/Factor 4 431.78*** 350.66***
Factor 2/Factor 4 123.23*** 42.11***
Note. Factor 1, Emotion Regulation, is comprised of the Emotion Reg-
ulation Q-Scale and the Emotion Regulation Checklist composite score;
Factor 2, Autonomy, is comprised of the Autonomy Q-Scale and Agency;
Factor 3, Agreeableness, is comprised of Agreeableness and Likability;
Factor 4: Neuroticism is comprised of Neuroticism and Anxiety/Depres-
sion.
' Difference from the x2 of the fully covaried model.
***p < .001.
217) = 81.12,/?<.001.In addition, five unidimensional models
comparing pairs of factors were computed: Model 1 compared
the Emotion Regulation and Autonomy factors; Model 2 com-
pared the Autonomy and Agreeableness factors; Model 3 com-
pared Jhe Emotion Regulation and Agreeableness factors; Model
4 compared the Emotion Regulation and Neuroticism factors;
and Model 5 compared the Autonomy and Neuroticism factors.
The statistics for these comparisons ranged from X 2 (15, N =
217) = 97.72, p < .001 to x 2 (15, N = 217) = 431.78, p <
.001. All five comparisons were significantly different from the
chi-square for the fully covaried model, suggesting that the con-
structs are statistically discriminable as assessed by these
measures.
Next, a confidence interval test was conducted to further ex-
plore the discriminant validity of the new emotion Q-scale. The
confidence interval was constructed of ±2 SEs around each of
the correlations of the two factors, then examined to see whether
these intervals overlapped in such a way that the two factors
were indiscriminable (Anderson & Gerbing, 1988). In none of
the tests conducted did the confidence interval indicate that the
two traits overlapped.
Group Differences: Well-Regulated Versus Dysregulated
Children
Groups of well-regulated versus dysregulated children were
identified with the Emotion Regulation Observations and the
two ERC subscales; these groups were composed of children
scoring 1 SD above or below the mean on these measures.
According to t tests, the new Q-scale effectively captured differ-
ences between these groups. Specifically, significant differences
were found between groups identified as either well-regulated
or dysregulated according to the ERC Lability/Negativity scale,
r(76) = 18.19, p < .001, and the ERC Regulation subscale,
f(69) = -12.66, p < .001. In addition, the new Q-scale distin-
guished between criterion groups as defined by the Emotion
Regulation Observations, r(68) = 6.63, p < .001, and the ERC
composite score (t = 17.62, p < .001; see Table 6).
 914 SHIELDS AND CICCHETTI
Table 6
Results oft Tests Evaluating Differences Between Criterion
Groups' Scores on the Emotion Regulation Q-Scale
Well-
regulated Dysregulated
Criterion emotion
measure M SD M SD t creativity, self-reliance, self-worth, and resourcefulness.
Lability/Negativity 65.10 4.80 41.01 6.81 18.20***
Regulation 63.67 5.78 43.75 8.54 -12.66***
Observations 55.81 8.34 42.64 10.81 6.63***
***/> < .001.
This article is intended solely for the personal use of the individual user and is not to be disseminated broadly.
This document is copyrighted by the American Psychological Association or one of its allied publishers.
Group Differences: Maltreatment Status
In addition, t tests evidenced significant differences between
maltreated and nonmaltreated children. As predicted, maltreated
children (M = 48.77) were more likely than were their nonmal-
treated peers (M = 52.44) to evidence emotion dysregulation
(t = 2.51, p < .05).
Developmental Differences in Emotion Regulation
The next set of analyses evaluated whether the new emotion
Q-scale would tap developmental differences in children's emo-
tion regulation. First, the correlation between children's age and
their Emotion Regulation Q-Scale scores was examined and
found to be not significant (r = .01, ns). This finding was
corroborated by median-split analyses in that no differences in
children's Q-scale scores were found for older versus younger
children. In addition, no age differences in children's ego resil-
iency scores were found.
Discussion
In Study 2, three sets of analyses were conducted to evaluate
the Emotion Regulation Q-Scale, and the pattern of results
strongly supported the construct validity of this new measure.
First, convergent validity was demonstrated in MTMM analyses,
evidencing substantial correlations between the new scale and
established measures of emotion regulation. That is, the Q-scale
was a strong predictor of emotional lability and negativity, sad
and negative mood states, and a tendency toward positive mood
and adaptive regulation, as assessed by independent observers'
and counselors' reports. On the whole, lesser correlations were
obtained between the emotion Q-scale and assessments of auton-
omy, suggesting that the Emotion Regulation Q-Scale captures
affective regulation and dysregulation rather than general adap-
tation related to optimal early development.
The specificity of this new measure was further underscored
by MTMM comparisons supporting its discriminant validity in
relation to ego resiliency. The possibility that this new scale
would overlap with or be subsumed by the higher order construct
of ego resiliency was suggested by commonalities across con-
struct definitions and shared items between the two Q-scales.
Ego resiliency is conceptualized as the aspect of children's func-
tioning that promotes a tolerable level of anxiety, an appropriate
degree of impulse control, and flexible adaptation in response
to changing situational demands. This notion of flexibility fos-
tering a relative homeostasis also is central to the construct of
emotion regulation as operationalized by the Q-sort. However,
the extreme items on the criterion resiliency sort also capture
more global aspects of children's functioning that may be asso-
ciated with, but are not central to, emotion regulation, including
Perhaps because these items also capture processes central to
autonomy (agency and self-determination), MTMM analyses
indicated that the Q-scale assessment of autonomy was indis-
criminable from that of ego resiliency. However, the Emotion
Regulation Q-Scale was a stronger predictor of three indexes
of emotion dysregulation than was the Ego Resiliency Q-Scale.
Although no differences were evident in the two scales' ability
to predict regulation and positive mood, the Ego Resiliency Q-
Scale was much less sensitive to indexes of negative mood,
emotional lability, and inflexible and maladaptive affective dis-
plays. This finding supports the specificity of the Emotion Regu-
lation Q-Scale and suggests that it is more sensitive to a broader
range of regulatory processes, including indicators of maladap-
tation and dysregulation. Therefore, the Emotion Regulation Q-
Scale makes a substantial contribution to existing Q-sort assess-
ments by providing a means by which to assess processes lead-
ing to both regulation and dysregulation in children.
Confirmatory factor analyses also suggested that this new
measure captured a distinct construct: Regulation as assessed by
the Emotion Regulation Q-Scale was statistically discriminable
from other key variables assessed through the Q-sort methodol-
ogy. That is, emotion regulation emerged as separable from the
potentially related (and confounded) constructs of autonomy,
neuroticism, and agreeableness. It was not possible to further
evaluate ego resiliency in this analysis, as no secondary assess-
ment of resiliency was available for pairing in the confirmatory
factor analysis. However, because autonomy overlapped sub-
stantially with ego resiliency, this finding supports by proxy
the assertion that this new assessment of emotion regulation is
discriminable from similar measures of ego resiliency.
As predicted, the Emotion Regulation Q-Scale also differenti-
ated between well-regulated and dysregulated groups as well as
between maltreated and nonmaltreated children. In addition, the
Q-scale appears to be equally informative with younger and
older children in the 6- to 12-year age range. Although these
findings suggest that the Q-scale may be an effective research
tool, analyses exploring its validity were limited to school-age
children who were at enhanced risk for negative developmental
outcomes. Therefore, exploring the validity and utility of this
measure among more normative school-age populations, as well
as among preschoolers, would be an important step. In addition,
continuing to evaluate the use of this measure among other
groups of children at heightened risk for dysregulation also is
critical.
Nevertheless, the Q-scale promises to be an important compo-
nent of assessment batteries targeting emotion regulation among
school-age children. In making a different type of methodology
available to emotion researchers, this new assessment increases
the diversity of extant measures. Strengths of the Q-sort method-
ology include its minimization of response bias and its use
across a wide age range. Furthermore, in generating comprehen-
sive profiles of individual personality functioning, this methodol-
 CRTTERION Q-SCALES
ogy facilitates evaluations of the target construct relative to other
salient personality processes. (Thus, although Q-scales enhance
the specificity of criterion sorts, raters should use the full 100-
item CCQ deck to preserve the unique benefits of the Q-sort
methodology.) Finally, this new Q-scale should enable research-
ers to explore questions regarding emotional development in
extant data sets that contain the CCQ. Therefore, the Emotion
Regulation Q-Scale promises to make a unique and important
contribution to investigations of regulation beyond the early
years of life.
References
This article is intended solely for the personal use of the individual user and is not to be disseminated broadly.
Achenbach, T. M. (1991). Manual for the Teacher's Report Form and
This document is copyrighted by the American Psychological Association or one of its allied publishers.
1991 Profile. Burlington: University of Vermont, Department of
Psychiatry.
Alessandri, S. M. (1991). Play and social behavior in maltreated pre-
schoolers. Development and Psychopathology, 3, 191-205.
Anderson, J. C , & Gerbing, D. W. (1988). Structural equation modeling
in practice: A review and recommended two-step approach. Psycho-
logical Bulletin, 103, 411-423.
Barnett, D., Manly, J. T, & Cicchetti, D. (1993). Denning child mal-
treatment: The interface between policy and research. In D. Cic-
chetti & S. L. Toth (Eds.), Advances in applied developmental psy-
chology: Vol. 8. Child abuse, child development and social policy
(pp. 7 - 7 3 ) . Norwood, NJ: Ablex.
Bentler, P. M. (1990). Comparative fit indices in structural models. Psy-
chological Bulletin, 107, 238-246.
Bentler, P. M. (1992). On the fit of models to covariances and methodol-
ogy to the Bulletin. Psychological Bulletin, 112, 400-404.
Block, J. (1961). The Q-sort method in personality assessment and
psychiatric research. Palo Alto, CA: Consulting Psychologists Press.
Block, J., & Block, J. H. (1980a). The California Child Q-set. Palo
Alto, CA: Consulting Psychologists Press. (Original work published
1969)
Block, J., & Block, J. H. (1980b). The role of ego-control and ego-
resiliency in the organization of behavior. In W. A. Collins (Ed.),
Development of cognition, affect, and social relations. The Minnesota
Symposia on Child Psychology (Vol. 13, pp. 39-101). Hillsdale, NJ:
Erlbaum.
Campbell, D. T, & Fiske, D. W. (1959). Convergent and discriminant
validation by the multitrait-multimethod matrix. Psychological Bulle-
tin, 56, 81-105.
Caspi, A., & Moffitt, T. (1995). The continuity of maladaptive behavior:
From description to understanding in the study of antisocial behavior.
In D. Cicchetti & D. J. Cohen (Eds.), Developmental psychopathol-
ogy: Volume 2. Risk, disorder, and adaptation (pp. 472-511). New
York: Wiley.
Cicchetti, D. (1990). The organization and coherence of socioemotional,
cognitive and representational development: Illustrations through a
developmental psychopathology perspective on Down syndrome and
child maltreatment. In R. Thompson (Ed.), Nebraska Symposium on
Motivation: Vol. 36. Socioemotional development (pp. 259—366).
Lincoln: University of Nebraska Press.
Cicchetti, D. (1993). Developmental psychopathology: Reactions, re-
flections, projections. Developmental Review, 13, 471—502.
Cicchetti, D., Ganiban, J., & Barnett, D. (1991). Contributions from
the study of high risk populations to understanding the development
of emotion regulation. In J. Garber & K. Dodge (Eds.), The develop-
ment of emotion regulation and dysregulation (pp. 15—49). New
\brk: Cambridge University Press.
Cicchetti, D., & Manly, J. T. (1990). A personal perspective on conduct-
ing research with maltreating families: Problems and solutions. In E.
915
Brody & I. Sigel (Eds.), Family research: volume 2. Families at risk
(pp. 87-133). Hillsdale, NJ: Erlbaum.
Cicchetti, D., & Rizley, R. (1981). Developmental perspectives on the
etiology, intergenerational transmission, and sequelae of child mal-
treatment. In R. Rizley & D. Cicchetti (Eds.), Developmental perspec-
tives on child maltreatment (pp. 32-59). San Francisco: Jossey-Bass.
Cicchetti, D., & Schneider-Rosen, K. (1986). An organizational ap-
proach to childhood depression. In M. Rutter, C. Izard, & P. Read
(Eds.), Depression in young people, clinical and developmental per-
spectives, (pp. 71-134). New "&rk: Guilford Press.
Cicchetti, D., & Sroufe, L. A. (1978). An organizational view of affect:
Illustration from the study of Down's syndrome infants. In M.
Lewis & L. Rosenblum (Eds.), The development of affect (pp. 309-
350). New York: Plenum Press.
Cohen, J., & Cohen, P. (1983). Applied multiple regression/correlation
analysis for the behavioral sciences. Hillsdale, NJ: Erlbaum.
Cummings, E. M., Hennessy, K. D., Rabideau, G. J., & Cicchetti, D.
(1994). Responses of physically abused boys to interadult anger in-
volving their mothers. Development and Psychopathalogy, 6, 31—41.
Deci, E. L., &Ryan, R. M. (1985). Intrinsic motivation and self-deter-
mination in human behavior. New %rk: Plenum Press.
Eisenberg, R , & Fabes, R. A. (1992). Emotion, regulation, and the
development of social competence. In M. S. Clark (Ed.), Review of
personality and social psychology: Vol. 14. Emotion and social behav-
ior (pp. 119-150). Newbury Park, CA: Sage.
Eisenberg, N., Fabes, R. A., Guthrie, I. K., Murphy, B. C , Maszk, P.,
Holmgren, R., & Suh, K. (1996). The relations of regulation and
emotionality to problem behavior in elementary school children. De-
velopment and Psychopathology, 8, 141 — 162.
Ekman, P., & Friesen, W. V. (1975). Unmasking the face: A guide to
recognizing emotions from facial cues. Palo Alto, CA: Consulting
Psychologists Press.
Erickson, M. E, Egeland, B., & Pianta, R. (1989). The effects of mal-
treatment on the development of young children. In D. Cicchetti &
V. Carlson (Eds.), Child maltreatment: Theory and research on the
causes and consequences of child abuse and neglect (pp. 647-684).
New York: Cambridge University Press.
Fox, N. A. (Ed.). (1994). The development of emotion regulation: Bio-
logical and behavioral considerations. Monographs of the Society for
Research in Child Development, 5 9 ( 2 - 3 , Serial No. 240).
Gianino, A., & Tronick, E. Z. (1988). The mutual regulation model:
The infant's self and interactive regulation and coping and defensive
capacities. In T. M. Field, P. M. McCabe, & N. Schneiderman (Eds.),
Stress and coping across development (pp. 4 7 - 6 8 ) . Hillsdale, NJ:
Erlbaum.
Gnepp, J. (1989). Children's use of personal information to understand
other people's feelings. In C. Saarni & P. L. Harris (Eds.), Children's
understanding of emotion (pp. 151-177). New Ifork: Cambridge Uni-
versity Press.
Gnepp, J., & Hess, D. L. R. (1986). Children's understanding of verbal
and facial display rules. Developmental Psychology, 22, 103-108.
Greenbaum, P. E., Dedrick, R. R, Prange, M. E., & Friedman, R. M.
(1994). Parent, teacher, and child ratings of problem behaviors of
youngsters with serious emotional disturbances. Psychological Assess-
ment, 6, 141-148.
Harris, P. L. (1989). Children and emotion: The development of psycho-
logical understanding. Oxford, England: Blackwell.
Hart, J., Gunnar, M., & Cicchetti, D. (1996). Altered neuroendocrine
activity in maltreated children related to symptoms of depression.
Development and Psychopathology, 8, 201-214.
Harter, S., & Buddin, B. J. (1987). Children's understanding of the
simultaneity of two emotions: A five-stage developmental acquisition
sequence. Developmental Psychology, 23, 288-299.
Hennessy, K. D., Rabideau, G. J,, Cicchetti, D., & Cummings, E. M.
 916 SHIELDS AND CICCHETTI
(1994). Responses of physically abused and nonabused children to
different forms of interadult anger. Child Development, 65, 815-828.
Howes, C , & Eldredge, R. (1985). Responses of abused, neglected,
and nonmaltreated children to the behaviors of their peers. Journal
of Applied Developmental Psychology, 6, 261-270.
Izard, C. (Ed.). (1982). Measuring emotions in infants and children.
New \brk: Cambridge University Press.
Jackson, D. N. (1969). Multimethod factor analysis in the evaluation
of convergent and discriminant validity. Psychological Bulletin, 72,
30-49.
John, O. P., Caspi, A., Robins, R. W., Moffitt, T. E., & Stouthamer-
Loeber, M. (1994). The "Little Five": Exploring the nomological
network of the Five-Factor Model of personality in adolescent boys.
Child Development, 65, 160-178.
Kagan, J. (1971). Change and continuity in infancy. New York: Wiley.
This article is intended solely for the personal use of the individual user and is not to be disseminated broadly.
Kaufman, J. (1991). Depressive disorders in maltreated children. Jour-
This document is copyrighted by the American Psychological Association or one of its allied publishers.
nal of the American Academy of Child and Adolescent Psychiatry,
30, 257-265.
Kazdin, A., Moser, J., Colbus, D., & Bell, R. (1985). Depressive symp-
toms among physically abused and psychiatrically disturbed children.
Journal of Abnormal Psychology, 94, 298-307.
Kenny, D. A., & Kashy, D. A. (1992). Analysis of the multitrait-
multimethod matrix by confirmatory factor analysis. Psychological
Bulletin, 112, 165-172.
Kopp, C. B. (1989). Regulation of distress and negative emotions: A
developmental view. Developmental Psychology, 25, 343-354.
Lewis, M., & Haviland, J. M. (Eds.). (1993). Handbook of emotions.
New \brk: Guilford Press.
Main, M., & George, C. (1985). Responses of abused and disadvantaged
toddlers to distress in agemates: A study in the day care setting.
Developmental Psychology, 21, 407—412.
Marsh, H. W. (1989). Confirmatory factor analyses of multitrait-
multimethod data: Many problems and a few solutions. Applied Psy-
chological Measurement, 13, 335-361.
Marsh, H. W., & Bailey, M. (1991). Confirmatory factor analysis of
multitrait-multimethod data: A comparison of alternative models. Ap-
plied Psychological Measurement, 15, 47-70.
McCoy, C. L., & Masters, J. C. (1985). The development of children's
strategies for the social control of emotions. Child Development, 56,
1214-1222.
McCrae, R. R., & John, 0. P. (1992). An introduction to the Five-Factor
Model and its applications. Journal of Personality, 60, 175-215.
Pekarik, E. G., Prinz, J. P., Liebert, D. E., Weintraub, S., & Neale, J. M.
(1976). The Pupil Evaluation Inventory: A sociometric technique
for assessing children's social behavior. Journal of Abnormal Child
Psychology, 4, 83-97.
Pianta, R. C. (1992). The Student-Teacher Relationship Scale. Char-
lottes ville: University of Virginia.
Pianta, R., & Steinberg, L. (1992). Teacher-child relationships and the
process of adjusting to school. New directions for child development:
Beyond the parent: The role of other adults in children's lives (pp.
61-80). San Francisco: Jossey-Bass.
Pollak, S., Cicchetti, D., Klorman, R., & Brumaghim, J. (in press).
Cognitive brain event-related potentials and emotion processing in
maltreated children. Child Development.
Rogosch, F , Cicchetti, D., & Aber, J. L. (1995). The role of child
maltreatment in early deviations in cognitive and affective processing
abilities and later peer relationship problems. Development and Psy-
chopathology, 7, 591-609.
Ryan, R. M. (1993). Agency and organization: Intrinsic motivation,
autonomy and the self in psychological development. In J. Jacobs
(Ed.), Nebraska Symposium on Motivation: Developmental perspec-
tives on motivation (Vol. 40, pp. 1-56). Lincoln: University of Ne-
braska Press.
Ryan, R. M., Deci, E. L., & Grolnick, W. S. (1995). Autonomy, relat-
edness and the self: Their relation to development and psychopathol-
ogy. In D. Cicchetti & D. J. Cohen (Eds.), Developmentalpsychopa-
thology: Vol. 1. Theory and methods (pp. 618-655). N e w a r k : Wiley.
Saarni, C. (1979). Children's understanding of display rules for expres-
sive behavior. Developmental Psychology, 15, 424-429.
Saarni, C. (1989). Children's understanding of strategic control of emo-
tional expression in social transactions. In C. Saarni & P. L. Harris
(Eds.), Children's understanding of emotion (pp. 181-208). New
\fork: Cambridge University Press.
Saarni, C. (1990). Emotional competence: How emotions and relation-
ships become integrated. In R. A. Thompson (Ed.), Nebraska Sympo-
sium on Motivation: Vol. 36. Socioemotional development (pp. 115-
182). Lincoln: University of Nebraska Press.
Saarni, C , & Harris, P. L. (1989). Children's understanding of emotion.
New %rk: Cambridge University Press.
Scherer, K. (1979). Acoustic concomitants of emotional dimensions:
Judging affect from synthesized tone segments. In S. Weitz (Ed.),
Nonverbal communication: Readings with commentary (pp. 105—
111). New \fork: Oxford University Press.
Schmitt, N., & Stults, D. M. (1986). Methodology review: Analysis of
multitrait-multimethod matrices. Applied Psychological Measure-
ment, 10, 1-22.
Shields, A. M., & Cicchetti, D. (1995, March). The development of an
emotion regulation assessment battery: Reliability and validity among
at-risk grade-school children. Poster session presented at the biennial
meeting of the Society for Research in Child Development, Indianapo-
lis, IN.
Shields, A. M., Cicchetti, D., & Ryan, R. M. (1994). The development
of emotional and behavioral self-regulation and social competence
among maltreated school-age children. Development and Psychopa-
thology, 6, 57-75.
Sroufe, L. A. (1983). Infant-caregiver attachment and patterns of mal-
adaptation in preschool: The roots of maladaptation and competence.
In M. Perlmutter (Ed.), Minnesota Symposia on Child Psychology:
Vol. 16. Development and policy concerning children with special
needs (pp. 41-83). Hillsdale, NJ: Erlbaum.
Sroufe, L. A. (1990). An organizational perspective on the self. In D.
Cicchetti & M. Beeghly (Eds.), The self in transition: Infancy to
childhood (pp. 281-307). Chicago: University of Chicago Press.
Thompson, R. A. (1994). Emotion regulation: A theme in search of a
definition. In N. A. Fox (Ed.), The development of emotion regulation:
Biological and behavioral considerations: Monographs of the Society
for Research in Child Development, 5 9 ( 2 - 3 , Serial No. 240), pp.
25-52.
Toth, S. L., Manly, J., & Cicchetti, D. (1992). Child maltreatment and
vulnerability to depression. Development and Psychopathology, 4,
97-112.
Underwood, M. K., Coie, J. D., & Herbsman, C. R. (1992). Display
rules for anger and aggression in school-age children. Child Develop-
ment, 63, 366-380.
Waters, E., Noyes, D. M., Vaughn, B. E., & Ricks, M. (1985). Q-Sort
definitions of social competence and self-esteem: Discriminant valid-
ity of related constructs in theory and data. Developmental Psychol-
ogy, 21, 508-522.
Wintre, M. G., & Vallence, D. D. (1994). A developmental sequence
in the comprehension of emotions: Intensity, multiple emotions, and
valence. Developmental Psychology, 30, 509-514.
Received February 13, 1996
Revision received May 2, 1997
Accepted May 2, 1997 •