Early Childhood Research Quarterly 17 (2002) 338–355

Multidimensional assessment of emotional and behavioral

adjustment problems of low-income preschool children:
development and initial validation
Megan Noone Lutz, John Fantuzzo∗ , Paul McDermott
Graduate School of Education, University of Pennsylvania, 3700 Walnut Street,
Philadelphia, PA 19104-6216, USA

Abstract
The Adjustment Scales for Preschool Intervention (ASPI) was developed and tested for use in
preschool programs serving low-income children. The ASPI is a measure of emotional and behav-
ioral adjustment problems observed within routine classroom situations. Principal components analyses
revealed five reliable behavioral dimensions: Aggressive, Withdrawn-Low Energy, Socially Reticent,
Oppositional, and Inattentive/Hyperactive and two higher-order dimensions: Overactive and Underac-
tive problem behaviors. Concurrent criterion validity of these dimensions was supported by multivariate
indicators of peer social competence and classroom behavior problems. Age and gender analyses in-
dicated that boys showed higher levels of overactive behavior than girls and that younger preschool
children evidenced more underactive and inattentive behavior than older preschool children. Situational
analyses indicated that situations requiring more initiation and self-regulation were associated with more
problematic behaviors.
© 2002 Elsevier Science Inc. All rights reserved.

Keywords: Multidimensional assessment; Preschool children; Behavioral adjustment

1. Introduction

Emotional and behavioral difficulties displayed by young children represent a growing con-
cern for early childhood professionals. Recent studies exploring the prevalence of these types
of problems in preschool populations have documented that larger numbers of children are

∗
Corresponding author. Tel.: +1-215-898-4790.
E-mail address: johnf@gse.upenn.edu (J. Fantuzzo).

0885-2006/02/$ – see front matter © 2002 Elsevier Science Inc. All rights reserved.
PII: S 0 8 8 5 - 2 0 0 6 ( 0 2 ) 0 0 1 6 8 - 0
 M.N. Lutz et al. / Early Childhood Research Quarterly 17 (2002) 338–355 339

displaying some type of emotional or behavioral maladjustment than previously believed

(Lavigne et al., 1996). Based on a comprehensive review of current literature, Campbell (1995)
concluded that approximately 10–15% of preschool children exhibit mild to moderate behavior
problems. Other studies suggest even higher prevalence rates, ranging from 20 to 37% (Pianta
& Caldwell, 1990; Rose, Rose, & Feldman, 1989). Moreover, young children with behavior
problems are more likely to demonstrate other concurrent difficulties, such as learning prob-
lems, problematic peer relations, and insecure attachments (Pianta & Caldwell, 1990; Vaughn,
Hogan, Lancelotta, & Shapiro, 1992).
In order to reach the largest number of children most in need of intervention, early iden-
tification efforts should be focused in areas with the highest concentration of children most
vulnerable for developing adjustment problems. There is a large body of empirical evidence
documenting that children from low-income, highly stressed families are at increased risk of de-
veloping emotional and behavioral difficulties (Dodge, Pettit, & Bates, 1994; Fantuzzo, Grim,
Mordell, McDermott, & Miller, 2001; Lavigne et al., 1996; Patterson, Kupersmidt, & Vaden,
1990). Therefore, it is critical that any measurement or assessment technology used for early
identification be appropriate for use with this most vulnerable population of children. Head
Start, the largest federally funded program for vulnerable young children, revised its Perfor-
mance Standards in 1997 to emphasize the importance of early detection of preschool children
with psychological adjustment problems (USDHHS, 1997). The Special Needs Section of the
Head Start Performance Standards indicates that children with emotional and behavioral dis-
orders are eligible for intensive special needs services. These standards define emotional and
behavioral disorders as:
A condition in which a child’s behavioral or emotional responses (to natural classroom situa-
tions) are so different from those of the generally accepted age-appropriate norms of children
with the same ethnic or cultural background as to result in significant impairment in social
relationships, self-care, educational progress, or classroom behavior. (Performance Standard,
1308.8a)

This definition makes clear that early childhood educators need sound, developmentally
and culturally appropriate measures that reflect principles and standards for early childhood
assessment (NAEYC, 1998; USDHHS, 1997). These measures need to describe children’s
responses to actual classroom situations that are observed by early childhood educators. More-
over, children’s difficulties should be determined empirically by comparing their classroom
behaviors to the norms of children who are ethnically and culturally similar.
The most commonly used assessment instruments for identifying children’s emotional and
behavioral problems are psychiatric symptom checklists completed by adults about children
(McDermott, 1993; Merrell, 1998). Though these checklists are efficient ways of screening
large numbers of children, they have been criticized as inappropriate for preschool children
because they identify children’s behavior out of context (Fantuzzo & Mohr, 1999; Sherrod,
1999). These scales present lists of symptoms and infer psychopathology from the number
of related symptoms observed and/or the observer’s general impression of the frequency or
intensity of symptoms [e.g., the Child Behavior Checklist (Achenbach, 1991) is a popular
example]. Such information does not clarify whether the symptomatic behavior is isolated to
specific circumstances (e.g., as when peers are physically aggressive or when authorities impose
340 M.N. Lutz et al. / Early Childhood Research Quarterly 17 (2002) 338–355

expectations for learning) or whether it is pervasive across varied circumstances (e.g., structured
teacher-directed or unstructured free play situations in the classroom). Lacking confirmation of
the contextual ethos for observed behavior, it is difficult to determine motivation and consider
remedial action.
Recent empirical studies bring into question the validity of asking early childhood educa-
tors to use checklists of psychiatric symptoms of mental disorders to describe low-income,
preschool children’s classroom behaviors (Fantuzzo et al., 2001; Lutz, 1999). Rather than
use these psychiatric symptom checklists, early childhood educators seriously underreport the
incidence of emotional and behavioral problems in the classroom. Teachers report that they
are reluctant to use these measures because they would like to avoid stigmatizing children
with labels that are not associated with needed classroom-based services (Lutz, 1999; Mallory
& Kerns, 1988; Piotrowski, Collins, Knitzer, & Robinson, 1994; Sinclair, Del-Homme, &
Gonzalez, 1993).
An alternative approach to child assessment has been demonstrated by McDermott (1994)
in the development of the Adjustment Scales for Children and Adolescents (ASCA). ASCA
was designed to assess youth 5–17-years-old and begins with behavioral content readily ob-
servable over a 1- or 2-month period within the classroom environments, but which requires no
inferences regarding children’s internal mediating psychological processes (such as thoughts
or feelings). It describes behavioral phenomena in the parlance of classroom teachers rather
than psychopathologists and, based on teachers’ recommendations, provides unique forms for
female and male children, respectively, each featuring appropriate gender referents (e.g., “she,”
“her,” etc. vs. “he,” “his”) to help teachers focus on a specific child rather than language struc-
ture. ASCA also applies a format that differs from all other empirical observation scales. The
format defines problem behavior through its multisituational occurrence. Rather than depend
upon respondent teachers’ estimates of frequency or severity for more global behaviors, the
instrument requires that teachers indicate which specific behaviors typify given contexts and
the verity and severity of a type of problem (i.e., construct or dimension of a scale) are drawn
from its pervasiveness in multiple contexts.
To illustrate with respect to the behavioral phenotype called oppositional defiance, man-
ifestation of the problem is confirmed empirically only when oppositional-type behavior is
emergent in 6.5 out of 11 possible classroom situations (e.g., when the teacher asks for help
with jobs, or when the child is expected to answer questions, or to show respect for the teacher,
or to play fairly in formal play, or get along with smaller/weaker children, or to work by oneself,
or when reacting to teacher attention or to correction, or when standing in line, or engaging
in unorganized play). The multisituational view toward problem behavior is consistent with
the work of Horn, Wagner, and Ialongo (1989) favoring the diagnostic import of pervasive
behavior and with the view that behavior emergent only in isolated contexts is far more likely
to be random or reactive than it is pathognomonic.
The ASCA method of assessing emotional and behavioral difficulties in school adjustment
is a promising method to consider for use in early childhood education. This method provides
a means for educators to report on observable behaviors of children by familiar knowledgeable
educators across multiple familiar situations. It determines children’s difficulties nomotheti-
cally affording educators with a capacity to use children’s ethnic and cultural group as a norm
reference.
 M.N. Lutz et al. / Early Childhood Research Quarterly 17 (2002) 338–355 341

The primary purpose of the present study was to investigate whether the ASCA method
could be used to produce a reliable and valid, multisituational measure of emotional and be-
havioral problems that is appropriate for use in early childhood educational programs serving
low-income preschool children. Partnerships were formed with urban, Head Start educators to
identify routine classroom situations and relevant behaviors that teachers observe in those sit-
uations. Principal components analyses were conducted to determine if psychologically mean-
ingful dimensions of emotional and behavioral problems emerged and to assess whether they
evidenced concurrent validity with previously validated constructs of preschool, classroom dif-
ficulties. A secondary purpose was to determine if empirically derived dimensions of problem
behaviors varied as a function of children’s age and gender. This research was a first step of a
research program to validate a problem behavior rating scale for low-income preschool children
derived from children’s behaviors in response to the demands of actual classroom situations.

2. Method

2.1. Participants

The 829 urban, Head Start children participated in this study. They ranged in age from 3.2
to 6.2 years (M = 4.6, SD = 0.6). The sample of children was divided almost equally between
boys and girls (48% male and 52% female). The racial distribution of these children was 75%
African American, 12% White, 8% Hispanic, 3% Asian, and 2% other. The participation rate
for this study was 97%.
These participants were enrolled in 46 Head Start classrooms geographically dispersed
throughout the city. The teachers in these classrooms volunteered to participate in this study.
The program’s six Education Coordinators, representing the six geographic clusters in the
program, recruited volunteer teachers. Participation was distributed across clusters as follows:
10, 9, 9, 9, 5, and 4 classrooms across the six clusters, respectively. Of the participating teachers,
90% were female. There was a large range of teacher experience, as 35% of the teachers had
been teaching less than 10 years, 27% between 10 and 20 years, and 38% over 20 years. Most
teachers had received either a Bachelors degree (44%) or Masters degree (47.9%). The majority
of teachers (66%) were White, and 31% were African American.
The sample was drawn from a large urban Head Start program in the Northeast. This program
serves nearly 5,000 families in over 260 full-day classrooms. The majority of the children
enrolled in these Head Start centers was African American (78%), while the remaining children
were White (8.5%), Hispanic (8.5%), Asian (3%), other (2%). Family income in the program
matches national proportions for urban Head Start programs, with annual income for 94% of
the families below $12,000. The age distribution of children in the program is 29, 68, and 2%
for 3-, 4-, and 5- or 6-year-olds, respectively.

2.2. Measures

2.2.1. Conners Teacher Rating Scale (CTRS-28; Conners, 1990)

The CTRS-28 was utilized to explore the concurrent validity of the Adjustment Scales
for Preschool Intervention (ASPI). This measure consists of 28 items rated on a
342 M.N. Lutz et al. / Early Childhood Research Quarterly 17 (2002) 338–355

four-point scale, and includes three subscales: Conduct Problems, Hyperactivity, and
Inattentive-Passive subscales. Normative data for the CTRS-28 are based on a study of
383 children aged 3–17 years (Goyette, Conners, & Ulrich, 1978). Internal consistency
estimates are not reported for the short form; however, an average reliability of .94 is re-
ported for the longer form of the same measure (CTRS-39). Interrater reliability of fac-
tor scores on the abbreviated form produced significant correlations ranging from .46 to
.57. Studies documenting evidence of predictive, discriminant, and concurrent validity have
been conducted and are reported in the manual for the Conners Rating Scales
(Conners, 1990). Overall these studies have shown that the CTRS related to other indi-
cators of externalizing or overactive behavior problems and that it has been effective in
predicting future hyperactivity and discriminating among numerous types of adjustment
problems.
Although the CTRS-28 is a symptom checklist and does not share the ASCA’s ability to as-
sess problem behaviors in context, it is a currently available standardized measure of preschool
problem behaviors. Moreover, the CTRS-28 was used in validity studies of the current ASCA,
and using consistent validation strategies will help in establishing the psychometric integrity
of the preschool version. Finally, the CTRS-28 has been tested with a sample of 640 children in
an urban Head Start Program (Fantuzzo et al., 2001). Exploratory factor analysis has revealed
three reliable and valid factors: Conduct/Adjustment, Hyperactivity, and Inattention/Passivity
(alpha’s .95, .92, and .67, respectively).

2.2.2. Penn Interactive Peer Play Scale

The PIPPS (Fantuzzo, Coolahan, Mendez, McDermott, & Sutton-Smith, 1998) is a teacher-
report instrument that was used to determine the amount and quality of children’s peer so-
cial interactions in the classroom as evidenced in the important context of play. The PIPPS
was developed in collaboration with the School District of Philadelphia Prekindergarten Head
Start teachers and parents specifically for use with low-income, urban Head Start children.
The 32 items of the PIPPS describe common play behaviors of preschool children that either
promote or hinder positive interactions with peers. Factor analysis of this measure yielded
three dimensions: Play Interaction, Play Disruption, and Play Disconnection. The PIPPS
scale as a whole demonstrates high internal consistency, based on Cronbach’s alpha (r =
.93). Internal consistency for each of the three dimensions is similarly high (r = .92, .91,
and .89 for the Play Interaction, Play Disruption, and Play Disconnection factors,
respectively).
Evidence has established concurrent validity for the PIPPS, using peer sociometrics, ob-
servations of peer play, language skills, and teacher-report indicators of conduct, learning
behaviors, temperament, and social skills (Coolahan, Fantuzzo, Mendez, & McDermott, 2000;
Fantuzzo et al., 1998; Mendez, 1999). Specifically, the PIPPS Play Interaction scale is associ-
ated with peer acceptance, observations of interactive peer play, high teacher reported social
skills, active engagement in classroom learning activities, strong adaptability and approacha-
bility, and good receptive vocabulary. Play Disconnection is linked with low peer acceptance,
lack of involvement with peers, inattention, passivity, and lack of motivation. Finally, Play
Disruption relates to low self-control, conduct problems, hyperactivity, and engagement in
solitary play.
 M.N. Lutz et al. / Early Childhood Research Quarterly 17 (2002) 338–355 343

2.3. Procedures

2.3.1. Scale development

The process of adapting the ASCA was conducted in accord with Gaskins’ (1994) approach
for culturally valid measurement development. The first stage in this approach was to develop
content in partnership with representatives of the target population of respondents. In order to
adapt the content of the ASCA for use in a preschool context, the original measure was reviewed
by a group of 10 Head Start professionals. These included teachers, special needs coordinators,
and administrators, all of whom had training and experience in early childhood education. After
explaining the rationale for developing the ASPI, each professional was asked to identify any
items or contextual categories that should be deleted or changed to fit the preschool setting.
Additionally, they were asked to provide any salient preschool-specific contexts or items they
felt should be added to the measure.
Based on feedback received from these professionals, several modifications were made to
the ASCA to create the ASPI. Of the 29 school-based situations that frame items on the ASCA,
four were considered inappropriate for application to preschool. These were “How does he/she
care for his/her books, etc.?” “Has he/she engaged in delinquent types of activities outside the
school?” “Has he/she engaged in objectionable or illicit activities within the school?” and “Does
he/she attend school regularly?” Additionally, the situation “Does he/she play fairly?” was
deleted, but two items associated with this question were incorporated into another situation:
“How does the child take part in games with other children?” These two items were: “Overly
rough with other children” and “Tries to cheat.” The remaining 24 situations were retained on
the ASPI, although many were reworded to reflect more appropriately the target age group. For
example, the ASCA situation “How does he/she sit at his/her desk?” was changed to “How
does the child sit during class-wide, teacher directed activities (e.g., story time, circle time,
etc.)?” and “Can he/she work by him/herself?” on the ASCA became “How does this child get
involved in classroom activities?”
Modifications were made at the item level as well. These were primarily minor rewording
such as changing “Takes things from others’ desks or lockers without permission” to “Takes
things belonging to other children without permission.” Other items were considered inappro-
priate and were deleted altogether. For example, in the situation “How does this child handle
conflicts with other children?” the behavioral description “Often used as a scapegoat or sub-
ject of ridicule by others” was not included on the ASPI since the majority of reviewers felt
it was not applicable to preschool children. Finally, a number of items considered descrip-
tive of preschool children’s behavior were added to the measure. These included “Clings to
the teacher,” “Causes a disturbance when not chosen for jobs,” “Insists on sitting next to the
teacher,” “Avoids eye contact with the teacher,” “Cries when corrected,” “Reacts negatively
to getting dirty,” “Bothers other children during teacher directed activities,” “Refuses to take
turns in games,” “Moves quickly from one activity to the other,” “Needs teacher assistance in
free play,” “Plays by self in free play,” “Refuses to share toys or other materials with peers,”
“Seeks teacher assistance during conflicts,” and “Uses physically aggressive means to handle
problems.” The final ASPI version included 24 contextual situations that framed 144 behav-
ioral descriptors. Like on the ASCA, descriptions of positive behavior were included among
the 144 items on the ASPI to allow teachers to identify children’s behavioral strengths as well
344 M.N. Lutz et al. / Early Childhood Research Quarterly 17 (2002) 338–355

as their needs. Two sample contextual situations and corresponding behavioral descriptions
are presented in Appendix A.
When the ASPI was presented to Head Start personnel for final review, it was received
enthusiastically. Head Start representatives indicated that the content of the measure used
terminology that was understandable to teachers and reflective of behaviors that could be
observed in a Head Start classroom. In addition, Head Start staff corroborated that the ASPI
format was nonoffensive in that it did not characterize children according to an illness-based
perspective. For these reasons, it was felt that teachers would complete the ASPI with minimal
reactivity, resulting in more valid assessment of children’s behaviors.

2.3.2. Data collection

Assessment packets were distributed to each participating teacher after an orientation and
training session. Each packet included an ASPI, a PIPPS, and a demographic questionnaire for
each child in the classroom. Additionally, a CTRS-28 was included for one boy and one girl
in each room. After a 5-week interval, Education Coordinators from Head Start and research
assistants collected assessment packets from all teachers.

2.4. Data analyses

A series of exploratory principal components analyses were conducted to resolve latent

structures in the ASPI. Retained factors were rotated to simple structure, first using orthogonal
(varimax and equamax) rotations. Resultant solutions were evaluated against the following
multiple criteria: (a) eigenvalues of unrotated factors must be greater than 1; (b) the ideal factor
solution must meet the constraints of Cattell’s scree Test; (c) each component must retain at least
five items with salient loadings (where loadings of at least .40 are considered salient), (d) each
retained component must demonstrate adequate internal consistency, with alpha coefficients
≥.70; (e) the final solution must minimize intercorrelation of retained unit-weighted factors;
(f) proposed factors must be psychologically meaningful. The final orthogonal solution was
then subjected to a series of oblique (promax) rotations, at variable levels of power. The best
promax solution (that which results in a peak of the hyperplane count and minimal interfactor
correlations), was compared to the final orthogonal solution. Retention of either the orthogonal
or oblique solution was based on maximum item coverage and minimum interfactor correlations
and items with salient loadings on more than one component.
Three additional procedures were conducted to verify the integrity of the proposed solution.
First, the specificity and error variance for each factor of the final solution were compared.
Specificity, or the reliable and unique variance associated with each component, was calculated
by subtracting the communality for the component from its alpha coefficient. Components
were considered valuable only if the amount of unique and reliable variance exceeded variance
associated with error, or one minus alpha. Next, oblique, multiple-group, principal-components
cluster analysis was conducted as a confirmatory procedure to further verify the final solution.
In this procedure, items were assigned to mutually exclusive clusters based on the final factor
solution. Confirmation of the solution occurs if the proportion of an item’s variance predicted
by items in its own cluster is greater than that predicted by items in the next closest cluster.
Finally, invariance and generalizability of the solution was conducted by obtaining solutions
 M.N. Lutz et al. / Early Childhood Research Quarterly 17 (2002) 338–355 345

for 10 random subsamples of the standardization sample, as well as samples derived on the
basis of child age and gender. Wrigley–Neuhause coefficients of congruence were calculated to
determine the degree of similarity of solutions for each subsample to that for the standardization
sample.1
To explore the concurrent validity of the new measure, two additional teacher-report mea-
sures of children’s behavior were collected. The CTRS-28 was completed on 102 children to
provide an estimate of: conduct problems, hyperactivity, and inattentive-passive. Additionally,
teacher ratings on the PIPPS were collected on 758 children, providing information about three
dimensions of children’s peer play behaviors: play interaction, play disconnection, and play
disruption. Canonical redundancy analysis was conducted to determine the degree of overlap
between the ASPI and the CTRS-28, as well as between the ASPI and the PIPPS. Addition-
ally, canonical analysis was used to explore the nature of the relationships between the ASPI
and the two other indicators of child functioning. Both convergent and divergent validity was
investigated based on hypothesized relationships. ASPI factors were hypothesized to relate
positively to the corresponding problem behavior factors on the CTRS-28. ASPI dimensions
were expected to show a moderate positive relationship with play disconnection and disruption
and a negative relationship with positive play interaction. These hypotheses were based on the
empirically demonstrated relationship between the constructs of problem behaviors and social
competence.
Significant age and gender differences in ASPI dimensions were investigated using a re-
peated measures analysis of variance. A three-way Analysis of Variance was conducted where
the first factor represented child age (older vs. younger children based on median split of 4.65
years) and the second factor child gender. The third was held as a repeated measure representing
the ASPI dimensions. The purpose of this analysis strategy was to examine solely interaction
effects of age group or gender with the ASPI dimensions.

3. Results

3.1. Construct validity of the ASPI

The first step in evaluating the construct validity of the ASPI was to inspect the prevalence
of each behavior in the sample of Head Start children. Of the 144 items, 122 items indicated
possibly problematic behaviors (less than 50% prevalence) and 22 were descriptive of higher
prevalence behavior (greater than 50% prevalence). The less than 50% prevalence distinction
was used to define problem dimensions. This distinction in the national representative study of
the ASCA (McDermott & Schaefer, 1996). Specifically, developmentally and demographically
generalizable syndromes must be based on both rare (essentially prevalence levels <2.1%) and
common (prevalence levels >2.1 and <50%) problem behaviors. This is because prevalence
levels vary significantly as a function of developmental level, sex, ethnicity, and socioeconomic
status (McDermott & Schaefer, 1996). Because the intended purpose of the ASPI was to identify
children demonstrating emotional or behavioral difficulties in the preschool classroom, the 22
higher prevalence items were excluded from further factor analyses in the content validation
of the scale.2
346 M.N. Lutz et al. / Early Childhood Research Quarterly 17 (2002) 338–355

The correlation matrix for the 122 low prevalence items was evaluated. According to
Bartlett’s chi-square criteria, the likelihood of an identity matrix was rejected (p < .0001),
allowing for investigation into latent structure. To do this, data for the sample of 829 Head Start
children were subjected to exploratory principal components analyses using both orthogonal
and oblique rotation methods. Orthogonal solutions were obtained using both varimax and
equamax rotation methods for one to six components. These solutions were evaluated based
on the five central criteria for retention and the degree to which they maximized exclusive item
coverage (the number of items with loadings ≥.40 on one and only one component). Specif-
ically, the eigenvalue of each unrotated component was greater than one, and the solution
met the constraints of Cattell’s scree Test. Furthermore, each of the five retained components
demonstrated adequate internal consistency, with Cronbach’s alpha coefficients of .92, .85, .79,
.78, and .79, respectively. Intercorrelation of retained unit-weighted component scores ranged
from −.06 to .62. The equamax, five-component structure was considered the most useful
solution that satisfied the criteria. A total of 73 items was retained on the final structure. The
five dimensions were: Aggressive, Withdrawn-Low Energy, Socially Reticent, Oppositional,
and Inattentive/Hyperactive.
The Aggressive dimension was comprised of 22 descriptions of rough or aggressive be-
haviors (e.g., Starts fights in free play; Has made unprovoked attacks on other children). The
second component of 18 items, Withdrawn-Low Energy described behaviors related to discon-
nection from activity, primarily due to low energy levels (e.g., Sluggish, apathetic in games;
Listless, seems unmotivated). The 12 items that loaded saliently on the third component, So-
cially Reticent, characterized shy or hesitant behaviors (e.g., Needs encouragement to join
in games; Friendly but shy with teacher). The fourth dimension, Oppositional, had 11 items
related to moody or controlling behavior (e.g., Helps with jobs unless in a bad mood; Cries
when corrected; Tries to dominate peers). The last component, Inattentive/Hyperactive was
comprised of 10 items describing inattention, impulsivity, and high levels of activity (e.g., An-
swers questions before taking time to think; Constantly restless). The specific items comprising
these behavioral dimensions are displayed in Table 1, along with their factor loadings.
To further test the integrity of the equamax solution, additional procedures were conducted.
First, the specificity and error variance for each component of the final solution were compared.
Specificity, or the reliable and unique variance associated with each component, is defined as the
component’s alpha coefficient minus its communality. Each dimension’s specificity exceeded
its error variance (see Table 2). Second, oblique, multiple-group, principal-components cluster
analysis was conducted as a confirmatory measure. In this procedure, the 73 items that loaded
on the five behavioral dimensions were assigned to five mutually exclusive clusters. Items
were allowed to migrate iteratively to clusters that better explained item variance. All items
remained with their hypothesized clusters, and the average item variance accounted for by its
own cluster was 2.3 times greater than the next best alternative cluster (see Table 1).
To determine whether the five behavioral dimensions obtained from the principal compo-
nents analysis conformed to the two broad types of behavior problems frequently discussed in
the literature (Achenbach, Edelbrock, & Howell, 1987), a second-order common factor anal-
ysis was conducted. Unit weighted raw scores were calculated for each dimension, and these
scores were submitted to common factor analysis with squared multiple correlations as the ini-
tial communality estimates. Two reliable factors were extracted. Three dimensions (Aggressive,
 M.N. Lutz et al. / Early Childhood Research Quarterly 17 (2002) 338–355 347

Oppositional, and Inattentive/Hyperactive) loaded appreciably on the first factor, Overactive.

The second factor, Underactive, was comprised of the other two dimensions (Withdrawn-Low
Energy and Socially Reticent) (see Table 2 for factor structure and interdimension correlations).
Finally, the replicability and generalizability of the five dimensions were tested with random
subsets of the standardization sample, as well as samples derived on the basis of child age and
child gender. Wrigley–Neuhaus coefficients of congruence (Harman, 1976) were calculated
to determine the degree of similarity of solutions for each subsample to that for the standard-
ization sample. Invariance, or the results of comparing models from random subgroups to
that for the norm sample was statistically supported. For each of the 10 random subgroups,
loadings were highly similar to those for the corresponding clusters of items derived from the
norm group with coefficients of congruence of .99 and were dissimilar when compared to the
other four, noncorresponding clusters (mean coefficients ranging from .14 to .42). Likewise,
coefficients of congruence comparing models for boys, girls, younger children, and older chil-
dren to the standardization sample indicated generalizability of the five dimensions to each of
these subgroups. Coefficients of congruence relating corresponding clusters ranged from .93
to .99, and the mean coefficients for noncorresponding clusters ranged from .11 to .43. Finally,
internal consistency coefficients were calculated for each age and gender subgroup, demon-
strating adequate reliability of each dimension for each group (alpha coefficients ranging from
.77 to .93).

3.2. Convergent and divergent validity

Canonical variance and redundancy analyses were conducted to determine the nature and
degree of overlap between the ASPI and the PIPPS and the CTRS-28. The canonical structures
for the ASPI with the PIPPS and the CTRS are presented in Tables 3 and 4, respectively.

3.2.1. Relationship between ASPI and PIPPS

Canonical variance analysis comparing standard scores for the five ASPI dimensions to the
three PIPPS factors indicated significant overlap between the two measures [Wilks’ lambda =
.21, F (15, 2071) = 104.94, p < .0001], and yielded three significant canonical correlations
(canonical R = .76, .69, and .26, respectively, p < .0001). The first variate pair consisted
of high positive canonical loadings for Aggressive, Oppositional, Inattentive/Hyperactive, and
Play Disruption. The two strongest loadings were for Aggressive and Play Disruption (.94 and
.98, respectively). Based on the pattern of relationships among these variables, the first variate
pair was named Interpersonal Disruption. The two variates in this pair had 57% common or
shared variance, as determined by the squared canonical correlation. The second variate pair,
Underactive Disconnection, was characterized by high positive loadings for Withdrawn-Low
Energy, Socially Reticent, and Play Disconnection. Additionally, Play Interaction had a strong
negative relationship with this variate pair. The two variates comprising this pair had 47%
variance in common. The last variate pair was named Oppositional Disconnection due to
moderate positive loadings for Oppositional and Play Disconnection. The proportion of shared
variance between these two variates was considerably lower, with only 7% in common.
To determine the magnitude of the multivariate relationship between the five ASPI dimen-
sions and the three PIPPS factors, the canonical redundancy between the two sets of variables
348 M.N. Lutz et al. / Early Childhood Research Quarterly 17 (2002) 338–355

was examined. As a whole, the ASPI dimensions explained slightly more of the variance in
the PIPPS (44%) than the PIPPS factors explained in the ASPI (38%).

3.2.2. Relationship between ASPI and CTRS

Canonical variance analysis relating standard scores for the five ASPI dimensions to the three
CTRS factors also revealed significant overlap between the two measures [Wilks’ lambda =
.15, F (15, 260) = 16.89, p < .0001] and produced three significant canonical correlations.
The first two canonical correlations (canonical R = .84 and .62, respectively, p < .0001) were
reflective of the relationship between the two sets of variables. However, the third canonical
correlation (canonical R = .37, p < .01) was driven primarily by ASPI variance as no CTRS
factors loaded appreciably on the appropriate variate. Therefore, this variate set was not relevant
to the issue of convergent and divergent validity.
The first variate pair was defined by strong positive loadings for the three ASPI dimensions
that comprise the Overactivity Scale with the strength of the relationship in the order of Ag-
gressive, Oppositional, and Inattentive/Hyperactive. All three CTRS-28 factors had positive
relationships to the variate. Conduct Problems and Hyperactivity had very strong canonical
loadings (>.90), but the loading for Inattention was moderate. This pattern of relationships led
to the name Overactive Conduct Problems. There was a large degree of overlap between the
two variates in the pair as there was 71% common variance. The second pair of variates which
shared 39% of the variance, was characterized by moderate positive relationships among ASPI
dimensions Withdrawn-Low Energy and Socially Reticent and the CTRS-28 Inattention factor.
This variate pair was named Underactive Inattention.
Canonical redundancy analysis for the relationship between the five APSI dimensions and
the three CTRS factors indicated that the ASPI explained 60% of the variance in the CTRS,
while the CTRS predicted 43% of ASPI variance.

3.3. Age and gender differences

Descriptive statistics on ASPI dimensions across age and gender are presented in Table 5.
A 2 (age) × 2 (gender) × 5 (ASPI dimension) Repeated Measures ANOVA yielded two
significant interactions: age × ASPI dimension, F (4, 3300) = 13.86, p < .00001, and
gender × ASPI dimension, F (4, 3300) = 10.86, p < .00001. The age × gender × ASPI
dimension interaction was not significant (F < 1). According to Tukey’s HSD, younger
children demonstrated significantly higher levels of Withdrawn-Low Energy, Socially Reticent,
and Inattentive/Hyperactive behaviors than did older children. There were no significant age
differences for Aggressive and Oppositional behaviors. Significant gender differences were
evident for Aggressive and Inattentive/Hyperactive behaviors, with boys showing higher levels
of both dimensions than girls.

3.4. Situational difficulties across children

Another unique feature of the ASPI is that it allows you to look at the routine early childhood
classroom situations that are associated with the most frequent manifestations of problematic
behaviors across children. To compute these frequencies, a count of situations was conducted
 M.N. Lutz et al. / Early Childhood Research Quarterly 17 (2002) 338–355 349

for each child. If a child had one or more behavior difficulty in a situation, the situation was
counted. Table 6 presents the percent of children who manifested difficulties in each of the
possible situations. The extent of difficulties evident in situations ranged from 10 to 49%
of the children. The situations that included a relatively high percent of children were situ-
ations that are considered more developmentally challenging for preschool children. These
classroom situations require children to use more complex child initiation and self-regulation
abilities in child- or teacher-directed group activities (i.e., free play, teacher directed group
activities, compliance with classroom jobs/responsibilities, and games). Situations with rela-
tively fewer children manifesting difficulties were ones with more specific one-to-one interac-
tions child–teacher interactions (i.e., greeting the teacher, reacting to correction, and seeking
teacher’s help).

4. Discussion

The primary objective of the present study was to address the need for a multidimensional
classroom measure of emotional and behavioral problems with clear content validity for early
childhood educators serving low-income children. Specifically, the ASPI was designed in
partnership with teachers to describe observable behaviors across classroom situations in a
large, urban Head Start program.
Exploratory factor analyses revealed five distinct behavioral dimensions measured by the
ASPI and two higher-order factors: Overactive and Underactive. The three Overactive dimen-
sions are Aggressive, Oppositional, and Inattentive/Hyperactive; the two Underactive dimen-
sions are Withdrawn-Low Energy, and Socially Reticent. These dimensions reflect patterns of
maladaptive responses to the natural challenges posed across regular and familiar classroom
situations. The concurrent criterion validity of these five dimensions was supported through
inspection of the relationships among these dimensions and two other multivariate indicators
of children’s peer social competence and classroom behavior problems.
The convergent and divergent validity of the three Overactive ASPI dimensions revealed
multivariate relationships that have been documented by previous empirical inquiry (Fantuzzo
et al., in press). As hypothesized the three overactive problem factors of the CTRS-28 related
significantly to the overactive dimensions of the ASPI. Similarly, the first canonical variate pair
relating the ASPI dimensions to the PIPPS factors was defined by salient loadings for the same
three overactive ASPI dimensions (Aggressive, Oppositional, and Inattentive/Hyperactive),
along with Play Disruption, the only PIPPS factor to describe overactive types of behaviors.
Furthermore, divergent validity was supported by loadings close to zero for factors theoretically
unrelated to underactivity: Withdrawn-Low Energy, Socially Reticent, Play Disconnection, and
Play Interaction. These findings are in accord with two recent studies conducted in urban Head
Start classroom settings (Coolahan, 1997; Mendez, 1999).
The ASPI dimensions related to underactivity are unique in the literature. No other teacher-
report scale of preschool problem behavior has documented a distinction between withdrawal
related to lethargy and withdrawal related to reticence. Some measures feature only one char-
acterization of internalizing behavior, such as “Anxious-Fearful” (Behar Problem Behavior
Questionnaire, Behar & Stringfield, 1974) or the “Inattentive/Passive” factor on the CTRS-28.
350 M.N. Lutz et al. / Early Childhood Research Quarterly 17 (2002) 338–355

Other measures have demonstrated empirical distinctions between two internalizing patterns
of behavior, but these have primarily reflected the difference between social withdrawal and
anxious or somatic behaviors, a distinction qualitatively different from that provided by the
ASPI (Achenbach et al., 1987; Merrell, 1995; Reynolds & Kamphaus, 1992).
Convergent and divergent validity was also demonstrated with the Underactive ASPI dimen-
sions. Canonical variance analysis between the ASPI and the PIPPS revealed strong positive
relationships among Withdrawn-Low Energy, Socially Reticent, and Play Disconnection and
a negative relationship with Play Interaction. This variate represents both convergent and di-
vergent validity for the two ASPI dimensions. Convergent validity is reflected in the positive
relationship of both Underactive ASPI dimensions to Play Disconnection. Previous empir-
ical work has documented the social disconnection of children with underactive behaviors
(Cillessen, Van Ijzendoorn, Van Lieshout, & Hartup, 1992; Coolahan, 1997). In fact, isolation
from peers and failure to play with others are common items associated with internalizing
factors on behavior checklists (Achenbach et al., 1987; Kohn, 1988). Furthermore, the strong
negative canonical loading of Play Interaction is an important indicator of divergent validity
for the two Underactive ASPI constructs. The literature regarding the social competence of
children with underactive problems has not only documented that these children are withdrawn
and disconnected from their peers, but also that they lack the requisite skills to maintain positive
peer interactions (Merrell, 1995; Olson & Rosenblum, 1998; Vaughn et al., 1992).
Relative to the knowledge base regarding overactive problem behaviors, preschool children’s
problems of underactivity have been neglected in the empirical literature (Campbell, 1995;
Merrell, 1994). These behaviors are comparatively difficult to identify and do not demand im-
mediate adult attention (Campbell, 1995). Therefore, continued and careful research is needed
to elucidate the various manifestations of underactive behavior. The unique picture of underac-
tivity offered by the ASPI is an important tool that can be used to complete our understanding
of these types of problems.
A secondary purpose of this study was to determine if ASPI dimensions varied as a function
of age and gender and to identify the classroom situations most frequently associated with
problem behaviors. The patterns of age and gender differences found on the ASPI dimen-
sions provide additional support for the validity of the dimensions, as these patterns comport
with previous research with Head Start children. In the current research, boys showed higher
levels of Aggressive behaviors and Inattentive/Hyperactive behaviors than girls. Furthermore,
younger children scored higher on the Withdrawn-Low Energy, Socially Reticent, and Inat-
tentive/Hyperactive dimensions than did older children. Coolahan et al. (2000) found that
preschool boys in Head Start evidenced more disruptive peer play behavior than girls did and
that younger preschool children were more withdrawn and disconnected in play. Similarly, in
a study investigating classroom problem behaviors in Head Start, Fantuzzo et al. (2001) found
that boys scored significantly higher than girls did on classroom ratings of hyperactivity and
that relatively younger Head Start children displayed higher levels of hyperactive and pas-
sive behaviors than their older classmates. These findings also comport with research outside
of Head Start indicating relationships between male sex and problem behaviors in preschool
children (Hart, DeWolf, Wozniak, & Burts, 1992; Pianta & Caldwell, 1990).
A distinctive feature of the ASPI is distinctive that it defines problematic behaviors by record-
ing the emergence of similar types of problems in distinctly different classroom situations. In
 M.N. Lutz et al. / Early Childhood Research Quarterly 17 (2002) 338–355 351

the present study, a descriptive look at the situations associated with more difficulties revealed
that situations requiring children to exhibit more initiation and self-regulation in a group con-
text were associated with more emotional and behavioral problem behaviors. These findings
correctly identified some of the most developmentally challenging classroom situations for
preschool children. Free play, circle-time instruction, and completing routine classroom “job”
responsibilities were situations in which children exhibited more problematic behaviors. This
is understandable since all of these situations require children to focus their attention in a so-
cial milieu and engage in instrumental social interactions. These findings draw attention to the
significance of considering how the transactions between children’s developmental capacities
and the nature of classrooms demands relate to observed emotional and behavioral difficul-
ties. Future research is necessary to understand these relationships and their implications for
curriculum and classroom management.

4.1. Implications for future research practice

Distinguishing features of the ASPI include its grounding in natural classroom situations
and its sensitivity to the target population for which it was developed. Earlier research has doc-
umented reluctance on the part of early childhood educators to use symptom checklists to de-
scribe the classroom behavior of preschool children (Mallory & Kerns, 1988). The ASPI method
involved co-constructing situations and behaviors within situations with early childhood edu-
cators, who were working with a vulnerable population of preschool children (Gaskins, 1994).
We focused exclusively on items including observable child behaviors in routine classroom
situations. These were familiar classroom situations and the child behaviors associated with
these situations were not perceived of as uncommon or bizarre. Moreover, any proposed item
that teachers were reluctant to use was adapted or removed. This co-construction reduced er-
ror related to teachers’ reactivity to offensive or alarming content. The discrete dimensions
that were empirically derived from the teacher reports on the ASPI indicate that the educators
understood the significance of the items embedded in the classroom situations and that they
were able to make fine distinctions between problem behaviors, which matched the latest early
childhood research literature on problem behaviors.
The above ASPI validation efforts constitute a first step in a comprehensive research program
to create better behavioral measures for vulnerable preschool children. As such, the present
findings are qualified by the assessment of validity with one method, at a single point in time,
and with one large urban sample. Future studies are needed to further establish the validity of
the ASPI. First, the convergent and divergent validity that we found in the current study were
obtained by using teacher-report rating scales. Using other methods or sources for gathering
information about children’s classroom behaviors would strengthen the concurrent validity of
the ASPI. Such alternative measures may include direct behavioral observations, or peer so-
ciometrics, and ratings from parents who have the opportunity to observe children’s behaviors
within classroom situations. Second, investigations of the temporal stability, interrater reliabil-
ity, and predictive validity of the ASPI are needed. It is recommended that Teacher Assistants
be used to test the interrater reliability. Typically Teacher Assistants in Head Start programs are
former Head Start parents, who are ethnically and culturally similar to the children. Using them
to assess interrater reliability would test if the ASPI descriptions of behaviors and situations
352 M.N. Lutz et al. / Early Childhood Research Quarterly 17 (2002) 338–355

have a common mean for raters who are culturally and ethnically different (Fantuzzo et al.,
1998). The hypothesis that ASPI dimensions predict a significant proportion of the variance
associated with future school maladjustment should be tested in future research. Finally, it is
important to assess the generalizability of ASPI dimensions with other vulnerable populations
of preschool children. These would include other culturally or linguistically distinct groups
of low-income urban preschool children (e.g., Hispanic or Asian) and low-income children
living in rural geographic locations. Here it would be important to follow the same partnership
process that was used in this study to ensure that behaviors and situations are observable, com-
mon, and have comparable meaning to the early childhood educators serving different groups
of children.
Early childhood educators could use ASPI data to contribute to an early identification and
early intervention mental health system. Such a system would include systematic screening pro-
cedures, including the ASPI, to assess the emotional and behavioral adjustment of all children
in a preschool program. Screening information could then be used to prioritize children’s needs
based on the degree of their demonstrated difficulties. Subsequently, comprehensive investiga-
tions could be conducted: (a) to explore the specific circumstances in the classroom context for
children, who were at least one standard deviation from the mean on ASPI dimensions, (b) to
assess other domains of child functioning (e.g., language, cognition, and motor development),
and (c) to evaluate the influences of other relevant contexts (i.e., home and neighborhood)
on emotional and behavioral problems. These more intensive assessments would inform the
generation of contextually relevant intervention strategies. In addition, ASPI data collected at
multiple points in time could inform program-wide planning and policies for children with
mental health needs. ASPI data on problematic situations, across teachers and children, could
be used to identify staff training opportunities or guide teachers’ sharing of effective strategies
used in these challenging classroom situations. Overall, the value of the ASPI method is that
it provides a means of obtaining important information about the emotional and behavioral
needs of low-income preschool children. Furthermore, practitioners and researchers who use
the ASPI are reminded of the developmental challenges that preschool children face every day
in adapting to real classroom situations.

Notes
1. The coefficient of congruence can be applied to any multivariate technique, which pro-
duces a matrix composed of elements which range from −1.00 to +1.00 (Guadagnoli &
Velicer, 1991). Given the choice of two such matrices (cluster loadings and component
loadings), cluster structures were used as the point of comparison because they consid-
ered only those items loading saliently on one of the retained components in the final
solution. This issue was particularly relevant to the present study due to the large number
(49) of nonloading or multiple-loading items. Variance associated with these trivial items
was excluded. In this process, the oblique, multiple-group, principal-components cluster
analysis was repeated for each subgroup. Resultant structures for the subgroups were
then compared to the structure obtained from the entire standardization sample.
2. Thus viable dimensions of problem behaviors that are generalizable across diverse groups
of children must rest on behaviors that vary in prevalence. This concept is commensurate
 M.N. Lutz et al. / Early Childhood Research Quarterly 17 (2002) 338–355 353

with the basic measurement principle that scale discrimination or test discrimination for
external criteria must rely on a broad band of item-total rs (.20–.80) that will enable the
different component items to relate differentially to a variety of external criteria (in this
case, manifestations of problematic child behavior) even though the component items
relate relatively homogeneously among themselves (internal consistency).

Acknowledgments

This research project was supported by a Head Start Research Scholars grant to the first
author and a Head Start/University Partnerships grant to the second author from the U.S.
Department of Health and Human Services. A special thanks goes to our collaborators at
Prekindergarten Head Start in the School District of Philadelphia: Director Jennifer Plumer
Davis, Assistant Director David Silbermann, Special Needs Coordinator Samuel Mosca, Edu-
cation Coordinators, and the following exemplary Head Start teachers and staff: Denise Ellis,
Catherine Link, Mary Ellen Hence, Judith McDowell, and Anice Dickerson-Watters.

Appendix A. ASPI directions and two sample contextual situations with

corresponding behavioral descriptions

Directions: After each question, there are several descriptions of behaviors children may
display. Fill in the circle beside any description that fits the child’s behavior over the past
month. For each question, mark as many descriptions as apply to the child. If no descriptions
apply, then do not fill in any circles for that question.

A.1. Sample 1: How does this child cope with new learning tasks?

◦ Has a happy-go-lucky attitude to every problem

◦ Charges in without taking time to think or follow instructions
◦ Approaches new tasks with caution, but tries
◦ Won’t even attempt it if he/she senses a difficulty
◦ Likes the challenge of something difficult
◦ Cannot work up the energy to face anything new

A.2. Sample 2: How is this child at free play/individual choice?

◦ Engages in appropriate activities

◦ Rather loud but not disruptive
◦ Is too timid to join in
◦ Disturbs others’ fun
◦ Wants to dominate and have his/her own way
◦ Starts fights and rough play
◦ Needs teacher assistance to get involved
354 M.N. Lutz et al. / Early Childhood Research Quarterly 17 (2002) 338–355

◦ Usually plays by him/herself

◦ Moves quickly from one activity to another

References

Achenbach, T. M. (1991). Child behavior checklist. Burlington, VT: University of Vermont.

Achenbach, T. M., Edelbrock, C., & Howell, C. T. (1987). Empirically based assessment of the behavioral/emotional
problems of 2- and 3-year-old children. Journal of Abnormal Child Psychology, 15, 629–650.
Behar, L., & Stringfield, S. (1974). The behavior rating scale for the preschool child. Developmental Psychology,
10, 601–610.
Campbell, S. B. (1995). Behavior problems in preschool children: A review of recent research. Journal of Child
Psychology and Psychiatry and Allied Disciplines, 36, 113–149.
Cillessen, A., Van Ijzendoorn, H. W., Van Lieshout, C., & Hartup, W. (1992). Heterogeneity among peer-rejected
boys: Subtypes and stabilities. Child Development, 63, 893–905.
Coolahan, K. C. (1997). Empirical relationships among parenting styles, determinants of parenting, and children’s
school readiness in urban Head Start families. Unpublished doctoral dissertation, University of Pennsylvania,
Philadelphia.
Coolahan, K., Fantuzzo, J., Mendez, J., & McDermott, P. (2000). Interactive peer play and readiness to learn:
Relationships between play competencies and classroom learning behaviors and conduct. Journal of Educational
Psychology, 92, 458–465.
Conners, C. K. (1990). Conners rating scales manual. Toronto, Ontario, Canada: Multi-Health Systems.
Dodge, K. A., Pettit, G. S., & Bates, J. E. (1994). Socialization mediators of the relation between socioeconomic
status and child conduct problems. Child Development, 65, 649–665.
Fantuzzo, J., Coolahan, K., Mendez, J., McDermott, P., & Sutton-Smith, B. (1998). Contextually-relevant validation
of peer play constructs with African American Head Start children: Penn Interactive Peer Play Scale. Early
Childhood Research Quarterly, 13, 411–431.
Fantuzzo, J. F., Grim, S., Mordell, M., McDermott, P. A., Miller, L., & Coolahan, K. (2001). A multivariate analysis
of the revised Connor’s Teacher Rating Scale with low-income, urban preschool children. Journal of Abnormal
Child Psychology, 29(2), 141–152.
Fantuzzo, J., & Mohr, W. (1999). Pursuit of wellness in Head Start: Making beneficial connections for children and
families. In D. Cicchetti, J. Rapapport, I. Sandler, & R. Weissberg (Eds.), The promotion of wellness in children
and adolescents. New York: Sage.
Gaskins, S. (1994). Integrating interpretive and quantitative methods in socialization research. Merrill-Palmer
Quarterly, 40, 313–333.
Goyette, C. H., Conners, C. K., & Ulrich, R. F. (1978). Normal data on revised Conners Parent and Teacher Rating
Scales. Journal of Abnormal Child Psychology, 6, 221–236.
Guadagnoli, E., & Velicer, W. (1991). A comparison of pattern matching indices. Multivariate Behavioral Research,
26, 323–343.
Hart, C. H., DeWolf, M., Wozniak, P., & Burts, D. C. (1992). Maternal and paternal disciplinary styles: Re-
lations with preschoolers’ playground behavioral orientations and peer status. Child Development, 63, 879–
892.
Harman, H. H. (1976). Modern factor analysis (3rd ed.). Chicago: University of Chicago Press.
Horn, W. F., Wagner, A. E., & Ialongo, N. (1989). Sex differences in school-aged children with pervasive attention
deficit hyperactivity disorder. Journal of Abnormal Child Psychology, 17, 109–125.
Kohn, M. (1988). Kohn Problem Checklist/Kohn Social Competence Scale. New York: Psychological Corporation.
Lavigne, J. V., Gibbons, R. D., Christoffel, K. K., Arend, R., Rosenbaum, D., & Binns, H. (1996). Prevalence rates
and correlates of psychiatric disorders among preschool children. Journal of the American Academy of Child
and Adolescent Psychiatry, 35, 204–214.
Lutz, M. N. (1999). Contextually relevant assessment of the emotional and behavioral adjustment of Head Start
children. Unpublished doctoral dissertation, University of Pennsylvania, Philadelphia.
 M.N. Lutz et al. / Early Childhood Research Quarterly 17 (2002) 338–355 355

Mallory, B. L., & Kerns, G. M. (1988). Consequences of categorical labeling of preschool children. Topics in Early
Childhood Special Education, 8, 39–50.
McDermott, P. A. (1993). National standardization of uniform multisituational measures of child and adolescent
behavior pathology. Psychological Assessment, 5, 413–424.
McDermott, P. A. (1994). National profiles in youth psychopathology: Manual of adjustment scales for children
and adolescents. Philadelphia: Edumetric and Clinical Science.
McDermott, P. A., & Schaefer, B. A. (1996). A demographic survey of rare and common problem behaviors among
American students. Journal of Clinical Child Psychology, 25, 352–362.
Mendez, J. L. (1999). Empirical relationships between developmental competencies and peer play behaviors
with African American Head Start children. Unpublished doctoral dissertation, University of Pennsylvania,
Philadelphia.
Merrell, K. W. (1994). Preschool and kindergarten behavior scales. Brandon, VT: Clinical Psychology Publishing.
Merrell, K. W. (1995). An investigation of the relationship between social skills and internalizing problems in early
childhood: Construct validity of the Preschool and Kindergarten Behavior Scales. Journal of Psychoeducational
Assessment, 13, 230–240.
Merrell, K. W. (1998). Assessing social skills and peer relations. In H. B. Vance (Ed.), Psychological assessment
of children: Best practices for school and clinical settings (2nd ed., pp. 246–276). New York: Wiley.
NAEYC (National Association for the Education of Young Children) & NAEC/SDE (National Association of Early
Childhood Specialists in State Departments of Education). (1998). Guidelines for appropriate curriculum content
and assessment in programs serving children ages three through eight (adopted November 1990). Washington,
DC: NAEYC.
Olson, S. L., & Rosenblum, K. (1998). Preschool antecedents of internalizing problems in children beginning
school: The role of social maladaptation. Early Education and Development, 9, 117–129.
Patterson, C. J., Kupersmidt, J. B., & Vaden, N. A. (1990). Income level, gender, ethnicity, and household compo-
sition as predictors of children’s school-based competence. Child Development, 61, 485–494.
Pianta, R. C., & Caldwell, C. B. (1990). Stability of externalizing symptoms from kindergarten to first grade and
factors related to instability. Development and Psychopathology, 2, 247–258.
Piotrowski, C. S., Collins, R. C., Knitzer, J., & Robinson, R. (1994). Strengthening mental health services in Head
Start. American Psychologist, 49, 133–139.
Reynolds, C. R., & Kamphaus, R. W. (1992). Behavior assessment system for children: Manual. Circle Pines, MN:
American Guidance Service, Inc.
Rose, S. L., Rose, S. A., & Feldman, J. F. (1989). Stability of behavior problems in very young children. Development
and Psychopathology, 1, 5–19.
Sherrod, L. R. (1999). Head Start and mental health: An argument for early screening and intervention. National
Head Start Association Dialogue, 2, 412–415.
Sinclair, E., Del-Homme, M., & Gonzalez, M. (1993). Systematic screening for preschool behavioral disorders.
Behavioral Disorders, 18, 177–188.
U.S. Department of Health and Human Services. (1997). Program performance standards for the operation of
Head Start programs by grantee and delegate agencies, 45 C.F.R. Part 1304, Federal Register, 61, 57186-57227.
Washington, DC: U.S. Government Printing Office.
Vaughn, S., Hogan, A., Lancelotta, G., & Shapiro, S. (1992). Subgroups of children with severe and mild behavior
problems: Social competence and reading achievement. Journal of Clinical Child Psychology, 21, 98–106.