Early Childhood Education Journal, Vol. 30, No.

4, Summer 2003 ( 2003)

Agreement and Stability of Teacher Rating Scales

for Assessing ADHD in Preschoolers

Sandra B. Loughran1,2

This study investigated the agreement and stability of 3 teacher rating Scales used to assess ADHD
in preschool children: the ADHD Rating Scale, the Child Attention Profile (CAP), and the Conners’
Teacher Rating Scale-28 (CTRS-28). A sample of suburban children (n = 60) was observed and
rated by their teachers and assistant teachers at preschool level (Time 1) and 4 years later at the
elementary school level (Time 2). Agreement among the rating scales and interrater agreement
between teacher and assistant teacher ratings yielded noticeably stronger correlations at Time 2
than at Time 1. Over the 4-year interval of the study, there was a significant change in the number
of children identified as potential ADHD risks. It is probable there were a high number of false-
positive indications in the preschool ADHD screenings. It is also possible that immature behavior
of preschool children may mimic ADHD behavior at the elementary school level.

KEY WORDS: ADHD; preschool; elementary; teacher rating scales.

INTRODUCTION early diagnosis. With the recent clarification of these

mandates (Part B of the Individuals with Disabilities Act
Attention Deficit/Hyperactivitiy Disorder (ADHD)
[IDEA]; Section 504 of the Rehabilitation Act of 1973)
has been recognized as a strong contributor to academic
regarding evaluation, there is an urgent need for reliable
underachievement and behavioral maladaptation (Gold-
educational assessments of children suspected of having
stein & Goldstein, 1993; Lerner, Lowenthal, & Lerner,
ADHD and a need to address these concerns as early
1995; Palfrey, Levine, Walker, & Sullivan, 1985). ADHD
as possible. Additionally, there has been an increasing
is a disorder that is marked by developmentally inappro-
interest in the assessment and treatment of very young
priate levels of inattention, impulsivity, and overactivity.
children since Public Law 99–457 mandated develop-
With a mean onset between 3 and 4 years of age (Ameri-
mental services for this age group (Shelton & Barkley,
can Psychiatric Association, 1994), there is a need to
1993).
clarify the behavioral criteria that put a preschool child
At the same time, excessive diagnosing and overpa-
at risk of this disorder.
thologizing in the assessment of ADHD in preschool
Given the legal mandates that allow ADHD chil-
children in our society continues to be a controversial
dren to be considered eligible for special services by
and ethical concern (American Psychiatric Association,
their school district when the diagnosis is accompanied
1994; Barkley, 1995). Many children between ages 3
by a documented developmental delay (Public Law 94–
and 5 years are inattentive, hyperactive, and impulsive
142), it is important to provide these children with an
at times as part of their normal development. However,
under certain conditions, these same behaviors are also
1
identified as behavioral descriptors of ADHD. The pos-
Elementary Education Department, Dowling College, Oakdale, Long
sibility of a misdiagnosis due to normal developmental
Island, NY.
2
Correspondence should be directed to Sandra B. Loughran, Dowling
variations is cause for concern.
College, Oakdale, Long Island, New York 11769; sandra.loughran A definitive set of criteria sensitive to the develop-
@eudoramail.com. mental level of preschool children is needed, but such

247
1082-3301/03/0600-0247/0  2003 Human Sciences Press, Inc.
248
criteria are lacking (Campbell, 1995). As a result, most
referrals for ADHD are made after a child’s entrance
into elementary school. A child who is truly ADHD is
almost certain to be ADHD at both preschool and at
elementary school level. The problem in identifying that
child in preschool is that the symptoms are masked by the
much wider range of acceptable behavior in preschool.
The change in setting from a preschool environment to a
more structured elementary school environment may re-
duce this masking effect and be a contributing factor to
the identification of true ADHD.
Teacher rating scales of behavior are considered to
be one of the most important components in the assess-
ment of ADHD and, as such, are among the most com-
monly used instruments for gathering school informa-
tion. These same rating scales also provide data for
developing an intervention plan for the individual child
who has been diagnosed with ADHD.
The questions that continue to plague the field and
that need further investigation are many. How are we
going to detect and adequately assess ADHD in pre-
school children? What instruments are going to be pre-
dictive of ADHD? Is there agreement among these in-
struments? Will the children who exhibit symptoms as
preschoolers continue to exhibit those same symptoms
at the elementary school level? In what ways, if any, is
early diagnosis helpful for the preschool population?
And, in what ways, if any, may misdiagnosis be harmful?
Because teacher rating scales are a primary source
of information, it is important to research their agree-
ment and stability over time when used with preschool
children. There is a continuing need for longitudinal
studies addressing the use of teacher rating scales as pre-
dictors of ADHD in preschool children.
Although most teacher rating scales that assess
ADHD are normed for school-aged children aged 5
years and older, the Conners’ Teacher Rating Scale
(CTRS; Goyette, Conners, & Ulrich, 1978) has been
normed for children as young as 3 years of age. Re-
search by Miller, Koplewicz, and Klein (1997) found
this instrument to be reliable with the preschool popula-
tion.
The present research compared the CTRS to two
other commonly used rating scales: The ADHD Rating
Scale (Du Paul, 1990) and the Child Attention Profile
(CAP), which was derived from the Child Behavior
Checklist-Teacher Report Form (Edelbrock & Achen-
bach, 1984). These comparisons serve two purposes.
First, agreement among several teacher rating scales
could prove greater reliability in assessing ADHD in
preschool children. Second, it is necessary to know if
rating scales used with preschool children will accu-
Loughran
rately predict the presence of ADHD later in childhood
as defined by DSM-IV (American Psychiatric Associa-
tion, 1994). The agreement among these scales with a
group of preschool children was examined over an ex-
tended period using ratings by the same teachers and
assistant teachers at preschool (Time 1) and again 4
years later in elementary school (Time 2).
One of the most important factors in the use of be-
havioral ratings scales is the raters’perceptions of what
is appropriate behavior in children. It was considered
that the same rater would be unlikely to have changed
significantly in that respect as opposed to the differences
that might be found in two different raters.
METHODOLOGY
This study examined the agreement and the stabil-
ity of three teacher rating scales administered at two dif-
ferent times with a group of children when they were in
preschool (Time 1) and again, 4 years later, when they
were in elementary school (Time 2). As indicated, the
same teachers and assistant teachers did the ratings at
both Time 1 and Time 2.
At Time 1, each child in the preschool was ob-
served and rated by his/her teacher and assistant teacher
using the three teacher rating scales. During a reunion
at Time 2, the children returned to the preschool, the
same environment in which they were rated at Time 1,
and were assigned to the peer group with whom they
had attended preschool. It is believed that the study bene-
fited from the greater control afforded through bringing
the same children together in a replicated environment.
An additional benefit was to reunite the participants in
an environment that closely approximated that in which
they had originally been rated.
PARTICIPANTS
The sample consisted of 60 children, 27 boys (47%)
and 33 girls (53%). The children were from a suburban,
upper-middle-class community where each of the chil-
dren had attended the same private preschool. The mean
age at Time 1 was 4.2 years and at Time 2, 8.9 years.
At Time 1, the children came from three different classes,
3-year-olds, 4-year-olds, and 5-year-olds.
There were three teachers and three assistant teach-
ers. The teachers had elementary education certification
with experience at both the preschool and elementary
school levels. The teachers had experience levels of 7,
10, and 30 years, respectively. The three assistant teach-
ers did not have professional training; however, they did
Assessing ADHD in Preschool
have prior experience working in various settings at both
elementary and preschool levels.
All teachers and assistant teachers had participated
in a staff development workshop on ADHD prior to
Time 1. The workshop addressed both assessment and
strategies for working with ADHD symptoms in the
classroom. They were also familiarized with the teacher
rating scales used in this research.
All of the teachers and assistant teachers engaged
in the study had uninterrupted employment over the 4-
year interim period at the preschool where the study was
conducted. During that time, none of them received any
special training in the field of ADHD. However, they
were in daily contact with a number of children who
evidenced ADHD symptoms, including some who were
formally diagnosed by outside agencies before leaving
the preschool.
At Time 1, the children ranged in age from 2.9 to
6.0 years. None of the children in this sample had been
diagnosed with ADHD prior to Time 1.
The ages of the children at Time 2 ranged from 7.6
to 10.9 years. At that time, the children were attending
public or private schools in the area, with the exception
of one child who was being schooled at home by a par-
ent. At Time 2, when the children were in grades 2
through 5, they were reunited with their same preschool
peer group. Prior to the reunion, two children had been
diagnosed with ADHD at elementary school. Neither
child was on medication nor receiving special services
at school. None of the teachers or assistant teachers were
informed of these diagnoses.
MEASURES
Three teacher rating scales were used in this study:
the ADHD Rating Scale (DuPaul, 1990), the CAP (Edel-
brock & Achenbach, 1984), and the CTRS-28 (Goyette
et al., 1978). The ADHD Rating Scale is a scale using
the 14 items of DSM-III-R for ADHD, which rates the
number of symptoms, a total score, an inattention/rest-
lessness score, and an impulsivity/hyperactivity score.
Observers rate the severity of symptoms on a 4-point
scale. The measures used in this study were the number
of symptoms presented in the ratings. For the older group,
the 6–12-year norms of 8 or more symptoms were used
as the cut-off score. While the recommended threshold
of 10 of 14 symptoms, which was not normed for pre-
schoolers but was recommended was used (Barkley,
1990).
The CAP is composed of 12 items taken from the
Child Behavior Checklist Teacher Report Form (Edel-
brock & Achenbach, 1984). Observers rate the severity
249
of these characteristics on a 3-point scale. This scale
also has a total score, an inattention score, and an over-
activity score. The total score was used for this study.
The normative cut-off point at the 93 percentile was the
threshold used. The only norms available for the CAP
were for 6–16-year age group. There was no differentia-
tion for preschool vs. elementary school ages.
The CTRS-28 is a 28-item questionnaire concerning
various types of child behavior problems and widely
used for clinical and research applications with children.
This scale is the only scale of the three scales normed
for use with 3-year-olds and with norms for the different
age groups in this study. It can be scored to yield four
factors: Conduct Problem, Hyperactivity, Inattention-
Passive, and Hyperactivity Index. Each factor is consid-
ered a scale and has been normed separately. The factors
for this study include the Hyperactivity Index, or total
score, the Inattentive-Passive factor, and the Hyperactiv-
ity factor. The descriptors of behavior are rated by the
observer on a 4-point scale. The Hyperactivity Index
was the scale used in this study, and the appropriate
norms were used for each group.
Written informed consent was obtained from each
child’s parents to participate in this study. Parents were
questioned as to any ADHD identification done by other
medical or educational professionals.
DATA COLLECTION PROCEDURES
The children were rated at Time 1 during the course
of normal attendance at the preschool. The raters were
cautioned against preconceived notions influencing the
rating of participants. In particular, at Time 2, they were
encouraged to rate behaviors as they observed them at
Time 2 only. The observations for Time 1 were made
over a longer period of time than observations had in
Time 2. It is believed that the benefits of having the
participants reunited in this unique set of circumstances
outweighed any limitation that might be associated with
a shorter period of observation. The Time 2 observations
were conducted over a one week period. During that
week, the children came in small groups (each with their
original class) and were observed for a single day. Each
day was 5 hours long.
Every effort was made to structure the reunion and
the day’s activities to afford the teachers and the assis-
tant teachers an opportunity to refamiliarize themselves
with their former students and to ensure the maximum
observation time. Age-appropriate activities were con-
ducted, including group-structured activities as well as
free-play and individual activities. The activities specifi-
cally addressed each question asked on the three teacher
250 Loughran

rating scales (e.g., a game with specific rules addressed Table I. t-Test Comparison of Scale Ratings Between Time 1
the questions on each of the scales pertaining to atten- and Time 2 by Teacher and by Assistant Teacher
tion as well as ability to get along with peers). Each Time 1 (N = 60) Time 2 (N = 60)
teacher and assistant teacher completed the teacher rat-
ing scales immediately following the reunion each day. Scale Ratings M SD M SD t p

Teacher
RESULTS ADHD 6.71 7.50 4.81 8.04 1.7 ns
CAP 4.52 5.1 3.07 4.69 2.3 .05
The results of comparing the performance of the CTRS-28 45.13 6.37 46.77 9.07 −1.5 ns
three rating scales are shown in Fig. 1. Although the Assistant Teacher
correlation among rating scales was higher at Time 2 ADHD 4.78 6.98 3.08 9.60 1.5 ns
CAP 3.37 5.11 1.80 3.53 2.7 .05
than at Time 1, the ratings were strongly correlated
CTRS-28 44.23 7.66 43.80 6.18 .5 ns
throughout the study with the CAP/CTRS-28 being the
highest (r = .95). Note. df = 59.
With respect to the stability of the individual rating
scales, the results of paired t tests between Time 1 and
Time 2 are shown in Table I. The t tests, when applied tween teachers and assistant teachers at Time 1 and
to the total population (N = 60) showed no significant Time 2 were: .61 vs. .75 for the ADHD Rating Scale,
changes in the ADHD Rating Scale and the CTRS-28 .62 vs. .80 for CAP, and .60 vs. .80 for CTRS-28.
between Time 1 and Time 2. CAP, however, did show Perhaps the most telling results are the number of
a significant change on both teacher ratings (t = 2.3, p = subjects identified by each rating scale as potentially
.05) and assistant teacher ratings (t = 2.7, p = .01). ADHD in preschool and 4 years later in elementary
Because both teachers and assistant teachers were school. Those results are shown in Fig. 3.
used in the study, a comparison was made between their There were nine children who met the criteria for
ratings for each of the rating scales at Time 1 and Time potential ADHD at Time 1 on one or more of the rating
2. The results of that comparison are shown in Fig. 2. scales by either teacher or assistant teacher, that is, at
Results of Pearson product-moment correlations be- Time 1, 15% of the sample was identified by at least

Fig. 1. Correlation between rating scales for teachers and assistant teachers at Time 1 and Time 2.
Assessing ADHD in Preschool 251

Fig. 2. Agreement between teachers and assistant teachers for different Rating scales at Time 1 and Time 2.

one rating. Significantly, only one child in that group
was later identified as a potential candidate for ADHD.
It is also interesting to note that one additional student
was identified at Time 2 who did not exceed the identifi-
cation threshold on the rating scales at Time 1. There
had been identification of ADHD for these two children
between Time 1 and Time 2 in their elementary schools,
although no special services were provided during that
interval. The parents of the child identified at Time 1
opted for further observation rather than early interven-
tion. The two children confirmed as meeting the ADHD
criteria represent 3% of the study sample, a figure in
close agreement with the DSM-IV estimate for preva-
lence of ADHD in the school population. Based on the
confirmation of the two subjects (by outside diagnosis),
the overall score for the rating process was; 1 correct, 1
missed, and 8 false positives at Time 1; 2 correct, 0
missed, and 0 false positives at Time 2.
DISCUSSION
The summary results shown in Fig. 3 might indi-
cate that CTRS-28 was the most conservative at Time 1
(least false positives) and was also the most consistent
over the 4-year interval. But the disparity between the
number of potential ADHD candidates indicated by the

Fig. 3. Summary of students meeting ADHD criteria on rating scales at Time 1 and Time 2.
252
different rating scales at Time 1 is not consistent with
the high degree of correlation between scales when ap-
plied to the total population.
An important factor that bears on the results is the
set of norms that were used in classifying a child as
potentially ADHD. As indicated earlier, only the CTRS-
28 had different norms for the two different age groups.
In the case of the ADHD Rating Scale, the appropriate
norms covering the age group at Time 2 were applied;
at Time 1, the recommended threshold of 10 of 14
symptoms of this scale, which was not normed for pre-
schoolers, was used (Barkley, 1990). For the CAP, only
one set of norms (male and female) for the age group
6–16 years was available and was used for both Time 1
and Time 2. The norms for all three rating scales differ-
entiated between male and female.
Considering that the overall agreement between rat-
ing scales was high (Fig. 1), it is unlikely that the dispar-
ity in the number of children identified by each of the
rating scales at Time 1 was due to any shortcoming of
the individual scales. It is most probable that the high
number of false positives noted in the ADHD Rating
Scale and the CAP at Time 1 were caused, in part, by
the lack of norms for preschool children.
Because ADHD is a chronic disorder, the children
who met the criteria at Time 1 but not at Time 2, for
whatever reason, confirm the cautions of previous re-
searchers about early labeling. Some children do, in fact,
outgrow the symptoms of ADHD that are presented in
preschool years (Campbell, 1995). Research by Fischer
and colleagues (Fischer, Barkley, Fletcher, & Smallish,
1993) indicated that hyperactivity might normalize with
behavioral adjustment across time. This supports the
maturational lag hypothesis for hyperactivity (Kins-
bourne, 1977) in that behavior comes closer to normal
with age and maturation in some children. This hypothe-
sis, in turn, supports the presence of the behaviors of
inattention, impulsivity, and hyperactivity that may very
well be part of the developmental process of early child-
hood.
All in all, it is believed that the high number of
false positives at the preschool level is attributable, in
the main, to two factors; the lack of preschool age norms
or threshold data in the ADHD Rating Scale and the
CAP, and the wide range of developmental variation in
a preschool child. All three rating scales were in close
agreement at the elementary level and at the preschool
level. Given a proper norm for the preschool age, they
should all be effective tools for preschool ADHD assess-
ment. The use of multiple rating scales for any assess-
ment might further improve the effectiveness or accu-
racy of a single assessment tool.
Loughran
A limitation of the study may be that all of the
children came from an upper-middle class community
that was characterized by ethnic and economic homoge-
neity. However, although fluctuations in the rate of ob-
served ADHD as a function of socioeconomic status have
been reported (Taylor, 1986; Trites, 1979), it should be
kept in mind that the present study addressed the rating
scales, not the population.
In conclusion, teacher rating scales provide a valu-
able piece of the information needed to evaluate and di-
agnose a child presenting the symptoms of ADHD in the
preschool setting and in the elementary school setting.
Agreement of observers on these scales is important
(Danforth, Barkley, & Stokes, 1991), because physicians
often use teacher rating scales in the initial assessment
of children in diagnosing ADHD as well as assessing the
efficacy of stimulant medication with ADHD children
(Barkley, Fischer, Newby, & Breen, 1988). Additionally,
for a portion of the preschool population, these teacher
rating scales may be used for monitoring purposes rather
than as diagnostic tools. This is important because as
this study has indicated, some preschoolers identified as
at risk for ADHD later demonstrated a lack of sufficient
symptoms to meet the diagnostic criteria.
The effect of diagnosis and treatment of all identi-
fied children of preschool age could have had profound
consequences and is clearly out of order. Labeling can
be limiting, if not damaging, to a child who is still devel-
oping—clearly a quandary for parents and teachers of
preschoolers. However, labeling may be in order if it
can have a prescriptive benefit, for example, facilitating
the availability of special services.
More research is needed in providing the preschool
child with a valid diagnosis. An accurate diagnosis is
crucial because children who could benefit from services
rendered to those with ADHD might be denied these
services if they are not identified. Clear definitions of
what constitutes ADHD behavior at the preschool level
would be of value for young children, their families, and
educators.
REFERENCES
American Psychiatric Association (1994). Diagnostic and statistical
manual of mental disorders (4th ed.). Washington, DC: Author.
Barkley, R. A. (1990). Attention deficit hyperactivity disorder: A
handbook for diagnosis and treatment. New York: Guilford
Press.
Barkley, R. A. (1995). Taking charge of ADHD: The complete author-
itative guide for parents. New York: Guilford Press.
Barkley, R. A., Fischer, M., Newby, R., & Breen, M. (1988). Develop-
ment of a multi-method clinical protocol for assessing stimulant
drug responses in ADHD children. Journal of Clinical Child Psy-
chology, 17, 14–24.
Assessing ADHD in Preschool
Campbell, S. B. (1995). Behavior problems in preschool children: A
review of recent research. Journal of Child Psychology and Psy-
chiatry, 36, 113–149.
Danforth, J. S., Barkley, R. A., & Stokes, T. F. (1991). Observations
of parent-child interactions with hyperactive children: Research
and clinical implications. Clinical Psychology Review, 11, 703–
727.
Du Paul, G. J. (1990). The ADHD Rating Scale: Normative data, reli-
ability, and validity. Unpublished manuscript, University of Mas-
sachusetts Medical Center at Worcester.
Edelbrock, C. S., & Achenbach, T. A. (1984). The teacher version of
the child behavior profile: Boys aged 6–11. Journal of Consulting
and Clinical Psychology, 52, 207–217.
Fischer, M., Barkley, R. A., Fletcher, K. E., & Smallish, L. (1993).
The stability of dimensions of behavior in ADHD and normal
children over an 8-year follow-up. Journal of Abnormal Child
Psychology, 21, 315–337.
Goldstein, S., & Goldstein, M. (1993). Managing attention disorders
in children: A guide for practitioners. New York: Wiley.
Goyette, C. H., Conners, C. K., & Ulrich, R. F. (1978). Normative
data on revised Conners’ parent and teacher rating scales. Journal
of Abnormal Child Psychology, 6, 221–238.
253
Kinsbourne, M. (1977). The mechanism of hyperactivity. In M. Blaw,
I. Rapin, & M. Kinsbourne (Eds.), Topics in child neurology (pp.
289–306). New York: Spectrum.
Lerner, J. W., Lowenthal, B., & Lerner, S. R. (1995). Attention deficit
disorders: Assessment and teaching. Pacific Grove, CA: Brooks/
Cole.
Miller, L. S., Koplewicz, H. S., & Klein, R. G. (1997). Teacher ratings
of hyperactivity, inattention, and conduct problems in preschool-
ers. Journal of Abnormal Child Psychology, 25, 113–119.
Palfrey, J. S., Levine, M. D., Walker, D. K., & Sullivan, M. (1985).
The emergence of attention deficits in early childhood: A pro-
spective study. Developmental and Behavorial Pediatrics, 6, 339–
348.
Shelton, T. L., & Barkley, R. A. (1993). Assessment of attention-defi-
cit hyperactivity disorder in young children. In J. L. Culbertson &
D. J. Willis (Eds.), Testing young children: A reference guide for
developmental, psychoeducational, and psychosocial assessments
(pp. 290–318). Austin, TX: Pro-ed.
Taylor, E. A. (1986). Childhood hyperactivity. British Journal of Psy-
chiatry, 149, 562–573.
Trites, R. L. (1979). Hyperactivity in children: Etiology, measurement,
and treatment implications. Baltimore: University Park Press.