Journal of Clinical Child & Adolescent Psychology, 42(6), 749–761, 2013

Copyright # Taylor & Francis Group, LLC
ISSN: 1537-4416 print=1537-4424 online
DOI: 10.1080/15374416.2013.773516


Clinical Usefulness of Observational Assessment
in the Diagnosis of DBD and ADHD in Preschoolers
Tessa L. Bunte and Sarah Laschen
Department of Psychiatry and Rudolf Magnus
Institute of Neuroscience, University Medical Center Utrecht

Kim Schoemaker
Department of Psychiatry and Rudolf Magnus
Institute of Neuroscience, University Medical Center Utrecht and
Department of Child and Adolescent Studies, Utrecht University

David J. Hessen and Peter G. M. van der Heijden
Department of Methodology and Statistics, Utrecht University

Walter Matthys
Department of Psychiatry and Rudolf Magnus
Institute of Neuroscience, University Medical Center Utrecht and
Department of Child and Adolescent Studies, Utrecht University

The aim of the present study was to investigate the clinical usefulness of an observa-
tional tool—the Disruptive Behavior Diagnostic Observation Schedule (DB-DOS)—in
the diagnosis of disruptive behavior disorders (DBD) and attention deficit=hyperactivity
disorder (ADHD) in preschoolers. We hypothesized that the DB-DOS may help
support the presumption of a diagnosis generated by the information from parents
and teachers (or other caregivers). Participants were referred preschool children with
externalizing behavioral problems (N ¼ 193; 83% male) and typically developing chil-
dren (N ¼ 58; 71% male). In view of the clinical validity study each child was given a
diagnosis of either DBD (N ¼ 40), or ADHD (N ¼ 54) or comorbid (DBD þ ADHD;
N ¼ 66) based on best-estimate diagnosis. The DB-DOS demonstrated good interrater
and test–retest reliability for DBD and ADHD symptom scores. Confirmatory factor
analysis demonstrated an excellent fit of the DB-DOS multidomain model of DBD
symptom scores and a satisfactory fit of ADHD symptom scores. The DB-DOS demon-
strated good convergent validity, moderate divergent validity, and good clinical validity
on a diagnostic group level for DBD and ADHD symptom scores. The Receiver
Operating Characteristic curve analyses revealed that for DBD the sensitivity and speci-
ficity are moderate and for ADHD good to excellent. The presumption of a diagnosis
based on information from parents, teachers, and cognitive assessment was supported

We are grateful to the parents and children who participated in this study. We especially thank Lauren Wakschlag, at the Department of Medical
Social Sciences, Feinberg School of Medicine, Northwestern University, Chicago, Illinois, for her cooperation in providing assessment procedures used
in this study and in training the Utrecht research group in administering and scoring the DB-DOS. We also thank Eva van der Kleij, Justa Kamstra, and
Susanne van Reedt Dortland, at the Department of Psychiatry, University Medical Center Utrecht, for their assistance with the data collection.
Correspondence should be addressed to Tessa L. Bunte, Department of Psychiatry, University Medical Center Utrecht, P.O. Box 85500, 3508
GA, The Netherlands. E-mail:

Hill. 2006). & Pickels. combining examiner. however. lems (e. American Psychi... Wakschlag. Recently. Although these newly parent-based behavioral observation paradigms pro- developed parent-report diagnostic interviews have vides complementary methods for incorporating the advanced identification of preschool psychopathology. a point Various observational tools have been developed of concern is the possibility of overidentification of pre. parents (Keenan et al. ents who consult a clinician would not accept that the Furthermore. 2012).. ODD comorbid with ADHD. 2007). however. including (semi)struc. 2008). American Psychiatric Association. behavior but also on parent behavior. [DSM–IV–TR]. losing his temper and physical aggression therefore has often been used to evaluate behavioral (Keenan et al. 2006). observational assessment focusing not only on child date) in the absence of these disorders is inappropriate. Goodman. the more extended (60 min) Disruptive Behavior although parent reports are based on children’s beha. conduct disorder (CD). The DPICS is an ing treatment (e.g. the DPICS is Diagnosing young children with ODD. Clearly. Ford. as most preschoolers exhibit at least some of the iceberg. and start. Briggs-Gowan.. 2005). a crucial issue in the 2000) diagnosis of ADHD in younger children has been use of direct observation.750 BUNTE ET AL. 2008).. Rabe- problems are the most common reasons for referral Hesketh. a limitation of obser- elementary school (Lahey et al. a . (Wakschlag et al.’’ Thus. or ADHD. 2000) in preschool children are similar observe the child in order to arrive at a diagnosis. pharmacotherapy with methylpheni.. Robinson & Eyberg. Posthumus et al. The question has tive of all problem behaviors (Le Couteur & Gardner. This is a manifestations of most of the symptoms of these disorders short (15 min) observational assessment of parent–child are not atypical per se when observed in a preschooler. as par- to those later in childhood (Egger & Angold. and parent based. interactive nature of social behavior into the assessment the limitations of sole reliance on parents as informants of clinical significance (Wakschlag. are amplified during the preschool years because young Wakschlag and colleagues (Wakschlag.g. 2008) developed behavior (Wakschlag. Even though clinicians do not expect to observe from normative transient disruptive behaviors within the each symptom. Therefore. et al. Clearly. not examiner based. In connection with this. the validity of the Diag. standardizing adult responses in a manner that presses tured interviews with the parents such as the Kiddie for a range of clinically salient behaviors in the child. Likewise. 2002). 2009). diagnosing a young child parent training for young children with conduct prob- with. Examiner-based ADHD is a challenge. Hill. 2007) and the Preschool Age Psychiatric child assessments. they lack the ecological validity of parent– Keenan et al. CD. 2004). 2008). CD.. Therefore. indeed been raised as to how one differentiates clinical 2008). and attention ing (Le Couteur & Gardner. such as The Dyadic Parent–Child Interaction Coding school children as having ODD. Diagnostic Observation Schedule (DB-DOS). Disruptive Behavior Disorder Schedule (K-DBDS. Furthermore. Our clinical experi- deficit=hyperactivity disorder (ADHD. The DB-DOS can be used to help support a presumption of a DBD and=or ADHD diagnosis in preschool children. for interaction that is sensitive to treatment change and example. as the System (DPICS. Epidemiological studies have demon.. 2008). there is evidence of the predictive validity decision on the presence or absence of a disorder would of ODD and CD in clinically referred preschool children be based only on parent and teacher reports. This makes clinical assessment of preschool on information from parents and teachers. direct observation (Wilens et al. 2008. that is not filtered through the perceptions of the parent strated that the prevalence rates of oppositional defiant may provide a different window on the child’s function- disorder (ODD). et al.. Hill. There is an increasing number of young children referred viors on a daily basis parent. that is.and Assessment (Egger et al.. however. by the DB-DOS in 60% for DBD and 75% for ADHD. they may want to see at least ‘‘the tip preschool age. assessment assessments are designed to be clinically sensitive by procedures have been developed. nostic and Statistical Manual of Mental Disorders (4th For clinicians who need to make a decision whether a ed. although the presumption of the of the behaviors that fall under the rubric of disruptive presence of ODD or ADHD may be generated based behavior. Among biased due to a number of factors such as personality these children. vation is its brevity and contextual dependency. attention deficit and disruptive behavior characteristics (Collishaw. direct obser- behavioral problems particularly subtle and complex vation may be used to support this presumption or not. for example. is how to weigh supported by a study demonstrating that the symptoms information from observation against information from and associated impairment are likely to persist well into parents and teachers. et al... child is diagnosed with a disorder. observations may be to mental health clinics (Howell & Teich. children cannot serve as informants about their own et al. 2011). 1981). in other Yet clinicians have long been reluctant to diagnose words. the behaviors observed need not be representa- young children with these disorders. want clinicians to ‘‘look at their child’’ themselves. ence suggests that many parents desire the clinician to atric Association...

that is. normative and disruptive behavior. Wakschlag. Therefore.. was agreement on one of the two diagnoses. & Angold. Finally. centers.8). score above the 90th percentile either on the Aggressive Briggs-Gowan. higher vocational for 16%. Furthermore. et al. the Netherlands.5% Asian. the procedure of best-estimate school children with behavioral problems. et al. and ADHD in preschool children. 2000). discuss all the findings. et al. Indeed. secondary for 25%. Our main range ¼ 40–72 days). presumption of a diagnosis generated by the information For the purpose of the clinical validity study. To examine The main aim of the present study was to examine the test–retest reliability. Schlichting. we was given a diagnosis of DBD.5 years 2. An interview with the parents about the reasons Participants for referral and the development of the child. SD ¼ 7. 2006).6. pediatricians. and cogni. whereas the correlation between DBD symptoms 1998) and the Peabody Picture Vocabulary Test–III–NL and hyperactivity-impulsivity in preschool children is (Dunn & Dunn. The information about the child consisted of the following: METHOD 1. 2005.. Hill. Children with a score in the clinical range either on Briggs-Gowan. to identify individual clinical was recruited from regular elementary schools and daycare cases. were referred by general practitioners. 2008). ADHD. each clinician had made a decision on the diagnosis of ined clinical validity on an individual level by determin. excluded. another diagnosis). The TD group (N ¼ 58) for clinical use.5-5 were an individual level. PRESCHOOLERS WITH DBD AND ADHD 751 standardized observational assessment of the preschool accounted for 6%. et al. ing.5-5 or C-TRF=1. In 75% of the chil- ined how often a diagnosis of DBD and=or ADHD dren there was agreement on the diagnosis. the particular child (DBD. qualitative. 0. Behavior scale or on the Attention Problems scale of The DB-DOS has been shown to be a reliable and the Child Behavior Checklist completed by parents valid tool to distinguish normative maladaptive beha. 2007). As there is no gold standard by which to diag- symptom scores in a European sample of referred pre. Achenbach & Rescorla. estimated with the average score on the Raven DB-DOS no attention has been paid to ADHD symp. it requires further research. an independent child psy- developing (TD) control group was also included in the chiatrist and clinical child psychologist consider the infor- study. the diagnosis was a discussion needed.. 193) and TD children (N ¼ 58) were included.5–5) and the Child Teacher Report Form com- viors from DBD symptoms (Wakschlag.5–5. Participants were 251 children aged 3. All the children (referred and TD) with an IQ ual clinical cases. assessing DBD symptoms in referred young children To study the reliability and convergent=divergent with behavior problems.5 to 5. were excluded. 24 children of the original group clinical usefulness of the DB-DOS in the diagnosis of were reassessed after 8 weeks (M ¼ 58 days. or comorbid DBD examined the reliability and validity of DBD and ADHD and ADHD. teachers. & Raven. Second. a semistructured DSM–IV– 12% Turkish=Moroccans. a typically diagnosis was used. 2007). ADHD. 2005). where 20 of these preschoolers were hypothesis was that the DB-DOS may help support the referred for behavior problems and four were TD children. and univer- behavior disorder [DBD]). both or no diagnosis= ing cutoff points of DBD and ADHD symptom scores.. primary ODD. The symptoms coded on the basis of the K-DBDS (M ¼ 55 months. and reach symptom scores on a group level by comparing the TD consensus (Lord et al. The DB-DOS is specifically sity for 19%. 80% were male. there was agreement on neither of both diagnoses. 2008). assessed in three interactional contexts: one parent Inclusion criteria for the referred patient group were a context and two examiner contexts (Wakschlag. However. Utrecht. Egger. In terms of parent education. 2% African American. Court. 2008. SD ¼ 7. Both clinicians were unfamiliar with the child. Third. Briggs-Gowan. that is.. (2008) investigated the clinical the Attention Problems scale or on the Aggressive Beha- validity of the DB-DOS on a group level and not on vior scale of the CBCL=1. Only when there was no agreement on which divide clinical from TD cases. toms.79 (Sterba. Child behavior is Department of Psychiatry. Hill. Wakschlag. while None of the participants were on medication. in research on the below 70. we exam. each child from parents and teachers (or other caregivers). First. and in 10% tive assessment was confirmed by the DB-DOS. 2008. we exam. which is required to identify individ. CD. and quanti. . Coloured Progressive Matrices (Raven. it seems evident to also include validity children referred with behavioral problems (N ¼ the assessment of ADHD symptoms in the DB-DOS. and TR-based parent interview for the assessment of 0. DBD and ADHD in preschool children. we examined clinical validity of DB-DOS mation about the child. 86% White.. in 15% there based on information from parents. intermediate child for diagnosing ODD and CD (also called disruptive vocational for 34%. nose a psychiatric disorder. et al. Children with behavior problems (N ¼ 193) designed to examine multiple. and tative aspects of preschool children’s behavior and well-baby clinics for clinical assessment at the Outpatient socioemotional functioning hypothesized to distinguish Clinic for Preschool Children with Behavioral Problems. Wakschlag. (Keenan et al. (CBCL=1. Before the consensus meet- group with DBD and ADHD groups. pleted by teachers or daycare caregivers (C-TRF=1.

Wakschlag. DBD ¼ disruptive behavior disorder.5 (6.6) 70.4 (8. In the involved in the Parent context..6 (6. the examiner also briefly leaves the room dren were positioned in the DBD. 2012).4) 101.7 (8. the NOS. are parallel across the mod- mated IQ (p < .7) Note. Schlichting. On doing his or her own work (Examiner Busy context). a N ¼ 40. instructions on flip cards.5-5 and Children were assessed during one morning session. a the DB-DOS and K-DBDS were administered. The scores of the Attention Problem scale and the Procedure Aggressive Behavior Scale of the CBCL=1. First. The Picture Vocabulary Test III–NL (Dunn & Dunn Medical Ethical Review Committee of the University 2005. ADHD ¼ attention deficit=hyperactivity disorder.9) 69. the basis of the consensus best-estimate diagnosis. the child is observed while and two CD symptoms) and ADHD were strictly working independently. CBCL ¼ Child Behavior Checklist. In this way the patients were cedures are explained to the parent before starting the divided into four groups: TD (N ¼ 58). b N ¼ 54.8) 53.1) 68. performing the three clinical groups. Snack Delay.0 (7. The videotaped observations of the child’s beha.5) 63. last- clinical validity study there were group differences in esti. The DB-DOS is a method for observa- Schoemaker et al.6 (11. 2008). The scores on the Raven Coloured Progressive received two small gifts.0) 75.8) 58. ing approximately 5 min.0 (11.752 BUNTE ET AL. the domains of Behavior Regulation (15 items).8 (10. Medical Center Utrecht.0 (10. c N ¼ 66. Schaffer et al. Briggs-Gowan. The scores on the Child Global Assessment lowed by the executive functions tasks. The characteris. Parents received a small well as the teacher=caregiver. and social play tasks.0 (3.1 (8. Pro- (DBD and ADHD) group.8 (9. In addition..1) 63. 6. defined as either three ODD symptoms and so-called Examiner Engaged context. within the at least one CD symptom or at least one ODD symptom context of minimal support. et al. The different tasks.1) TRF Attention 52. or DBD. 1983). 1998) and the Peabody the parents was gathered before participating. the exam- TR criteria of one of the DBDs (ODD. ADHD. or comorbid to probe potential covert rule-breaking behaviors. CD.  p < . Hill. with the examiner being busy applied (American Psychiatric Association. Then. that is. Written informed consent from Matrices (Raven et al.4 (7. TRF ¼ Teacher Report Form. The primary caregiver was tics of the four groups are displayed in Table 1. The measure of the impairment of the functioning of tasks were administered in a quiet and nondistractible the child’s behavior.8) 101.7) 69. 2008. including frustration. vior during the psychological tasks.4) TRF Aggression 52.3) 101. who performed compliance. IQa is an estimate based on mean Raven and Peabody.5 (9. and parents are provided with simply worded ADHD (N ¼ 54) and comorbid (N ¼ 66).001 (compared with the typically developing group). 2005).7) CBCL Aggression 50. chil.7) 62.4 (10. tional assessment of disruptive behavior in pre-school 7.8 (2.. DBD (N ¼ 40).3) 53. and Shape School) and two working memory Measures tasks (Delayed Alternation and Nine boxes. It is divided into three interactional contexts: two examiner contexts and one To reach a clinical consensus diagnosis the DSM–IV– parent context.3) 67.1) CBCL Attention 50. internalization of rules.5 (1.4 (11. children (Wakschlag.5-5.4 (9... iner is ostensibly responsive to child behavior. The results of the executive function tasks: three inhibition tasks (GoNoGo task. 3. The four groups did not differ system for behavior problems consists of problems in in age or gender. financial compensation for participating. 2000).e.3) 75.. After a break Schedule (CGAS.7 (6. the C-TRF=1. i. The DB-DOS coding similarly to each other.4) 54. tasks.1 (11. . with the TD group significantly out. general intellectual functioning was assessed fol- 4. ules.1) % Boys 71 85 76 86 IQa 112. and TABLE 1 Means (and Standard Deviation) for the Demographics and Control Variables in the Four Groups DBDa ADHDb DBD þ ADHDc M (SD) M (SD) M (SD) M (SD) Age (Months) 56. In the first examiner context.2 (12.001).1) 61. DB-DOS.3 (11. et al. filled out by the parents as room with a one-way mirror. and children 5.0 (3. approved this study.

CD ¼ 0. The Pervasive Develop- group of Wakschlag trained our study group for admin.8%) and the CGAS teacher questionnaire mately 15% of the videotaped sessions were randomly (6. Confirmatory factor analyses were used ation. CGAS.. tions. and practice administra. Convergent and the following Cronbach alpha’s (Cronbach.75. For coding. Disruptive behavior symptoms teacher=caregivers reports (CGAS. and personal bur- observed as behaviors and scored during the obser. mental scale of the ECI was used in view of examining istration and coding of the DB-DOS. Hill. In the present study Cronbach ive dimensions of disruptive behavior for the ratings. To provide a global assessment of function- Competence scale. and Impulsivity (two items). a DSM-based checklist. Parents reports on the 0 ¼ normative behavior and 1 ¼ normative misbehavior) summary score of the Social Skills Rating Scale (SSRS. we developed 1983). with the TD group. For the scoring we used the developmentally behavior as proposed by Wakschlag et al. DB-DOS administration and coding. which could be mother assesses the social. and attention problems were assessed using the scores To assess clinical validity. CBCL=1. parents and teachers=caregivers completed the behavior problems domains. in preschool children it is not only the frequency but also the intensity of such behaviors that distinguishes clinical problems from normative behavior. lations of DB-DOS domain scores and multimethod assessment of child functioning as evaluated by parent (K-DBDS. 2000) and the C-TRF=1. 2000) DBD and ADHD symptoms in preschool to test the fit of the multidomain model of disruptive children. with lower 10 ADHD items organized in terms of problem behaviors scores indicating greater impairment. DB-DOS codes are ratings of child behavior ranging from 0 to 3 and comprising two categories: typical (code Social Skills Rating Scale. with the exception of the ECI teacher ques- the child’s clinical status. and clinically concerning behavior (code 2 ¼ of concern Gresham & Elliot. reliability (Wakschlag. 1992) completed by the those DSM–IV–TR criteria for ADHD. 0. The items were developed on Scale (Sheeber & Johnson. For the present study two criterion Statistical Analyses coders trained an independent team of research students For all scales used the percentage of missing data was at for reliable coding. Gadow & Sprafkin. in this study we concentrate on the ing. (Wakschlag. extending the DB-DOS. Coders were independent and blind to most 4. The ADHD domain Impact on family scale. Furthermore.91. as alpha was 0. examined on a diagnostic of the Attention Problem scale and the Aggressive group level. present study Cronbach alpha was 0.5-5. den resulting from child behavior problems. at test and retest. DBD ¼ 0. 2007). 2008). ECI) and CBCL=C-TRF. Interrater agreement and test–retest reliability at the domain level and by context were assessed using intra- K-DBDS. Following Wakschlag.1%. live present study Cronbach alpha was 0. 2000).5-5. In the present study we found ing the newly proposed ADHD domain. while controlling for age and gender .64. SSRS. related to Inattention (four items).8%). In the istration included a review of the DB-DOS manual.. divergent validity of DBD and ADHD scores. In the vation (see the appendix). et al. DB-DOS uses the quantitative and the qualitat- module of the DB-DOS. C-TRF=1.. analyses of (co)variance were conducted to Behavior Scale of the CBCL=1.83.. Training for admin. a semistructured clinical interview mean level differences in domain scores and by context assessing DSM–IV–TR (American Psychiatric Associ. reliability was established via 80% exact item-level agreement with the coders of the study group of Wakschlag. 1951).79.5-5 (Achenbach & compare the DBD and ADHD groups (with and without Rescorla.5-5 (Achenbach & comorbid children) and the comorbid group separately Rescorla.77 (parent) and and videotaped observations. tionnaire (4. 2008). and an extension of this model includ- possible to DSM–IV–TR. The CGAS is scored from 0 to 100. Parents and teachers= caregivers reported on child behavior symptoms with the Early Child Inventory (ECI. ADHD ¼ 0. approxi. financial. 1951): divergent validity were examined by computing the corre- ODD ¼ 0. ECI). modified approach. 1990) were used to evaluate children’s and 3 ¼ atypical). Hyperactivity (four items). nonclinician version of the CGAS (Schaffer et al. This scale was completed during the parent context. We also examined (Keenan et al. For the reliability analyses the internal consist- selected for double coding to monitor ongoing interrater ency was examined with Cronbach alpha (Cronbach.82 (teacher). The study 1996). it is designed to adhere as closely as Hill. The Impact on Family consists of these 10 items. Early Child Inventory. et al. Parents were administrated the K-DBDS class correlation coefficients (ICCs). Each item is rated separately for each social skills. Although there is a CGAS.95. PRESCHOOLERS WITH DBD AND ADHD 753 Anger Modulation (six items).

Because estima- with each other. and checked ency in child behavior across the different DB-DOS con- if a child diagnosed only with ADHD (DBD). All nal consistency (see Table 2).69–.58) relative to associa- DBD (ADHD) diagnosis on the DB-DOS. Model 2 was an extended model that includes the ADHD domain. on two-sided tests. these items were excluded from domain DB-DOS score is required to divide clinical from nonclini.5-5.92.40). 1989). Group membership was based statistically (Lord et al. 2012).52–. Information on M ¼ . range ¼ . according to the across domains and contexts both (see Table 2) in terms DB-DOS.95) and by context (ICC child’s symptoms in the outpatient clinic. For Pearson correlations among the DB-DOS domains these analyses we used data from the total sample.91) tion.. both in terms of total domain scores exclusively on the basis of the direct observation of the (ICC M ¼ . domain scores (Cronbach a M ¼ . we computed the percentage agreement between the best-estimate diagnosis DBD and=or ADHD (or no diagnosis) and the DB-DOS Domain Intercorrelations Across Contexts diagnosis DBD and=or ADHD (or no diagnosis).86–. diagnosis (see participants). Across the information from parents. Five items (all in the on consensus best-estimate diagnosis. et al.77–. and cognitive assessment. the child’s everyday behavior from parents and teachers= caregivers should also be taken into account. range ¼ . Test–Retest Reliability of Domain Scores To examine how the identification of the presence Test–retest analyses indicated moderate reliability or absence of DBD and=or ADHD. will get an ADHD (DBD) significantly higher across the two examiner contexts diagnosis based on the DB-DOS but will not receive a (Pearson correlation ranged. Hill. both in terms of total cutoffs. beneath the 90th percentile on the Attention Problems scale and on the Aggressive Behavior scale of the Confirmatory Factor Analyses CBCL=1. range ¼ . across context indicated modest to substantial consist- including children who are not diagnosed. domain scores tended to decrease slightly givers). For this. In both models.64. cal cases for DBD and=or ADHD. Model 1 was the multidomain model of disruptive behavior as proposed by Wakschlag. On the other hand. using fewer than 10 times in the clinical concerning range: Bonferroni corrections. Interrater Reliability ate. from this perspective a high specificity is essential. Internal Consistency The optimal cutoffs were determined by finding the values Overall the three domains (Behavioral Regulation.5-5 and C-TRF=1. The analyses were Behavioral Regulation domain) were rated as present carried out with and without controlling for IQ. Statistical significance was based verbal aggression (two items). that led to a balance between sensitivity (percentage true Anger Modulation. teachers (or other care- the contexts.92. specificity (>70%) is required (Kim & Lord. diagnosing a child with a psychiatric disorder while there is no diagnosis clearly is inappropri.754 BUNTE ET AL.94). result in some degree of misclassifica.82. (2008).82. three of RESULTS the observed domain scores were regressed on the latent Anger Modulation construct. range ¼ .97). Patients groups were not compared spiteful behavior.88–. may be used in supporting or rejecting a of total domain scores (ICC M ¼ . range ¼ . the fit of two models. as this was not an aim of the study.59–. Receiver Operating Characteristic (ROC) curve analysis was used.44–. of course. For clinical use a high sensitivity and an acceptable and by context (Cronbach a M ¼ . based texts. Cross-context associations within a domain were on best-estimate diagnosis. based on and by context (ICC M ¼ .71) possible diagnosis of DBD and=or ADHD.18–.80). three other observed domain Domain Level Reliability scores were regressed on the latent Behavioral Regulation Diagnostic observation measures may include clinically construct. level analyses. and sneaky behavior. for the DBD total group. and the latent Anger Modulation and Beha- salient items with an occurrence too low to be evaluated vioral Regulation constructs were regressed on the latent . directed aggression. using best-estimate in the second testing. tes of reliability can be unduly biased by differences To use the DB-DOS as a clinical assessment tool on an between a single pair of raters for items with a very low individual level. range ¼ . and ADHD) exhibited good inter- positives) and specificity (percentage true negatives). We used the BR=DBD total score cutoff points for the DBD diagnosis and the Confirmatory factor analyses were conducted to test ADHD total score cutoff point for the ADHD diagnosis. an appropriate cutoff point of the occurrence.64. It showed excellent interrater reliability as Table 2 We note that in clinical practice diagnosis will not be based illustrates. We assumed tions between each of the examiner contexts and the the TD children have no diagnosis based on their score Parent context (Pearson correlation ranged.

The latent ADHD and Behavioral Regulation onto Disruptive Behavior were construct was allowed to correlate with the latent constrained to be equal.151. tion results of the two models are shown in Figure 1.0) ADHD M .3) 2.86 . The errors of these domain scores were again approximation ¼ . Hill. Following Wakschlag. EA ¼ Examiner Engaged Context. the factor loadings of Anger Modulation domain scores from the same context.0) 11.5) 3. root mean square error of struct.92 .3 (4.59 Examiner Engaged . parative fit index ¼ .8) 2.52 3. The parameter estima- scores from the same context were allowed to correlate. v2(5) ¼ 8.000. PC ¼ Parent context.94 .7 (5. et al.946. three new observed domain As in the study of Wakschlag.95 M .1 (3.0 (2.63 Examiner Engaged .2) 3. PRESCHOOLERS WITH DBD AND ADHD 755 TABLE 2 Domain Score Reliability Domain Cronbach’s aa Interrater ICCb Test–Retest ICCc Test M (SD) Retest M (SD) Behavioral Regulation M .0 (4.051. root mean square error of on the latent Hyperactivity=Impulsivity construct.88 .2) Examiner Busy . HI ¼ Hyperactivity=Impulsivity.0) 12.3 (4. The fit of Model 2 could be the latent Inattention and Hyperactivity=Impulsivity considered satisfactory. p ¼ . the fit scores were regressed on the latent Inattention construct.71 Examiner Engaged . v2(33) ¼ 74. com- constructs were regressed on the latent ADHD con. and approximation ¼ .6 (4.91 .5 (4. Model 2 extends Model 1.94 .57 7. DBD ¼ disruptive behavior disorder.2 (2.073.6 (1.3) Parent . FIGURE 1 Confirmatory factor analysis. and Parent Context (N ¼ 28). .74 11.88 M . Disruptive Behavior construct. of Model 1 was excellent.86 . (2008). AM ¼ Anger Modulation. The errors of observed domain Disruptive Behavior construct.2 (2.0) Parent .75 .4) Parent .8 (2.1.8 (6. ADHD ¼ attention deficit=hyperactivity disorder.62 5.6) 6.1) Note. Note. ADHD ¼ attention deficit=hyperactivity disorder.992.81 .92 . ICC ¼ intraclass correlation coefficient. et al.91 M .95 . b Examiner Engaged (N ¼ 39). Examiner Busy (N ¼ 47). (2008).53 1.77 . here. BR ¼ Behavior Regulation.7) Examiner Busy .94 .8) 10.7 (5.92 M 0.41.80 12.1 (5.87 .3) Examiner Busy . compara- three other new observed domain scores were regressed tive fit index ¼ .77 M . allowed to correlate with the errors of other observed Hill.6) 3.68 11. EB ¼ Examiner Busy context.79 M . c N ¼ 24.84 .96 .8 (3. IN ¼ Inattention.69 . a N ¼ 251.5 (4.1) Anger Modulation M .5 (4. p ¼ .64 14.82 .68 3.

35 . EE ¼ Examiner Engaged context.28 . K-DBDS ¼ Kiddie Disruptive Behavior Disorder Schedule. with the TD group for the Examiner Engaged context. IFS ¼ Impact on Family Scale.28 .34 . First we determined a cutoff lower but significant as well. Modulation domain and the parent and teacher= Using a total DBD score of 24 yielded a sensitivity of caregiver questionnaires.29 .24 . who according to clinical con- relations were found between the majority of DB-DOS sensus were diagnosed with DBD. and the Par- the TD group and the clinical groups without control. sensitivity of 77% means that Divergent validity.08 .33 . Clinical validity.15 .24 .06 .11 .11 Teacher=Caregiver Reported CGAS .07 . On the basis of this DBD cutoff score.85.32 . .22 .19 . Specificity of 69% means that 69% of the (SSRS.09 .23 . Validity differed from the TD group on all DB-DOS scores. Correlations between the Behavioral Regulation required.32 .25 .05.19 .34 . All clinical groups ROC analysis yielded an AUC of . We used the total DBD group (DBD and domain and the parent and teacher=caregiver question- DBD þ ADHD both consensus best-estimate diagnosis) naires. The results were similar for be determined (i.18 .32 .14 . ADHD ¼ attention deficit=hyperactivity disorder.16 .24 .28 .29 . .27 ECI (PDD) .19 .81. Table 3). To investigate clinical validity on a Second.21 .12 . 95% CI [. according to consensus best-estimate diagnosis.24 .36 . Impact on Family appropriate cutoff point of the DB-DOS score is Scale).17 .74.26 . the estimated area under as expected.5-5.  p < .21 . were significant and the ROC analysis.28 . Correlations between point of the total (Examiner Engaged.37 .44 IFS (M) .29 .32 . Family Scale (Mother). 20 . Correlations between the Anger Modu- curve (AUC) values and their 95% confidence intervals lation domain and the questionnaires were slightly (CI) were satisfactory.17 . To examine clinical validity on an individual level.01. signifi- Behavior Regulation Examiner Engaged context and cant correlations were established between DB-DOS the ADHD group with respect to the Anger Modulation domain scores and parent and teacher=caregiver Examiner Engaged=Examiner Busy context.30 . with the exception of the CBCL=1.25 .25 . CGAS ¼ Clinical Global Assessment Scale.14 . CBCL=1.40 . C-TRF=1.41 .30 .37 .31 SSRS . PC ¼ Parent context. an CGAS.30 .18 . questionnaires of symptoms and impairment (K-DBDS.30 .24 .17 .39 CGAS .42 ECI-PDD .41 TRF .45 .26 .22 .22 . three out of four children.14 .18 .21 .13 .15 .35 . . 95% CI [.18 .88].17 .20 .30 .14 . The the analyses when controlling for IQ.38 . TD children will not be diagnosed with DBD. part of the total DBD score). we explored whether a better cutoff point of the examine group differences in DB-DOS scores between total (Examiner Engaged.34 .26 . Examiner Busy.23 Note. and 31% will be diagnosed with DBD while the diagnosis is not given.  p < .78.25 . The ROC analysis correlations between the Behavioral Regulation=Anger yielded an estimated AUC of .44 .42 . Small but significant minor cor. and the absolute values were higher than the Regulation and Anger Modulation). analyses of (co)variance were conducted to tory.30 .15 .17 .13 .11 .24 .46 CBCL . CBCL ¼ Child Behavior Checklist.41 .e. As Table 3 illustrates. SSRS ¼ Social Skills Rating Scale.14 . IFS (M) ¼ Impact on.12 ..34 .10 . as this low-sensitivity=specificity was unsatisfac- group level.5-5 Aggressive and the total ADHD group (ADHD and DBD þ ADHD Behavior scale and Behavioral Regulation domain in consensus best-estimate diagnosis).10 . EB ¼ Examiner Busy context. ECI-PDD.28 .17 .5-5. ent context together) Behavior Regulation score could ling for IQ (see Table 4).23 . ECI (PDD) ¼ Early Child Inventory (Pervasive Developmental Disorder). with the exception of the DBD group with respect to the Convergent validity.91]. the ADHD scales and the questionnaires were signifi- and the Parent context together) DBD score (Behavior cant.27 .15 . Scores by Context and Multi-Informant Assessments of Child Disruptive Behavior and Functioning Behavioral Regulation Anger Modulation ADHD EE EB PC Total EE EB PC Total EE EB PC Total Parent Reported K-DBDS . 77% and a specificity of 69% (see Table 5).34 . will receive a DBD scores and parent and teacher=caregiver questionnaires diagnosis.29 . Examiner Busy.41 .09 . Overall.16 .756 BUNTE ET AL. TABLE 3 Association (Pearson r Correlation) of the Disruptive Behavior Diagnostic Observation Schedule (DB-DOS).

4) 13. 95% CI ADHD Totalb 32 91 69 76 88 [.0 (9.9 (1. 95% CI [.9 (4.77].7 (3. a ¼ DBD vs.8) 7.5 disruptive behavior disorder.  p < .3 (4. f N ¼ 120.0) 13. using a total Anger Modulation score of 13 Score 3. TD.6 (5. Fifth we examined whether the cutoff a N ¼ 106.9) 5.8) (ab) (de) (c) AM tot 9. b ¼ ADHD vs.9 (5. Using a 38 79 95 94 82 total Behavioral Regulation score for the 3.0 (4.4) (abcde) BR tot 10.5–4.4 (4.0) 6.4 (4. This cutoff Positive Negative point for Behavior Regulation was slightly better than DB-DOS Predictive Predictive the one for total DBD score.8) 14.7) (acde) AM PC 4. a total Anger Modulation 18 63 86 94 70 score of 11 yielded a sensitivity of 62% and a specificity Total BR 13 90 72 76 88 of 64%.60.01.2) (abcde) ADHD tot 27.9 (5.0) 13.1 (3.9) 7. 16 77 81 80 78 The ROC analysis yielded an AUC of 0. and the Parent context) for the total DBD.3 (4.6 (2.4 (4.3 (9. Fourth we examined whether the Score 34 87 79 81 86 Behavior Regulation cutoff point for younger and older 36 83 83 83 83 children differed in specificity and sensitivity.7 (8. d N ¼ 66.7) 23.8) 6.9 (4.7 (4.8) 45.83. Bold numbers indicate maximized sensitivities and specifici.2) 9.4) 4.2) 3.3) 4.5 years of age.8) 21.2 (11.7) 3. These cutoff points for Anger Modulation Total ADHD 33 89 72 76 87 were unsatisfactory.4) 4.9 (4.3 (4.2) 4. .7 (4.4 (3.6 (9.1 (3. c ¼ DBD þ A A DHD vs.1) 10. b N ¼ 40.5.1 (4.4 (2.3 (8.5-year-old children of 14 yielded a sensitivity of 87% ties depending on criteria used in selecting cut-off scores. point for referred patients with clinical scores on the b N ¼ 120. sensitivity was 67% and specificity DBD þADHD.1) 14.2) 13.4) 4.5–5 (T score above 70) differed in specificity .4) (abcde) AM EE 1.1 (9.3) 15. DBD ¼ disruptive behavior disorder.69.  p < .0) 2. TRF 1.8 (4.3 (4.5 (4.7) 14.5 (2.8) 11. ADHD ¼ attention deficit=hyperactivity disorder.5) 5.4 (4.5) 10.05. .1 (3.2) 5. DBD total ¼ DBD only and DBD þ ADHD.4) (abcde) Note.4 (2.9 (5.9 (5.4 (4.7 (2.3 (3. BR ¼ Behavior Regulation.2 (4.2) 45.5 (5.89.0) 18.8 (8. e N ¼ 106.0 (3.1 (4.4) 6.4) 16.3) 14.5) 16.7 (2.5 14 87 76 78 85 Years 15 81 79 79 81 yielded a sensitivity of 55% and a specificity of 74%.9) 9. We have explored the role DBD Totala Score Sensitivity Specificity Value Value of different contexts (Examiner Engaged. a N ¼ 58.1) 3.4 (3. TD ¼ typically developing. TD.9) (abcde) BR PC 5.1) 20.8 (3. ADHD ¼ attention deficit=hyperactivity disorder.2) (bcde) BR EB 2.7) 15.1 (3.95].4) Note.9) 4. BR ¼ Behavior Regulation.  p < .6) 15. TD.1 (9.7) 7.0 (6.4 (6.0) 2. whereas for 4.4) 10.6) 11. TD.2) 13.0) (b) (a) (cde) ADHD EE 8. ADHD Total ¼ ADHD only and ADHD þ DBD.7 (5. tot ¼ total. and the Parent context together) Anger Modu- 16 68 81 78 72 lation score would be better.g.0) 12. We explored Total BR 12 87 60 69 82 also whether the total (Examiner Engaged.001.05.2) 6.9) 3. TABLE 5 Using a total Behavior Regulation score of 13 yielded a Receiver Operating Characteristic Curve sensitivity of 83% and a specificity of 72%.5) 6.8) 6.1) 14.1) 3.2 (4. PC ¼ Parent context. 4.1) 41. 28 67 79 76 71 but the results of the sensitivity=specificity balance for 31 56 83 77 65 the different contexts were unsatisfactory.7 (4.2 (1.6) (a) (bcde) ADHD PC 9. AM ¼ Anger Modulation.6 (7.1) 10.8) 15. and the Anger Modulation scores.3 (10.4 (3.6 (11.7) (ae) (cd) AM EB 2.2 (4.9 (4.7 (8. The ROC analysis yielded an AUC of . ADHD Total¼ ADHD only and ADHD þ DBD. DBD total ¼ DBD only and to 5. Examiner Total DBD 20 85 59 67 80 Busy. TD.4) 42.. PRESCHOOLERS WITH DBD AND ADHD 757 TABLE 4 Means (and Standard Deviations) of the DB-DOS Domains in the Six Groups DBD only and ADHD only and TDa DBDb ADHDc DBD þ ADHDd DBD þ ADHDe DBD þ ADHDf M (SD) M (SD) M (SD) M (SD) M (SD) M (SD) BR EE 2. DBD ¼ and a specificity of 76% (see Table 5). Examiner Score 13 83 72 75 81 Busy. the Score 24 77 69 71 75 Behavior Regulation.4 (3.4 (4.2) 35. (ae) means p value for a and e lower than . EB ¼ Examiner Busy context.4 (9.4 (5.8 (9. EE ¼ Examiner Engaged context. e. was 76%. c N ¼ 54.7 (3.6) (a) (bcde) ADHD EB 9. d ¼ DBD total vs.7 (5. e ¼ ADHD total vs.

95% CI [. the percentage agree. on the basis of the ADHD score.5 years).. Likewise.S. Another explanation could be that young ADHD should be investigated. CGAS parents or and the ADHD group with respect to the Anger Modu- teachers=caregivers (above or below 60). the results for DBD. Briggs-Gowan.92. ing to clinical consensus were diagnosed with ADHD The main aim of the present study was to examine will receive an ADHD diagnosis. The clinical cutoff point for ADHD symptom scores The results of reliability (internal consistency. will or ADHD (or no diagnosis). the percentage agreement was computed DBD score. The better results for referred and the DB-DOS ADHD diagnosis (or not) based on patients with clinical scores on the TRF 1. This may be due to the cross-situational of reliability and validity of the ADHD symptom scores character of ADHD symptoms (Gadow & Drabick. 2007). study by adding a just one context (at home or at school). and sensitivity for the different modules. For the best-estimate diagnosis ADHD (or not) younger preschoolers. The results of confirma. age group (4.5–5 (T score above 70) showed a better sensitivity group on DB-DOS DBD and ADHD symptom scores. children. 2012). exception of the DBD group can be explained by the pared to the total DBD group. using the Behavioral Regulation between the best-estimate diagnosis DBD and=or ADHD total score. This can be Thus. using a the exception of the ADHD group is in line with expecta- total (Examiner Engaged.5–5. who according (or no diagnosis) and the DB-DOS diagnosis DBD and= to clinical consensus were diagnosed with DBD. et al. and 21% will be diagnosed with the Behavior Regulation total score appeared to have ADHD while the diagnosis is not given. Categorization of the total DBD group in sex. will receive an ADHD diagnosis. receive a DBD diagnosis. Furthermore. The referred With respect to the clinical validity of the DB-DOS on children with DBD with clinical scores on the TRF a group level. and specificity balance compared to the total DBD with the exception of the DBD group with respect to group. slightly better sensitivity=specificity than the total Finally. Thus. their impulsive behavior in a novel context. et al. and CBCL lation Examiner Engaged=Examiner Busy context. In contrast. the Behavior Regulation Examiner Engaged context IQ (IQ above or below 104). This is important. the current study not only repli. The Regulation contexts and the Anger Modulation Parent ROC analysis yielded an AUC of . who according to clinical consensus were diag- gent and divergent) of DBD symptom scores in the nosed with ADHD.758 BUNTE ET AL. maybe because older pre- off point of the DBD total score of 24 (see Table 5) is schoolers can better inhibit their own behavior than 60%. The percentage agreement. highly structured context with an unknown examiner. comorbidity with patient clinic. children). Thus.96]. test–retest reliability) and validity (conver.88. Thus. 2008) in a U. tions. DBD diagnoses appeared to be better for ADHD than Hill.5 years) than in the older 5) is 59% and the percentage agreement based on the cut.S. be more difficult to elicit DBD symptoms than ADHD as in the clinical assessment of preschool children with symptoms in an observational procedure at an out- externalizing behavior problems. symptoms in preschool children (Sterba et al. diagnosed with ADHD. and the Par.5–4. four out of five children. By contrast the Anger Modu- in this study group (referred and typically developing lation total score was unsatisfactory. 2008. ODD symptoms may be present in cated but also extended the U. Indeed. context compared with the TD group. sample. are satisfactory.5–5 (T score the cutoff point of 34 (see Table 5). Besides. it may reliable and valid ADHD domain. 2012). . Examiner Busy. sensitivity of explained by the high correlation of DBD and ADHD 87% means almost nine out of 10 patients who accord. all clinical groups differed from the TD 1. current study in a European sample are similar to the Based on ROC curve analyses the sensitivity and speci- findings of the studies by Wakschlag and colleagues ficity for the ADHD diagnosis when compared with (Wakschlag. according to consensus While exploring the clinical cutoff point for DBD. symptoms according to both parent and teacher had more mental health concerns than youth with symp- toms according to either parent or teacher (Gadow & DISCUSSION Drabick.. almost nine out of 10 reliability.. children with ADHD are typically unable to control tory factor analyses for ADHD domain are satisfactory. higher DB-DOS symptom scores in all the three Behavior tivity of 87% and a specificity of 79% (see Table 5). above 70) are in line with evidence that youth with ment is 75%. best-estimate diagnosis. the (above or below 70) did not show better results com. interrater appeared to be good. Specificity of clinical validity on an individual level using clinical 79% means that 79% of the TD children will not be cutoff points for DBD and ADHD symptom scores. Wakschlag. between the best-estimate DBD diagnosis (or sensitivity of the clinical cutoff point for Behavior not) and the DB-DOS DBD diagnosis (or not) based Regulation appeared to be somewhat better in the on the cutoff point of the BR total score of 13 (see Table younger age group (3. a cutoff point of 34 was suitable (see Table 5). For the ADHD domain. children with ADHD showed significantly ent context together) ADHD score of 34 yielded a sensi. whereas a .

further research is required to dimensional conceptualization of psychopathology. in line with mind that in everyday clinical practice the decision of the Wakschlag. Our because preschool children show fast developmental main hypothesis was that the DB-DOS may help sup. accurate identification of ADHD in preschoo- and cognitive assessment are strengths of the present lers is crucial as treatment of these children may include study. the newly developed ADHD preschool children necessitates a combination of infor- domain. Second. in line with the Wakschlag. and for ADHD. diagnosing a child with ODD. generated by the information from parents. teachers (or other caregivers). study the predictive validity of the DB-DOS. First. (or not). 2011). and cognitive assessment. which means that for that in this data set no subscales of ADHD (i. et al... and cognitive ECI-PDD part and the Behavioral Regulation=Anger assessment. et al.. Hill. et al. the correlation between the ents. 2008. and cognitive assessment to discuss sex differences. Gowan. overuse of psychostimulants to decrease symptoms of whereas the use of DBD diagnoses in young children ADHD in the absence of a clear diagnosis (Rey & has been questioned. 6 months) is not an appropriate alternative with the parents (Matthys & Lochman. small but significant minor givers). first study that gives a quantitative indication of what may be expected from a standardized direct obser- Implications for Practice and Research vation of the child within the context of a broad clini- cal assessment for preschool children suspected for First. to help support (or not) a presumption of a DBD and= The replication of the study by Wakschlag and or ADHD. et al. Inatten- nearly three of four children the diagnosis ADHD (or tion and Hyperactivity=Impulsivity) are used in the clinical not).. (2008) study. 2008. the DB-DOS may be used as an observational tool DBD and ADHD. and cognitive assessment using the social skills are less well developed in children with DBD best-estimate diagnosis. Wakschlag. Briggs- information from parents. (2008) study. Gowan. Fam- based on the results of a single measure but on the iliarity with the tasks and parents promising presents for combination of results from multiple measures such their child’s good behavior may explain the lower score as a standardized parent and teacher=caregiver rating on the retest. teachers (or other caregivers). 2003). Children (or not) by the DB-DOS. PRESCHOOLERS WITH DBD AND ADHD 759 child with ODD or CD may be less likely to display their to preschool disruptive behavior has been developed symptoms in an unfamiliar context. generated by the information from parent. as caregivers). Third. When investigating the clinical validity of the The study also has limitations. teachers (or other priate measure to examine divergent clinical validity. & Landau. generated by the information from the par. developmentally based approach it should be considered whether a more practical and . and cognitive assessment. one may question whether the SSRS is an appro- by the information from parents.. CD. teachers (or other caregivers). or ADHD is not In our study we organized the retest after 8 weeks. a multidimensional. colleagues (Wakschlag. mation from parent. and ADHD (Ronk. as an alternative. 2008. 2008) in a European sample The complexity of diagnosing DBD and ADHD in of referred children. a longer period for the retest scales and a (semi)structured DSM-oriented interview (e. mation about the child’s behavior not only from a par- tinguish clinical from nonclinical cases. Briggs- ADHD or no diagnosis. Hill. Fourth. et al. changes. Therefore mately six out of 10 children diagnosed with DBD we added the ECI-PDD part in view of studying diver- (or not). Third. Second. Hund. has been confirmed tion subtype diagnosis. To our knowledge this is the diagnosis. gent validity. As expected. Briggs-Gowan. et al.e.. teachers (or other care. 2010). Wakschlag. 2012). the determination of cutoff points to dis. Wakschlag. DOS scores and parent reports of social skills on the mately 60% the DBD diagnosis (or not). the DBD diagnosis has been confirmed Modulation=ADHD symptom scores was weak. generated SSRS. and the ent interview=questionnaire and a teacher=caregiver determination of the importance of the DB-DOS to questionnaire but also from an observation of the child support (or not) a presumption of a DBD and=or (Le Couteur & Gardner. results of the study score supported nearly 75% of the ADHD diagnosis thus are not generalizable to children with IQs below 70. in the course of studying divergent port the presumption of a diagnosis generated by the clinical validity. generated by the infor. 2008). In this study group correlations were found between the majority of DB- the DB-DOS BR=DBD total score supported approxi. generated by the information from the parents validity study because of the low (N ¼ 6) ADHD inatten- and teachers (or other caregivers). (Wakschlag et al. based on a Sawyer. the test–retest DB-DOS on an individual level we should keep in reliability was only moderate for DBD. This means that for approxi. However. based on consensus best-estimate (or not) by the DB-DOS. The DB-DOS ADHD total with an IQ below 70 were excluded. the very small number of girls constrains the ability teachers (or other caregivers).g. Hill. In the present study the DB-DOS is examined pharmacotherapy and concerns have been raised about in the context of DSM-IV–TR-based diagnoses.. Another limitation of this study is using the best-estimate diagnosis.

& Dunn. 1005–1013. Raaijmakers. C. & Schopler. New Autism Diagnostic Interview– multidimensional. K. . A. 457–468. Journal of the American Academy Amsterdam. developmentally-based approach to preschool Revised algorithms for toddlers and young pre-schoolers from 12 disruptive behavior. W. 63. UK: Blackwell. H. A.. A... (2006). G. A. (2012). Danis. Amsterdam. (2008). F. Assessment of social (III–NL) [Dutch version by L. Pine. B. A. Washington. E. M. A. J. doi: 10. & Drabick. Tolan.. C. E. Hund. 622–631. A. M.. Oppositional defiant disorder and conduct disorder in childhood. beyond parent interviews and parent=teacher question. W. Lahey. 1228–1231. E. Test–retest reliability of the Preschool Age Schlichting. S. S. (2008).. Youth & Families.. 42. Hill. (2012). Goode. and evaluations. A. Leventhal. G. 313–337. S. (2000). University of Vermont. Walter.760 BUNTE ET AL. Briggs-Gowan. Journal of the American Wakschlag. 82–93... C. Three-year predictive validity of ment of preschool disruptive behavior. Jordan. J. M.. & Pickles.. J. Peabody Picture Vocabulary Test Ronk. Predictive validity of DSM–IV oppositional B. A. M. G. 48.. J. Disruptive Behavior Diagnostic Observation Schedule (DB–DOS). Rabe-Hesketh. M. Duax. text rev. L.. R. Psychiatry. Hill. ily Scale for assessing families with behaviorally difficult children. 40. J. M. S. Gresham. J. (2007). Massetti. Journal of the American Academy of Child and Adolescent & Donald. disorders in preschool children: Presentation. Observational Assess- defiant and conduct disorders in clinically referred preschoolers.. Anger and irritability symptoms Matthys. (2010). L. Brasic. & Eyberg.. Hill... Robinson. . Manual for the ASEBA Blackwell. (2006).. Donald. J. Journal of the American Academy of Child and Adolescent 2014–2020. & Gadow. British Journal of Psychiatry. (1996). & Rescorla. J. C. syndromes. Burns. D. R.. J. . P. 53..... Wakschlag. American Journal of Psychiatry.. 155–159. A. Keeler. nosed at 4–6 years of age. 50. P. A children’s global assessment scale Egger. of Child and Adolescence Psychiatry. & Lord. Sustained effects of incredible years as a American Psychiatric Association... 49. pp.. H. C.. Sheeber. 284–286. K.). R. . (2011). 632–640.. 161. . K. Psychometrika... B. J. (2004). S. Bird. Briggs-Gowan. 487–500..1007=s10803-011-1213-1 Wakschlag. Mawhood. Le Couteur. Ehrhardt. Carter.... Psychiatric Assessment (PAPA).. L. M. M. . M. Circle Journal of Child Psychology and Psychiatry. Danis. F. H.. H. E.. Stevenson. L.. Wakschlag. of Consulting and Clinical Psychology. 297–334. Putting theory to the test: Modeling a Kim.. Oxford. Shulman. Ford. C.. Are psychostimulant drugs being munity factors and child psychopathology informant-specific and used appropriately to treat child and adolescent disorders? The informant-general? Journal of Child Psychology and Psychiatry. Taylor. (1983). 51. 111–119. J.... (2012). G. W. Leventhal. S.. 8.)... A. Bunte. T. preschool forms and profiles.. S. S. Early childhood inventories manual. H. S. M. Journal of the American Academy of Child and to 47 months of age. Erkanli. Court. (2007). A. B. Raven Coloured Collishaw. & Raven. A.. 593–604. Carter. S. Lee.. . Risi. Schoemaker. (2000). J. Journal of Autism and Developmental Disorders. Diagnostic specificity Howell. 1073–1085. (2003).. 571–580. Humphries.. K. (2008). Common emotional and behavioral Abnormal Child Psychology. & Elliot. (1990)... Keenan. A. Autism from 2 to 9 years of age. & Briggs-Gowan. Ambrosini. & Lochman. Social Skills Rating System. M... Angold. T. S. 45. D. B. Journal of Autism and Developmental Disorders. .. Journal of Abnormal Child Psychology. .. 35. view of using the DB-DOS in clinical practice. (1981).. . S. Loney. Briggs-Gowan. & Angold. B. Fisher. J. Dekovic. Chen. epidemiology. L. Egger. Diagnostic and statistical preventive intervention in preschool children with conduct manual of mental disorders (4th ed.. & A. Fifth. Espy. and Schaffer.. S.. J. D. Chichester. (1951). & Teich. (1998).. M. H. 47. Journal of Child Psychology and Psychiatry. Scott. (1992). nosology.. M. L. 47. L. Rutter. M. .. Part II: Validity of the Journal of Child Psychology and Psychiatry. the competence of boys with attention deficit=hyperactivity disorder: Netherlands: Harcourt Assessment.. A. H. & Sprafkin. Danis. Peabody Picture Vocabulary Test III–NL. host responses. A. S. Use of structured interviews Wakschlag. less time-consuming way of coding may be developed in Lord. Rutter... Thapar ‘‘disruptive’’ in preschool behavior: What diagnostic observation (Eds. Keenan. Rey. L. S. K... C. Journal tests. Keenan. E.. S. H. Henry. B. Coyne. UK: Oxford Psychologist Press.. S. Keenan... incremental knowledge gained from the DB-DOS Lord. H.. J. Boeldt. (2009). L.. K. Coefficient alpha and the internal structure of interaction coding system: Standardization and validation. H. UK: Wiley- Achenbach. 182. Goodman. S.. 52.. Children. Goud.. D. 829–840. & Angold. Progressive Matrices. Humphries. Clinical Child and Family Psychology Review. Journal of Child Psychology and Psychiatry. 40. M. Burlington: Research Center for Posthumus. schedule: A standardized observation of communicative and social behavior. The dyadic parent–child Cronbach. L. Egger.. Further evidence of the reliability and validity of Psychiatry. S. family and com. Disruptive Behavior Diagnostic Observation Schedule (DB–DOS). Duax. J. DiLavore. Variations in Medicaid mental and nonspecificity in the dimensions of preschool psychopath- health service use and cost for children. (2012). S.. S. Heemsbergen. L. K.. A. Adolescent Psychiatry. How far are associations between child. A.. Defining the Bishop. J.. M. (2008). D. 694–701. Author.. 17. L. Gadow. & Matthys. (2011). & Gardner. Thurm. can teach us. B. ment of Preschool Disruptive Behavior. 220–228. the (2006). Autism diagnostic observation naires is a future research direction as well. Dunn. 19. Potts. Applicability of the Impact on Fam- Stony Brook. NY: Checkmate Plus. 47. K. C. 538–549. Pelham.. W. . Raven. 185–212. & (CGAS). Journal of Child Psychology and Psychiatry. H.. L. . Rutter’s child and adolescent psychiatry (5th ed. C. Part I: Reliability of the DSM–IV attention deficit hyperactivity disorder in children diag. 16. Executive function deficits in preschool children among youth with ODD: Cross-informant versus source-exclusive with ADHD and DBD. A. in Mental Health. Academy of Child and Adolescent Psychiatry. DC: problems. 47–55. Sterba. & Pickels. 183–201.. M. Keenan. M. L. Problematic peer entry. & Aluwahlia. S. Hill. Carter. Wiebe. . (1989). L. Pines. 271–288). Journal of Abnormal Child Psychology. REFERENCES Matthys. Schlichting]. 39. In M. L. (2005). B. K.. 245–250. D... Oxford.. Archives of General Psychiatry. L. K. E. Kipp. the Netherlands: Hartcourt Assessment. Carter. Observational assess- S. C. J. L. D. (2012). Archives of General Psychiatry. . M.. D. (2005).. & Sawyer.. J. DSM–IV ODD and CD in preschool children. Danis.. & Johnson. E. and observational methods in clinical settings. Maassen. S. 40. & Landau. J. D. MN: American Guidance Service. S. van Engeland. (2005). P. A. . Administration and Policy ology. 26. H. Journal of Egger.

difficulty waiting (verbal) . Difficulty sustaining attention 10. E.. J. Talks excessively Inattention Impulsivity 1.. T. S31–S36. J.. Monuteaux.. Difficulty waiting (non verbal) 2. Careless=inattention to details 9. Patterns of psychopathology and dysfunction 4. Difficulty remaining seated and vigorous motor activity (major movements=hyperactivity) 7. Journal of Developmental & Behavioral Pediatrics. & 3. M. Biederman. squirms (minor movements) APPENDIX 6. (2002). Interrupts. Hyperactivity 5.. J. Brown. Difficulty playing quietly DB–DOS ADHD items 8. Easily distracted Spencer. S. Fidgets. Easily bored in clinically referred preschoolers. 23. T. PRESCHOOLERS WITH DBD AND ADHD 761 Wilens. Prince.

download. or email articles for individual use. users may print. .Copyright of Journal of Clinical Child & Adolescent Psychology is the property of Taylor & Francis Ltd and its content may not be copied or emailed to multiple sites or posted to a listserv without the copyright holder's express written permission. However.