You are on page 1of 18

Journal

of Anxiety

Pergamon

Diwrdcrs, Vol. 10, No. 2, pp. 97-114.1996 Copyright &, 1996 Elsevier Science Ltd Printed in the USA. All rights msetwd 0887-618396 $15.00 + .Ml

SSDI 0887-61%5(95)00039-9

Anxiety

Rating for ChiMren - Revised: Reliability and Validity


GAIL A. BERNSTEIN, M.D.
Psychiatry. University of Minnesota Medical School

Division

of Child and Adolescent

RossD.
Department of Psychiatry,

CROSBY.

PH.D. Medical School

University

of Minnesota

AMY R. Pt%RWIFN, B.A.


Division of Child and Adolescent Psychiatry. University of Minnesota Medical School

CARRIE M. BORCHARDT, M.D.


Division of Child Md Adolescent Psychiatry, University of Minnesota Medical School

Abstract - The purpose of this investigation was to define the psychometric properties of the Anxiety Rating for Children - Revised (ARC-R), a clinician rating scale for the assessment of anxiety symptoms in chiidren and adolescents. The ARC-R is comprised of an Anxiety subscale and a Physiological subscale. In a clinical sample (N = 22). the test-retest reliability and interrater reliability were investigated. In a nonoverlapping clinic sample of school refusers (N = 199). the internal reliability and convergent, divergent, and discriminant validity were evahtated. Test-retest (r = .93) and interrater reliability (r = .95) were excellent. There was good internal reliability of items (Cronbachs alpha = .80). The Anxiety subscale of the ARC-R correlated

Dr. Bernsteins effort on this manuscript was supported in part by Grant R29 MH46534 from the National Institute of Mental Health. The authors thank John Hopwood, M.A. and Suzy Peterson, B.A. for their assistance in administering the rating scales and Lois Laitinen, M.B.A., M.M. for manuscript preparation. Dr. Crosby is currently at NCS Assessments,Minnetonka, MN. Amy Perwien is currently a graduate student in the Department of Clinical and Health Psychology at the University of Florida. Requests for reprints should be sent to Gail A. Bernstein, M.D., Director, Division of Child and Adolescent psychiatry, Box 95 UMHC, 420 Delaware St SE, Minneapolis, MN 55455. 97

98
somewhat depression criminated ment was the ARC-R for clinical higher with self-report

G. A. BERNSTEIN

ET AL.

anxiety

instruments

(r = .62)

than

with

self-report disinstruof

instruments (r = .54-S6). between children with and shown Anxiety and to be a reliable subscale research purposes

The Anxiety subscale of the instrument without an anxiety disorder. This new of clinician-rated The potential anxiety. The validity uses of this

measure

was demonstrated. are highlighted.

instrument

INTRODUCTION Anxiety disorders are among the most prevalent disorders in child and adolescent psychiatry (Costello, 1989; Kashani & Grvaschel, 1990). However, only in very recent years has attention been focused on anxiety disorders in children and adolescents (Last, 1992). As more attention has shifted to the area of childhood anxiety disorders, methods of assessment need to be improved and expanded. To adequately assessfor anxiety in children and adolescents, it is critical to obtain information with a variety of types of instruments. The methods that are currently available for assessment of childhood anxiety disorders include structured and semistructured diagnostic interviews (Chambers et al., 1985; Herjanic 8z Reich, 1982; Hodges, McKnew, Burbach, & Roebuck, 1987; Silverman & Nelles, 1988). self-report instruments (Bernstein & Garfinkel, 1992; Ollendick, 1983; Reynolds & Richmond, 1978; Spielberger, 1973), and parental report instruments (Achenbach, 1991; Wirt, Lachar, Kliendinst, & Seat, 1977). The Anxiety Rating for Children - Revised (ARC-R) is the only clinician rating scale, to our knowledge, which was specifically developed to assessanxiety symptoms in children and adolescents. The ARC-R uses symptom clusters similar to those in the Hamilton Anxiety Rating Scale (Hamilton, 1959) which was designed for use in adults. The Hamilton Anxiety Rating Scale is used as an outcome measure in many treatment studies of anxiolytic medications (Maier, Buller, Philipp, & Heuser, 1988). It has recently been demonstrated to be reliable and valid in an adolescent sample (Clark & Donovan, 1994). It has been noted that interview measures are often advantageous because they minimize differences in subjects interpretations of questions (Clark & Donovan, 1994). These scales are important assessment tools because they integrate. both the clinicians experience and expertise and the childs report of anxiety symptoms. In addition, the ARC-R provides the clinician with a measure of the severity of cognitive and somatic aspects of anxiety rather than being based solely on DSM-N (American Psychiatric Association, 1994) criteria. While parents can serve as a valuable source of information, the symptoms experienced by children and adolescents with anxiety disorders are often internally or subjectively felt. Therefore, it is imperative that the children be interviewed about their anxiety symptoms. In a review of parent-child agreement in clinical assessment, Klein (1991) concluded that the concordance between childrens and parents reports of anxiety disorders is generally poor. In a study by Herjanic and Reich (1982), children reported significantly more subjective

ARC-R:

RELIABILITY

AND

VALIDITY

99

symptoms, such as anxiety and depression, than the parents described in their children. These factors underscore the importance of obtaining information relevant to anxiety from the childs perspective. Although anxiety may be manifested and reported by children and adolescents as somatic symptoms (Beidel, Christ, & Long, 1991; Last, 1991), few anxiety instruments systematically assess for somatic complaints. The ARCR, which includes a physiological subscale, provides the clinician with information about the child or adolescents somatic symptoms, in addition to information about psychological anxiety. Of the anxiety instruments designed for children and adolescents, only the Revised Childrens Manifest Anxiety Scale (RCMAS; Reynolds & Richmond, 1978) has a physiological subscale. The physiological items of the RCMAS and ARC-R differ, with the ARC-R providing a more comprehensive assessment of somatic complaints. Beidel et al. (1991) found that children with anxiety disorders endorse a variety of different physical complaints. The purpose of this investigation was to define the psychometric properties of the ARC-R in clinical samples and to examine the utility of the ARC-R as an assessment instrument and as a screening measure for anxiety. It was predicted that reliability and validity data would support the use of this measure in children and adolescents. Based on previous research of gender differences on anxiety measures (Bernstein & Garfinkel, 1992; Ollendick, 1983; Ollendick, Ring, & Frary, 1989; Reynolds & Richmond, 1985), it was further hypothesized that females would score higher than males on the ARC-R.
Development of the Instrument

The original ARC (Erbaugh, 1984) was developed for use as a clinician rating scale of anxiety in children and adolescents. The Anxiety subscale of the original ARC included the following items: anxious mood, cognitive, tension, fears, separation anxiety, depressed mood, and sleep disturbance. The Physiological subscale included: muscular, sensory, cardiovascular, respiratory, gastrointestinal, genitourinary, and autonomic. A behavioral observation item was also included in the original ARC. The items included in each subscale were consistent with the concept of measuring anxiety that is used in the Hamilton Anxiety Rating Scale (Hamilton, 1959). The ARC grouped symptoms together by category. The depressed mood item of the original ARC was deleted in the ARC-R because the content overlapped with the content of the Childrens Depression Rating Scale - Revised (CDRS-R; Poznanski, Freeman, & Mokros, 1985). It was felt that the revised version of the ARC would be a purer measure of anxiety without the depression item. In addition, the sleep disturbance item was deleted due to poor psychometric properties (e.g., corrected item-to-total correlation of .32 for sleep disturbance; .44-.68 for other items). Due to interest in the investigation of somatic complaints, six physiological anxiety items were revised in 1992. The genitourinary item was excluded because the content did

100

0. A. BBBNSTBlN Br AL.

not seem to be applicable to children and adolescents. After these changes, the instrument was referred to as the ARC-R. The ARC-R is intended to be administered by clinicians and mental health professionals. The ARC-R allows the clinician to interview the child in a semistructured fashion and rate the severity of the childs anxiety symptoms throughout the interview. In the ARC-R, a number of specific questions for children and adolescents focusing on anxiety symptoms occurring within the past week are included for each item. Each item is rated by the clinician according to symptom severity on a Likert scale. Ratings include: not present (0), mild (1). moderate (2), severe (3), and very severe (4). Criteria for each of the severity ratings are included. As an example, the separation anxiety item and the associated rating criteria are provided in Table 1. In the ARC-R, the Anxiety subscale score is based on five anxiety items (anxious mood, cognitive, tension, fears, separation anxiety), and the Physiological subscale score is based on six physiological items (muscular, sensory, cardiovascular, respiratory, gastrointestinal, autonomic). The sum of the two subscales is the total Anxiety score. There are 11 items with a possible score of 0
TABLE 1 SEPARATION ANXIB~Y ITEM FXOMARC-R When youre not with your folks, are you afraid or worried about something bad happening to them? Do you worry about something bad happening to you when youre apart? What has this been like in the past week? How many times have you been absent in tbe last week? What is it like for you when you know its time to go to school? Do you get stomachaches and headaches? How do you feel when youre on the way to school? Have you been leaving school to go home during the school day? When youre home, do you stay close to your mother (parent)? Do you follow your mom or dad around tbe house or yard? How do you feel when your parent(s) go out without you? Does it make you nervous or afraid to be.alone in the house? Do you sleep in a room alone? Do you feel afraid or nervous about sleeping alone? Do you sleep with a brother or sister, or with your parent(s)? Do you go into your parent(s) room during the night? How do you feel about sleeping away from your home and family (e.g., at friends or relatives homes, at camp)? 0 = NOT PRESENT - not at all anxious; no difftculties with separations I = MILD - feels anxious about some usual separations from patent figures. No social impairment. 2 = MODERATE -quite anxious about routine separations; some social impairment or restriction of ageappropriate activities 3 = SEVERE - intense dependency and separation anxiety; functioning impaired due to anxiety, marked social impairment or restriction 4 = VERY SEVERE - panic attacks or tantrums in context of separation. Intense separation anxiety, extreme dependency, grossly disabling and resulting in incapacity for age-appropriate functioning

ARC-R:

RELIABILITY

AND

VALIDITY

101

to 4 for each item. The range of total scores is 0 to 44. Administration instrument requires approximately 30 min. METHOD

of the

The psychometric properties of the ARC-R were studied in two separate studies. In one study, the stability of the ARC-R across administrations (testretest reliability) and between raters (interrater reliability) was examined in 22 children and adolescents with anxiety and/or depressive disorders. In a second study of 199 children and adolescents in the School Refusal Clinic, the convergent and divergent validity of the Anxiety subscale of the ARC-R was examined via correlation analyses with other measures of anxiety and depression. The discriminant validity of the ARC-R Anxiety subscale was examined in diagnostic subgroups of these outpatients. Study 1: Test-Retest and Interrater Reliability Subjects. Twenty-two subjects participated in the reliability study. Written informed consent for participation was obtained from each subject and a parent. Subjects were informed that the purpose of the study was to learn about and to define the test properties of the ARC-R. Nineteen of the subjects completed both interviews. Approximately two-thirds of the subjects (N = 14) were child and adolescent outpatients who had previously been evaluated in the School Refusal Clinic. Approximately one-third of the subjects (N = 8) were inpatients on the Child and Adolescent Psychiatry Unit. Inclusion criteria for inpatients were admission diagnoses (including provisional and rule-out diagnoses) of anxiety disorder, mood disorder or adjustment disorder with emotional features. The sample included 13 females (59%) and 9 males (41%). Mean age was 15.2 + 2.1 years with a range of 11 to 18 years. Racial composition was 21 Caucasians and 1 Asian. Procedure. Two B.A.-level research assistants were trained by the first author in administration of the ARC-R interview. The training involved several components including: explanation of the ARC-R, observations of live ARC-R interviews, and practice administrations of the instrument to patients. In the reliability study, ARC-R interviews were completed with one research assistant giving the interview and the other observing the interview in the same room. Both raters independently and simultaneously scored the interviews. All interviews were tape recorded. Interviewers were not blind to the purpose of the study. However, multiple interviews were scheduled during the week to decrease the likelihood that an interviewer would remember information from a particular subject. Subjects were scheduled for two interviews administered by the same interviewer, 2-5 days apart. The same interviewer was used to minimize changes in scores

102

G. A. BERNSTHN

FaT AL.

based on different interviewers. The 2- to 5&y interval was chosen to minimize changes in anxiety level merely as a function of time and because the ARC-R measures current anxiety (i.e., within the past week). Subjects were paid at the end of each interview. Study II: Internal Reliability and Validity

Subjects. Subjects were 84 females (42.2%) and 115 males (57.8%) ranging in age from 6 to 17. The mean age was 13.1 f 2.4 years. Demographics of the School Refusal Clinic sample have been described in previous publications (Bernstein, 1991; Bernstein, Svingen, & Garfinkel, 1990, Bernstein & Garfinkel, 1992). Procedure. Data from 199 outpatients evaluated in the School Refusal Clinic over 8 years (1984-1992) were included. Approval from the Institutional Review Board to review records and existing data was obtained. As part of a comprehensive 3-hr outpatient assessment, the children and adolescents were administered the original ARC and the CDRS-R, clinician rating scales, by a masters-level psychologist. The children and adolescents also completed selfreport rating scales for anxiety and depression. The same psychologist, who was blind to clinical information about the subjects, administered all the ARCS, CDRS-Rs and self-report measures. Only the first 26 outpatients had the Physiological subscale of the ARC administered due to time constraints in the Clinic. Parents were not present during the testing. The clinician rating scales and the self-report instruments were given in the following order: Childrens Depression Inventory (CDI; Kovacs, 1981), Revised Childrens Manifest Anxiety Scale (RCMAS; Reynolds & Richmond, 1978), Visual Analogue Scale for Anxiety - Revised (WA-R; Bernstein & Garfinkel, 1992), CDRS-R, ARC, and Childrens Depression Scale (CDS; Lang & Tisher, 1978). Prior to initiating the self-report instruments, the psychologist informally assessed each childs reading level by having the child read some of the items aloud. If reading difficulty was apparent, the psychologist read all the items to the child. The psychologist remained in the room to answer questions during the administration of the self-report instruments. To evaluate the discriminant validity of the ARC-R anxiety subscale, a subset of the patients (N = 127) was assigned DSiU-III-R (American Psychiatric Association, 1987) diagnoses based on independent review of 95% of the charts by the first author and either a child and adolescent psychiatry resident or medical student. The charts contained extensive clinical data because the focus in the School Refusal Clinic is on thorough diagnostic assessment for anxiety and mood disorders. Information reviewed included the outpatient evaluation summary and treatment notes. Diagnoses were based on data in the charts documenting the presence of symptoms and not based on specific diagnoses in the chart. The raters were blind to scores on the ARC-R and to scores

ARC-R:

RELJABlLlTy

AND

VALIDllY

103

on other rating scales. After the first chart reviews, interrater agreement for assigning subjects to diagnostic groups (anxiety disorder, anxiety and depressive disorders, depressive disorder, and no anxiety or depressive disorder) was 85% (100 of 118). Disagreements in diagnostic group assignment were resolved by a second independent chart review by each rater and subsequent discussion between the raters after the second chart review. Gf the 127 cases reviewed, 73 (57.5%) had an anxiety disorder. Of the subjects with an anxiety disorder, 30 had a comorbid depressive disorder (major depression or dysthymia). Thirty-one (24.4%) had a depressive disorder only, and 23 (18.1%) had no anxiety or depressive disorder. In the no anxiety or depressive disorder group, eight (34.8%) had oppositional defiant disorder, six (26.1%) had conduct disorder, one (4.3%) had attention-deficit hyperactivity disorder, and eight (34.8%) had no diagnosis. The discriminant validity analysis included three groups: anxiety disorder with or without comorbid depressive disorder, depressive disorder only, and no anxiety or depressive disorder. It was expected that any subject with an anxiety diagnosis, regardless of the presence of a comorbid depressive disorder, would have an elevated ARC-R Anxiety subscale score. Therefore, all subjects with an anxiety disorder, with or without a depressive disorder, were grouped together. Psychometric Instruments. Revised Childrens Manifest Anxiety Scale (RCMAS; Reynolds & Richmond, 1978): The 37 statements on this self-report anxiety instrument require yes or no responses. The RCMAS includes three anxiety subscales and a Lie subscale. Adequate construct validity (Reynolds, 1980) and test-retest reliability (Wisniewski, Mulick, Genshaft, & Coury, 1987) have been demonstrated. In the adolescent group, the RCMAS was also found to be a valid measure of anxiety (Lee, Piersel, Friedlander, & Collamer, 1988). Visual Analogue Scale for Anxiety - Revised (VAA-R; Bernstein & Garfinkel, 1992). Each of the 11 items in this anxiety self-report is followed by a continuous line on which respondents are asked to place a mark to show how jittery/nervous or steady they feel. Reliability and validity have been studied and values are in the acceptable ranges (Bernstein & Garfinkel, 1992). Childrens Depression Inventory (CDI; Kovacs, 1981). This is a 27-item self-report measure of depression. One month test-retest reliability was good in a nonclinical sample (Kovacs, 1981). The instrument has high internal consistency (Saylor, Finch, Spirito, & Bennett, 1984). Convergent validity (Sham, Naylor, & Alessi, 1990) and discriminant validity (Moretti, Fine, Haley, 8 Marriage, 1985) have been demonstrated. Childrens Depression Scale (CDS; Lang & Tisher, 1978). This self-report depression scale includes 66 statements which are written on individual cards. Each card is sorted into one of five boxes (labeled from very right to very wrong). The test-retest reliability at 7-10 days was good in a nonclinical sample (Tisher & Lang, 1983). Adequate internal consistency and concurrent validity have been demonstrated (Knight, Hensley, & Waters, 1988; Rotundo

104

G.A.BERNSWNETAL.

& Hensley, 1985). The CDS differentiates between children with and without a diagnosis of depression (Knight et al., 1988). Childrens Depression Rating Scale - Revised (CDRS-R; Poxnanski et al., 1985): This is an l&item clinician-rated scale for depression. High test-retest reliability has been demonstrated (Poznanski et al., 1984). The instrument correlates highly with other measures of depression supporting the convergent validity of the scale. The CDRS-R discriminates depressed versus nondepressed children (Poxnanski et al., 1984). Statistical Analysis Test-retest reliability was determined by Pearson correlation coefficients between scores obtained at the two test administrations. Interrater reliability for subscale and total scores was assessed using intraclass correlation coefficients (Bartko & Carpenter, 1976). The internal reliability of the instrument was assessedusing Cronbachs alpha. The association between measures was assessed using Pearson correlations. Differences in ARC-R Anxiety subscale scores between diagnostic groups were determined using one-way analysis of variance. Receiver operating characteristic (ROC) analysis (Hanley & McNeil, 1982) was performed to examine the diagnostic utility of the ARC-R Anxiety subscale. This analysis is useful in characterizing the ability of a diagnostic instrument to distinguish patients with a psychiatric disorder (true positives) from those without the disorder (true negatives) (Hsiao, Bartko, & Potter, 1989; Murphy et al., 1987). ROC analysis evaluates the test performance of a diagnostic instrument across the full range of test scores. For any given cutoff score on a diagnostic instrument, the following values can be calculated: (a) the sensitivity, or true positive rate, is the proportion of subjects with the disorder scoring at or above that cutoff, (b) the specificity, or true negative rate, is the proportion of subjects without the disorder scoring below that cutoff, and (c) the cumphnent of sensitivity (i.e., one minus sensitivity), or the false negative rate, is the proportion of subjects with the disorder scoring below that cutoff, and (d) the complement of specificity, or false positive rate, is the proportion of subjects without the disorder scoring at or above that cutoff. The ability of a diagnostic instrument to distinguish patients with a disorder from those without the disorder can be graphically displayed using an ROC curve. An ROC curve plots sensitivity (true positive rates) on the vertical axis and one minus specificity (false positive rates) on the horizontal axis across all possible cutoff scores. The diagonal line on the ROC curve, where true positive rates are equal to false positive rates, represents the line of no information. The greater the distance between the diagnostic tests ROC curve and the line of no information, the better the diagnostic performance of the instrument. Statistical methods are available for estimating the area under the ROC curve (AUC) and its standard error (Hanley & McNeil, 1982). The AUC provides an estimate of the probability that a randomly chosen subject with an anxiety diagnosis will

ARC-R:

RELIABILITY

AND

VALIDITY

105

have a higher ranking on the ARC-R Anxiety subscale than a randomly chosen subject without an anxiety diagnosis (Hanley & McNeil, 1982). In the present study, RGC analysis was used to determine the ability of the ARC-R Anxiety subscale to distinguish subjects with an anxiety diagnosis from those subjects without an anxiety diagnosis, Subjects with an anxiety disorder with or without a comorbid depression were included because it was expected that all subjects with an anxiety disorder regardless of the presence of a depressive disorder would have an increased score on the ARC-R Anxiety subscale. The AUC and standard error were calculated on the basis of the binormal curve (Hanley and McNeil, 1982). RESULTS
Study I: Test-Retest and Interrater Reliability

Test-retest reliability data were obtained at a mean of 3.1 f .9 days on 19 sub jects. For the Anxiety subscale score, the Pearson correlation coefficient comparing scores at the first test administration with scores at the second administration was .97 (Table 2). For the Physiological subscale score, the Pearson correlation coefficient was .82. Pearson correlation coefficient was .93 for the total Anxiety score. All correlations are at p < .OOl level. Group mean total Anxiety score at the first testing was 13.5 + 9.6 and 12.3 + 8.3 at the second administration. Interrater reliability, based on the first test administration, showed an intraclass correlation coefficient of .97 for the Anxiety subscale score, .93 for the Physiological subscale score, and .95 for the total Anxiety score (Table 2). These correlations are significant at p < .OOl level.
Study II: Internal Reliability and Validity

Average interitem correlation on the Anxiety subscale was .43 (range = .35-66); corrected item-to-total correlations ranged from .47 (separation anxiety) to .68 (anxious mood). Cronbachs alpha was .79 for the five Anxiety items (N = 199).
TABLE 2
TlSbhZlEST AND hTBRRATER hIABiLITY

Test-Retest Reliability* N= 19 Anxiety Subscale .9l Physiological Subscale .82 Total


SCOE.

Interrater Re.liabilityb N= 19 Anxiety Subscale .91 Physiological Subscale .93 Total score .95

.93

WI Pearson conelation coefficients at p < .OOl. bul intrachiss comehtions at p < .OCJI _

106

G. A. BeRNSTEIN ET AL.

Average interitem correlation for the Physiological subscale was .28 (range = .08 to SO); corrected item-to-total correlations ranged from .34 (gastrointestinal) to -57 (muscular). Cronbachs alpha was .69 for the six Physiological items (N = 26). Cronbachs alpha was .80 when the two subscales were combined (N = 26). All analyses of the validity of the measure used the subscales included in the ARC-R. The ARC-R is basically a subset of items from the ARC. The ARC was revised to form the ARC-R by deleting the depression, sleep disturbance, genitourinary, and behavioral observation items. The validity results are based on the ARC-R Anxiety subscale, not the total ARC-R scale. Pearson correlation coeffkients between the ARC-R Anxiety subscale (sum of the five anxiety items) and subjects scores on Anxiety and Depression rating scales scores are summarr zed in Table 3. The correlations between the ARC-R Anxiety subscale and scores on the self-report anxiety instruments were higher (r = .62) than the correlations between the ARC-R Anxiety subscale and scores on the self-report depression instruments (r = 54-56). However, the difference between these correlations is not significant based on the z statistic. All correlations in Table 3 are at p < .OOl level. To explain the .67 correlation between the Anxiety subscale of the ARC-R and CDRS-R, the analysis was repeated with anxiety disorder only patients and resulted in a correlation of r = .58. For depression only patients the correlation
was r = .66.

Pearson correlation coefficients between the ARC-R Physiological subscale (includes six items) and anxiety measures were not significant, although the correlation between the RCMAS Physiological subscale and the ARC-R Physiological subscale approached significance (r = .35, p = .093). Significant
TABLE 3
CORRELATIONS~ BETWEEN THE AICOEIY ANXETY AND DEPRIWION RATHWX SUBSCALE OF THE ARC-R AND S~ALEX IN A CLINIC SAMPLEI r N

Anxiety Scales Revised Childrens Manifest Anxiety Scale Subscales Physiological Worry/Oversensitivity Fear/Concentration Visual Analogue for Anxiety - Revised Depression Scales Childrens Depression Inventory Childrens Depression Scale Childrens Dcpmsion Rating Scale - Revised

.62 45 .61 .52 .62 .54 56 .67

185

99 185 175 187

aAll Pearson correlation coeffkients at p < .OO 1. br = .58 for subjects with anxiety disorder only (N = 43).

ARC-R:

RELIABILITY

AND

VALIDITY

107

correlations were found between the ARC-R Physiological subscale and the CD1 (r = .41, p = 048) and the CDRS-R (r = 52, p = .007). The correlation of the Anxiety and Physiological subscales of the ARC-R was 52 (p = 406,
N = 26).

Mean Anxiety subscale scores were significantly higher for females (6.8 f 4.0) than males (5.3 + 3.4) (F = 8.06, df = 1, 197, p = .005) (Table 4). Mean Physiological subscale scores were also higher for females (5.5 f 2.7) than males (3.8 f 2.9), although the results did not reach statistical significance (F = 2.41, df = 1, 24, p = .134). Similarly, females scored significantly higher than males on all other rating scales (Table 4). The mean score for the entire sample (N = 199) on the Anxiety subscale was 5.9 f 3.7, with a range from 0 to 18. The mean score on the Physiological subscale (N = 26) was 4.6 f 2.9, with a range from 0 to 9. Analysis of variance (ANOVA) showed a significant difference in anxiety subscale score among the three diagnostic groups (F = 9.96, df = 2, 124, p = .OOOl). The mean score for the anxiety disorder group was highest at 6.2 it 3.6. The mean score for the depression group was 4.5 f 2.8 and the mean score for the no anxiety or depressive disorder group was 3.0 f 2.0. Following ANOVA, the Tukey-HSD Multiple Range Test was completed to evaluate pairwise comparisons between groups. The anxiety disorder group scored significantly higher than each of the other two diagnostic groups on the anxiety subscale. There was no significant difference on Anxiety subscale score between the depressive disorder and no anxiety or depressive disorder groups.

COMPARISON

OF

TABLE FEMALES AND

4
hhLE.S ON RATING SCALES

Females M Anxiety Scales ARC-R Anxiety Subscale ARC-R Physiological Subscale RCMAS VAA-R Depression Scales CD1 CDS CDRS-R SD N M

Males SD N F 4f P

6.8 5.5 11.5 5.3 12.7 64.5 42.2

4.0 2.7 7.4 2.1 9.2 26.4 13.5

84 12 76 42 79 74 79

5.3 3.8 8.3 4.0 8.2 51.7 36.7

3.4 2.9 4.9 1.9 4.9 23.6 10.9

115 14 109 64 106 101 108

8.06 2.41 12.60 10.96 18.73 11.40 9.71

1,197 1,24 1, 183 1,104 1, 183 1.173 1, 185

.005 .134 <.OOl .OOl 401 <.oOl .002

Note. ARC-R = Anxiety Rating for Children - Revised. RCMAS = Revised Childrens Manifest Anxiety Scale, VAA-R = Visual Analogue Scale for Anxiety - Revised, CD1 = Childrens Depression Inventory, CDS = Childrens Depression Scale, CDRS-R = Childrens Depression Rating Scale -Revised.

108

G. A. BERNSTEIN

ITI- AL.

The ROC analysis evaluating the ability of the ARC-R Anxiety subscale to distinguish subjects with an anxiety diagnosis from those without an anxiety diagnosis is presented in Figure 1. The figure presents sensitivity (true positive rate) and one minus specificity (false positive rate) values for cutoffs between 1 and 12. The AUC is .70 (SD = .05), which is significantly higher than the line of no information (p < .oOl). The ROC curve can be used to determine, for a given cutoff score, what percentage of subjects with and without an anxiety disorder scored at or above (or conversely, below) that point. For example, the values corresponding to a cutoff score of 4 are: sensitivity = .75; one minus specificity = .42. This would indicate that 75% of subjects with an anxiety disorder scored 4 or higher (i.e., sensitivity = .75) while only 25% scored below 4. Similarly, 58% of all subjects without an anxiety disorder scored below 4 (i.e., specificity = .58), while 42% scored 4 or above. A cutoff value of 3 would result in a sensitivity of .82 and a specificity of .31; a cutoff value of 5 would result in a sensitivity of .67 and a specificity of .6 1. DISCUSSION The ARC-R, to the best of our knowledge, is the only clinician rating scale for anxiety that is specifically designed for use in children and adolescents.

0.4

WC = .70 SD=.06

0.0

0.2

0.4 (7 - Specificity)

0.6

0.8

1.0

FIG. 1. ARC-R

AMuen

Smsax.e

ROC CURVE.

ARC-R:

RELIABILITY

AND

VALIDITY

109

The test-retest reliability of r = .93 for the total Anxiety score indicates stability of the measure over a short period of time for which the instrument assesses. Test-retest reliability was evaluated by using older children and adolescents (11-18 years old). Reliability of the ARC-R may have been influenced by the subjects ages because reliability has been shown to increase with age (Edelbrock, Costello, Dulcan, Kalas, & Conover, 1985; Schwab-Stone, Fallon, Briggs, & Crowther, 1994). Interrater reliability of .95 indicates agreement of raters when using the instrument, The instrument also has good internal reliability (Cronbachs alpha of .80). These properties indicate that the ARC-R is a reliable measure of clinician rated anxiety. Correlations between the Anxiety subscale of the ARC-R and RCMAS (r = .62) (Table 3) and between the anxiety subscale of the ARC-R and the VAA-R (r = .62) suggest all three anxiety scales are measuring similar constructs and support the convergent validity of the instrument. Gf the three subscales on the RCMAS, the ARC-R Anxiety subscale correlates best with the Worry/Oversensitivity subscale (r = .61), which has been identified as the subscale on the RCMAS that best identifies children with anxiety disorders (Mat&on, Bagnato, & Brubaker, 1988). In a study of referred children, only the wonyloversensitivity factor of the RCMAS significantly distinguished the anxiety group from the psychiatric control group (Mattison et al., 1988). The correlations between the Anxiety subscale of the ARC-R and selfreport anxiety instruments (r = .62) were higher than the correlations between the Anxiety subscale of the ARC-R and self-report depression instruments (r = .54-.56) (Table 3), providing some support for the divergent validity of the instrument. Although the correlation between the ARC-R Anxiety subscale and CDRS-R was r = .67, this may be explained by the observations that both are clinician ratings and that there is a substantial rate of comorbidity of anxiety and depression in this sample. Scoring high on both anxiety and depression instruments may reflect the child or adolescent experiencing both anxiety and depressive symptoms (Bernstein & Garfinkel, 1992). More symptomatic children and adolescents often have high scores on multiple measures as a function of their severity (Bernstein, 1991). If only subjects with pure anxiety disorders were included, the correlation between the ARC-R Anxiety subscale and CDRS-R was lower (r = S8). Furthermore, anxiety instruments do not always measure pure anxiety symptomatology, nor do depression instruments measure pure depressive symptoms (Bernstein & Garfinkel, 1992). Moderate correlations between anxiety and depression instruments are consistently reported in assessment of children and adolescents (Strauss, Lease, Last, & Francis, 1988). Nevertheless, the finding that there are moderate correlations between the ARC-R Anxiety subscale and the depression scales suggest a potential shortcoming of this instrument. Further investigation of the relationship between the ARC-R and measures of depression in a larger, pure anxious sample is needed to evaluate the divergent validity of the ARC-R.

110

G. A. B@RNSTElN

GT AL.

The Physiological subscale of the ARC-R only correlated significantly with the CD1 (r = .41) and the CDRS-R (r = .52). Due to the small sample size used to evaluate the Physiological subscale (N = 26), these correlations and other correlations between the Physiological subscale and measures of anxiety and depression should be viewed as preliminary. However, it appears that somatic complaints may be symptoms of depression as well as symptoms of anxiety. In a study by McCauley, Carlson, and Calderon (1991) exploring the relationship between anxiety, depression, and somatic complaints, the presence and severity of depression predicted somatic complaints, even when anxiety was controlled. Furthermore, a substantial number of the subjects who met criteria for depression had moderate to severe somatic complaints. The authors concluded that the association between depression and somatic complaints could not be accounted for by anxiety. Females scored significantly higher than males on the Anxiety subscale. Females also were higher than males on the Physiological subscale, although this finding was not statistically significant. The gender differences on the ARCR are consistent with gender findings on other anxiety measures (Bernstein & Garilnkel, 1992; Ollendick et al., 1989; Reynolds & Richmond, 1985). Children and adolescents with a DSM-III-R anxiety disorder had significantly higher scores on the Anxiety subscale than those without an anxiety disorder. This finding provides data for the discriminant validity of the Anxiety subscale of the ARC-R. However, a limitation of this study is the use of retrospective chart reviews for establishing the diagnoses of the subjects. The methodology would have been stronger if structured psychiatric interviews had been used for diagnoses. Therefore, the conclusions about discriminant validity need to be viewed tentatively. The diagnostic utility of the ARC-R Anxiety subscale in screening for anxiety disorder in psychiatric patients is suggested by the ROC analysis. The Anxiety subscale was shown to significantly differentiate between psychiatric patients with and without anxiety disorders. Cutoff points of 4 or 5 appear most appropriate in providing an optimal balance between sensitivity and specificity. Further study is needed to determine the diagnostic utility of the ARC-R in a nonpsychiatric setting. Somatic complaints are often a manifestation of anxiety. Because it measures somatic symptoms, the ARC-R may prove to be particularly useful with the changes made from DSM-III-R (American Psychiatric Association, 1987) to DSM-ZV (American Psychiatric Association, 1994). Criteria for separation anxiety disorder, generalized anxiety disorder, and panic disorder emphasize physical symptoms associated with anxiety. Therefore, with DSM-ZV, more attention will be focused on somatic symptoms as we diagnose children and adolescents with anxiety symptoms. Studies evaluating somatic symptoms in anxious children suggest that further investigation, with an instrument such as the ARC-R, will be fruitful. Several studies have noted that somatic complaints are common in children

ARC-R:

RELIABILITY

AND

VALIDITY

111

with separation anxiety disorder and panic disorder (Last, 1991; Livingston, Taylor, & Crawford, 1988). In an investigation including 76 children, anxious children reported significantly more physical symptoms than normal controls, and several of the symptoms differentiated the two groups (Beidel et al., 1991). As to children and adolescents with recurrent abdominal pain, higher levels of anxiety in these children compared to those in normal children have been reported (Hodges, Kline, Barbero, 8z Woodruff, 1985; Walker & Greene, 1989; Wasserman, Whitington, & Rivara, 1987). In our experience with the ARC-R, some youth appear to minimize their anxiety symptoms on the five psychological anxiety items, but readily endorse multiple symptoms on the six physiological items. Perhaps they find it more acceptable to report physical symptoms than emotional symptoms. The use of only 26 outpatients in the evaluation of the Physiological subscale is a shortcoming of the validation study. However, data on the psychometric properties of this subscale are currently being collected. The entire ARC-R instrument is currently being used as an outcome measure in a treatment study of schoolrefusing adolescents with comorbid anxiety disorder and major depression. It will be of interest in the study to examine the relationship between change on the Anxiety subscale and change on the Physiological subscale. The ARC-R is a reliable instrument for clinician-rated anxiety in children and adolescents. The validity of the Anxiety subscale has been established. This instrument fills the void of a much needed clinician rating scale for anxiety. It provides the advantage of assessing both psychological and physiological symptoms of anxiety. Its psychometric properties justify the use of this rating scale as a clinical instrument for assessing anxiety and demonstrate its usefulness as a screening instrument. As a research instrument, the ARC-R may also prove to be an effective tool in evaluating the efficacy of treatments for anxiety in children and adolescents. The ARC-R complements the other anxiety instruments currently available by providing a perspective that is not bound by DW-ZV criteria, as in sbrucRued and semistru~ interviews. It also overcomes some of the potential weaknesses of self-report and parental report measures because it integrates the clinicians expertise with the child or adolescents report of symptoms. The ARC-R would be most useful as part of a comprehensive assessment which includes a diagnostic interview, self-report measures of anxiety and depression, and parental report measures. REFERENCES
Achenbach, T. M. (1991). Manual for the child behavior checklist/4-(18) and 1991 profile. Burlington, VT: University of Vermont Department of Psychiatry. American Psychiatric Association (1987). Diagnostic and statistical manual of mental disorders (3rd ed.. rev.). Washington, DC: Author. American Psychiatric Association (1994). Diagnostic and statistical manual of mental disorders (4th ed.). Washington. DC: Author. Bartko, J. J.. & Carpenter. W. T. (1976). On the methods and theory of reliability. Journal of NervousandMentalDiseose, 163.307-317.

112

G. A. BERNSTEIN

El- AL.

Beidel, D. C., Christ, M. A. Cr., & Long, P. J. (1991). Somatic complaints in anxious children. Journal of Abnormal Child Psychology, 19.659670. Bernstein, G. A. (1991). Comorbidity and severity of anxiety and depressive disorders in a clinic sample. Journal of the American Academy of Child and Adolescent Psychiatry, 30.43-50. Bernstein, G. A., & Garfnkel, B. D. (1992). The Visual Analogue Scale for Anxiety - Revised: Psychometric properties. Journal of Anxiety Disorders, 6,223-239. Bernstein, G. A., Svingen, P. H., & Garfinkel, B. D. (1990). School phobia: Patterns of family functioning. Journal of the American Academy of Child and Adolescent Psychiatry, 29.24-30. Chambers, W. J., Puig-Antich. J.. Hirsch, M., Paez, P., Ambrosini. P. J., Tabrizi, M. A., & Davies, M. (1985). The assessmentof affective disorders in children and adolescents by semistructured interview: Test-retest reliability of the schedule for affective disorders and schizophrenia for school-age children, present episode version. Archives of General Psychiatty, 42,6%-702. Clark, D. B.. & Donovan, J. E. (1994). Reliability and validity of the Hamilton Anxiety Rating Scale in an adolescent sample. Journal of the American Academy of Child and Adolescent
Psychiatry, 33,354-360.

Costello, E. J. (1989). Child psychiatric disorders and their correlates: A primary care pediatric sample. Joumal of the American Academy of Child and Adolescent Psychiatry, 28.851-855. Edelbrock, C., Costello, E. J., Dulcan, M. K.. Kalas, R, & Conover, N. C. (1985). Age differences in the reliability of the psychiatric interview of the child. Child Development, 56265-275. Erbaugh, S. E. (1984). Anxiety Rating for Children. Unpublished instrument. Hamilton, M. (1959). The assessment of anxiety states by rating. British Journal of Medical
Psychology, 32,50-55.

Hanley, J. A., & McNeil, B. J. (1982). The meaning and use of the area under a Receiver Operating Characteristic (ROC) curve. Diagnostic Radiology, 143.29-36. Hetjaoic, B., & Reich, W. (1982). Development of a structured psychiatric interview for children: Agreement between child and parent on individual symptoms. Journal of Abnormal Child Psychology, 10,307-324. Hodges, K., Kline, J. J., Barbero, G., & Woodruff, C. (1985). Anxiety in children with recurrent abdominal pain and their parents. Psychosomatics, 26859-866. Hodges, K.. McKnew, D.. Burbach, D. J., & Roebuck L. (1987). Diagnostic concordance between the Child Assessment Schedule (CAS) and the Schedule for Affective Disorders and Schizophrenia for School-Age Children (K-SADS) in an outpatient sample using lay interviewers. Joumal of the American Academy of Child and Adolescent Psychiatry, 26.654-661. Hsiao, J. K., Bartko, J. J., & Potter, W. 2. (1989). Diagnosing diagnoses: Receiver operating characteristic methods and psychiatry. Atchive of Geneml Psychiatry, 46.664667. Kashani, J. H., & Grvaschel, H. (1990). A community study of anxiety in children and adolescents. American Journal of Psychiatry, 147.313-318. Klein, R. G. (1991). Parent-child agreement in clinical assessment of anxiety and other psychopathology: A review. Journal of Anxiety Disorders, 5,187-l 98. Knight, D., Hensley, V. R. & Waters, B. (1988). Validation of the Childrens Depression Scale and the Childrens Depression Inventory in a pmpubettal sample. Journal of Chitif Psychology and
Psychiatry and Allied Disciplines. 29,853-863.

Kovacs. M. (1981). Rating scalesto assessdepression in school-aged children. A&I Paedop@tiotrica, 46.305-315. Lang, M., & Tisher, M. (1978). Chitdrens Depression Scale. Melbourne: Australian Council for Educational Research. Last, C. G. (1991). Somatic complaints in anxiety disordered children. Journal ofAnriety Disonicrs, 5125-138. Last, C. G. (1992). Anxiety disorders in childhood and adolescence. In W. M. Reynolds (Ed.), Internalizing disorders in &&en and adolescents. New York: John Wiley and Sons. Lee, S. W., Pier&, W. C., Friedlander, R. & Collamer, W. (1988). Concurrent validity of the Revised Childrens Manifest Anxiety Scale (RCMAS) for adolescents. Educational and
Psychological Measurement, 48.429-433.

ARC-R:

RELIABILITY

AND

VALIDITY

113

Livingston, R. Taylor, J. L., & Crawford, S. L. (1988). A study of somatic complaints and psychiatric diagnosis in childten. Journal of the American Academy of Child ad Aabkmwt Psych&y, 27,
185-187.

McCauley, E., Carbon. G. A., & Calderon, R. (1991). The role of somatic complaints in the diagnosis of depression in children and adolescents. Journul of the American Academy of Child
and Adolescent Psych&y, 38.631-635.

Maier, W., Buller, R., Philipp, M., % Heuser, I. (1988). The Hamilton Anxiety Scale: Reliability, validity and sensitivity to change in anxiety and depressive disorders. Joum~l ofA$xtive
Disorders, 14,61.

Mattison, R. E., Bagnato, S. J., & Brubaker, B. H. (1988). Diagnostic utility of the Revised Childrens Manifest Anxiety Scale in children with DSM-III anxiety disorders. Jourrud of
Anriefy Disonlers, 2.147-155.

Mote& M. M., Fine, S., Haley, G., Bi Marriage, K. (1985). Childhood and adolescent depression: Child-report versus parent-report information. Journal of the American Academy of Child
Psychiatry, 24,298-302.

Murphy, J., Berwick, D., Weinstein, M., Borus. J., Budman. S., 8i Klerman, G. (1987). Performance of screening and diagnostic tests. Archives of General Psych&y, 44.550-555. Ollendick, T. H. (1983). Reliability and validity of the Revised Fear Survey Schedule for Children (FSSC-R). Behuviour Research and Therapy, 21,685-692. Ollendick. T. H., King, N. J., & Frary, R. B. (1989). Feats in children and adolescents: Reliability and generalizability across gender, age and nationality. Behnviour end Research Therapy, 27, 19-26. Poznanski, E. 0, Freeman, L. N., & Moktos, H. B. (1985). Childrens Depression Rating Scale Revised. Psychopharmacology Bulletin, 4.979-989. Poznanski, E. 0.. Grossman, J. A., Buchsbaum, Y., Banegas, M., Freeman, L., & Gibbons, R. (1984). Preliminary studies of the reliability and validity of the Childrens Depression Rating
Scale. Joumol of the American Academy of Child Psychiatry. 23,191-197.

Reynolds, C. R. (1980). Brief reports: Concurrent validity of What I Think and Feel: The Revised Childrens Manifest Anxiety Scale. Joumnt of Consulting and Clinical Psychology, 48,
774-775,

Reynolds, C. R., & Richmond, B. 0. (1978). What I think and feel: A revised measure of childrens manifest anxiety. Journal of Abnormal Child Psychology, 6.271-280. Reynolds, C. R., & Richmond, B. 0. (1985). Revised Childrens Manifest Anxiety Scale (RCMAS) menuul. Los Angeles, CA: Western Psychological Services. Rotundo, N., & Hensley, V. R. (1985). The Childrens Depression Scale. A study of its volidiry.
Joum~l of Child Psychology and Psychiatry, 26.917-927.

Saylor, C. F., Finch, A. J., Spirito, A., & Bennett, B. (1984). The Childrens Depression Inventory: A systematic evaluation of psychometric properties. Journal of Consulting and Cfinical
Psychology, 52,955-967.

Schwab-Stone, M., Fallon, T.. Briggs, M., Crowther, B. (1994). Reliability of diagnostic reporting for children aged 6-1 I years: A test-retest study of the Diagnostic Interview Schedule For children -Revised. American Journal of Psychiatry, 151.1048-1054. Shain, B. N., Naylor, M. & Alessi. N. (1990). Comparison of self-rated and clinician-rated measures of depression in adolescents. American Jourmal of Psychinrry, 147.793-795. Silverman, W. K., & Nelles, W. B. (1988). The Anxiety Disorders Interview Schedule for Children. Journal of the American Academy of Child and Adolescenr Psychintry. 27,772-778. Spielberger, C. (1973). Manual for the State-Trait Anxiety Inventory for Children. Palo Alto: Consulting Psychologists Press. Strauss, C. C., Lease, C. A., Last, C. G., & Francis, G. (1988). Overanxious disorder: An examination of developmental differences. Journal ofAbnomw[ Child Psychology, 16,433-443. Tisher, M., & Lang, M. (1983). The Childrens Depression Scale: Review and further developments. In D. P Cantwell & G. A. Carlson (Eds.), Affective disorders in childhood and ndofescence: An update (pp. 181-203). New York: Spectrum Publications.

114

0. A. BlXNSIElN

ET AL.

Walker, L. S., & Greene, J. W. (1989). Childreo with recurrent abdominal pain and their parents: More somatic complaints, anxiety, and depression than other patient families? Journal of
Pediatric Psychology, 14.23 l-243.

Wasserman, A. L., Whitington, R F., & Rivara, F. P (I 988). Psychogenic basis for abdominal pain in children and adolescents. Journal of the American Academy of Child and Adolescent Psychiatry, 27,179-l 84. Wisniewski, J. J., Mulick, J. A., Genschaft. J. L., & Coury, D. L. (1987). Test-retest reliability of the Revised Childrens Manifest Anxiety Scale. Perceptaal and Motor Stilk. 65.67-70. Wti. R. D., Lachar, D., Klieodinst, J. K., & Seat, F? D. (1977). Multidimensional description of child personality: A manaal for the Personali?y Inventory for Children. Las Angeles: Western Psychological Services.

You might also like