Testing the Cross-Ethnic Construct Validity

of the Brief Symptom Inventory

Maanse Hoe
John Brekke
University of Southern California

Objective: The purpose of the present study was to examine the cross-ethnic construct validity of the Brief
Symptom Inventory (BSI). Method: The sample consisted of 1,166 individuals diagnosed with severe and persis-
tent mental illness who were receiving treatment in community-based mental health programs. Multiple-group con-
firmatory factor analysis was used to test measurement invariance of the BSI’s second-order factor model across
three ethnic groups (African Americans, Caucasians, and Latinos). Results: The data supported the configural
invariance of the BSI’s second-order factor model as well as the full metric invariance and the partial metric
invariance. Conclusions: The present study presented substantial empirical evidence for the construct validity of
the BSI’s second-order factor model across the three ethnic groups.

Keywords: Brief Symptom Inventory (BSI); second-order factor; measurement invariance; people with severe
and persistent mental illness (SPMI); confirmatory factor analysis

The purpose of the present study was to examine the
construct validity of the Brief Symptom Inventory (BSI;
Derogatis, 1993) across three ethnic groups (African
Americans, Caucasians, and Latinos) for individuals diag-
nosed with severe and persistent mental illness (SPMI) in
community-based mental health programs. Measure-
ment invariance across ethnic groups is critical in men-
tal health services and research because cultural factors
are associated with health-related behaviors and symptom
presentations (U.S. Department of Health and Human
Services, 2001). Understanding cross-ethnic symptom
variations also helps clinicians accurately assess clients,
which can contribute to providing culturally relevant
interventions. Cross-ethnic comparisons can be inter-
preted validly when the measures used are invariant
across ethnic groups because measurement invariance
leads to unbiased scientific and clinical inferences about
differences among individuals and groups (Horn &
McArdle, 1992). Central to measurement invariance is
the construct validity of measures across ethnic groups.
Note: Address correspondence to Maanse Hoe, PhD, School of Social Work,
University of Southern California, 669 West 34th Street, MRF Los Angeles,
CA 90089-0411; e-mail: hoe@usc.edu.
Research on Social Work Practice, Vol. 19 No. 1, January 2009 93-103
DOI: 10.1177/1049731508317285
© 2009 Sage Publications
Several studies emphasize the need for assessment tools
that are valid across ethnic groups (Eddy, 1998; Krause,
2005; Ramirez, Ford, Stewart, & Teresi, 2005;
Snowden, 2003; Switzer, Wisniewski, Belle, Dew, &
Schultz, 1999).
The BSI is used widely as a measure of psychological
symptoms. For example, an Ovid Medline search yielded
more than 300 publications since 2000 that used the BSI,
and PsycINFO listed more than 1,900 publications since
2000 that have used the BSI. It has particular relevance to
social work practice and to research on people diagnosed
with SPMI. Study topics for the SPMI population have
included changes in psychiatric symptoms (Margolese,
Carlos Negrete, Tempier, & Gill, 2006), substance abuse
(Primm et al., 2000), quality of life (Ritsner, 2003), invol-
untary outpatients (Swartz, Swanson, Wagner, Burns, &
Hiday, 2001), treatment effectiveness (Kingsep, Nathan,
& Castle, 2003), service delivery systems (Mares, Young,
McGuire, & Rosenheck, 2002), and measurement studies
(O’Malia, McFarland, Barker, & Barron, 2002; Preston &
Harrison, 2003).
A previous study (Hoe & Brekke, in press) examined
the cross-ethnic invariance of a secondary factor model
of the BSI. A secondary factor model deals with factor
scores rather than item scores; therefore, it is now
important to test the second-order factor model based on
item-level variances, which will allow for confidence in
both the item- and factor-level invariance of the BSI.

93
94 RESEARCH ON SOCIAL WORK PRACTICE
The present study tested the second-order factor model
of the BSI and the measurement invariance of the model
across African Americans, Caucasians, and Latinos. The
second-order factor model of the BSI has been hypoth-
esized but not tested, and its measurement invariance
across ethnic groups is unknown. This is a critical issue
because when construct validity and measurement
invariance across ethnic groups are not confirmed, the
presence of measurement bias because of ethnicity can
confound a wide range of analyses leading to biased
results and inaccurate conclusions across a range of
issues.
Hypothetical Factor Structure of the BSI
The BSI was developed by Derogatis (1975) for
assessing psychological symptoms. This inventory is an
abbreviated version of the Symptom Checklist (SCL-
90-R; Derogatis, 1977). The BSI consists of nine subscales
representing nine dimensions of psychiatric symptoms,
which are somatization, obsessive-compulsive, interper-
sonal sensitivity, depression, anxiety, hostility, phobic
anxiety, paranoid ideation, and psychoticism. These fac-
tors were recovered in a factor analytic study of the inter-
nal structure of the BSI (Derogatis & Melisaratos, 1983)
using principal components analysis. However, a number
of researchers have reported that the nine factors of the
subscales in the BSI or in the SCL-90-R are not recov-
ered in their samples (Benishek, Hayes, Bieschke, &
Stoffelmayr, 1998; Bonynge, 1993; Boulet & Boss,
1991; Coelho, Strauss, & Jenkins, 1998; Cyr, McKenna-
Foley, & Peacock, 1985; Hafkenscheid, 1993; Hayes,
1997; Heinrich & Tate, 1996; Holcomb, Adams, &
Ponder, 1983; Johnson, Murphy, & Dimond, 1996;
Kellett, Beail, Newman, & Hawes, 2004; Piersma, Boes,
& Reaume, 1994; Ruiperez, Ibanez, Lorente, Moro, &
Ortet, 2001; Schwartzwal, Weisenberg, & Solomon,
1991). Furthermore, many of these studies have reported
that a single factor accounts for most of the observed
variance among the BSI (Boulet & Boss, 1991; Piersma
et al., 1994) or SCL-90-R items (Bonynge, 1993; Cyr
et al., 1985), which suggests that the BSI might indicate
general psychological distress rather than multidimen-
sional symptoms.
There are two possible explanations for the variations
in the factor structure of the BSI across studies
(Ruiperez et al., 2001). The first explanation is that dif-
ferences in the factor analysis procedures across studies
might result in the variations. Exploratory methods such
as principal components analysis, which is common in
the previous factor analytic studies cited above, only
indirectly test hypotheses regarding factor structure
(Long, Harring, Brekke, Test, & Greenberg, 2007)
because they are used mainly for data reduction rather
than for finding the internal structure of a psychological
construct (see details in Reise, Waller, & Comrey,
2000). The second explanation is that the use of differ-
ent samples might have resulted in the variations in the
factor structure of the BSI (Schwartzwal et al., 1991,
cited in Ruiperez et al., 2001). This explanation is par-
ticularly true in studies using multiethnic samples
(Aroian, Patsdaughter, Levin, & Gianan, 1995; Coelho
et al., 1998; Ruiperez et al., 2001).
Given that studies have questioned the construct
validity of the BSI in general and when applied to ethnic
samples, we proposed to test the second-order factor
model. From the classical test theory perspective, a
second-order factor model is considered a viable alter-
native to an oblique factor model (i.e., multiple factor
models with factor correlations). Essential to the notion
of a second-order factor model is the possibility of the
existence of a single second factor in the BSI. In this
regard, it is known that the nine subscales of the BSI are
correlated to a notable degree (Derogatis, 1993). The
substantive correlations among the nine subscales sug-
gest the possibility of a second-factor model for items of
the BSI as discussed in Chen, Sousa, and West (2005).
The second-order factor model holds nine primary fac-
tors representing the nine symptom dimensions of the
BSI but adds the assumption of a higher common factor
causing the nine primary factors (see Figure 1).
A second-order factor model is plausible for the
internal structure of the BSI. Specifically, the second-
order factor model is superior to the oblique nine-factor
model of the BSI because (a) it represents correlations
among the nine symptom dimensions of the BSI in a
more parsimonious way with fewer parameters (Chen
et al., 2005); (b) hierarchical solutions (i.e., the second-
order model in the context of the present study) are
appropriate for many psychological instruments as most
psychological constructs are composed of multiple, cor-
related facets (Floyd & Widaman, 1995); and (c) the
second-order factor model provides a scientific founda-
tion to support the prevailing practice of using the BSI
in research and clinical practices. For example, in clini-
cal practice, clinicians have used symptom profile
analyses by using the nine separate subscale scores but
also a total symptom score (Derogatis, 1993). Researchers
have used the Global Severity Index (GSI) of the BSI,
which is a total mean score of the 53 BSI items. They
also often have selected one or several of the nine sub-
scales by using the practice of item parceling (combining
items within scales or subscales, which in turn represent
variables of interests).
 Hoe, Brekke / BRIEF SYMPTOM INVENTORY 95

e1 e2 e3 e4 e5 e6 e7 e8 e9 e10 e11 e12 e13 e14 e15 e16 e17 e18 e19 e20 e21 e22 e23 e24 e25 e26 e27 e28 e29

2 7 23 29 30 33 37 5 15 26 27 32 36 20 21 22 42 9 16 17 18 35 50 1 12 19 38 45 49

e50 SOM e51 O-C e52 I-S DEP e54 ANX

e53

Symptom

e55 HOS e56 PHOB e57 PAR e58 PSY

6 13 40 41 46 4 10 24 48 51 3 14 34 44 53
8 28 31 43 47

e30 e31 e32 e33 e34 e35 e36 e37 e38 e39 e40 e41 e42 e43 e44 e45 e46 e47 e48 e49

Figure 1: The Hypothesized Second-Order Factor Model of the BSI

NOTE: The numbers in the observed variables (squares) indicate the BSI item numbers. The labels in the latent variables (ovals) indicate the
BSI factors: Symptom refers to a second-order common factor representing general psychological distress, SOM refers to a primary factor rep-
resenting somatization, O-C refers to obsessive-compulsive, I-S refers to interpersonal sensitivity, DEP refers to depression, ANX refers to anx-
iety, HOS refers to hostility, PHOB refers to phobic anxiety, PAR refers to paranoid ideation, and PSY refers to psychoticism. BSI = Brief Symptom
Inventory.

All of these applications of the BSI can be subsumed
by the second-order factor model from the classical test
theory perspective. Theoretically speaking, the second-
order factor model of the BSI could yield a valid esti-
mate of psychological distress because (a) the common
factor in the second-order factor model is composed of
the shared variance of primary factors (i.e., the first-
order factors) after eliminating the unique variance of
each primary factor, (b) the second-order factor model
allows for the simultaneous testing of the invariance of
subscales and of the items across groups of interest. This
is useful particularly when subscales are commonly
used, as is the case with the BSI.
Measurement Invariance
Measurement invariance is a logical prerequisite to
making decisions that address group differences because
finding differences or similarities across individuals and
groups cannot be interpreted clearly unless measure-
ment invariance is present (Horn & McArdle, 1992).
Measurement invariance is one of the best ways for test-
ing the conceptual and scale equivalence of measures in
mental health research (U.S. Department of Health and
Human Services, 2001) because measurement invari-
ance is, by definition, indicative of the equivalence of
measured constructs across two or more independent
groups (Chen et al., 2005, p. 472).
Measurement invariance is conceptualized by con-
straining several elements in hypothesized factor struc-
ture models. These elements are the number of factors,
a factor pattern, the magnitudes of factor loadings,
unique variances, and the covariances (or correlations)
between factors. According to Jöreskog (1979a), the
factor analytical model is represented as follows:
y = Λf + z, (1)
where y is a vector of order p of observed scores, f is a
vector of k of latent common factor scores, z is a vector
of order p of unique scores, and Λ is a p × k matrix of
factor loadings. The assumptions of classical test theory
are that E(f) = E(z) = 0, E(ff`) = Φ, which is the factor
correlation matrix, and E(zz′) is Ψ, which is the diago-
nal matrix of unique factors. Based on these assump-
tions, the correlation matrix of y, Σ(yy′), is:
Σ = ΛΦΛ′ + Ψ. (2)
This equation represents the fundamental idea of
factor analysis, which is that latent common factors
account for all correlations among the observed scores
96 RESEARCH ON SOCIAL WORK PRACTICE
in y. When factor analysis is extended to simultaneous
factor analysis in several populations, as in this study,
Equation 2 (Jöreskog, 1979b, p. 191) becomes:
Σ(g) = Λ(g)Φ (g) Λ′(g) + Ψ (g), (3)
where g = 1, 2, . . . , G groups. In Equation 3, Σ(g) is the
correlation matrix at Group g; Λ(g) is the factor pattern
matrix containing the factor loadings at Group g; Φ(g) is
the factor correlation matrix at Group g; and Ψ (g) is the
diagonal matrix of unique factors at Group g.
Measurement invariance is examined by testing
whether the internal structures of measures are the same
across groups, which is called factorial invariance.
Different types of factorial invariance can be specified
by constraining elements of Equation 3 (i.e., configura-
tions and parameter values) to be the same across groups.
There are two types of factorial invariance: configural
invariance and metric invariance (Horn & McArdle, 1992).
Configural invariance means that the factor structure
underlying items is equivalent across groups. It requires
the same factor pattern matrices, Λ(g) in Equation 2, in
order to have the same factor configuration across all
groups, but it allows the magnitudes of factor loadings
and their measurement error variances to vary across all
groups. In the context of this study, all items of the BSI
should load in the same factor pattern, which is invari-
ant across the three groups.
Metric invariance is conceptualized by placing addi-
tional equality constraints across different groups on the
parameters of a configural invariance model. Weak met-
ric invariance is defined by the presence of equal factor
loadings across groups but with the error variance
allowed to vary across groups. In the context of the pre-
sent study, weak metric invariance requires that the
same BSI items load onto the same factors with equal
factor loadings across the three ethnic groups, but both
the factor correlation and the measurement errors are
free to vary across groups. Strong metric invariance is
defined by placing an additional equality constraint on
the correlations among the measured variables. Last,
strict metric invariance is defined by placing additional
equality constraints on the measurement errors.
Horn and McArdle (1992) have stated that metric
invariance is less reasonable to expect in human mea-
surements because only the configuration of zero and
nonzero pattern coefficients (i.e., configural invariance)
realistically can be expected to remain invariant across
groups. Therefore, Byrne, Shavelson, and Muthen (1989)
suggested a practical invariance model, which is called
partial metric invariance. Partial metric invariance
requires configural invariance but allows some loadings
on items to be different across groups, which contrasts
with full metric invariance in which all loadings on
items should be the same across groups. The partial met-
ric invariance assumes that the noninvariant items would
not affect cross-group comparisons to any significant
extent (Cheung & Rensvold, 1999), which was demon-
strated in Byrne et al. (1989). The partial metric invari-
ance is particularly relevant in cross-cultural studies
(Byrne & Watkins, 2003; Cheung & Rensvold, 1999;
Steenkamp & Baumgartner, 1998; Vandenberg, 2002).
METHOD
Study Participants
The sample consisted of 1,166 individuals diagnosed
with SPMI in community-based mental health programs.
The sample data were extracted from data routinely
gathered by county mental health staff and entered into
a management information system administered by the
Mental Health Association of Los Angeles, California.
All participants in the sample were receiving treatment
for SPMI from county-funded clinics. To be eligible for
services, an individual had to meet all of the following
criteria: (a) diagnosis of schizophrenia, depression,
bipolar disorder, or borderline personality disorder;
(b) severe functional impairment; and (c) at least one
psychiatric hospitalization in the year prior to treatment.
Despite the diagnostic heterogeneity, this is a sample
that can be described as having a severe and persistent
mental illness. All participants were administered the
BSI as part of routine psychosocial assessments during
treatment. Approval for this study was received from the
appropriate university Institutional Review Board.
There were on average 2.53% nonresponses across
the 53 items of the BSI. Scores for the nonresponse cases
were derived by using a multiple imputation method in
NORM (Schafer & Graham, 2002). Sample characteris-
tics as well as the mean of the BSI subscales and GSI
scores for each group are presented in Table 1. There
were more male than female participants in the sample,
and this did not vary by ethnic group. The average age
for all participants in the sample was 42.48 (SD = 10.18).
The mean differences of age across the three ethnic
groups were tested using ANOVA with Sheffé’s post hoc
test, indicating a statistically significant mean difference
( p = .007). However, the mean difference of 1.7 years
does not represent a substantial age difference across the
three groups.
 Hoe, Brekke / BRIEF SYMPTOM INVENTORY 97

TABLE 1: Sample Characteristics and Scores on the BSI for Three Ethnic Groups (N = 1,166)

African American Caucasian Latino Total Sample

Variable (n = 399) (n = 511) (n = 256) (N = 1,166)

Age
M 42.81 43.22 40.62 42.48
SD 9.2 10.5 10.81 10.18
Minimum 20 20 22 20
Maximum 67 68 68 68

n (%) n (%) n (%) n (%)

Gender
Male 227 (56.9) 290 (56.8) 138 (53.9) 655 (56.2)
Female 172 (43.1) 221 (43.2) 118 (46.1) 511 (43.8)

M SD M SD M SD M SD

Global Severity Index 0.88 0.75 1.13 0.87 0.94 0.78 1.00 0.81
Somatization 0.63 0.71 0.89 0.88 0.76 0.80 0.77 0.99
Obsessive-Compulsive 1.12 0.93 1.36 1.03 1.15 0.99 1.23 1.04
Interpersonal Sensitivity 0.96 1.00 1.28 1.09 1.03 0.98 1.11 1.01
Depression 0.97 0.96 1.25 1.07 1.00 0.95 1.10 0.95
Anxiety 0.85 0.85 1.19 1.02 1.00 0.92 1.02 0.89
Hostility 0.72 0.84 0.91 0.94 0.75 0.89 0.81 0.97
Phobic Anxiety 0.74 0.83 1.05 1.05 0.92 0.98 0.91 1.00
Paranoid Ideation 1.10 1.00 1.22 1.03 1.05 0.97 1.14 0.92
Psychoticism 0.89 0.89 1.11 0.97 0.86 0.88 0.97 0.81

NOTE: BSI = Brief Symptom Inventory.

Measure
The BSI is designed to assess psychological distress
and consists of nine subscales measuring the primary
dimensions of psychopathological symptoms (Derogatis,
1993). The nine subscales are somatization, obsessive-
compulsive, interpersonal sensitivity, depression, anxi-
ety, hostility, phobic anxiety, paranoid ideation, and
psychoticism. The BSI has 53 items, with 49 items
endorsed to the nine subscales. All items have a 5-point,
Likert-type scale that shows degrees of psychological
distress ranging from not at all to extremely. A Global
Severity Index (GSI), which is the average score of all
53 items, can be derived to represent the depth of gen-
eral psychological distress. Figure 1 shows the hypothe-
sized second-order factor model of the BSI. In Figure 1,
the numbers in the squares indicate the observed BSI
items according to the BSI manual (Derogatis, 1993).
The labels in the ovals represent latent factors respond-
ing to the nine subscales of the BSI. The small circles
connected to the observed BSI items refer to error vari-
ances of all observed variables (i.e., measurement
errors), corresponding to Ψ in Equation 2.
Data Analysis
Because there has been no previous study testing a
second-order factor model of the BSI, the data analysis
was conducted in two steps. In the first step, the hypoth-
esized second-order factor model was tested in the
Caucasian sample only (n = 511) using confirmatory
factor analyses. The Caucasian group was selected
because it was the largest group among the three ethnic
groups and was assumed to be reasonably homogeneous.
Testing measurement invariance would not proceed if a
single-group confirmatory factor analysis did not fit into
the data because the lack of fit would indicate that even
configural invariance would not hold across the three
ethnic groups.
In the second step, measurement invariance tests
across the three ethnic groups were conducted using
multiple-group confirmatory factor analyses (CFAs).
The multiple-group CFA is known as the best method
for testing measurement invariance because it provides
a chi-square statistic and goodness-of-fit indices empha-
sizing a priori model testing (Ployhart & Oswald, 2004).
Two types of measurement invariance (configural
invariance and full metric invariance) were tested sepa-
rately to determine the best fit model to the data. Partial
metric invariance also was tested as advocated above.
Because the BSI is composed of a 5-point, Likert-
type scale with a left-censored skewed distribution, it is
hard to assume a multivariate normal distribution of all
the items. To bypass this problem, unweighted least
squares (ULS) estimation was used in testing the
98 RESEARCH ON SOCIAL WORK PRACTICE
hypothesized factor models. ULS estimation does not
require observed variables to be normal in the model
identification (Browne, 1982). All factor analyses were
performed with AMOS 7 (Arbuckle, 2006). Multiple fit
indices are recommended to be used to evaluate overall
goodness-of-fit in models (Arbuckle, 2006; Kline,
2005). The Goodness-of-Fit Index (GFI), the Normed
Fit Index (NFI), and the Standardized Root Mean Square
Residual (SRMR) were used to decide whether or not
the hypothesized models would fit the data as ULS does
not provide some model fit indices, such as the
Comparative Fit Index (CFI) and root mean square error
of approximation (RMSEA) value, in Amos. Following
conventional recommendations (Arbuckle, 2006; Kline,
2005), a value higher than 0.95 for the GFI and NFI
indicates an acceptable model, and a model with a value
lower than 0.08 for the SRMR is accepted.
RESULTS
Single-Group Confirmatory Factor Analyses
The goodness-of-fit indices for the second-order
factor model were excellent (χ2 = 3545.25, df = 1118,
NFI = .99, GFI = .99, SRMR = .04), suggesting that the
second-order factor model would be a very plausible
model for explaining covariances among item scores as
well as covariances among the nine subdimension
scores. In other words, the excellent fit indicated that the
second-order factor model was supported in the
Caucasian sample of individuals with SPMI. Item and
primary factor loadings as well as their variances
accounted for are presented in Table 2. In the factor
analysis literature, items with loadings of less than .30
or .40 are indicative of a meaningless item for a factor
of interest (Floyd & Widaman, 1995; O’Keefe, Mennen,
& Lane, 2006). There were no items or primary factors
showing loadings of less than .40 in the estimated
second-order factor model of the BSI.
Measurement Invariance Across Three Ethic Groups
The second-order factor model of the BSI was tested
for invariance across the three ethnic groups of African
Americans, Caucasians, and Latinos using multiple-
group confirmatory factor analyses. Testing the configural
invariance produced negative variance in the residuals
of the psychoticism factor in African Americans and
Latinos, although Caucasians did not have the problem
of negative variance. Estimations with negative vari-
ances are considered improper solutions in confirmatory
factor analyses and estimated factor models, and thus
should be rejected (Wothke, 1993). However, negative
variances do not mean necessarily that a hypothesized
factor model is misspecified because the estimation
problem of negative variances might be caused by sam-
pling errors (Anderson & Gerbing, 1984; Chen, Bollen,
Paxton, Curran, & Kirby, 2001; Hair, Anderson, Tatham,
& Black, 1998; McDonald, 2004). A recommended solu-
tion to the problem of negative variances is that a model
should be reestimated with selected parameters being
constrained to zero or a small positive value. Following
Anderson and Gerbing’s (1988) suggestion, we con-
strained the residuals of the psychoticism factor to .005
and reestimated the configural invariance of the second-
order factor model.
The second-order factor model with the constraint of
.005 on the residuals of the psychoticism factor fit very
well to the data in terms of configural invariance (χ2 =
8371.50, df = 3357, NFI = .986, GFI = .988, SRMR =
.045), suggesting that the second-order factor model
was valid across the thee ethnic groups. Furthermore, it
fit well to the data in the metric invariance of item load-
ings (χ2 = 10847.33, df = 3437, NFI = .982, GFI = .984,
SRMR = .050) and the metric invariance of primary
factor loadings (χ2 = 11971.25, df = 3453, NFI = .980,
GFI = .982, SRMR = .052). The metric invariance of
primary factor loadings, however, produced negative
variances in the residuals of the anxiety factor and the
phobic anxiety factor for the group of African
Americans only. These results indicated that the internal
structure of the second-order factor model was fully
invariant between Latinos and Caucasians, so-called full
metric invariance, but was not invariant in the group of
African Americans. As the present data did not support
the full metric invariance across the three ethnic groups,
we proceeded to test partial metric invariance following
the recommended procedure of assessing measurement
invariance in Steenkamp and Baumgartner (1998) as
well as Vandenberg (2002). To test the partial metric
invariance of the second-order factor model, two equal-
ity constraints on the primary factor loadings of the anx-
iety factor and the phobic anxiety factor were set to be
free across the three ethnic groups. Testing partial met-
ric invariance resulted in a good model fit (χ2 =
11563.93, df = 3449, NFI = .981, GFI = .983, SRMR =
.051). Fit indices of the three invariance models dis-
cussed above are presented in Table 3.
In summary, as seen in Table 3, the results show that
the SRMR values for all three invariance models are lower
than .08, and that NFI and GFI for all three invariance
models exceed the recommended minimum value of
.95. The satisfactory goodness of fit with the configural
 Hoe, Brekke / BRIEF SYMPTOM INVENTORY 99

TABLE 2: Loadings in the Hypothesized Second-Order Factor Model of the BSI as Obtained in Caucasians

Item Number Item Description Loading

Somatization factor
2 Faintness or dizziness .54
7 Pains in heart or chest .59
23 Nausea or upset stomach .68
29 Trouble getting your breath .66
30 Hot or cold spells .65
33 Numbness or tingling in parts of your body .73
37 Feeling weak in parts of your body .81
Obsessive-Compulsive factor
5 Trouble remembering things .67
15 Feeling blocked in getting things done .73
26 Having to check or double-check what you do .69
27 Difficulty making decisions .76
32 Your mind going blank .72
36 Trouble concentrating .77
Interpersonal-Sensitivity factor
20 Your feelings easily hurt .72
21 Feeling that people are unfriendly or dislike you .77
22 Feeling inferior to others .74
42 Feeling very self-conscious with others .75
Depression factor
9 Thoughts of ending your life .61
16 Feeling lonely .71
17 Feeling blue .80
18 Feeling no interest in things .75
35 Feeling hopeless about the future .79
50 Feelings of worthlessness .82
Anxiety factor
1 Nervousness or shakiness inside .60
12 Suddenly scared for no reason .73
19 Feeling fearful .75
38 Feeling tense or keyed up .75
45 Spells of terror or panic .76
49 Feeling so restless you couldn’t sit still .71
Hostility factor
6 Feeling easily annoyed or irritated .75
13 Temper outbursts that you could not control .75
40 Having urges to beat, injure, or harm someone .60
41 Having urges to break or smash things .67
46 Getting into frequent arguments .72
Phobic Anxiety factor
8 Feeling afraid in open spaces .68
28 Feeling afraid to travel on buses, subways, or trains .67
31 Having to avoid certain things, placers, or activities because they frighten you .71
43 Feeling uneasy in crowds .78
47 Feeling nervous when you are left alone .76
Paranoid Ideation factor
4 Feeling others are to blame for most of your troubles .51
10 Feeling that most people cannot be trusted .71
24 Feeling that you are watched or talked about by others .75
48 Others not giving you proper credit for your achievement .66
51 Feeling that people will take advantage of you if you left them .76
Psychoticism factor
3 The idea that someone else can control your thought .40
14 Feeling lonely even when you are with people .70
34 The idea that you should be punished for your sins .64
44 Never feeling close to another person .69
53 The idea that something is wrong with your mind .72
The Second-Order factor
SOML Latent variable refers to somatization .81
O-CL Latent variable refers to obsessive-compulsive .93
I-SL Latent variable refers to interpersonal-sensitivity .96
DEPL Latent variable refers to depression .93
ANXL Latent variable refers to anxiety .98
HOSL Latent variable refers to hostility .96
PHOBL Latent variable refers to phobic anxiety .93
PARL Latent variable refers to paranoid ideation .90
PSYL Latent variable refers to psychoticism .99

NOTE: BSI = Brief Symptom Inventory.

100 RESEARCH ON SOCIAL WORK PRACTICE
TABLE 3: Fit Indices of the Partial Metric Invariance of the Second-Order Factor Model Across Three Ethnic Groups
Invariance Model
1. Configural invariance
2. Metric invariance of item loadings
3. Partial metric invariance of item loadings and primary factor loadings
Model difference between
Models 1 and 2
Models 2 and 3
NOTE: SRMR = standardized root mean square residual; GFI = Goodness-of-Fit Index; NFI = Normed Fit Index. Δχ2 = differences in chi-squares
of models; Δdf = differences in degree of freedom of models. ΔIFI = model differences in Bollen’s Incremental Fit Index (IFI). IFI values for each
invariance model are not calculated in Amos.
invariance model means that the second-factor model of
the BSI is extracted in each of the three ethnic groups,
which, in turn, supports the construct validity of the
BSI. The full metric invariance of item loadings and the
partial invariance of primary factor loadings support that
the nine dimension subscale scores in the BSI can be
compared across the three ethnic groups: African
Americans, Caucasians, and Latinos.
Comparisons of the Three Invariance Models
As the three invariance models fit well into the data,
these models were compared to determine the best-fit-
ting model to the data. Comparing structural equation
models is conducted conventionally using chi-square
difference tests (Bentler & Bonett, 1980). However,
Cheung and Rensvold (2002) in their simulation study
have recommended using the change in the goodness-
of-fit index in comparing measurement invariance mod-
els rather than the chi-square difference test considering
that the chi-square difference test is sensitive to large
sample sizes. We applied the differences in Bollen’s
Incremental Fit Index (ΔIFI) based on the simulation
results in Cheung and Rensvold (2002). A value of –.01
was used as the critical value for indicating model dif-
ference between two invariance models. The configural
invariance model was used as a baseline model, and two
model comparisons were conducted. The first compari-
son was intended to examine whether or not metric
invariance with equality constraints of the item loadings
across the three ethnic groups would be a statistically
better-fit model than the configural invariance model.
The second comparison was to test whether or not par-
tial metric invariance would be a statistically better-fit
model than the metric invariance model of item load-
ings. The model comparison results presented in Table 3
indicated that the metric invariance model was the best-
fit model. It was a significantly better-fit model over the
configural invariance model (ΔIFI = –.002) but was not
χ2 df SRMR GFI NFI
8371.50 3,357 .045 .988 .986
10,847.33 3,437 .050 .984 .982
11,563.93 3,449 .051 .983 .981
ΔIFI
–.002
–.017
significantly different from the partial invariance model
(ΔIFI = –.017).
There was no statistically significant difference
between the metric invariance of item loadings and the
partial invariance of primary factor loadings. However,
we decided to hold the partial invariance model as a
plausible model from a theoretical and practical per-
spective. In addition, the critical value of –.01 in ΔIFI is
with an alpha of .01. Thus, the value of –.017 in ΔIFI
between the metric invariance of item loadings and the
partial metric invariance might be significant with an
alpha of .05. It should be noted that we used the critical
value of ΔIFI according to Cheung and Rensvold
(2002), who did not provide a critical value of ΔIFI with
an alpha of .05.
DISCUSSION AND APPLICATION
TO SOCIAL WORK
There are two main findings in the present study. First,
the construct validity of the BSI was confirmed because
the second-order factor model showed excellent fit to the
sample of Caucasians. This is the first empirical evidence
supporting the BSI’s second-order factor structure. The
second finding was that the cross-ethnic construct valid-
ity of the BSI’s second-order factor model was supported
by the presence of measurement invariance across
African Americans, Caucasians, and Latinos. The data
supported the configural invariance of the BSI’s second-
order factor model as well as the full metric invariance of
item loadings and the partial metric invariance of primary
factor loadings, presenting substantial empirical evidence
for the construct validity of the BSI’s second-order factor
model across the three ethnic groups.
Our findings have four important implications for
using the BSI with the population of individuals with
SPMI. First, the BSI can be used with African
Americans, Caucasians, and Latinos in measuring their

psychological distress in community-based mental
health programs. Second, the nine subscales of the BSI
can be used in practice and research across the three eth-
nic groups. The multidimensional structure of the BSI
validates the practice of selecting one or more subscales
to use as symptom variables in research or practice. In
addition, the multidimensionality supports using the
BSI subscales for symptom profile assessment, which is
the most important use of the BSI in practice.
Third, a higher common symptom factor of the BSI
exists that can be used for clinical and research pur-
poses. The higher common symptom factor of the BSI is
operationalized by the GSI, which is used frequently in
practice and research. The GSI is an average total score
of the BSI items, and its psychometric soundness is sup-
ported directly by the BSI’s second-factor model.
Fourth, the BSI can be useful for comparing differences
in the mean level of symptoms across the three ethnic
groups, which confirms the BSI’s utility for both clini-
cal and research purposes. This practical implication is
induced from our finding that the full metric invariance
of item loadings in the second-order factor model and
the partial metric invariance of primary factor loadings
both fit into the data. It should be noted that partial met-
ric invariance enables the measure of interest to be used
to test group mean difference without any loss of the sci-
entific validity of the measure as demonstrated by Byrne
et al. (1989).
In summary, our findings suggest that the BSI can be
used as a reliable and valid symptom measure in assess-
ing people with SPMI in community-based mental
health programs and to compare quantitative differences
in the magnitude of psychological distress across differ-
ent ethnic groups. The BSI can be used with confidence
in practice settings where practitioners require an
assessment of symptoms across ethnic groups with
SPMI. Likewise, the BSI can be used by social work
researchers in studies that target ethnic minority status
and severe mental disorder. Our findings endorse the
utility of the BSI in mental health practice and research,
which has particular relevance to social work where a
strong commitment to both ethnic minorities and to
highly vulnerable populations exists.
In considering the implications of our findings for
using the BSI, it is important to understand the meaning of
measurement invariance for mental health practice and
research. First, social work practitioners who intend to
compare clinical symptoms in clients with different indi-
vidual backgrounds, such as ethnicity, should withhold
their clinical judgments until the symptom measure is
proved to be equivalent across different individual back-
grounds. Second, social work researchers should not
Hoe, Brekke / BRIEF SYMPTOM INVENTORY 101
address any research question regarding cross-ethnic
symptom difference without confirming the measurement
invariance of a symptom measure of interest. Little (1997)
identified two categories of measurement invariance in
cross-cultural studies. The first category refers to factorial
invariance, which includes configural invariance and met-
ric invariance of measures, and the second category con-
cerns between-group differences, such as means scores
across groups and the covariances among variables. We
recommend that social work researchers pay careful atten-
tion to testing measurement invariance. Third, the present
study demonstrated the use of partial metric invariance,
which is very practical in research. As partial invariance is
used widely in cross-cultural research, we recommended
its use in social work research. Finally, emphasis should
be placed on the pivotal role of measurement invariance in
implementing evidence-based practice in community-
based mental health programs as measurement has been
said to be the foundation of evidence-based practice
(Margison et al., 2000). The invariance of valid and reli-
able measures is a necessary condition for assessing
clients, evaluating their change, and comparing different
interventions and treatments.
Despite this being the first test of the second-order
factor model of the BSI, this study has several limitations.
First, we used three ethnic groups assuming that each of
the three groups was homogeneous in terms of their
background culture. However, heterogeneity may exist
in each of the three groups. For example, some Latino
groups might differ substantially in terms of their coun-
try of origin. Thus, future research should control for the
confounding effects of possible subgroups among
Latinos and African Americans. Second, the present
study was not intended to test possible alternative mod-
els in terms of the factor structures of the BSI, such as
the six-factor model of the BSI (Ruiperez et al., 2001).
Those alternative factor models also might fit ethnic
samples and therefore need to be investigated in future
research. Fourth, the negative variance we found might
be caused by the relatively small sample size of African
Americans (n = 399) as well as Latinos (n = 256).
Loehlin (2004) stated that a sample with 100 partici-
pants or fewer is good with high communalities, a small
number of factors, and a relatively large number of indi-
cators per factor, whereas a sample of 300 or even 500
participants might be required with lower communali-
ties, more factors, and fewer indicators per factor. We
assumed that the current study data met Loehlin’s rec-
ommendations for sample size; however, Bentler and
Chou (1987) noted that researchers might go as low as
five cases per parameter estimate but only if the data are
normally distributed with no missing data or outliers.
102 RESEARCH ON SOCIAL WORK PRACTICE
Future research should attempt to replicate the sec-
ond-order factor model of the BSI using large samples
in both clinical and nonclinical populations. This will
address the generalizability of the second-order model.
Finally, we did not test the invariance of the BSI across
gender or diagnostic groups. Although all participants
were diagnosed with a severe and persistent mental ill-
ness, had a recent psychiatric hospitalization, and seri-
ous functional impairment within the public mental
health system, we urge further research to address
invariance across gender and diagnostic groups.
REFERENCES
Anderson, J. C., & Gerbing, D. W. (1984). The effect of sampling
error on convergence, improper solutions, and goodness-of-fit
indices for maximum likelihood confirmatory factor analysis.
Psychometrika, 49, 155-173.
Anderson, J. C., & Gerbing, D. W. (1988). Structural equation mod-
eling in practice: A review and recommended two-step approach.
Psychological Bulletin, 103, 411-423.
Arbuckle, J. L. (2006). AMOS 7.0 update to the Amos user’s guide.
Chicago: SPSS.
Aroian, K. J., Patsdaughter, C. A., Levin, A., & Gianan, M. E.
(1995). Use of the Brief Symptom Inventory to assess psycholog-
ical distress in three immigrant groups. International Journal of
Social Psychiatry, 41, 31.
Benishek, L. A., Hayes, C. M., Bieschke, K. J., & Stoffelmayr, B. E.
(1998). Exploratory and confirmatory analyses of the Brief
Symptom Inventory among substance abusers. Journal of
Substance Abuse, 10, 103-114.
Bentler, P. M., & Bonett, D. G. 1980. Significance tests and good-
ness of fit in the analysis of covariance structures. Psychological
Bulletin, 88, 588-606.
Bentler, P. M., & Chou, C.-P. (1987). Practical issues in structural
modeling. Sociological Methods Research, 16, 78-117.
Bonynge, E. R. (1993). Unidimensionality of SCL-90-R scales in
adult and adolescent crisis samples. Journal of Clinical
Psychology, 49, 212-215.
Boulet, J., & Boss, M. W. (1991). Reliability and validity of the Brief
Symptom Inventory. Psychological Assessment, 3, 433-437.
Browne, M. W. (1982). Covariance structures. In D. M. Hawkins
(Ed.), Topics in multivariate analyses. New York: Cambridge
University Press.
Byrne, B. M., & Watkins, D. (2003). The issue of measurement
invariance revisited. Journal of Cross-Cultural Psychology, 34,
155-175.
Byrne, B. M., Shavelson, R. J., & Muthen, B. (1989). Testing for the
equivalence of factor covariance and mean structures: The issue
of partial measurement invariance. Psychological Bulletin, 105,
456-466.
Chen, F., Bollen, K. A., Paxton, P., Curran, P. J., & Kirby, J. B.
(2001). Improper solutions in structural equation models: Causes,
consequences, and strategies. Sociological Methods Research,
29, 468-508.
Chen, F. F., Sousa, K. H., & West, S. G. (2005). Teacher’s corner:
Testing measurement invariance of second-order factor models.
Structural Equation Modeling: A Multidisciplinary Journal, 12,
471-492.
Cheung, G. W., & Rensvold, R. B. (1999). Testing factorial invari-
ance across groups: A reconceptualization and proposed new
method. Journal of Management, 25, 1-27.
Cheung, G. W., & Rensvold, R. B. (2002). Evaluating goodness-of-
fit indexes for testing measurement invariance. Structural
Equation Modeling: A Multidisciplinary Journal, 9, 233-255.
Coelho, V. L. D., Strauss, M. E., & Jenkins, J. H. (1998).
Expression of symptomatic distress by Puerto Rican and Euro-
American patients with depression and schizophrenia. Journal of
Nervous and Mental Disease, 186, 477-483.
Cyr, J. J., McKenna-Foley, J. M., & Peacock, E. (1985). Factor struc-
ture of the SCL-90-R: Is there one? Journal of Personality
Assessment, 49, 571-578.
Derogatis, L. R. (1975). Brief Symptom Inventory. Baltimore:
Clinical Psychometric Research.
Derogatis, L. R. (1977). The SCL-90 manual I: Scoring, administra-
tion and procedures for the SCL-90. Baltimore, MD: Clinical
Psychometric Research.
Derogatis, L. R. (1993). BSI, Brief Symptom Inventory: administra-
tion, scoring & procedures manual. Minneapolis, MN: National
Computer Systems.
Derogatis, L. R., & Melisaratos, N. (1983). The Brief Symptom
Inventory: An introductory report. Psychological Medicine, 13,
595-605.
Eddy, D. M. (1998). Performance measurement: Problems and solu-
tions. Health Affairs, 17, 7-25.
Floyd, F. J., & Widaman, K. F. (1995). Factor analysis in the devel-
opment and refinement of clinical assessment instruments.
Psychological assessment, 7, 286-299.
Hafkenscheid, A. (1993). Psychometric evaluation of the Symptom
Checklist (SCL-90) in psychiatric inpatients. Personality and
Individual Differences, 14, 751-756.
Hair, J. F., Anderson, R. E., Tatham, R. L., & Black, W., C. (1998).
Multivariate data analysis (5th ed.). Upper Saddle River, NJ:
Prentice Hall.
Hayes, J. A. (1997). What does the Brief Symptom Inventory mea-
sure in college and university counseling clients? Journal of
Counseling Psychology, 44, 360-367.
Heinrich, R. K., & Tate, D. G. (1996). Latent variable structure of the
Brief Symptom Inventory in a sample of persons with spinal cord
injuries. Rehabilitation Psychology, 41, 131-147.
Hoe, M., & Brekke, S. J. (in press). Testing the measurement invari-
ance of the Brief Symptom Inventory (BSI) across three ethnic
groups for individuals diagnosed with severe and persistent men-
tal illness. Social Work Research.
Holcomb, W. R., Adams, N. A., & Ponder, H. M. (1983). Factor
structure of the Symptom Checklist-90 with acute psychiatric
inpatients. Journal of Consulting and Clinical Psychology, 51,
535-538.
Horn, J. L., & McArdle, J. J. (1992). A practical and theoretical
guide to measurement invariance in aging research. Experimental
Aging Research, 18, 117-144.
Johnson, L. C., Murphy, S. A., & Dimond, M. (1996). Reliability,
construct validity, and subscale norms of the Brief Symptom
Inventory when administered to bereaved parents. Journal of
Nursing Measurement, 4, 117-127.
Jöreskog, K. G. (1979a). Basic ideas of factor and component analy-
sis. In J. Magidson (Ed.), Advances in factor analysis and struc-
tural equation models (pp. 5-20). Cambridge, MA: Abt Books.

Jöreskog, K. G. (1979b). Simultaneous factor analysis in several
populations. In J. Magidson (Ed.), Advances in factor analysis
and structural equation models (pp. 189-206). Cambridge, MA:
Abt Books.
Kellett, S., Beail, N., Newman, D. W., & Hawes, A. (2004). The
factor structure of the Brief Symptom Inventory: Intellectual dis-
ability evidence. Clinical Psychology and Psychotherapy, 11,
275-281.
Kingsep, P., Nathan, P., & Castle, D. (2003). Cognitive behavioural
group treatment for social anxiety in schizophrenia. Schizophrenia
Research, 63, 121-129.
Kline, R. B. (2005). Principles and practice of structural equation
modeling (2nd ed.). New York: Guilford.
Krause, M. S. (2005). How the psychotherapy research community
must work toward measurement validity and why. Journal
Clinical Psychology, 61, 269-283.
Little, T. D. (1997). Mean and covariance structures (MACS) analy-
sis of cross-cultural data: Practical and theoretical issues.
Multivariate Behavioral Research, 32, 53-79.
Loehlin, J. C. (2004). Latent variable models: An introduction to
factor, path, and structural equation analysis (4th ed.). Mahwah,
NJ: Lawrence Erlbaum.
Long, D. J., Harring R. Brekke, S. J., Test, M, A., & Greenberg, J.
(2007). Longitudinal construct validity of Brief Symptom
Inventory subscales in schizophrenia. Psychological Assessment,
19, 298-308.
Mares, A. S., Young, A. S., McGuire, J. F., & Rosenheck, R. A.
(2002). Residential environment and quality of life among seri-
ously mentally ill residents of board and care homes. Community
Mental Health Journal, 38, 447-458.
Margison, F. R., Barkham, M., Evans, C., McGrath, G., Clark, J. M.,
Audin, K., et al. (2000). Measurement and psychotherapy:
Evidence-based practice and practice-based evidence. British
Journal of Psychiatry, 177, 123-130.
Margolese, H. C., Carlos Negrete, J., Tempier, R., & Gill, K. (2006).
A 12-month prospective follow-up study of patients with schizo-
phrenia-spectrum disorders and substance abuse: Changes in psy-
chiatric symptoms and substance use. Schizophrenia Research,
83, 65-75.
McDonald, R. P. (2004). Respecifying improper structures.
Structural Equation Modeling: A Multidisciplinary Journal, 11,
194-209.
O’Keefe, M., Mennen, F., & Lane, C. J. (2006). An examination of
the factor structure for the Youth Self Report on a multiethnic
population. Research on Social Work Practice, 16, 315-325.
O’Malia, L., McFarland, B. H., Barker, S., & Barron, N. M. (2002).
A level-of-functioning self-report measure for consumers with
severe mental illness. Psychiatric Services, 53, 326-331.
Piersma, H. L., Boes, J. L., & Reaume, W. M. (1994).
Unidimensionality of the Brief Symptom Inventory (BSI) in adult
and adolescent inpatients. Journal of Personality Assessment, 63,
338-344.
Ployhart, R. E., & Oswald, F. L. (2004). Applications of mean and
covariance structure analysis: Integrating correlational and exper-
imental approaches. Organizational Research Methods, 7, 27.
Hoe, Brekke / BRIEF SYMPTOM INVENTORY 103
Preston, N. J., & Harrison, T. J. (2003). The Brief Symptom Inventory
and the positive and negative Syndrome Scale: Discriminate valid-
ity between a self-reported and observational measure of psy-
chopathology. Comprehensive Psychiatry, 44, 220-226.
Primm, A. B., Gomez, M. P., Tzolova-Iontchev, I., Perry, W.,
Crum, R. M., Primm, A. B., et al. (2000). Chronically mentally ill
patients with and without substance use disorders: A pilot study.
Psychiatry Research, 95, 261-270.
Ramirez, M., Ford, E. M., Stewart, L. A., & Teresi, A. J. (2005).
Measurement issues in health disparities research. Health Service
Research, 40, 1640-1657.
Reise, S. P., Waller, N. G., & Comrey, A. L. (2000). Factor analysis
and scale revision. Psychological assessment, 12, 287-297.
Ritsner, M. M. D. P. D. (2003). Predicting changes in domain-spe-
cific quality of life of schizophrenia patients. Journal of Nervous
& Mental Disease, 191, 287-294.
Ruiperez, M. A., Ibanez, M. I., Lorente, E., Moro, M., & Ortet, G.
(2001). Psychometric properties of the Spanish version of the BSI.
European Journal of Psychological Assessment, 17, 241-250.
Schafer, J. L., & Graham, J. W. (2002). Missing data: Our view of
the state of the art. Psychological Methods, 7, 147-177.
Schwartzwal, J., Weisenberg, M., & Solomon, Z. (1991). Factor
invariance of SCL-90-R: The case of combat stress reduction.
Psychological Assessment, 3, 385-390.
Switzer, G. E., Dew, M. A., & Bromet, E. J. (1999). Issues in mental
health assessment. In C. S. Aneshensel & J. C. Phelan (Eds.),
Handbook of the sociology of mental health (pp. 81-104). New
York: Kluwer/Plenum.
Snowden, L. R. (2003). Bias in mental health assessment and inter-
vention: Theory and evidence. American Journal of Public
Health, 93, 239-243.
Steenkamp, J.-B. E. M., & Baumgartner, H. (1998). Assessing
measurement invariance in cross-national consumer research.
Journal of Consumer Research, 25, 78-90.
Swartz, M. S. M. D., Swanson, J. W. P. D., Wagner, H. R. P.
D., Burns, B. J. P. D., & Hiday, V. A. P. D. (2001). Effects of
involuntary outpatient commitment and depot antipsychotics on
treatment adherence in persons with severe mental illness.
Journal of Nervous & Mental Disease, 189, 583-592.
Switzer, G. E., Wisniewski, S. R., Belle, S. H., Dew, M. A., &
Schultz, R. (1999). Selecting, developing, and evaluating research
instruments. Social Psychiatry and Psychiatric Epidemiology,
34, 399-409.
U.S. Department of Health and Human Services. (2001). Mental
health: Culture, race, and ethnicity—A supplement to mental
health: A report of the surgeon general. Rockville, MD: U.S.
Department of Health and Human Services, Substance Abuse and
Mental Health Services Administration, Center for Mental Health
Services.
Vandenberg, R. J. (2002). Toward a further understanding of and
improvement in measurement invariance methods and proce-
dures. Organizational Research Methods, 5, 139-158.
Wothke, W. (1933). Nonpositive definite matrices in structural mod-
eling. In K. A. Bollen & J. S. Long (Eds.), Testing structural
equation models (pp. 256-293). Newbury Park, CA: Sage.