You are on page 1of 6

European Journal of Psychological

Assessment
Factorial Invariance of the DASS-21 Among Adolescents
in Four Countries
David Mellor, Eugenia V. Vinet, Xiaoyan Xu, Norul Hidayah Bt Mamat, Ben Richardson, and
Francisca Román
Online First Publication, July 25, 2014. http://dx.doi.org/10.1027/1015-5759/a000218

CITATION
Mellor, D., Vinet, E. V., Xu, X., Hidayah Bt Mamat, N., Richardson, B., & Román, F. (2014, July
25). Factorial Invariance of the DASS-21 Among Adolescents in Four Countries. European
Journal of Psychological Assessment. Advance online publication.
http://dx.doi.org/10.1027/1015-5759/a000218
Original Article

Factorial Invariance of the DASS-21


Among Adolescents in Four
Countries
David Mellor,1 Eugenia V. Vinet,2 Xiaoyan Xu,3 Norul Hidayah Bt Mamat,4
Ben Richardson,1 and Francisca Román2
1
Deakin University, Melbourne, Australia, 2University de La Frontera, Temuco, Chile,
3
Sichuan Normal University, Chengdu, PR China, 4UCSI University, Kuala Lumpur, Malaysia

Abstract. The use of scales to measure constructs in populations or contexts other than that in which they were established is highly
controversial. Despite this, the use of scales without reference to ‘‘local’’ psychometric data is still widespread. In this study we examined the
factor structure of the short 21-item form of the Depression, Anxiety, and Stress Scales (DASS-21), when it was applied to adolescent samples
recruited from high schools in Australia (N = 371), Chile (N = 448), China (N = 558), and Malaysia (N = 388). Multigroup confirmatory
factor analyses revealed that the purported three-factor structure of the DASS-21 was supported in each location with structural invariance
across locations. While convergent and divergent validity studies are required to support this finding, the DASS-21 appears to be suitable for
use with adolescents in these locations.

Keywords: DASS-21, adolescents, depression, anxiety, stress, Australia, Chile, China, Malaysia

A fundamental issue faced by researchers and clinicians cultures. Whereas in the West sadness and feelings of
who wish to apply an instrument developed in one cultural worthlessness predominate, somatic symptoms such as lack
context in another is whether the constructs under examina- of appetite and difficulty sleeping predominate in non-
tion exist in the new setting. Dana (1993) observed that the Western populations.
method generally used to establish cross-cultural validity of A small number of studies have investigated the factor
tests is replication of the factor structure. If the factor structure of the DASS-21. In a clinical study in Canada,
dimensions are stable, and present an invariant structure Antony, Bieling, Cox, Enns, and Swinson (1998) replicated
across cultures, then cross-cultural validity may be inferred. the purported three-factor structure using exploratory factor
The aim of the current study is to investigate the struc- analysis with participants with various psychological condi-
tural validity of the 21-item short form of the 42-item tions. However, in another Canadian study using confirma-
Depression, Anxiety, and Stress Scale (DASS; Lovibond tory factor analysis (CFA), this time with depressed
& Lovibond, 1995a) when it is used with adolescents in patients, Clara, Cox, and Enns (2001) were unable to find
four different cultural contexts, including that in which a three-factor solution that met all the fit criteria. Studies
the DASS was developed. The DASS-21, which includes in other contexts have also not fully supported the factor
seven rather than 14 items for each subscale, has a number structure of the DASS-21. For example, Daza, Novy,
of advantages over the full length DASS, in that it is more Stanley, and Averill (2002) found a different three-factor
acceptable for respondents with limited concentration, and structure when they applied CFA to data collected from
it omits items from the full 42-item DASS that have previ- people with anxiety disorders in Spain. Also in Spain,
ously been identified as problematic (Henry & Crawford, Bados, Solanas, and Andrs (2005) concluded on the basis
2005). of CFA of data obtained from a student sample, that the
Of particular interest in the cross-cultural application of three-factor model is ‘‘acceptable but not good enough.’’
the DASS-21 is that the dimensions it purportedly mea- Henry and Crawford’s (2005) confirmatory factor anal-
sures, depression, anxiety, and stress, are aspects of adjust- ysis of nonclinical data in Australia revealed that the model
ment that have been suggested to manifest differently in with optimal fit had a quadripartite structure, which
eastern cultures relative to western cultures. For example, included a general factor of psychological distress in addi-
Marsella, Sartorius, Jablensky, and Fenton (1985) reported tion to the depression, anxiety, and stress factors. Henry and
that the symptomatology of depression varies across Crawford concluded that the DASS-21 subscales are valid,

Ó 2014 Hogrefe Publishing European Journal of Psychological Assessment 2014


DOI: 10.1027/1015-5759/a000218
2 D. Mellor et al.: Factorial Invariance of the DASS-21 Among Adolescents

Table 1. Age and gender distribution across the four samples


Male Female Total
N Mean Age (SD) Age range N Mean Age (SD) Age range N Mean Age (SD) Age range
Australia 177 14.51 (1.87) 12–18 184 14.60 (1.90) 12–19 371 14.56 (1.88) 12–19
Chile 170 15.76 (1.43) 13–19 278 15.84 (1.35) 13–19 448 15.81 (1.38) 13–19
China 210 15.33 (2.02) 11–19 346 16.28 (2.13) 12–19 558 15.92 (2.13) 11–19
Malaysia 186 14.89 (1.63) 12–19 202 14.95 (1.92) 12–19 388 14.92 (1.79) 12–19

but that each of them also assesses a more general dimen- indicating a lack of convergent validity (Hashim et al.,
sion of psychological distress or negative affect. 2011).
Studies with adolescents have reported similar results to In summary, the DASS-21 has good general support in
those with clinical and community samples of adults. relation to its factor structure when it has been evaluated in
Willemsen, Markey, Declerq, and Vanheule’s (2011) predominately Western contexts, with researchers to date
CFA study with a large sample of Belgian adolescents generally accepting the three-factor structure despite some
revealed a good fit with the original three-factor model pro- reports of high inter-factor correlations. The purpose of this
posed by Lovibond and Lovibond (1995b), and exception- study is to investigate the degree to which the factor struc-
ally good fit with the hierarchical model proposed by Tully, ture is supported when it is used with adolescents in four
Zajac, and Venning (2009) who, like Henry and Crawford countries. Specifically, our aim is to test the construct valid-
(2005), contended that depression and anxiety are highly ity of the DASS-21 when it used in adolescent samples in a
related due to the common latent factor of negative affect. Western context (Australia), a Latino context (Chile), and
To compare the fit of these two models, an S-B v2 two Eastern contexts (China and Malaysia).
difference test was conducted. It was found that
Tully et al.’s hierarchical model fit best the data. The cor-
relations between the three factors were very high
(depression-anxiety = 0.82 for boys and 0.78 for girls;
depression-stress = 0.89 for boys and 0.78 for girls; and
Method
anxiety-stress = 0.90 for boys and 0.93 for girls). This
appears to be an important limitation since the DASS-21 Participants
aims to discriminate between depression, anxiety, and stress
(Willemsen et al., 2011). The sample of 1765 participants was recruited from second-
Similarly, Szab (2010) conducted a CFA within an ary schools in Australia, Chile, China, and Malaysia.
adolescent Australian sample. The one-factor and two- It included 371 students from Australia, 448 from Chile,
factor models yielded high and significant Chi-squares 558 from China, and 388 from Malaysia. Table 1 shows
indicating poor fit. However the three-factor (depression, the gender and age distribution across the four subsamples.
anxiety, and stress) model showed good fit, which was even The Australian sample was recruited from schools in a
better when a four-factor model which incorporated ‘‘nega- middle class suburb in eastern Melbourne, the capital city
tive affect’’ as proposed by Tully et al. (2009) was exam- of the state of Victoria. The Chilean sample was recruited
ined. However, correlations between factors were again from middle school in Temuco, the capital of Araucana
found to be too high (depression-anxiety = .81; depres- Region. It included students from municipal, subsidized pri-
sion-stress = 0.87; and anxiety-stress = .93). vate, and other private schools, and therefore included stu-
Hashim, Golok, and Ali’s (2011) recent study reported dents of different socioeconomic status. In Malaysia, the
on the use of the DASS-21 within a large Malaysian participants were recruited from schools located in middle
adolescent sample. CFA revealed a reasonable fit overall class suburbs of Kuala Lumpur, the national capital.
(one-factor model). However, the differentiation between The Chinese data were collected in urban Chengdu, a large
depression, anxiety, and stress was weak as a result of city in Southwest China, and the capital of Sichuan prov-
extremely high intercorrelations which were approach- ince. Thus, all participants were drawn from large urban
ing collinearity (depression-anxiety = 0.83; depression- environments.
stress = 0.87; and anxiety-stress = 0.84). Additionally,
an analysis involving a four-factor model (depression,
anxiety, stress, and negative affect) was conducted, since Measures
negative affect is common to the other three factors,
and can thereby determine shared variance of items The DASS-21 is made up of three seven-item subscales that
between the three DASS-21 factors. This analysis measure the three dimensions specified in Lovibond and
showed nonsignificant unstandardized factor loadings on Lovibond’s (1995a) tripartite model of affect: low positive
each of the three factors for 19 of 21 items, thereby affect (Depression), physiological hyperarousal (Anxiety),

European Journal of Psychological Assessment 2014 Ó 2014 Hogrefe Publishing


D. Mellor et al.: Factorial Invariance of the DASS-21 Among Adolescents 3

and negative affect (Stress). The psychometric properties In Malaysia the principals of five schools in the state of
of original 42-item version of the DASS are well- Selangor were approached to allow students from their
established (see Antony et al., 1998; Brown, Chorpita, schools to participate in the study. Four schools agreed,
Korotitsch, & Barlow, 1997; Clara et al., 2001; Lovibond and a similar procedure to that described above was
& Lovibond, 1995b), and the short form maintains these followed.
properties. Item examples include ‘‘I experienced trembling
in the hands’’ (Anxiety), ‘‘I felt that I had nothing to look
forward to’’ (Depression), and ‘‘I found it hard to wind Analytic Procedure
down’’ (Stress). A four-point response scale ranging from
0 to 3 allows respondents to indicate how much each state- Multiple group confirmatory factor analysis in Mplus v6.11
ment applied to them over the past week. Scores for each was used to test for measurement invariance across the four
subscale are obtained by adding the responses to the rele- samples. Given the response scale of the DASS, it was most
vant items. Lovibond and Lovibond (1995a) suggested that appropriate to treat indicator variables as ordinal categorical
these scores may then be doubled to make them directly rather than continuous and thus the model was estimated
equivalent to those derived from the full version of the using a weighted least squares approach (WLSMV).
DASS because the DASS-21 encompasses the full range Comparing all four groups simultaneously in a single model
of symptoms measured by the full version. introduces considerable complexity into the invariance test-
In Chile, China, and Malaysia the DASS-21 was trans- ing and for this reason, invariance was tested by comparing
lated into languages appropriate for the setting – Spanish, the Australian sample separately to the other three samples
Chinese, and Bahasa Malaysia, the national language, and (Chilean, Chinese, and Malaysian). The Australian sample
the language of instruction in schools. Translation followed was chosen as the baseline for comparison as the DASS
a rigorous process of translation back translation (Brislin, was developed in Australia and has been extensively vali-
1980; MuÇiz & Hambleton, 1996; Werner & Campbell, dated in that population. For each comparison, three models
1970). were evaluated: (i) an unconstrained model where loadings
and thresholds were free to vary across groups, (ii) a par-
tially constrained model where factor loadings were con-
Procedure strained to equality across groups while thresholds were
free to vary, and (iii) a fully constrained model where load-
Approval to undertake this project was obtained from the ings and thresholds are equal across groups. Measurement
relevant university ethics committees in each setting. invariance is present when the more restricted model does
In addition, approvals to conduct the research in schools not fit the data significantly worse than the less restricted
were obtained from the relevant education authorities. model. Fit of the nested models was compared using crite-
In Australia, schools in the eastern suburbs of Melbourne ria recommended by Chen (2007) for large sample sizes.
were contacted and invited to participate. Two schools Specifically, invariance is present when the more restricted
which showed an interest and were able to provide an model shows a drop in CFI of  0.01 supplemented by an
appropriate sample size, were provided with plain language increase in the RMSEA of  0.015. Overall model fit was
descriptions of the project to be sent home with students. evaluated using v2, CFI, TLI, and the RMSEA (SRMR is
Questionnaires were sent out to each school, where they not available when WLSMV estimation is used). Hu and
were administered by class teachers across a range of year Bentler (1999) recommend that CFI and TLF values of
levels to all students whose parents had consented to their 0.90 or greater indicate adequate fit (and over 0.95 indicates
involvement. All adolescents also completed a consent excellent fit). RMSEA values of less than 0.06 indicate
form, then the anonymous questionnaire in small, well- good fit.
spaced groups. The questionnaire took approximately
5 min to complete.
In Chile, a randomly selected range of municipal, subsi-
dized private, and private schools were approached to par- Results
ticipate in the study. After securing each principal’s
agreement, parents were asked to sign an informed consent Data screening revealed a small number of missing values
form to permit their child to participate in the study. (items were missing between 0 and 2.6%, average 1.84%)
Students whose parents were agreeable were asked for their which were replaced using a single imputation strategy
consent, and if they agreed they completed the DASS-21 (expectancy maximization). Mean subscale scores for par-
under the supervision of one of the teachers and a research ticipants within each country are presented in Table 2.
assistant in the student’s classrooms. The average response Results from the main analyses showed that, except in some
time for the DASS-21 was 5–8 min. cases where the RMSEA was slightly above 0.06, in all
In China, data were collected from one school in the models fit surpassed recommended cut-offs for adequate
city of Chengdu. After obtaining the principal’s and teach- fit. These results indicated that the three-factor structure
ers’ agreement, questionnaires were administered by class- was an appropriate fit for the data (see Table 3). Applying
room teachers to those students whose parents provided recommendations from Chen (2007), measurement invari-
consent. ance was clearly present between Australia and China as

Ó 2014 Hogrefe Publishing European Journal of Psychological Assessment 2014


4 D. Mellor et al.: Factorial Invariance of the DASS-21 Among Adolescents

Table 2. Mean DASS subscale scores grouped by gender Table 4. Standardized factor loadings
and country
Australia China Malaysia Chile
Mean Mean Mean
Depression (a = .89) (a = .81) (a = .82) (a = .85)
depression (SD) anxiety (SD) stress (SD)
Item 3 0.75 0.64 0.60 0.73
Australia Item 5 0.69 0.60 0.63 0.50
Females 5.48 (5.05) 4.51 (3.95) 5.92 (4.75) Item 10 0.81 0.65 0.76 0.91
Males 3.98 (4.65) 3.55 (3.77) 3.71 (4.06) Item 13 0.87 0.79 0.77 0.79
Chile Item 16 0.85 0.64 0.67 0.57
Females 6.72 (5.02) 5.50 (3.98) 7.55 (4.28) Item 17 0.90 0.73 0.77 0.85
Males 4.79 (4.09) 4.05 (3.23) 5.93 (3.62) Item 21 0.82 0.77 0.67 0.92
China Anxiety (a = .80) (a = .75) (a = .74) (a = .72)
Females 5.65 (4.34) 6.51 (3.91) 7.68 (4.41) Item 2 0.41 0.51 0.42 0.39
Males 5.28 (4.37) 6.08 (4.43) 6.59 (4.66) Item 4 0.55 0.58 0.48 0.44
Malaysia Item 7 0.72 0.52 0.50 0.61
Females 5.69 (3.97) 6.25 (3.75) 6.72 (3.99) Item 9 0.79 0.68 0.71 0.66
Males 5.56 (4.43) 5.30 (3.91) 6.24 (4.30) Item 15 0.84 0.80 0.77 0.78
Item 19 0.64 0.52 0.62 0.55
Item 20 0.85 0.65 0.67 0.73
Stress (a = .85) (a = .80) (a = .80) (a = .79)
well as for Australia and Malaysia (see Table 4). However, Item 1 0.60 0.64 0.66 0.40
the situation was less clear for the Australia/Chile compar- Item 6 0.66 0.63 0.51 0.46
ison where the constrained models showed a CFI drop of Item 8 0.78 0.72 0.73 0.59
greater than 0.01 relative to the less constrained model. Item 11 0.82 0.80 0.74 0.57
However, this change was not accompanied by a change Item 12 0.81 0.78 0.80 0.91
in the RMSEA of greater than 0.015 and thus it was con- Item 14 0.76 0.55 0.57 0.60
cluded that measurement invariance was present. All three Item 18 0.79 0.52 0.59 0.71
factors were highly correlated (between 0.60 and 0.71).

invariance across the four contexts in which the study


was conducted and thus, scores from these different groups
Discussion can be meaningfully compared.
Our finding is consistent with the studies that have
In this study we set out to investigate the structural validity reported general, although not unequivocal support for the
of the DASS-21 when it is used with adolescents in four three-factor structure of the DASS-21 in various contexts
different cultural contexts. The instrument is a simple and including, Australia and Malaysia. However, in contrast
easy way to screen for elevated levels of depression, anxi- to the approach of other studies, which have tended to treat
ety, and stress, making it a useful tool for mental health the item level data as continuous, our data were treated as
practitioners such as school counsellors. To date however, ordinal categorical. Given the four-point nature of DASS
there have been few studies of its structural validity when items, this is a more appropriate way to model these data.
it is applied to adolescents, particularly in contexts other Thus, we may have greater confidence that the DASS-21
than that in which it was developed, Australia. Our analyses can be meaningfully used with adolescents in the four con-
established support for the existing three-factor structure. texts in which our study was undertaken. Despite this, it is
Results also established that there was measurement notable that the three factors were highly correlated with

Table 3. Fit indices for confirmatory factor analyses


v2 (df) CFI TLI RMSEA
Australia vs. China
Unconstrained model 1101.3 (372) 0.950 0.943 0.065
Partially constrained model (thresholds free) 1107.3 (390) 0.950 0.947 0.063
Fully constrained model 1287.4 (429) 0.941 0.942 0.066
Australia vs. Malaysia
Unconstrained model 774.8 (372) 0.970 0.966 0.053
Partially constrained model (thresholds free) 759.2 (390) 0.972 0.970 0.050
Fully constrained model 934.1 (429) 0.962 0.963 0.056
Australia vs. Chile
Unconstrained model 894.4 (372) 0.964 0.960 0.059
Partially constrained model (thresholds free) 1088.7 (390) 0.952 0.948 0.066
Fully constrained model 1446.3 (429) 0.930 0.932 0.076

European Journal of Psychological Assessment 2014 Ó 2014 Hogrefe Publishing


D. Mellor et al.: Factorial Invariance of the DASS-21 Among Adolescents 5

each other, suggesting that there is a considerable lack of Henry, J. D., & Crawford, J. R. (2005). The short-form version
differentiation between the constructs being measured. of the Depression Anxiety Stress Scales (DASS-21):
While this study included data from large samples in Construct validity and normative data in a large non-clinical
sample. British Journal of Clinical Psychology, 44,
each of the locations, it is not possible to determine the rep- 227–239. doi: 10.1348/014466505X29657
resentativeness of each sample, and having established that Hu, L., & Bentler, P. M. (1999). Cutoff criteria for fit indexes in
the factor structure of the DASS-21 is invariant across the covariance structure analysis: Conventional criteria versus
settings, further studies are now required to establish the new alternatives. Structural Equation Modeling, 6, 1–55.
convergent and divergent validity of the subscales for ado- doi: 10.1080/10705519909540118
lescent respondents in the various contexts. Such studies Lovibond, P. F., & Lovibond, S. H. (1995a). Manual for the
Depression Anxiety Stress Scales. Sydney, Australia: Psy-
could use other established measures of the three constructs chology Foundation of Australia.
targeted by the DASS-21, or clinical diagnoses if suffi- Lovibond, P. F., & Lovibond, S. H. (1995b). The structure of
ciently large samples of clinically diagnosed participants negative emotional states: Comparison of the Depression
could be accessed. Anxiety Stress Scales (DASS) with the Beck Depression and
Anxiety Inventories. Behaviour Research and Therapy, 33,
335–343. doi: 10.1016/0005-7967(94)00075-U
Acknowledgments Marsella, A. J., Sartorius, N., Jablensky, A., & Fenton, R. E.
(1985). Crosscultural studies of depressive disorders: An
The study reported here was in part supported by grant overview. In A. Kleinman & B. Good (Eds.), Culture and
number 80110035 from the Atraccin de Capital Humano depression: Studies in the anthropology and cross-cultural
Avanzado Del Extrajero (Attraction of Advanced Human psychiatry of affect and disorder (pp. 299–324). Berkeley,
CA: University of California Press.
Capital from Abroad) program of the Chilean Comisin MuÇiz, J., & Hambleton, R. K. (1996). Directrices para la
Nacional De Investigacin Cientfica Y Tecnolgica. traduccin and adaptacin de los tests [Guidelines for the
translation and adaptation of tests]. Papeles del Psiclogo,
66, 63–70.
References Szab, M. (2010). The short version of the Depression Anxiety
Stress Scales (DASS-21): Factor structure in a young
Antony, M. M., Bieling, P. J., Cox, B. J., Enns, M. W., & adolescent sample. Journal of Adolescence, 33, 1–8. doi:
Swinson, R. P. (1998). Psychometric properties of the 10.1016/j.adolescence.2009.05.014
42-item and 21-item versions of the Depression Anxiety Tully, P., Zajac, I., & Venning, A. (2009). The structure of
Stress Scales in clinical groups and a community sample. anxiety and depression in a normative sample of younger
Psychological Assessment, 10, 176–181. doi: 10.1037/1040- and older Australian adolescents. Journal of Abnormal Child
3590.10.2.176 Psychology, 37, 717–726. doi: 10.1007/s10802-009-9306-4
Bados, A., Solanas, A., & Andrs, R. (2005). Psychometric Werner, O., & Campbell, D. (1970). Translating, working
properties of the Spanish version of Depression, Anxiety and through interpreters, and the problem of decentering. In
Stress Scales (DASS). Psicothema, 17, 679–683. doi: R. Naroll & R. Cohen (Eds.), A handbook of method in
10.1016/j.jad.2008.01.023 cultural anthropology. New York, NY: Natural History
Brislin, R. (1980). Translation and content analysis of oral Press.
andwritten materials. In H. C. Triandis & J. W. Berry Willemsen, J., Markey, S., Declercq, F., & Vanheule, S. (2011).
(Eds.), Handbook of cross-cultural psychology: Methodol- Negative emotionality in a large community sample of
ogy (Vol. 2, pp. 389–444). Boston, MA: Allyn & Bacon. adolescents: The factor structure and measurement invari-
Brown, T. A., Chorpita, B. F., Korotitsch, W., & Barlow, D. H. ance of the short version of the depression anxiety stress
(1997). Psychometric properties of the Depression Anxiety scales (DASS-21). Stress & Health: Journal of the Interna-
Stress Scales (DASS) in clinical samples. Behaviour tional Society for the Investigation of Stress, 27, e120–e128.
Research and Therapy, 35, 79–89. doi: 10.1348/ doi: 10.1002/smi.1342
014466506X158996
Chen, F. F. (2007). Sensitivity of goodness of fit indexes to lack
of measurement invariance. Structural Equation Modeling:
A Multidisciplinary Journal, 14, 464–504. doi: 10.1080/
10705510701301834
Clara, I. P., Cox, B. J., & Enns, M. W. (2001). Confirmatory Date of acceptance: March 11, 2014
factor analysis of the Depression Anxiety Stress Scales in Published online: July 25, 2014
depressed and anxious patients. Journal of Psychopathology
and Behavioral Assessment, 23, 61–67. doi: 10.1023/
A:1011095624717
Dana, R. (1993). Multicultural assessment perspectives for
professional psychology. Boston, MA: Allyn & Bacon.
Daza, P., Novy, D. M., Stanley, M. A., & Averill, P. (2002). David Mellor
The Depression Anxiety Stress Scale-21: Spanish translation
and validation with a Hispanic sample. Journal of Psycho- School of Psychology
pathology and Behavioral Assessment, 24, 195–205. doi: Deakin University
10.1023/A:1016014818163 Burwood 3125
Hashim, H., Golok, F., & Ali, R. (2011). Factorial validity and Australia
internal consistency of Malaysian Adapted Depression Tel. +61 3 9244 3742
Anxiety Stress Scale – 21 in an adolescent sample. Fax +61 3 9244 6858
International Journal of Collaborative Research on Internal E-mail mellor@deakin.edu.au
Medicine & Public Health (IJCRIMPH), 3, 29–39.

Ó 2014 Hogrefe Publishing European Journal of Psychological Assessment 2014

You might also like