This article was downloaded by: [Northeastern University]

On: 16 November 2014, At: 06:49

Publisher: Routledge
Informa Ltd Registered in England and Wales Registered Number: 1072954
Registered office: Mortimer House, 37-41 Mortimer Street, London W1T 3JH,
UK

International Journal of Testing

Publication details, including instructions for
authors and subscription information:
http://www.tandfonline.com/loi/hijt20

The Resilience Scale for

Adults: Construct Validity and
Measurement in a Belgian
Sample
a b
Odin Hjemdal , Oddgeir Friborg , Stéphanie Braun
c c c
, Chantal Kempenaers , Paul Linkowski & Pierre
d
Fossion
a
Norwegian University of Science and Technology ,
Trondheim, Norway
b
University of Tromsø , Tromsø, Norway
c
Free University of Brussels , Erasme Hospital ,
Brussels, Belgium
d
Free University of Brussels , Brugmann Hospital ,
Brussels, Belgium
Published online: 23 Feb 2011.

To cite this article: Odin Hjemdal , Oddgeir Friborg , Stéphanie Braun , Chantal
Kempenaers , Paul Linkowski & Pierre Fossion (2011) The Resilience Scale for Adults:
Construct Validity and Measurement in a Belgian Sample, International Journal of
Testing, 11:1, 53-70, DOI: 10.1080/15305058.2010.508570

To link to this article: http://dx.doi.org/10.1080/15305058.2010.508570

PLEASE SCROLL DOWN FOR ARTICLE

Taylor & Francis makes every effort to ensure the accuracy of all the
information (the “Content”) contained in the publications on our platform.
However, Taylor & Francis, our agents, and our licensors make no
representations or warranties whatsoever as to the accuracy, completeness,
or suitability for any purpose of the Content. Any opinions and views
 expressed in this publication are the opinions and views of the authors, and
are not the views of or endorsed by Taylor & Francis. The accuracy of the
Content should not be relied upon and should be independently verified with
primary sources of information. Taylor and Francis shall not be liable for any
losses, actions, claims, proceedings, demands, costs, expenses, damages,
and other liabilities whatsoever or howsoever caused arising directly or
indirectly in connection with, in relation to or arising out of the use of the
Content.

This article may be used for research, teaching, and private study purposes.
Any substantial or systematic reproduction, redistribution, reselling, loan,
sub-licensing, systematic supply, or distribution in any form to anyone is
expressly forbidden. Terms & Conditions of access and use can be found at
http://www.tandfonline.com/page/terms-and-conditions
Downloaded by [Northeastern University] at 06:49 16 November 2014
 International Journal of Testing, 11: 53–70, 2011
Copyright
C Taylor & Francis Group, LLC
ISSN: 1530-5058 print / 1532-7574 online
DOI: 10.1080/15305058.2010.508570

The Resilience Scale for Adults:

Construct Validity and Measurement
in a Belgian Sample
Downloaded by [Northeastern University] at 06:49 16 November 2014

Odin Hjemdal
Norwegian University of Science and Technology, Trondheim, Norway

Oddgeir Friborg
University of Tromsø, Tromsø, Norway

Stéphanie Braun, Chantal Kempenaers, and Paul Linkowski

Free University of Brussels, Erasme Hospital, Brussels, Belgium

Pierre Fossion
Free University of Brussels, Brugmann Hospital, Brussels, Belgium

The Resilience Scale for Adults (RSA) was developed and has been extensively
validated in Norwegian samples. The purpose of this study was to explore the
construct validity of the Resilience Scale for Adults in a French-speaking Belgian
sample and test measurement invariance between the Belgian and a Norwegian
sample. A Belgian student sample (N = 363) completed the RSA, the Hopkins
Symptom Checklist-25 (HSCL-25), and Sense of Coherence Scale (SOC-29). A
Norwegian second sample (N = 315) was included in the analyses of invariance of
the RSA. There were expected positive and negative significant correlations with
SOC-29 and HSCL-25, respectively. The metric invariance was supported, with the
exception of one of the six RSA factors.The findings demonstrate that the RSA
may be a valid and reliable self-report measure of protective factors and further
the results also indicated cross-cultural validity for the RSA in a French-speaking
Belgian sample.

Keywords: Belgian, Belgium, cross-cultural validity, HSCL-25, resilience, Re-

silience Scale for Adults (RSA), SOC-13

Correspondence should be sent to Odin Hjemdal, Department of Psychology, Norwegian University

of Science and Technology, N-7491 Trondheim, Norway. E-mail: odin.hjemdal@svt.ntnu.no
 54 HJEMDAL ET AL.

Resilience research has identified a plethora of factors that conserve mental health
despite exposure to health hazards or psychosocial adversity. Central in the work
to advance research on resilience is the identification of influential protective and
vulnerability factors (Luthar, 2006). The Resilience Scale for Adults (RSA) was
developed to capture a set of fundamental protective factors (Hjemdal, Friborg,
Martinussen, & Rosenvinge, 2001). It has been well validated in Norwegian sam-
ples (Friborg, Hjemdal, Rosenvinge, & Martinussen, 2003; Friborg, Hjemdal,
Martinussen, & Rosenvinge, 2009; Hjemdal, Friborg, Stiles, Rosenvinge, &
Martinussen, 2006). The present study is the first to explore the construct validity
Downloaded by [Northeastern University] at 06:49 16 November 2014

of the RSA in a French-speaking sample and whether item scores are equivalently
valid across cultures.
Empirical evidence from leading researchers indicate a consensus on three over-
arching resilience characteristics: (1) individual positive dispositional attributes,
(2) family support and coherence, and (3) external support systems outside the
family (Garmezy, 1983; Werner, 1989, 1993; Werner & Smith, 1982, 1992). The
dispositions of resilient individuals are many and contain a wide variety of pro-
tective factors such as effective use of skills, high social maturity, achievement-
orientation, social perceptiveness (Werner & Smith, 1992), and pro-social behavior
(Blum, 1998). They also hold a positive self-concept and high degree of internal
locus of control (Cederblad, 1996; Dahlin & Cederblad, 1993) and are more op-
timistic, emotionally stable, possess good interpersonal skills, and view life as
meaningful and filled with purpose. They pursue higher education and have a
well-developed ability to plan and organize (Clausen, 1993). On an interpersonal
level the resilient families are more cohesive and give more emotional support
(Werner & Smith, 1992, 2001). The norms and rules are clear, and to a larger
extent these families share values compared to other families (Cederblad, Dahlin,
Hagnell, & Hansson, 1993). Resilient individuals often also experience social sup-
port from outside the family in times of crises (Werner & Smith, 1982, 2001), with
women eliciting and providing more social support. A total review of resilience
characteristics and resilience research is beyond the scope of this article, but for
an updated overview of resilience research see Masten and Wright (2010).
Resilience has been defined as a relatively good outcome or functioning, despite
experience with adverse situations shown to carry significant risk for maladjust-
ment or even psychopathology (Luthar, Cicchetti, & Becker, 2000; Masten &
Reed, 2002; Rutter, 2000). This definition focuses on the outcome and the process
that leads to a good outcome. In our view, the resilience construct also includes
the antecedent protective factors that heighten the probability of a good outcome.
Higher levels of protective resilience factors have been found to indicate lower lev-
els of psychological symptoms and to a certain extent absence of psychopathology
(e.g., Friborg et al., 2009; Hjemdal et al., 2006).
The aim in developing the RSA was essentially to identify influential protec-
tive factors based on the resilience research. A large literature review identified
 RESILIENCE SCALE FOR ADULTS IN A BELGIAN SAMPLE 55

a plethora of protective resilience factors (Hjemdal et al., 2001). These were

categorized into 15 different clusters of protective factors; 295 items were gen-
erated. A series of studies using exploratory and confirmatory factor analyses
generated a six-factor structure with 33 items, namely: (1) Perception of self,
(2) Planned future, (3) Social competence, (4) Structured style, (5) Family co-
hesion, and (6) Social recourses (Friborg, Barlaug, Martinussen, Rosenvinge, &
Hjemdal, 2005; Friborg et al., 2003; Hjemdal et al., 2001; Hjemdal et al., 2006).
The reliability of the initial version (Friborg et al., 2003; Hjemdal et al., 2001)
as well as for the revised version using a semantic differential response scale in-
Downloaded by [Northeastern University] at 06:49 16 November 2014

stead of a Likert scale has been adequate (Friborg et al., 2005; Friborg, Martinssen,
& Rosenvinge, 2006; Hjemdal et al., 2006). It differentiates healthy individuals
from psychiatric patients (Friborg et al., 2003) and correlates positively with con-
structs such as dispositional optimism, self-esteem, and sense of coherence, and
correlates negatively with symptoms of disturbance in mood (depression and anxi-
ety), negative automatic thoughts, and levels of hopelessness (Friborg & Hjemdal,
2004). Higher RSA scores have been found to be associated with a healthy Bigfive
personality profile (Friborg et al., 2005) as well as less subjective reports of stress
and pain in an experimental study of pain (Friborg et al., 2006). Prospectively the
RSA predicted fewer symptoms of depression and anxiety following stressful life
events (Hjemdal et al., 2006). It has been argued that resilience and the RSA fac-
tors may only be counterparts of vulnerability and psychiatric symptoms. A recent
study found that the RSA factors both share and have unique aspects compared to
traditional measures of vulnerability and symptoms (Friborg, 2009). Collectively
the results indicate that the RSA consist of essential protective factors that con-
tribute to explaining important aspects of mental health in Norwegian samples. It
is thus important to explore if the RSA represents culturally common universals
by using samples from other countries. The present study uses a Belgian sample.
Belgium is a part of a Western European culture, while Norway represents the
northernmost Scandinavian culture. Besides the fact that Norway is a small coun-
try with only 4.5 million inhabitants, while French is spoken in 29 countries of
136 million people, the differences in language structure and cultural history are
notable.
First, Norwegian is part of the Germanic group of languages, while French
belongs to the Romanian languages in the Indo-European group of languages.
Second, the cultural differences are historically rooted since Norway is part
of the arctic region. Because of these differences, it is very likely that a transla-
tion from Norwegian to French may change the meaning of some of the items.
Hence, a cross-cultural validation trial exploring the psychometric properties is
clearly warranted to substantiate that the scale works comparably well in an Indo-
European language culture and context. As a part of a cross-cultural validation,
exploring the factorial invariance is important. It may indicate whether individuals
from different countries ascribe different meanings to the same set of questions,
 56 HJEMDAL ET AL.

making comparisons between samples from different countries less meaningful.

Measurement equivalence may be examined on different levels.
At the most stringent level, support of scalar invariance makes it possible to
infer that any mean score differences across samples from different countries can
be attributed differences in the latent construct rather than measurement error or
different item thresholds. This is important if the test scores are planned to be used
as end-point or outcome variables. However, as resilience factor scores are more
interesting as predictors of mental health status, it is sufficient to examine for metric
invariance (i.e., equivalence of factor loadings). Support of metric invariance tells
Downloaded by [Northeastern University] at 06:49 16 November 2014

us that an equal amount of increase in raw scores indicate an equal increase in the
latent factor score in both countries; thus individuals from both groups interpret
the item similarly. This is a prerequisite for comparing beta coefficients from
regression analyses. Hence, metric invariance was of most concern in the current
study, although other forms of equivalence were tested as well. Finally, it was
tested whether the latent means between the countries were different.
The present study is the first to explore the psychometric properties and the
factorial validity of the RSA in a French-speaking Belgian sample. The established
six-factor structure of the RSA was expected to replicate. Configural invariance is
supported if the similar six-factor model explains covariation between RSA items
scores equally well in both cultures, which was certainly expected. Support of
metric invariance was also expected (equivalence of factor loadings), which will
make comparisons of regression coefficients meaningful across cultures. Support
of invariance in item score reliability (equivalence in residual variances) and
scalar invariance (equivalence in item mean scores) was not expected or regarded
as necessary to support the use of the RSA scale as a tool to predict health outcome
status in another culture.
The construct validity in the Belgian sample will be explored by comparing
levels of resilience to levels of depressive and anxiety symptoms. These are the
most common psychological symptoms in the general population and among
psychiatric patients (Kessler et al., 1994; World Health Organization, 2003). The
French version (Pichot, Wildelöcher, & Pull, 1989) of the Hopkins Symptom
Checklist (HSCL-25; Derogatis, Lipman, Rickels, Uhlenhuth, & Covi, 1974) was
included. For the assessment of positive mental health, the French version (Gana
& Garnier, 2001) of the Sense of Coherence (SOC-13; Antonovsky, 1993) was
included as a measure of overall ability to adapt and find life meaningful and
purposeful. The cross-cultural validity for both measures is well established.

METHOD

Subjects
The sample contains two subsamples, one Belgian (N = 363) and one Nor-
wegian (N = 315). The Belgian sample was from Free University of Brussels
 RESILIENCE SCALE FOR ADULTS IN A BELGIAN SAMPLE 57

first-year university students (158 males—43.5%, and 205 females—56.5%) be-

longing to medical or paramedical faculties (214 from medicine—59%, and 149
from physiotherapy—41%). Their age ranged from 17 to 25 years (M = 18.9;
SD = 1.5). All students were native French speakers or had spoken French since
their childhood (before the age of ten years). Female participants (M = 18.74
years, SD = 1.62) were significantly younger than male participants (M = 20.51
years, SD = 8.59) (Student’s t = −2.92, p < .01).
Participants in the Norwegian sample was from the Norwegian University
of Science and Technology attending bachelor courses in social sciences (73
Downloaded by [Northeastern University] at 06:49 16 November 2014

males—23.17%, and 242 females—77.46%). Their age ranged from 17 to 44

years (M = 22.30, SD = 3.24). Female participants (M = 22.13 years, SD = 3.27)
were not significantly younger than male participants (M = 22.84 years, SD =
3.11) (Student’s t = −1.62, p > .05).

Procedure
The RSA was translated into French by two independent bilingual individuals with
extensive knowledge of Norwegian and French and was then back-translated by two
new independent individuals. The two back-translations were compared and minor
linguistic corrections consisting of grammatical or orthographical corrections of
some words or sentences were made in the French version if there was a mismatch
between the two versions. Corrections were undertaken to preserve the meaning
of the original version. The assessment took place during a scheduled lecture, and
the students were informed that their participation was voluntary and anonymous.

Instruments
The following demographical information was collected: gender, age, mother
tongue.

The Hopkins Symptom Check List-25 (HSCL-25). The HSCL-25

(Derogatis et al., 1974) is a brief 25-item version of the Symptom Check List
(SCL-90-R) (Derogatis, 1983; Derogatis, Lipman, Rickels, Uhlenhuth, & Covi,
1973). This self-report questionnaire rates the presence of depressive and anxiety
symptoms using a 4-point Likert scale ranging from 1 (not at all) to 4 (very much).
It contains 13 depression items, 10 anxiety items, and 2 somatic items and higher
scores indicating higher levels of psychiatric/affective symptoms. The scale has
proven highly reliable (Cronbach alpha = .91), including in Norwegian samples
(Lavik, Laake, Hauff, & Solberg, 1999; Moum, 1998). It has been found to be a
valid screening tool for caseness (Glass, Allan, Uhlenhuth, Kimball, & Borinstein,
1978; Hough, Landsverk, & Jacobsen, 1990), including cross-culturally (Hinton,
Chen, Tran, Newman, & Lu, 1994; McKelvey & Webb, 1997; Mollica, Wyshak,
 58 HJEMDAL ET AL.

De Marnefe, Khuon, & Lavelle, 1987). A French version of HSCL anxiety items
has been previously utilized in French samples in a large epidemiological study
with 1,003 participants (Pichot et al., 1989). A large study (N = 3890) utilized an
extended version of the HSCL-25 in a Belgian sample and evaluated it to be a reli-
able and valid measure of emotional distress (Bean, Derluyn, Eurelings-Bontekoe,
Broekaert, & Spinhoven, 2007).

Sense of Coherence (SOC-13). The SOC-13 is a brief version of the SOC-

29 self-report questionnaire that measures Antonovsky’s (1993) construct Sense
Downloaded by [Northeastern University] at 06:49 16 November 2014

of Coherence. It measures a general positive intrapersonal adjustment shown to

be important in preserving good mental health derived from studies of concentra-
tion camp survivors from the Second World War. Sense of Coherence comprises
the psychological constructs comprehensibility (cognitive), manageability (in-
strumental/behavioral), and meaningfulness (motivational) (see e.g., Eriksson &
Lindström, 2005). Scores are reported along a seven-point semantically differenti-
ated scale with semantic positive and negative at each endpoint, and higher scores
indicating higher levels of SOC and better individual adaptation. The scale is uti-
lized in more than 20 countries and has proven to be highly reliable (Cronbach’s
alphas ranging from .82 to .95) (Antonovsky, 1993). The SOC-29 has previously
shown positive significant correlations with the RSA and negative correlations
with the HSCL-25 (Friborg et al., 2003). Further, it has correlated significantly
negatively with current depression, experienced stress, and trait anxiety (Frenz,
Carey, & Jorgensen, 1993; Sammallahti, Holi, Komulainen, & Aalberg, 1996). A
French version of the SOC-13 has shown satisfactory reliability and indications
of convergent and discriminate validity (Gana & Garnier, 2001).

Resilience Scale for Adults (RSA). The RSA (Friborg et al., 2003; Hjemdal
et al., 2001) is a 33-item self-report scale for measuring protective resilience
factors among adults (Friborg et al., 2005; Friborg & Hjemdal, 2004). Several
studies have found the RSA to be both reliable and valid (Friborg et al., 2005;
Friborg & Hjemdal, 2004; Friborg et al., 2003, 2006, 2009; Hjemdal et al., 2006).
The revised version of the RSA (Friborg, Martinussen, & Rosenvinge, 2006) used
in the present study applies a seven-point semantic differential scale in which each
item has a positive and a negative attribute at each end of the scale continuum. To
reduce acquiescence biases half of the items are reversely scored. Higher scores
indicate higher levels of protective resilience factors. A five-factor structure was
initially reported; however; later confirmatory factor analyses have indicated that
splitting one of the five factors in two gave a better fit. The final version has
a six-factor solution (Friborg et al., 2005; Hjemdal et al., 2006) with factors
named: (1) Perception of self (Cronbach’s α = .74), (2) Planned future (α = .73),
(3) Social competence (α = .83), (4) Structured style (α = .80), (5) Family cohesion
(α = .80), and (6) Social resources (α = .74) (Friborg et al., 2005; Hjemdal et al.,
 RESILIENCE SCALE FOR ADULTS IN A BELGIAN SAMPLE 59

2006). The content of the factors were as follows: (1) Perception of self contains
items that measure confidence in their own abilities and judgements, self-efficacy
and realistic expectations; (2) Planned future measures the ability to plan ahead,
have a positive outlook, and be goal oriented; (3) Social competence contains items
measuring levels of social warmth and flexibility, ability to establish friendships,
and the positive use of humor; (4) Structured style measures the preference of
having and following routines, being organized, and the preference of clear goals
and plans before undertaking activities; (5) Family cohesion measures whether
values are shared or discordant in the family and whether family members enjoy
Downloaded by [Northeastern University] at 06:49 16 November 2014

spending time with each other, have an optimistic view of the future, have loyalty
toward each other, and have the feeling of mutual appreciation and support; and
(6) Social resources measure availability of social support, whether they have
a confidante outside the family (such as friends or other family members that
appreciate and encourage them), and whether they may turn to someone outside
the family for help if needed. These factors cover the three consensus-based
overarching categories of protective factors associated with resilience.

Statistics
Descriptive statistics, correlations, and analyses of reliability were performed us-
ing SPSS 15.0. Inferential tests were based on a p-value < .05 to dismiss the
null-hypothesis. As in previous publications, all items were significantly nega-
tively skewed. The kurtosis of 22 of the 33 items were also non-normal (Mardia’s
coefficient of kurtosis = 66.2, p < .001). Non-normality in resilience scores is
a normal phenomenon. However, it tends to narrow the standard errors of the
parameter estimates. To adjust the error band, an asymptotic covariance matrix
was estimated using PRELIS, which is part of LISREL 8.80 (Jöreskog & Sörbom,
2006), and it was included as a weight matrix to adjust the covariance matrix.
Parameters were estimated using the robust maximum likelihood method, which
provides Satorra-Bentler rescaled chi-square statistics (SB χ 2). Following Hu and
Bentler (1999) and Marsh, Hau, and Wen (2004), the comparative fit index (CFI)
and root mean square residual (RMSEA) were evaluated in addition to SB χ 2 when
assessing model fit. A CFI > 0.95 and RMSEA < 0.06 indicate a reasonably good
model fit.
As item variances and factor loadings varied across the RSA items, the tau-
equivalence assumption of Cronbach’s alpha—equal true score variance (Raykov,
2001), was violated thus slightly underestimating the true reliability of the RSA
scores. The assumption that all items have equal true score variances may be
relaxed by allowing the factor loadings to be estimated freely in LISREL. The true
score reliability may then be estimated as a ratio of true score variance (variance
of unstandardized factor loadings) to observed score variance (Raykov, 2001).
 60 HJEMDAL ET AL.

A multigroup CFA approach was taken to determine the degree of invariance in

test scores across the Belgian and the Norwegian sample, as it allows for statistical
testing of differences in test parameters across cultures (Byrne, 2010; Ployhart &
Oswald, 2004). Identification of the models is ensured by fixing the variance of
either the latent factor score or one of the factor loadings to 1. As fixing the factor
variances represents a too stringent test of metric equivalence, which in practice
constrains both the factor variance and the factor loadings, the factor loading
with the smallest non-significant difference between the samples were fixed to 1.
Measurement invariance was tested by specifying increasingly restrictive models.
Downloaded by [Northeastern University] at 06:49 16 November 2014

First, configural invariance was tested by examining whether the same factor model
showed an adequate model fit in both samples (e.g., RMSEA < .06). Second, metric
invariance was tested by constraining the factor loadings to be equal across the
samples. Third, invariant measurement errors (equal item score reliability) were
examined by constraining the error variances equally. Fourth, scalar variance was
tested by constraining the intercepts (mean values) equally.
As the increasingly restrictive models estimate the same parameters as in the less
constrained models, they are nested within the comparison model. As the nested
model estimates fewer parameters with more degrees of freedom, the inequality
of models may be tested statistically by comparing whether the increase in chi-
squares is significantly larger than the increase in degrees of freedom. Due to
non-normality in the data, these difference tests were based on rescaled Satorra-
Bentler chi-square values (Satorra & Bentler, 2001). If significant differences
emerged, post-hoc analyses on a factor or an item level were performed to identify
the source of misfit. However, as minor model misspecifications may be overly
sensitive for differences given a large enough sample, changes in the RMSEA and
the CFI index were also consulted.

RESULTS

Psychometric Characteristics of RSA

Table 1 presents the means, standard deviations, and reliability estimates of the
measurement instruments. In the Belgian sample, the alpha for the total RSA
score was .84, but varied between .78 and .63 for the subscale sum scores. As
the tau-equivalence assumption of Cronbach’s alpha was slightly violated, more
correct reliability estimations are also presented. The Raykov’s rho estimates were
slightly higher differences varying from .80 to .65.

Mean Differences in RSA Scores across Gender

Gender differences in some of the RSA factor mean scores emerged, indicat-
ing that males scored significantly higher than females on Perception of self
 Downloaded by [Northeastern University] at 06:49 16 November 2014

TABLE 1
Means, Standard Deviations, Test Score Reliability, and Pearson’s Correlations between HSCL-25, SOC-13, and RSA Scores (N = 363)

Belgium Norway Belgium

Mean SD Mean SD g α ρ 1 2 3 4 5 6 7 8 9

1 Age 19.55 5.94

2 HSCL total 1.63 .42 .89 −.37
3 SOC-13 4.86 .89 .79 .11 −.56
4 RSA total 5.29 .68 5.32 .71 −.04 .84 .01 −.43 .61
5 Perception of self 4.77 1.07 4.90 1.18 −.12 .71 .72 .24 −.59 .55 .71
6 Planned future 5.11 1.23 4.98 1.33 .10 .71 .72 .06 −.43 .49 .66 .51
7 Social competence 5.50 .94 5.33 1.05 .17∗ .63 .69 −.01 −.24 .38 .66 .42 .34
8 Structured style 4.62 1.24 4.59 1.20 .02 .63 .65 −.14 .06 .14 .44 .09 .17 .13
9 Family cohesion 5.34 1.13 5.47 1.06 −.12 .78 .80 −.10 −.12 .30 .60 .24 .19 .17 .21
10 Social resources 6.02 .84 6.16 .76 −.17∗ .74 .75 −.16 −.26 .44 .75 .35 .36 .46 .22 .47

Note. ∗ p < .05, g = Hedge’s g (effect size), α = Cronbach’s alpha, ρ = Raykov’s rho based on congeneric scores.
Correlations above > .09 are significant at p < .05, and above > .12 at p < .01.

61
 62 HJEMDAL ET AL.

(M = 5.02 vs. M = 4.52) (t = 4.65, p < .001), while females scored higher than
men on Structured style (M = 4.40 vs. M = 4.78) and Social resources (M = 5.90
vs. M = 6.01) (t = 3.00, p < .01, and t = 2.09, p < .05, respectively).

Mean Differences across Countries

Confirmatory factor analyses and measurement invariance. As re-
silience factors tend to covary, all factors were allowed to correlate. The original
six-factor model represented the base model (M1a and M1b in Table 2). Model
Downloaded by [Northeastern University] at 06:49 16 November 2014

fit was poor in absolute terms (SB χ 2480 = 825.6, p < .001). However, as mod-
els almost always include some degree of misspecification, which the chi-square
index easily detects and rejects given a large enough sample, the RMSEA index
was relied on to judge whether configural invariance was adequately supported in
both countries. In the Belgian sample, the model fit was acceptable (RMSEA =
.043 and CFI = .959). To rule out alternative models, a five-factor model (which
was the first RSA model published in 2001) and a one factor model were tested,
but these models resulted in poorer fit. In the Norwegian sample, the six-factor
model did not fit the data as well as in the Belgian sample (SB χ 2480 = 902.9,
p < .001), but was still within acceptable limits (RMSEA = .053 and CFI = .957).
Hence, configural invariance was adequately supported in both countries. Table 3
presents the standardized factor loadings. The baseline model, combining the two
datasets in a multigroup analysis (M2), was also adequate in terms of a sufficiently
low RMSEA index (.048).
The most important test of invariance was the analysis of the metric invariance.
In the second model (M3), all factor loadings were constrained equal. This model
resulted in a poorer fit in terms of the SB χ 2 difference test, while the RMSEA
index did not change much. To identify the non-invariant items, modification
indices were examined. All of the items within the RSA factor Structured style
appeared to have significantly different factor loadings (items 30–33). This was
also apparent in Table 3. Setting up a new model (M3a), which allowed these
factor loadings to vary freely between the countries, did not produce a significant
change in the chi-square difference test. The remaining items were thus considered
invariant and measuring the latent correlates equivalently between the countries.
Equivalence of test score reliability was not supported since the SB χ 2 difference
test was significant (model M4 was worse than M3). To rule out that freely
estimated factor loadings (items 30–33) had an influence on item score reliability
(M4), the same four factor loadings as in model M3a were allowed to vary freely
(model M4a), but it produced a worse misfit. Equivalence in score reliability is
a rather stringent test of equivalence and is very seldom completely supported
in psychological measures. The modification indices were used to identify items
showing a significant difference in item score reliabilities between the groups.
By allowing the following 14 error variances to vary freely between the groups
 Downloaded by [Northeastern University] at 06:49 16 November 2014

TABLE 2
Evaluations of Measurement Invariance across Belgium and Norway

Compared
Model Type of Test with χ2 SB χ 2 df εa CFI df SB χ 2  εa CFI

M1a Belgium 996.92 825.56 480 .0430 .9592

M1b Norway 1195.42 902.93 480 .0530 .9567
M2 Baseline (both models) 2192.34 1732.04 960 .0480 .9577
M3 Factor loadings, λall equal M2 2317.21 1817.55 987 .0491 .9544 27 78.03∗∗∗ .0011 −.0033
M3a λ4 free M2 2242.61 1762.20 984 .0476 .9573 24 32.49 .0004 −.0004
M4 Item errors, δ all equal M3 2514.46 1997.31 1020 .0524 .9464 33 197.25∗∗∗ .0033 −.0080
M4a δ all equal , λ4 free M3a 2451.95 1947.94 1017 .0512 .9489 33 209.34∗∗∗ .0036 −.0083
M4b δ 14 free, λ4 free M3a 2267.58 1789.54 1003 .0474 .9569 19 24.97 −.0002 −.0004
M5 Intercepts/means, τ all equal , δ 14 free , λ4 free M4b 2548.81 2076.25 1036 .0536 .9429 33 281.23∗∗∗ .0062 −.0140
M5a τ 12 free, δ 14 free , λ4 free M4b 2290.50 1819.74 1024 .0472 .9564 20 22.92 −.0002 −.0005
M6 Latent means equal, κ all equal, τ 12 free, δ 14 free , λ4 free Same as model M5a
M6a Latent means different, κ 6 free, τ 12 free, δ 14 free , λ4 free M6 2284.52 1811.37 1018 .0473 .9565 6 5.98 .0001 .0001

Note. ∗ p < .05, ∗∗ p < .01, ∗∗∗ p < .001. SB χ 2 = Satorra-Bentler rescaled chi-square, εa = Root mean square error of approximation, CFI = Comparative
fit index,  = change in statistical values. Model 3a = all four items from RSA structured style allowed free, Model 4a = 14 measurement errors allowed
free: item 1, 4, 8, 11, 18, 21, 23–25, 27, 29, 31–33, Model 5a = 12 intercepts allowed free: item 3, 6, 7, 9, 10, 12, 13, 19, 21–23, 25.

63
 64 HJEMDAL ET AL.

TABLE 3
Standardized Factor Loadings in Both Countries

Belgium (N = 363) Norway (N = 315)

Items 1 2 3 4 5 6 1 2 3 4 5 6

1 PS .47 .62
2 PS .67 .76
3 PS .60 .63
4 PS .49 .76
5 PS .63 .75
Downloaded by [Northeastern University] at 06:49 16 November 2014

6 PS .42 .57
7 PF .53 .48
8 PF .75 .72
9 PF .57 .72
10 PF .65 .77
11 SC .41 .46
12 SC .39 .40
13 SC .69 .84
14 SC .71 .87
15 SC .41 .45
16 SC .46 .57
17 FC .67 .60
18 FC .57 .76
19 FC .75 .73
20 FC .55 .64
21 FC .65 .64
22 FC .56 .53
23 SR .46 .57
24 SR .72 .72
25 SR .57 .55
26 SR .47 .45
27 SR .79 .79
28 SR .58 .61
29 SR .54 .59
30 SS .72 .36
31 SS .59 .46
32 SS .44 .81
33 SS .45 .78

Note. PS = Perception of self, PF = Planned future, SC = Social competence, FC = Family

cohesion, SR = Social resources and SS = Structured style.

(model 4b), 1, 4, 8, 11, 18, 21, 23–25, 27, 29, 31–33, the fit was comparable with
model M3a (see Table 2).
Scalar invariance was tested (M5) by equally constraining all intercepts. Support
of scalar invariance makes direct comparisons of observed mean scores values
across countries possible. As expected, complete support was not present because
 RESILIENCE SCALE FOR ADULTS IN A BELGIAN SAMPLE 65

the SB χ 2 difference test between model M5 and M4b was significant. Items with
different mean values were identified again by checking the modification indices.
Allowing the following 12 items to have different intercepts (model 5a), 3, 6,
7, 9, 10, 12, 13, 19, 21–23, 25, the fit was comparable with model M4b. This
indicates that direct comparisons of sumscore means between the countries will
be confounded with measurement errors and different intercepts. To account for
these factors when testing for differences in latent means between the countries,
the measurement errors and intercepts were allowed to vary freely in the final
analysis of latent mean differences.
Downloaded by [Northeastern University] at 06:49 16 November 2014

Because all the kappa coefficients (parameters for the latent means) were
equally constrained in the first test (M6) and thus not estimated, the baseline
latent mean model was exactly the same as model M5a. The latent means for the
six resilience factors in the Norwegian group were then allowed to vary (M6a),
but the improvement in model fit was non-significant (SB χ 26 = 5.98, p = .43).
However, when conducting six t-tests on the RSA factor sumscores between the
countries, which in practice treats the measurement model completely invariant,
Social competence and Social resources had significantly higher and lower scores
in Norway and Belgium, respectively, though with small effect sizes. Hence, these
differences are more likely to be confounded with the psychometric properties of
the scale rather than reflecting real differences on a construct level.

Validity of the RSA

In order to explore the concurrent validity of the total RSA and the factor scores in
the Belgian sample, all correlation coefficients between the HSCL, the SOC, and
the RSA were estimated (see Table 1). As expected, the correlations between the
HSCL and the RSA subscales were significant and of a moderate to large negative
size. For the RSA factors, all but one were significantly negatively associated
with HSCL-25, the exception being the non-significant correlation with RSA
Structured style. The highest negative correlations were between HSCL-25 and
the RSA factors Perception of self and Planned future. The correlation between
the SOC-13 and the RSA total score was significant and positive, and of large
size. Among the subscale factors, Perception of self , Planned future, and Social
recourses had the highest associations with the SOC-13.

DISCUSSION

This is the one of the two first studies reporting on the cross-cultural validity of
the RSA, and it is the first using a French speaking sample. Another study will be
reporting on the cross-cultural validity for the RSA in an Iranian sample shortly
 66 HJEMDAL ET AL.

(Jowkar, Friborg, & Hjemdal, 2010). Overall, the results indicate that the psycho-
metric properties and the construct validity of the RSA are adequate. The most im-
portant finding is that metric invariance test (equality of factor loadings) received
adequate support with the exception of the RSA factor Structured style. Use of the
raw scores as indicators of protective factors in Belgium may therefore be expected
to measure the latent traits in equal amounts as in Norway, which is reassuring.
The factor structure of the RSA has been extensively tested in Norwegian
samples, and these studies consistently supported a six-factor model (e.g. Friborg
et al., 2009; Hjemdal et al., 2006). The present study conducted in Belgium
Downloaded by [Northeastern University] at 06:49 16 November 2014

confirmed that the same six-factor model fitted the data best. The degree of model
misspecification in terms of RMSEA was within acceptable limits, and it was even
smaller in Belgian than in a Norwegian sample (Hjemdal et al., 2006).
Regarding measurement invariance the six-factor model received adequate sup-
port in both countries, thus supporting configural invariance. Metric invariance was
partially supported. The main source of misfit was related to the items from the
RSA factor Structured style. All of these items had significantly different factor
loadings between the countries. Removing the items belonging to the factor Struc-
tured style (reducing the scale from 33 to 29 items) led to the SB chi-square no
longer being significant (SB χ 224 = 27.55, p = .28). The factor loadings for the
items belonging to the first five RSA factors may therefore be considered invari-
ant. This is an important finding because it makes it possible for researchers in
Belgium to conclude that a one-point higher score on the RSA measurement scale
indicates an equal amount of change in the latent trait in Norway. Subjects thus
interpret the item wording similarly in both countries. One may therefore expect
that the test scores from the five first RSA factors in the Belgian version correlate
comparably with other psychological constructs, as has been previously reported
in Norwegian samples. Item reliabilities (measurement errors) and mean values
(item intercepts) were, however, different for 13 of the 33 items. These items were
not associated with particular factors. Support of scalar invariance would be useful
to achieve as direct mean comparisons between the countries become possible.
Statistical significant differences in factor sum scores were only found for two
of the six RSA factor (higher Social competence scores in Belgium and higher
Social resources scores in Norway). The differences were only small in effect size,
and because the latent mean analysis, controlling for different item reliabilities
and intercepts, did not reveal any differences in latent means, this observed score
difference may be interpreted as a response leniency (bias) rather than a real dif-
ference. Furthermore, complete scalar invariance is seldom achieved, and as the
most important mean value comparisons often are performed within rather than
between samples. This is not considered a serious flaw. In conclusion, the Belgian
RSA scores are mostly comparable with Norwegian prediction coefficients rather
than differences in outcome mean scores.
 RESILIENCE SCALE FOR ADULTS IN A BELGIAN SAMPLE 67

The Cronbach’s alpha of the total RSA score was .84 but varied between .63
and .78 for the six RSA factor sumscores. The true scale reliability estimates
based on congeneric scores were slightly higher. In sum, the reliability of five of
the factors was satisfactory but rather low for the sixth factor Structured style.
Since the tests for metric invariance also indicated that this factor is a weak one,
these items should be used with caution in Belgium, or alternatively, completely
removed from the scale or revised in a future version. Future studies are needed
to identify the best solution.
The construct validity of the RSA was supported by finding significantly neg-
Downloaded by [Northeastern University] at 06:49 16 November 2014

ative correlations with a measure of affective disturbances (Hopkins Symptom

Check List) and positively with a measure of general adaptation (Sense of Co-
herence). Correlations were within moderate to strong in size. Thus, construct
validity seems to be supported. These results are in accordance with the construct
of resilience as representing the presence of protective recourses associated with
good adaptation and mental health as well as the relative absence of psychiatric
symptoms found in Norwegian samples (Friborg et al., 2003, 2009; Hjemdal et al.,
2006). Gender differences appeared for the following RSA factors Perception of
self , Structured style, and Social resources. Male participants rated themselves
as higher on Perception of self , while women rate themselves higher on Struc-
tured style and Social resources. Previous studies (Friborg et al., 2003; Hjemdal
et al., 2006) have identified similar gender differences with regard to Perception
of self and Social resources, which are consistent with studies of gender differ-
ences. Women experience more stress when partners or close ones experience
stressful life situations than men (Cross & Marcus, 1993). Other studies have
identified similar gender differences. Men reported feeling personally more com-
petent than women, while women rated themselves as being more extraverted,
experiencing more social support and larger social networks, and being more
trustful and gregarious (Feingold, 1994). The present study is, however, the first
report of gender differences for the RSA factor Structured style. Women rate
themselves as more structured than men. Further studies with Belgian samples
are needed to explore if this gender difference will prevail in more heterogeneous
samples.
One limitation of the present study is the young age of the participants and that
all were university students, which implies caution with regard to generalizing the
results to the general adult population in Belgium. Further validity studies of the
RSA on more heterogeneous samples in terms of age and occupation may address
this uncertainty. However, the six-factor structure of the RSA was confirmed using
confirmatory factor analysis, and the RSA scores had a pattern of intercorrelations
similar to that of those reported previously. All together, the results support the
psychometric properties and the validity of the RSA in a French-speaking sample.
The results also indicate that the protective factors included in the RSA may be
 68 HJEMDAL ET AL.

relevant across cultures. Further studies are needed to explore if these protective
resilience factors are universals shared by other cultures.

REFERENCES

Antonovsky, A. (1993). The structure and properties of the Sense of Coherence Scale. Social Science
of Medicine, 36(6), 725–733.
Bean, T., Derluyn, I., Eurelings-Bontekoe, E., Broekaert, E., & Spinhoven, P. (2007). Validation of
the multiple language versions of the Hopkins Symptom Checklist-37 for refugee adolescents.
Downloaded by [Northeastern University] at 06:49 16 November 2014

Adolescence, 42(165), 51–71.

Blum, R. W. M. (1998). Healthy youth development as a model for youth health promotion. Journal
of Adolescent Health, 22, 368–375.
Byrne, B. M. (2010). Structural equation modeling with AMOS: Basic concepts, applications, and
programming. New York: Routledge/Taylor & Francis Group.
Cederblad, M. (1996). Fifty years of epidemiologic studies in child and adolescent psychiatry in
Sweden. Nordic Journal of Psychiatry, 50, 55–66.
Cederblad, M., Dahlin, L., Hagnell, O., & Hansson, K. (1993). Salutogenic childhood factors reported
by middle-aged individuals: Follow-up of the children from the Lundby study grown up in families
experiencing three or more childhood psychiatric risk factors. European Archives of Psychiatry and
Clinical Neuroscience, 244, 1–11.
Clausen, J. A. (1993). American lives: Looking back at the children of the Great Depression. New
York: The Free Press.
Cross, S. E., & Marcus, H. R. (1993). Gender in thought, belief, and action: A cognitive approach.
In I. A. E. Beall & R. J. Sternberg (Eds.), The psychology of gender (pp. 80–81). New York: The
Guilford Press.
Dahlin, L., & Cederblad, M. (1993). Salutogenesis–protective factors for individuals brought up in a
high-risk environment with regard to the risk for a psychiatric or social disorder. Nordic Journal of
Psychiatry, 47, 53–59.
Derogatis, L. R. (1983). SCL-90-R: Administration, scoring, and procedures manual, II. Towson, MD:
Clinical Psychometric Research.
Derogatis, L. R., Lipman, R. S., Rickels, E. H., Uhlenhuth, E. H., & Covi, L. (1973). SCL-90-R: An
outpatient psychiatric rating scale. Psychopharmacological Bulletin, 9, 13–28.
Derogatis, L. R., Lipman, R. S., Rickels, K., Uhlenhuth, E. H., & Covi, L. (1974). The Hopkins
symptom checklist (HSCL)—a self-report symptom inventory. Behavioral Science, 19(1), 1–15.
Eriksson, M., & Lindström, B. (2005). Validity of Antonovsky’s sense of coherence scale: A systematic
review. Journal of Epidemiology & Community Health, 59(6), 460–466.
Feingold, A. (1994). Gender differences in personality: A meta-analysis. Psychological Bulletin, 116,
429–456.
Frenz, A. W., Carey, M. P., & Jorgensen, R. S. (1993). Psychometric evaluation of Antonovsky’s Sense
of Coherence scale. Psychological Assessment, 5(2), 145–153.
Friborg, O., Barlaug, D., Martinussen, M., Rosenvinge, J. H., & Hjemdal, O. (2005). Resilience in
relation to personality and intelligence. International Journal of Methods in Psychiatric Research,
14, 29–40.
Friborg, O., & Hjemdal, O. (2004). Resilience as a measure of adaptive capacity. Journal of the
Norwegian Psychological Association, 41, 206–208.
Friborg, O., Hjemdal, O., Rosenvinge, J. H., & Martinussen, M. (2003). A new rating scale for adult
resilience: what are the central protective resources behind healthy adjustment? International Journal
of Methods in Psychiatric Research, 12(2), 65–76.
 RESILIENCE SCALE FOR ADULTS IN A BELGIAN SAMPLE 69

Friborg, O., Hjemdal, O., Martinussen M., & Rosenvinge, J. H. (2009). Empirical support for resilience
as more than the counterpart and absence of vulnerability and symptoms of mental disorder. Journal
of Individual Differences, 30, 138–151.
Friborg, O., Hjemdal, O., Rosenvinge, J. H., Martinussen, M., Aslaksen, P. M., & Flaten, M. A. (2006).
Resilience as a moderator of pain and stress. Journal of Psychosomatic Research, 61, 213–219.
Friborg, O., Martinussen, M., & Rosenvinge, J. H. (2006). Likert-based versus semantic differential-
based scorings of positive psychological constructs: A psychometric comparison of two versions of
a scale measuring resilience. Personality and Individual Differences, 40(5), 873–884.
Gana, K., & Garnier, S. (2001). Latent structure of the sense of coherence scale in a French sample.
Personality and Individual Differences, 31, 1079–1090.
Garmezy, N. (1983). Stressors of childhood. In N. Garmezy & M. Rutter (Eds.), Stress, coping &
Downloaded by [Northeastern University] at 06:49 16 November 2014

development in children. (pp. 43–84). New York: McGraw-Hill.

Glass, R. M., Allan, T., Uhlenhuth, E. H., Kimball, C. P., & Borinstein, D. I. (1978). Psychiatric screen-
ing in a medical clinic: An evaluation of a self-report inventory. Archives of General Psychiatry,
35(10), 1189–1195.
Hinton, W. L. N., Chen, Y. C., Tran, C. G., Newman, T. B., & Lu, F. G. (1994). Screening for major
depression in Vietnamese refugees: A validation and comparison of two instruments in a health
screening population. Journal of General Internal Medicine, 9, 202–206.
Hjemdal, O., Friborg, O., Martinussen, M., & Rosenvinge, J. H. (2001). Preliminary results from the
development and validation of a Norwegian scale for measuring adult resilience. Journal of the
Norwegian Psychological Association, 38(4), 310–317.
Hjemdal, O., Friborg, O., Stiles, T. C., Rosenvinge, J. H., & Martinussen, M. (2006). Resilience pre-
dicting psychiatric symptoms: A prospective study of protective factors and their role in adjustment
to stressful life events. Clinical Psychology and Psychotherapy, 13, 194–201.
Hough, R. L., Landsverk, J. A., & Jacobsen, G. F. (1990). The use of psychiatric screening scales to
detect depression in primary care patients. In C. C. Attkisson & J. M. Zich (Eds.), Depression in
primary care: Screening and detection (pp. 139–154). New York: Routledge.
Hu, L., & Bentler, P. M. (1999). Cutoff criteria for fit indexes in covariance structure analysis: Con-
ventional criteria versus new alternatives. Structural Equation Modeling, 6, 1–55.
Jöreskog, K. G., & Sörbom, D. (2006). LISREL 8.8 for Windows. Lincolnwood, IL: Scientific Software
International, Inc.
Jowkar, B., Friborg, O., & Hjemdal, O. (2010). Cross-cultural validation of the Resilience Scale for
Adults (RSA) in Iran. Scandinavian Journal of Psychology, 51, 418–425.
Kessler, D. C., McGonagle, K. A., Zhao, S., Nelson, C. B., Eshleman, S., Wittchen, H. U., et al. (1994).
Lifetime and 12-month prevalence of DSM-III-R psychiatric disorders in the United States: Results
from the National Comorbidity Survey. Archives of General Psychiatry, 51, 8–19.
Lavik, N. J., Laake, P., Hauff, E., & Solberg, Ø. (1999). The use of self-reports in psychiatric studies
of traumatized refugees: Validation and analysis of HSCL-25. Nordic Journal of Psychiatry, 53(1),
17–20.
Luthar, S. S. (2006). Resilience in development: A synthesis of research across five decades. In D.
Cicchetti & D. J. Cohen (Eds.), Developmental psychopathology: Risk, disorder and adaptation
(Vol. 3, pp. 739–795). Hoboken, NJ: John Wiley & Sons, Inc.
Luthar, S. S., Cicchetti, D., & Becker, B. (2000). The construct of resilience: A critical evaluation and
guidelines for future work. Child Development, 71(3), 543–562.
Marsh, H. W., Hau, K. T., & Wen, Z. (2004). In search of golden rules: Comment on hypothesis-testing
approaches to setting cutoff values for fit indexes and dangers in overgeneralizing Hu and Bentler’s
(1999) findings. Structural Equation Modeling, 11, 320–341.
Masten, A. S., & Reed, M. G. J. (2002). Resilience in development. In C. R. Snyder & S. J. Lopez
(Eds.), Handbook of positive psychology (pp. 74–88). New York: Oxford University Press, Inc.
 70 HJEMDAL ET AL.

Masten, A. S., & Wright, M. O. D. (2010). Resilience over the lifespan: Developmental perspectives on
resistance, recovery and transformation. In J. W. Reich, A. J. Zautra, & J. S. Hall (Eds.), Handbook
of adult resilience (pp. 213–238). New York: The Guilford Press.
McKelvey, R. S., & Webb, J. A. (1997). A prospective study of psychological distress related to refugee
camp experience. Australian and New Zealand Journal of Psychiatry, 31(4), 549–554.
Mollica, R. F., Wyshak, G., De Marnefe, D., Khuon, F., & Lavelle, J. (1987). Indochinese version of the
Hopkins Symptom Checklist-25: A screening instrument for psychiatric care of refugees. American
Journal of Psychiatry, 144(4), 497–500.
Moum, T. (1998). Mode of administration and interviewer effects in self-reported symptoms of anxiety
and depression. Social Indicators Research, 45(1–3), 279–318.
Pichot, P., Wildelöcher, D., & Pull, J. C. (1989). Epidémiologie de l’anxiété dans la population générale
Downloaded by [Northeastern University] at 06:49 16 November 2014

francaise. Psychiatry and Psychobiology, 4, 257–266.

Ployhart, R. E., & Oswald, F. L. (2004). Applications of mean and covariance structure analysis:
Integrating correlational and experimental approaches. Organizational Research Methods, 7, 27–65.
Raykov, T. (2001). Estimation of congeneric scale reliability using covariance structure analysis with
non-linear constraints. The British Journal of Mathematic and Statistical Psychology, 54, 315–323.
Rutter, M. (2000). Resilience reconsidered: Conceptual considerations, empirical findings, and policy
implication. In J. P. Shonkoff & S. J. Meisels (Eds.), Handbook of early childhood intervention (2nd
ed., pp. 651–682). New York: Cambridge University Press.
Sammallahti, P. R., Holi, M. J., Komulainen, E. J., & Aalberg, V. A. (1996). Comparing two self-report
measures of coping: The Sense of Coherence Scale and the Defense Style Questionnaire. Journal
of Clinical Psychology, 52(5), 517–524.
Satorra, A., & Bentler, P. (2001). A scaled difference chi-square test statistic for moment structure
analysis. Psychometrika, 66, 507–514.
Werner, E. E. (1989). High-risk children in young adulthood: A longitudinal study from birth to 32
years. American Journal of Orthopsychiatry, 59, 72–81.
Werner, E. E. (1993). Risk, resilience and recovery: Perspectives from the Kauai longitudinal study.
Development and Psychopathology, 5, 503–515.
Werner, E. E., & Smith, R. S. (1982). Vulnerable but invincible. A longitudinal study of resilient
children and youth. New York: McGraw-Hill.
Werner, E. E., & Smith, R. S. (1992). Overcoming the odds. High risk children from birth to adulthood.
London: Cornell University Press.
World Health Organization. (2003). Mental health evidence and research (MER) [Electronic Version].
Retrieved from http://www.who.int/mental health/evidence/en/