Journal of Clinical Epidemiology 68 (2015) 551e562

A brief form of the Perceived Social Support Questionnaire (F-SozU)

was developed, validated, and standardized
oren Kliema,*, Thomas M€
S€ oßlea, Florian Rehbeina, Deborah F. Hellmanna, Markus Zengerb,
Elmar Br€ahlerb,c
a
Criminological Research Institute of Lower Saxony, L€utzerodestraße 9, 30161 Hannover, Germany
b
Department of Medical Psychology and Medical Sociology, University of Leipzig, Ph.-Rosenthal-Str. 55, 04103 Leipzig, Germany
c
Department of Psychosomatic Medicine and Psychotherapy, University of Mainz, Untere Zahlbacher Str. 8, 55131 Mainz, Germany
Accepted 3 November 2014; Published online 13 November 2014

Abstract
Objectives: Development of a brief instrument (F-SozU K-6) for the measurement of perceived social support in epidemiologic con-
texts by shortening a well-established German questionnaire (F-SozU K-14).
Study Design and Setting: The development of the F-SozU K-6 consisted of two phases; phase 1: the F-SozU K-14 was presented to a
general population sample representative for the Federal Republic of Germany (N 5 2,007; age: 14e92 years). Six items for the short form
were selected based on the maximization of coefficient alpha. Phase 2: the new short form (F-SozU K-6) was evaluated and standardized in
an independent second population survey (N 5 2,508, age: 14e92 years).
Results: The F-SozU K-6 showed very good reliability and excellent model fit indices for the one-dimensional factorial structure of the
scale. Furthermore, strict measurement invariance was detected allowing unbiased comparison of means and correlation coefficients and
path coefficients between both sexes across the full lifespan from adolescence (14e92 years). Well-established associations of perceived
social support with depression and somatic symptoms could be replicated using the short form.
Conclusion: The F-SozU K-6 presents a reliable, valid, and economical instrument to assess perceived social support and can thus be
effectively applied within the frameworks of clinical epidemiologic studies or related areas. Ó 2015 Elsevier Inc. All rights reserved.

Keywords: Perceived social support; Self-report questionnaire; Standardization; Social Support Questionnaire short form (F-SozU K-6); Measurement invari-
ance; Psychometrics; Confirmatory Factor Analysis

1. Introduction short forms assessing physical health or physical con-

In the field of clinical epidemiology, economizing self-
assessment instruments seems of particular relevance. This
is especially true for large population samples, with the ne-
cessity to assess a variety of relevant constructs and given
space constraints due to reasons of costs and acceptance.
One solution to this problem could be to include short
forms of well-established instruments, which are highly
correlated with their long versions. In the past, numerous
Conflict of interest: None.
Funding: The study was authorized by the Ethics Committee of the
Medical Faculty of the University of Leipzig (Az.: 050/13-03.05.2013).
The study was financed by internal funds of the Department for Medical
Psychology and Medical Sociology of the University Clinic of Leipzig.
* Corresponding author. Tel.: þ49-(0)511-34836-70; fax: þ49-(0)511-
34836-10.
E-mail address: Soeren.Kliem@kfn.de (S. Kliem).
http://dx.doi.org/10.1016/j.jclinepi.2014.11.003
0895-4356/Ó 2015 Elsevier Inc. All rights reserved.
straints [1e4] and psychopathology (eg, depression
[5e7], anxiety [6,8], somatoform disorder [3,9], or post-
traumatic stress symptoms [10,11]) have been either well
established or recently developed. In addition to these clin-
ically relevant measures, short forms of more general con-
structs are needed that possibly (1) maintain or induce
pathology, (2) moderate or mediate the outcome of medical
or psychotherapeutic interventions, or (3) could be seen as
secondary outcome measures (eg, quality of life or global
functioning).
Decades of research have shown that perceived social
support plays an essential role in preventing mental and
physical illness [12e17]. Correspondingly, a current
meta-analysis [12], which evaluated three major compo-
nents of social relationships, shows that regarding mortal-
ity, the importance of the functional aspects of social
relationship (ie, received and perceived social support)
may be rated as comparable to other well-researched risk
552 S. Kliem et al. / Journal of Clinical Epidemiology 68 (2015) 551e562
What is new?
Key findings
 Decades of research have shown that perceived so-
cial support plays an essential role in preventing
mental and physical illness. In this study, the
six-item brief social support questionnaire, F-SozU
K-6, was developed and evaluated based on two in-
dependent surveys, representative of the general
population of Germany.
 The F-SozU K-6 showed very good reliability, and
excellent model fit indices were detected for the
one-dimensional factorial structure of the scale.
What this adds to what was known?
 The F-SozU K-6 provides an economical and reli-
able instrument for evaluating the degree of
perceived social support.
 Based on measurement invariance analyses, the
F-SozU K-6 allows comparison of means and cor-
relation coefficients, as well as path coefficients
within structural equation models between both
sexes across the full lifespan (14e92 years).
What is the implication and what should change
now?
 The application of the F-SozU K-6 within the
frameworks of clinical epidemiologic studies or
related areas is supported.
factors, such as smoking or regular alcohol consumption,
and even surpasses the importance of other risk factors,
such as obesity or physical inactivity.
Furthermore, social support can be awarded to have rele-
vance in medical settings, for example, the development
and progression of cardiovascular disease [18], compliance
with medical regimens [19], and a decreased length of hos-
pitalization [20].
1.1. Measures of social support
The importance of the concept of social support is also
reflected in the number of measures developed for its
assessment. However, available instruments for assessing
perceived social support seem to be unsuited in the frame-
work of clinicaleepidemiologic studies due to the number
of items [eg, MSPSS (Multidimensional Scale of Perceived
Social Support) [21], SPS (Social Provisions Scale) [22],
DUFSS (Duke-UNC Functional Social Support Question-
naire) [23], ASSIS (Arizona Social Support Interview
Schedule) [24], PSSS (Perceived Social Support Scale)
[25], SSQ (Social Support Questionnaire) [26]]; low
validity or reliability [eg, short forms of the OSSS (Oslo
Social Support Scale) [27,28], short form of the SPS
[29]]; or elaborated scoring [NSSQ (Norbeck Social Sup-
port Questionnaire) [30], SSQ-6 [31]] or they have not been
conceived [eg, mMOS-SS (modified Medical Outcomes
Study Social Support Survey) [32], DUFSS-10 [33]] or
even evaluated (DUFSS-8 [34], DUFSS-6 [35], NSSQ
[30], SSQ-6 [31]) in the general population.
1.2. The German Social Support Questionnaire
In German-speaking countries, the Social Support Ques-
tionnaire (F-SozU) by Fydrich et al. [36] is well accepted to
assess general social support in the general population and
in clinical trials. Since the 1980s, it is primarily used in
research contexts in clinical psychology, psychotherapy,
medical sociology, health psychology, and behavioral med-
icine [37]. Following Barrera [38], Heller and Swindle [39],
and House [40], the authors conceptualize social support as
perceived or anticipated support from the social network.
This cognitive approach goes back to Cobb [41] and fo-
cuses on the assessment by the recipient of social support.
Several studies have shown that in clinical and epidemio-
logic contexts, this perspective attains higher significance
than formal or structural network characteristics. The F-
SozU assesses social support in the natural environment
(general social support) that excludes help from health care
professionals [37].
From an individual perspective, statements regarding
perceived or anticipated social support are rated on a
five-point Likert scale, ranging from 1 (does not apply) to
5 (exactly applicable). These statements cover generalized
experiences rather that concrete situations. A long version
with 54 items (S-54) [36,37] and a short version with 22
items (K-22) [37] cover three central characteristics of so-
cial support: practical and material (instrumental) support
(being able to receive practical help with daily problems,
for example, borrowing something, receiving practical
advice, being relieved of tasks), emotional support (being
liked and accepted by others, being able to show feelings,
experiencing sympathy), and social integration (belonging
to a circle of friends, undertaking ventures together,
knowing people with similar interests). These dimensions
can be interpreted as subscales and combined to a total
score of general perceived social support. Although also
containing items from all the three dimensions, another
short version comprising 14 items (K-14) [37,42] focuses
exclusively on general perceived social support, which in
this instrument is not further differentiated. Hence, the au-
thors suggest an unidimensional interpretation of a total
score. Quality criteria of the F-SozU K-14 are overall
convincing, showing high consistency of the instrument
(a 5 0.94) and satisfactory selectivity between 0.55 and
0.76 [42]; a 1-week retest reliability of 0.96 is specified.
All short forms were generated by selecting items based
on psychometric properties [37].
 S. Kliem et al. / Journal of Clinical Epidemiology 68 (2015) 551e562
Perceived social support as measured by the F-SozU
has been shown to be associated with social competence,
social insecurity, psychopathological symptoms as well
as several social, and sociodemographic variables [eg,
gender (higher values for woman), relationship status
(higher values for being in a relationship), educational
status (higher values for individuals with university
degree)] in accordance with its theoretical framework
and hence provides support for construct validity [37].
These relations could be replicated using the short forms
[37].
1.3. Aims of the study
For use within the context of clinical epidemiology, it
would be desirable to have an economical (ie, low comple-
tion and scoring time) instrument convincingly covering
various areas of perceived social support that is still charac-
terized by high reliability and validity. The goals of this
study were hence to (1) develop a brief form for assessing
social support (F-SozU K-6) based on the F-SozU K-14,
using a representative community sample, and (2) subse-
quently examine its psychometric properties and stan-
dardize this brief form in a second representative
population sample. To the best of our knowledge, to date,
no shorter versions of the F-SozU than the F-SozU K-14
have been developed and validated.
2. General method
2.1. Overview
The development of the brief form was carried out in
two phases.
Phase 1: In the first phase, the F-SozU K-14 was admin-
istered to a community sample. Based on the data collected
in phase 1, the items were evaluated and the short form
F-SozU K-6 was compiled.
Phase 2: In a second general population survey, the
newly developed brief form (F-SozU K-6) was adminis-
tered and evaluated. Norms regarding age and gender were
compiled.
2.2. Procedure
The overall design of the two surveys was highly similar.
Both surveys were conducted by the University of Leipzig
and were carried out by the same contractor [an indepen-
dent institute for opinion and social research (USUMA,
Berlin)] using the same procedure. A final sample size of
2,500 participants was intended in both surveys. Aims of
these annual surveys were to (a) assess prevalence rates
of a variety of relevant physical or mental disorders and
related risk behaviors (descriptive epidemiology), (b)
examine causes and conditions of these disorders (analytic
epidemiology), and (c) analyze psychometric properties
553
and providing German norms for clinicalepsychological
instruments. Hence, in this study, detailed information of
procedure and sample characteristics are only provided
for phase 2 where the F-SozU K-6 was used, please refer
to Fydrich et al. [42] for more details on the survey from
phase 1.
3. Phase 1: development of the short form of the
F-SozU
3.1. Method of phase 1
3.1.1. Study design and participants
The development of the brief form was carried out based
on data from a representative sample of the Federal Repub-
lic of Germany from a 2003 survey by the University of
Leipzig [N 5 2,007 aged between 14 and 92 years; for a
more detailed description, see 42] and authorized by the
Ethics Committee of the Medical Faculty of the University
of Leipzig.
3.1.2. Statistical analyses
Criteria for the shortened scale were a very good coeffi-
cient alpha value, unidimensionality of the instrument
(necessary to calculate a total score), and a small number
of items (max. six items) to provide an economic measure-
ment of perceived social support. The shortened scale
should preferably contain the same number of items for
each of the three dimensions originally postulated. Original
wording was maintained for all items. We used Hayes’ al-
phamax macro [43] to establish an optimal combinations of
items regarding coefficient alpha (the alphamax algorithm
maximizes coefficient alpha over every possible combina-
tion of items). Subsequently, confirmatory factor analysis
(CFA) was conducted to compare potential abbreviated
item sets. The CFA estimation followed the procedure
mentioned in the method section.
3.2. Results of phase 1
Internal consistency as expressed by Cronbach’s alpha
was a 5 0.94. The mean sum score of the F-SozU K-14
was 3.97 (standard deviation (SD) 5 0.97), and the full
range from 1 to 5 points was exploited. To allow for com-
parison with the short form, Table 1 lists the item character-
istics of the F-SozU K-14.
4. Phase 2: psychometric evaluation of short form
F-SozU K-6
The newly created brief questionnaire (F-SozU K-6) was
subsequently analyzed based on an independent survey that
was again representative of the Federal Republic of Ger-
many with regard to its psychometric parameters. The
554 S. Kliem et al. / Journal of Clinical Epidemiology 68 (2015) 551e562

Table 1. Mean (M), SD, item difficulty (Pi), corrected itemetotal correlation (rit), and group differences for the F-SozU K-14 items and total scores
Group
Item Total Men Women difference
English German M SD Pi rit M SD Pi rit M SD Pi rit t
I can easily find someone Ich finde ohne weiteres 4.08 0.91 77 0.65 4.03 0.97 76 0.66 4.12 0.87 78 0.65 2.46*
who can look after my jemanden, der sich um
home when I’m not there. meine Wohnung
ku€mmert, wenn ich mal
nicht da bin.
There are people who Es gibt Menschen, die mich 4.12 0.82 78 0.71 4.08 0.82 77 0.69 4.15 0.82 79 0.72 2.31*
accept me the way I am ohne Einschr€ankung so
without reservations. nehmen wie ich bin.
I receive a lot of Ich erfahre von anderen viel 3.91 0.88 73 0.73 3.85 0.91 71 0.74 3.96 0.86 74 0.71 3.10*
understanding and Verst€andnis und
security from others. Geborgenheit.
There is someone very close Ich habe einen sehr 4.18 0.89 80 0.73 4.14 0.91 79 0.74 4.22 0.88 81 0.73 2.12*
to me whose help I can vertrauten Menschen,
always count on. mit dessen Hilfe ich
immer rechnen kann.
If I need to, I can borrow Bei Bedarf kann ich mir 4.06 0.87 77 0.67 4.00 0.90 75 0.67 4.10 0.85 78 0.68 3.07*
something from friends ohne Probleme bei
or neighbors without any Freunden oder Nachbarn
problems. etwas ausleihen.
I have friends/relatives who Ich habe Freunde/ 3.93 0.86 73 0.76 3.87 0.87 72 0.73 3.99 0.85 75 0.77 3.48
will definitively take time Angeh€orige, die sich auf
to listen if I need jeden Fall Zeit nehmen
someone to talk to. und gut zuh€oren, wenn
ich mich aussprechen
m€ochte.
I know several people with Ich kenne mehrere 3.95 0.91 74 0.71 3.93 0.88 73 0.69 3.97 0.93 74 0.73 1.13
whom I like to do things. Menschen, mit denen ich
gerne etwas unternehme.
I have friends/relatives who Ich habe Freunde/ 3.75 1.03 67 0.69 3.60 1.06 65 0.68 3.88 1.00 72 0.70 6.70**
sometimes simply give Angeh€orige, die mich
me a hug. einfach mal umarmen.
When I am sick, I can ask Wenn ich krank bin, kann 4.03 0.86 76 0.71 3.96 0.88 74 0.71 4.08 0.84 77 0.71 3.52**
friends/relatives to ich ohne Z€ogern Freunde/
handle important things Angeh€orige bitten,
for me without hesitation. wichtige Dinge fu € r mich
zu erledigen.
If I’m very depressed, I Wenn ich mal sehr 3.97 0.90 74 0.76 3.86 0.92 72 0.75 4.06 0.87 77 0.77 5.58**
know who I can turn to. bedru€ ckt bin, weiß ich,
zu wem ich damit ohne
weiteres gehen kann.
There are people who share Es gibt Menschen, die 3.97 0.90 74 0.73 3.89 0.95 72 0.71 4.04 0.85 76 0.75 4.09**
both joy and sorrow with Freude und Leid mit mir
me. teilen.
I have some friends/ Bei manchen Freunden/ 3.96 0.91 74 0.71 3.92 0.90 73 0.71 4.00 0.92 75 0.70 2.34*
relatives with whom I can Angeh€origen kann ich
be quite playful. auch mal ganz
ausgelassen sein.
There is someone close to Ich habe einen vertrauten 4.09 0.91 77 0.71 4.05 0.93 76 0.73 4.12 0.893 78 0.70 1.99*
me in whose presence I Menschen, in dessen
feel comfortable without N€ahe ich mich ohne
any reservations. Einschr€ankung wohl
€hle.
fu
There is a group of people Es gibt eine Gruppe von 3.61 1.10 65 0.55 3.63 1.07 66 0.57 3.59 1.13 65 0.54 1.05
(friends, clique) that I Menschen (Freundeskreis,
belong to and whom I Clique), zu der ich geh€ore
meet often. und mit der ich mich
h€aufig treffe.
F-SozU K-14 mean score 3.97 0.68 d d 3.91 0.69 d d 4.02 0.67 d d 3.86**
Abbreviation: SD, standard deviation.
*P ! 0.05, **P ! 0.001. WHO-conform forwardebackward translation from German to English independently carried out by (in each case) two
native speakers of the target language. Discrepancies were discussed in an expert panel with the result of a consensual solution.
 S. Kliem et al. / Journal of Clinical Epidemiology 68 (2015) 551e562 555

representative sample of this survey furthermore served as a

norming group.

4.1. Method of phase 2

4.1.1. Study design and participants
Data collection took place between April and June 2013.
Sampling was conducted using a threefold random selection
procedure: First, 258 sample points, representing different
nonoverlapping inhabited areas in Germany, were randomly
selected [44] from a German community inventory stratified
according to the BIK Aschpurwis þ Behrens GmbH (BIK)
classification system. The BIK is measuring the grade of ur-
banization and the geographic distribution [45]. Second,
target households were selected randomly through the
random route procedure with a specified starting address.
Trained interviewers (k 5 198) were provided with a con-
crete street and a corresponding starting house number.
Beginning with this address, every third household on the
respective street was identified and contacted with the aim
of conducting an interview. For this purpose, the addresses
that had been identified were entered into an address list.
Third, the random selection of the target person in the house-
holds was done with the help of a Kish selection grid to
determine the target person. Before the target person was
randomly selected, the interviewer checked criteria for inclu-
sion (age  14 years and sufficient ability to understand
written German language) for every potential participant. Af-
ter giving full information on the study and data security,
informed consent was obtained. Following a structured
sociodemographic interview, participants completed
self-report questionnaires on physical and psychological
symptoms in the presence of (but without any interference
from) the interviewer. The interviews were conducted at
the participants’ homes. Reasons for nonparticipation and
corresponding figures can be obtained from Fig. 1.
At the institute, interviews and questionnaires were
checked for completeness. Before data entry, the correct fil-
ter procedure was checked by an encoder, and if the cir-
cumstances were ambiguous, they were immediately
corrected. After having been recorded, the data were
checked in a second step based on the original question-
naire and, if necessary, corrected. Interviewers were
controlled by sending prestamped postcards to the partici-
pants (37%, randomly chosen). About 53% of the postcards
were returned; all of them confirmed proper conduct of the
interview. The initial sample consisted of 4,360 persons, of
which 2,508 (57.5%) participated in the full study. All pro-
cedures were authorized by the Ethics Committee of the
Medical Faculty of the University of Leipzig.
Fig. 1. Flowchart of sampling procedure and reasons for nonparticipa-
tion (phase 2).
4.1.2. Measures
4.1.2.1. Demographic questionnaire. Age, gender, family
characteristics, student and employment status, and house- stratification index was created according to the following
hold income were surveyed. Based on the method of the scores: school education (1: no certificate, secondary
research alliances within rehabilitation science, a social school; 2: high school, technical college; 3: university
556 S. Kliem et al. / Journal of Clinical Epidemiology 68 (2015) 551e562
entrance diploma), lifetime professional status (1: laborer;
2: clerical worker, public servant; 3: freelancer), household
income (1: !V1,250; 2: V1,250eV2,000; 3: OV2,000).
4.1.2.2. The Patient Health Questionnaire 2, German
version. The Patient Health Questionnaire (PHQ-2)
[46] is a two-item self-administered version of the
PRIME-MD [47], a diagnostic instrument for common
mental disorders. The PHQ-2 constitutes a depression mod-
ule, which scores the two DSM-5 main criteria for major
depression from 0 (not at all) to 3 (nearly every day).
PHQ-2 sum scores range from 0 to 6, whereas higher values
indicate higher burden of depression. In the present study,
the German version of the PHQ-2 [48] was applied. The
translation of the German version followed state-of-the-art
procedures in cross-cultural assessment [49]. In a recent
population-based study [6] and in the present study, the
PHQ-2 reached high internal consistency (a 5 0.75; study
at hand: a 5 0.80).
4.1.2.3. The Giessen Subjective Complaints List, German
version. To assess somatic symptom strain, we used the
short form of the Giessen Subjective Complaints List
(Gießener Beschwerdebogen GBB-8; [50]). This inventory
comprises eight items: for example, stomach ache, back
pain, headaches, feeling tired, trouble sleeping, and dizzi-
ness. Each symptom is rated on a Likert scale from 1
(never) to 5 (always). GBB-8 sum scores range from 8 to
40, whereas higher values indicate higher somatic burden.
In the study at hand, the GBB-8 reached a high internal
consistency, a 5 0.89.
4.1.2.4. The Generalized Anxiety Disorder Scale-2,
German version. In the Generalized Anxiety Disorder
Scale -2 (GAD-2; [51,52]), two main symptoms of general-
ized anxiety disorder are assessed within the last 2 weeks
using two questions on a four-point scale from 0 (not at
all) to 3 (almost every day). GAD-2 sum scores range from
0 to 6, whereas higher values indicate higher burden of
generalized anxiety. In the present study, the German
version of the GAD-2 [6] was applied. The translation of
the German version followed state-of-the-art procedures
in cross-cultural assessment [49]. The GAD-2 showed high
internal consistency (a 5 0.82; study at hand: a 5 0.79) in
the general population [6].
4.1.3. Statistical analyses
Internal consistency of the F-SozU K-6 is reported as co-
efficient a. To determine selectivity, the correlation of the
respective item with the sum of all other items was computed
(itemerest correlations). Item difficulty coefficients were
calculated as quotients of the sum of the item values that
were obtained and the sum of the maximum achievable item
values, multiplied by 100. We applied chained equation
modeling [53] using the following variables: gender, age,
monthly net income, educational status, and partnership
status to estimate missing data (proportion of missing values
of analyzed items: 0.1  0.4%). To avoid nonexisting item
values, the estimated values (^y) were corrected by predictive
mean matching (ie, the observed values closest to the pre-
dicted value were chosen). We used the R package mice
[54] for this analysis. To replicate well-established associa-
tions and hence provide evidence of validity of the F-SozU
K-6, correlation coefficients with the PHQ-2 depression in-
ventory, the GAD-2 anxiety inventory, and an inventory for
somatic symptom strain, the GBB-8, were calculated. Based
on the results of earlier surveys, the following hypotheses
were formulated: (1) depression level should be higher in in-
dividuals with lower perceived social support scores [55,56],
(2) anxiety level should be higher in individuals with lower
perceived social support [55,56], and (3) somatic symptom
strain should be higher in individuals with lower perceived
social support [12].
We examined a simple general factor model with all
items loading on one factor using CFA. This model allows
for summarizing the item scores to a total score. Because of
significant deviations from a multivariate normal distribu-
tion, the robust maximum likelihood estimation with a
mean-adjusted chi-square test statistic (SatorraeBentler
c2) was applied, which has been shown to be robust to
the violation of normality [57]. Because inconsistent obser-
vations in surveys (eg, due to a sloppy answering style) can
bias statistical results and distort conclusions [58], we con-
ducted multivariate outlier detection. We used the newly
developed item pairebased outlier score for rating scales
(Gþ; [59]), which is defined as the number of weighted
Guttman errors [60] to determine outlier scores. Gþ repre-
sents the degree to which a respondent is more favorable
toward less popular items and is most useful for
one-dimensional scales [59]. The outlier scores were then
classified as discordant or not discordant using Tukey’s
boxplot [61] for outlier detection.
To evaluate goodness of fit of the relevant model, we
considered four different criteria. Although the standard-
ized root mean square residual (SRMR), root mean square
error of approximation (RMSEA), and 90% confidence in-
terval assess absolute model fit, the two additionally calcu-
lated criteria [Comparative Fit Index (CFI) and Tucker
Lewis Index (TLI)] are measures of relative model fit,
compared with the ‘‘null’’ model. RMSEA and SRMR
values !0.050 represent a close fit, values between 0.050
and 0.080 represent a reasonably close fit, and
values O0.080 represent an unacceptable model [62,63].
Regarding CFI and TLI, Hu and Bentler [63] suggested a
CFI and TLI O0.900 for an adequate fit and a CFI and
TLI O 0.950 for a good model fit.
Furthermore, we conducted several measurement invari-
ance tests across gender (group 1 5 men; group
2 5 women), age [group 1 ! 51 years (median
split)  group 2], and gender  age interaction (group 1
women and !51 years; group 2 women and 51 years;
group 3 men and !51 years; group 4 men and
 S. Kliem et al. / Journal of Clinical Epidemiology 68 (2015) 551e562 557

51 years). Measurement invariance tests were performed
using the sequential strategy discussed by Meredith and
Teresi [64]. First, we tested a configural invariance model.
Configural invariance refers to the equivalence of the facto-
rial structure. It is given if the analyzed constructs show the
same dimensionality and, additionally, the observed vari-
ables are correlated with the same latent constructs in both
groups. Configural invariance is a necessary, but not suffi-
cient, condition to expect an unbiased comparison of mea-
surements between groups.
Second, we tested the weak invariance model by con-
straining the estimated factor loadings to be equal across
groups. If empirical support for weak invariance is pro-
vided, it allows for comparing structural relationships [eg,
correlation coefficients, structural (path) coefficients] be-
tween latent constructs in groups. Third, the strong invari-
ance model was tested by constraining both intercepts
and loadings to be equal across groups. This level of invari-
ance enables the comparison of means of the latent
construct between groups.
Finally, we tested the strict invariance model by
constraining the loadings, intercepts, and item error vari-
ances to be equal across groups. Different residual
variances in groups can have two possible consequences.
On one hand, it can lead to different reliabilities of indices
in those groups. On the other hand, it can affect decisions in
screening processes that depend on the expression of a
construct, resulting in different error rates (ie, sensitivity,
specificity) for different groups [65]. As recommended by
Chen [66], CFI differences with a cutoff value of
DCFI O 0.01 were used for testing the different stages of
measurement invariance. Data analysis was carried out with
the R (R Core Team, 2013, Vienna, Austria) packages lav-
aan [67] and mice [54].
4.2. Results
4.2.1. Sample characteristics
Participants’ mean age was M 5 49.67 years
(SD 5 18.30) with a range of 14e92 years; n 5 96
(3.8%) participants were not German regarding nationality.
Further sample details can be found in Table 2.
4.2.2. Item characteristics
Table 3 displays means and SD for the items of the F-
SozU K-6. In addition, item difficulties ( pi) are shown. In
the total sample, the difficulty values varied between
pi 5 68 (I receive a lot of understanding and security from
others.) and pi 5 79. (There is someone very close to me
whose help I can always count on.) The mean sum score

Table 2. Demographic characteristics of the study sample

Sample characteristics Total sample (N [ 2,508) Men (N [ 1,174) Women (N [ 1,334)
Age
Mean (SD) 49.67 (18.30) 49.16 (18.18) 50.12 (18.44)
Median 50.00 50.00 50.00
Range 14e92 14e92 14e92
Age group, N (%)
!25 yr 257 (10.2) 134 (11.4) 123 (9.2)
25e34 yr 360 (14.4) 152 (12.9) 208 (15.6)
35e44 yr 382 (15.2) 180 (15.3) 202 (15.1)
45e54 yr 445 (17.7) 213 (18.1) 232 (17.4)
55e64 yr 454 (18.1) 225 (19.2) 229 (17.2)
65e74 yr 381 (15.2) 177 (15.1) 204 (15.3)
75 yr 229 (9.1) 93 (7.9) 136 (10.2)
Living with a partner, N (%) 1,315 (52.4) 663 (56.5) 652 (48.9)
Years of education, N (%)
8 yr 942 (37.6) 432 (36.8) 510 (38.2)
9e11 yr 1,023 (40.8) 453 (38.6) 570 (42.7)
12 yr 455 (18.1) 238 (20.3) 217 (16.3)
Current student 78 (3.1) 45 (3.8) 33 (2.5)
Missing 10 (0.4) 6 (0.5) 4 (0.3)
Employment status
Pupil/student 192 (7.7) 103 (8.8) 89 (6.7)
Working (!35 h) 1,259 (50.2) 651 (52.8) 608 (45.6)
Unemployed 189 (7.6) 85 (7.2) 104 (7.8)
Homemaker 104 (4.1) 4 (0.3) 100 (7.5)
Retired 745 (29.7) 329 (28.0) 416 (31.2)
Household income in V
!1,250 517 (20.6) 197 (17.3) 320 (24.0)
1,250e2,500 1,156 (45.7) 527 (44.9) 619 (46.4)
2,500 769 (97.0) 417 (35.5) 352 (26.4)
Missing 76 (3.0) 33 (2.8) 43 (3.2)
Abbreviation: SD, standard deviation.
558 S. Kliem et al. / Journal of Clinical Epidemiology 68 (2015) 551e562

Table 3. Mean (M), SD, item difficulty (Pi), corrected itemetotal correlation (rit), and group differences for the F-SozU K-6 items and total scores
Group
Item Total Men Women difference
English German M SD Pi rit M SD Pi rit M SD Pi rit t
I receive a lot of Ich erfahre von anderen viel 3.73 0.93 68 0.70 3.69 0.94 67 0.68 3.78 0.91 70 0.67 2.40*
understanding and Verst€andnis und
security from others. Geborgenheit.
There is someone very close Ich habe einen sehr 4.17 0.94 79 0.76 4.15 0.94 79 0.75 4.18 0.94 80 0.74 0.75
to me whose help I can vertrauten Menschen,
always count on. mit dessen Hilfe ich
immer rechnen kann.
If I need to, I can borrow Bei Bedarf kann ich mir 3.98 0.94 75 0.69 3.95 0.95 74 0.65 4.00 0.93 75 0.68 1.33
something from friends ohne Probleme bei
or neighbors without any Freunden oder Nachbarn
problems. etwas ausleihen.
I know several people with Ich kenne mehrere 4.02 0.95 76 0.68 4.04 0.95 76 .64 4.01 0.94 75 0.69 0.69
whom I like to do things. Menschen, mit denen ich
gerne etwas unternehme.
When I am sick, I can ask Wenn ich krank bin, kann 4.10 0.93 78 0.80 4.09 0.92 77 0.77 4.10 0.93 78 0.79 0.35
friends/relatives to ich ohne Z€ogern Freunde/
handle important things Angeh€orige bitten,
for me without hesitation. wichtige Dinge fu€r mich
zu erledigen.
If I’m very depressed, I Wenn ich mal sehr 4.06 0.97 77 0.79 4.02 1.00 76 0.79 4.10 0.94 78 0.75 1.87
know who I can turn to. bedru€ckt bin, weiß ich,
zu wem ich damit ohne
weiteres gehen kann.
F-SozU K-6 mean score 4.01 0.76 d d 3.99 0.77 d d 4.03 0.75 d d 1.36
Abbreviation: SD, standard deviation.
*P ! 0.05. WHO-conform forwardebackward translation from German to English independently carried out by (in each case) two native
speakers of the target language. Discrepancies were discussed in an expert panel with the result of a consensual solution.

of the F-SozU K-6 was 4.01 (SD 5 0.76), with a range
from 1 to 5 points. At the item level, there were no statis-
tically significant differences in the item values between
men and women. In addition, the corrected itemetotal cor-
relation values (rit) are listed in Table 3. The value charac-
teristics in the total sample, which were between rit 5 0.68
(I know several people with whom I like to do things.) and
rit 5 0.80 (When I am sick, I can ask friends/relatives to
handle important things for me without hesitation.), can
be regarded as very satisfactory. Fig. 2 illustrates the depen-
dence of the F-SozU K-6 sum score from age and sex. A
two-factorial analysis of variance with the factors sex and
age (eight levels: corresponding to the norm allocation)
showed a significant main effect of age, F (6,

Fig. 2. The F-SozU K-6 values depending on age and gender (A) Women, (B) Men.
 S. Kliem et al. / Journal of Clinical Epidemiology 68 (2015) 551e562 559

Table 4. Normative data from the general population (N 5 2,508) for the F-SozU-6
Total
F-SozU mean 14e91 yr 14e24 yr 25e34 yr 35e44 yr 45e54 yr 55e64 yr 65e74 yr ‡75 yr
score (N [ 2,508) (N [ 257) (N [ 360) (N [ 382) (N [ 445) (N [ 454) (N [ 381) (N [ 229)
1.00 0 0 0 0 0 0 1 0
1.17 0 0 1 0 0 0 1 0
1.33 0 0 1 0 0 0 1 0
1.50 1 0 1 1 0 0 1 0
1.67 1 0 2 1 1 0 1 0
1.83 1 0 2 1 1 1 1 1
2.00 2 0 3 2 2 1 1 1
2.17 3 1 3 3 3 2 2 3
2.33 3 1 5 4 4 3 3 5
2.50 5 1 6 5 6 3 4 8
2.67 7 2 7 7 9 7 6 11
2.83 10 4 10 9 10 9 9 16
3.00 14 7 13 11 15 14 13 22
3.17 17 9 18 14 20 17 15 27
3.33 21 12 22 18 25 22 18 31
3.50 25 14 25 22 28 27 24 36
3.67 31 18 29 26 37 32 30 43
3.83 38 23 38 34 42 39 40 50
4.00 49 36 48 44 54 52 49 61
4.17 57 44 55 53 63 60 59 65
4.33 66 55 61 60 70 68 70 74
4.50 73 66 67 68 77 76 77 81
4.67 82 77 76 79 85 84 84 87
4.83 89 87 85 86 92 90 91 92
5.00 !99 !99 !99 !99 !99 !99 !99 !99
Normative data are presented as F-SozU-6 mean scores with corresponding percentiles. Percentiles are shown for the total sample and for
subsamples based on age and gender.

2,480) 5 8.14, P ! 0.001, a nonsignificant main effect of Regarding the CFI differences, strong invariances can be
gender, F (1, 2,480) 5 1.68, P 5 0.195, and a nonsignifi- assumed for gender, age, and age  gender interaction.
cant age  gender interaction effect, F (6,2,480) 5 1.37, Regarding strict invariance, the relevant CFI difference for
P 5 0.221. Thus, in addition to general norms, age- age  gender interaction was slightly above the cutoff value
specific norms will be provided (Table 4). Because the of DCFI O 0.01 as recommended by Chen [66]. For practical
empirical distribution of the F-SozU K-6 sum scores devi- reasons, strict invariance can therefore be assumed.
ated from a normal distribution, the F-SozU K-6 sum scores
were transformed into percentiles.
4.2.6. Construct validity
4.2.3. Internal consistency To determine evidence for validity of the F-SozU K-6,
Internal consistency for the total sample was a 5 0.90 correlation coefficients were calculated with related in-
(men: a 5 0.90; women: a 5 0.90). struments. As can be seen in Table 5, there were low
but substantial correlations between the F-SozU K-6 and
other self-rating inventories in the expected direction:
4.2.4. Factorial validity
Higher perceived social support was associated with
Based on the Gþ statistic, n 5 26 (1.0%) individuals
were excluded from this analysis. CFA revealed very good
fit parameters for the general factor model. All assessed
indices showed an adequate to very good model fit for Table 5. Correlation coefficients between the F-SozU-6 and other self-
the total sample [SRMR 5 0.024; RMSEA 5 0.068, 90% rating questionnaires
confidence interval (0.060, 0.077), CFI 5 0.978, Self-rating questionnaires F-SozU-K6 PHQ-2 GAD-2 GBB-8
TLI 5 0.964]. Factor loadings were high (0.70e0.84). F-SozU-K6 1 d d d
Thus, it can be assumed that all items are meaningful indi- PHQ-2 0.26*** 1 d d
cators of the latent construct. GAD-2 0.21*** 0.76*** 1 d
GBB-8 0.26*** 0.56*** 0.56*** 1
Abbreviations: PHQ-2, Patient Health Questionnaire 2; GAD-2,
4.2.5. Factorial invariance
Generalized Anxiety Disorder Assessment-2; GBB-8, Giessen Subjec-
The results of the measurement invariance analysis tive Complaints List-8.
regarding age, gender, and age  gender are depicted in Spearman’s correlation coefficient was used.
Supplementary Material (e-Table 1) at www.jclinepi.com. ***P ! 0.01.
560 S. Kliem et al. / Journal of Clinical Epidemiology 68 (2015) 551e562
lower depression (PHQ-2), lower generalized anxiety
(GAD-2), and lower somatic symptom strain (GBB-8).
5. Discussion
In the present study, the six-item short form of the
German Social Support Questionnaire (F-SozU) was devel-
oped based on a representative German population sample.
The newly developed questionnaire (F-SozU K-6) was sub-
sequently evaluated and standardized in another indepen-
dent population sample that was also representative of the
general German population. The reliability of the short
form was found to be comparable to the longer versions
of the German Social Support Questionnaire (F-SozU
K-14, F-SozU K-22) [37]. The selectivity of the F-SozU
K-6 was excellent and as expected for a shortened instru-
ment, slightly surpasses the range of the F-SozU K-14
[42]. The mean value of the F-SozU K-6 is similar to the
F-SozU K-14 [37] with a somewhat higher standard devia-
tion, closer to the standard deviation of a standard normal
distribution, thus indicating a higher degree of differentia-
tion. As the kurtosis of the F-SozU K-6 sum scores is lower
than the kurtosis of the F-SozU K-14, it seems unlikely that
the higher standard deviation is a result of a high number of
extreme values. In accordance with associations found with
the F-SozU and its related short forms (F-SozU K-14 and
F-SozU K-22), we found comparable negative correlations
with depression, anxiety, and somatic symptom strain [37].
The one dimensionality of the F-SozU K-6 was confirmed
on the basis of CFA. Hence, a unidimensional interpretation
of the total score is given comparable to the F-SozU K-14
[42]. Furthermore, the analyses showed comparable factor
structures in the subsamples that were studied (with regard
to age, gender, and age  gender). The existence of strict
invariance and the associated possibility of unbiased com-
parison of means and correlation coefficients, as well as
path coefficients within structural equation models between
the aforementioned groups, appear to be particularly
relevant.
5.1. Limitations
Despite the number of strengths of this study, for
example, the large sample size and its representativeness,
certain limitations should be mentioned. First, the response
rate was only 57.5%. However, compared with clinical
studies, a lower response rate is quite common in general
population studies. Furthermore, the response rate was
comparable to other general population surveys [5,68,69].
Over and above, a selection bias seems unlikely because
the study sample corresponds to data from the general
population with regard to demographic characteristics. Sec-
ond, because the study sample is representative of the
general population of Germany, comparisons only with
Western European and white American populations seem
appropriate. In future research, investigators should try to
replicate our findings in different (eg, Chinese or South Af-
rican) and also more heterogeneous cultures.
5.2. Conclusion
In summary, in spite of some limitations, the F-SozU
K-6 provides an economical and reliable instrument for
evaluating the degree of perceived social support. It is
conceivable that it could be used within the framework of
clinical epidemiologic studies. Based on a German repre-
sentative population sample, norm values and pragmatic
cutoff points can be provided as a practical classification
of available social support. The inventory also provides
for an undistorted comparison of measurement values from
both sexes across the full lifespan (14e92 years).
Acknowledgments
Authors’ contributions: S.K., M.Z., and E.B. had full ac-
cess to all the data in the study and take responsibility for
the integrity of the data and the accuracy of the data
analysis.
Supplementary data
Supplementary data related to this article can be found at
http://dx.doi.org/10.1016/j.jclinepi.2014.11.003.
References
[1] Ware JE Jr, Sherbourne CD. The MOS 36-item short-form health
survey (SF-36): I. conceptual framework and item selection. Med
Care 1992;30:473e83.
[2] Gierk B, Kohlmann S, Kroenke K, Spangenberg L, Zenger M,
Br€ahler E, et al. The Somatic Symptom Scalee8 (SSS-8) a brief
measure of somatic symptom burden. JAMA Intern Med 2014;
174:399e407.
[3] Kroenke K, Spitzer RL, Williams JB. The PHQ-15: validity of a
new measure for evaluating the severity of somatic symptoms. Psy-
chosom Med 2002;64:258e66.
[4] Ware JE Jr, Kosinski M, Keller SD. A 12-item short-form health sur-
vey: construction of scales and preliminary tests of reliability and
validity. Med Care 1996;34:220e33.
[5] Kliem S, M€oßle T, Zenger M, Br€ahler E. Reliability and validity of
the Beck depression inventorydfast screen for medical patients in
the general German population. J Affect Disord 2013;156:236e9.
[6] L€owe B, Wahl I, Rose M, Spitzer C, Glaesmer H, Wingenfeld K,
et al. A 4-item measure of depression and anxiety: validation and
standardization of the Patient Health Questionnaire-4 (PHQ-4) in
the general population. J Affect Disord 2010;122:86e95.
[7] Kroenke K, Spitzer RL. The PHQ-9: a new depression diagnostic
and severity measure. Psychiatr Ann 2002;32:1e7.
[8] Spitzer RL, Kroenke K, Williams JB, Lowe B. A brief measure for
assessing generalized anxiety disorder: the GAD-7. Arch Intern
Med 2006;166:1092.
[9] Rief W, Hiller W. A new approach to the assessment of the treat-
ment effects of somatoform disorders. Psychosomatics 2003;44:
492e8.
[10] Kr€oger C, Kliem S. Screening for posttraumatic stress disorder:
comparison of the diagnostic efficiency of two self-rating measures.
Eur J Psychol Assess 2014;30:93e9.
 S. Kliem et al. / Journal of Clinical Epidemiology 68 (2015) 551e562
[11] Breslau N, Peterson EL, Kessler RC, Schultz LR. Short screening
scale for DSM-IV posttraumatic stress disorder. Am J Psychiatry
1999;156:908e11.
[12] Holt-Lunstad J, Smith TB, Layton JB. Social relationships and mor-
tality risk: a meta-analytic review. PLoS Med 2010;7:e1000316.
[13] Cohen S, Janicki-Deverts D. Can we improve our physical health by
altering our social networks? Perspect Psychol Sci 2009;4:375e8.
[14] House JS, Landis KR, Umberson D. Social relationships and health.
Science 1988;241:540e5.
[15] Kessler RC, McLeod JD. Social support and mental health in com-
munity samples. San Diego, CA, US: Academic Press; 1985.
[16] Lakey B, Orehek E. Relational regulation theory: a new approach to
explain the link between perceived social support and mental health.
Psychol Rev 2011;118:482.
[17] Uchino BN, Cacioppo JT, Kiecolt-Glaser JK. The relationship be-
tween social support and physiological processes: a review with
emphasis on underlying mechanisms and implications for health.
Psychol Bull 1996;119:488.
[18] Barth J, Schneider S, von K€anel R. Lack of social support in
the etiology and the prognosis of coronary heart disease: a sys-
tematic review and meta-analysis. Psychosom Med 2010;72:
229e38.
[19] DiMatteo MR. Social support and patient adherence to medical
treatment: a meta-analysis. Health Psychol 2004;23:207.
[20] Lett HS, Blumenthal JA, Babyak MA, Catellier DJ, Carney RM,
Berkman LF, et al. Social support and prognosis in patients at
increased psychosocial risk recovering from myocardial infarction.
Health Psychol 2007;26:418.
[21] Kazarian SS, McCabe SB. Dimensions of social support in the
MSPSS: factorial structure, reliability, and theoretical implications.
J Community Psychol 1991;19:150e60.
[22] Cutrona CE, Russel DW. The provisions of social relationships and
adaptation to stress. In: Jones WH, Perlman D, editors. Advances in
Personal Relationships. (2nd edn), Vol. 1. Greenwich CT: J.S.
House; 1987.
[23] Broadhead W, Gehlbach SH, De Gruy FV, Kaplan BH. The Duke-
UNC Functional Social Support Questionnaire: measurement of so-
cial support in family medicine patients. Med Care 1988;26:
709e23.
[24] Barrera M. A method for the assessment of social support networks
in community survey research. Connections 1980;3:8e13.
[25] Procidano ME, Heller K. Measures of perceived social support from
friends and from family: three validation studies. Am J Community
Psychol 1983;11:1e24.
[26] Sarason IG, Levine HM, Basham RB, Sarason BR. Assessing social
support: the social support questionnaire. J Pers Soc Psychol 1983;
44:127.
[27] Van Lente E, Barry MM, Molcho M, Morgan K, Watson D,
Harrington J, et al. Measuring population mental health and social
well-being. Int J Public Health 2012;57:421e30.
[28] Naeem F, Irfan M, Zaidi Q, Kingdon D, Ayub M. Angry wives,
abusive husbands: relationship between domestic violence and psy-
chosocial variables. Womens Health Issues 2008;18:453.
[29] Cutrona CE, Russell DW. The provisions of social relationships and
adaptation to stress. Adv Pers Relat 1987;1:37e67.
[30] Norbeck JS, Lindsey AM, Carrieri VL. The development of an in-
strument to measure social support. Nurs Res 1981;30:264e9.
[31] Sarason IG, Sarason BR, Shearin EN, Pierce GR. A brief measure of
social support: practical and theoretical implications. J Soc Pers Re-
lat 1987;4:497e510.
[32] Moser A, Stuck AE, Silliman RA, Ganz PA, Clough-Gorr KM. The
eight-item modified Medical Outcomes Study Social Support Sur-
vey: psychometric evaluation showed excellent performance. J Clin
Epidemiol 2012;65:1107e16.
[33] Li F, Godinet MT, Arnsberger P. Protective factors among families
with children at risk of maltreatment: follow up to early school
years. Child Youth Serv Rev 2011;33:139e48.
561
[34] Castle H, Slade P, Barranco-Wadlow M, Rogers M. Attitudes to
emotional expression, social support and postnatal adjustment in
new parents. J Reprod Infant Psycol 2008;26:180e94.
[35] Bovier PA, Chamot E, Perneger TV. Perceived stress, internal re-
sources, and social support as determinants of mental health among
young adults. Qual Life Res 2004;13:161e70.
[36] Fydrich T, Geyer M, Hessel A, Sommer G, Br€ahler E. Frageb-
ogen zur sozialen Unterst€utzung (F-SozU): Normierung an ei-
ner repr€asentativen Stichprobe. Social Support Questionnaire
(F-SozU): Norms of a representative sample. Diagnostica
1999;45:212e6.
[37] Fydrich T, Sommer G, Br€ahler E. Fragebogen zur sozialen Un-
terst€utzung (F-SozU). Social Support Questionnaire (F-SozU).
G€ottingen: Hogrefe 2007.
[38] Barrera M Jr. Distinctions between social support concepts, mea-
sures, and models. Am J Community Psychol 1986;14:413e45.
[39] Heller K, Swindle RW. Social networks, perceived social support, and
coping with stress. Prev Psychol Theory Res Pract 1983;87e103.
[40] House JS. Social support and social structure. Sociol Forum 1987;1:
135e46.
[41] Cobb S. Social support as a moderator of life stress. Psychosom
Med 1976;38:300e14.
[42] Fydrich T, Sommer G, Tydecks S, Br€ahler E. Fragebogen zur sozia-
len Unterst€utzung (F-SozU): Normierung der Kurzform (K-14). So-
cial Support Questionnaire (F-SozU): Standardization of short form
(K-14). Z Med Psych 2009;18:43e8.
[43] Hayes A. A computational tool for survey shortening applicable to
composite attitude, opinion, and personality measurement scales.
Chicago, IL: Midwestern Association for Public Opinion Research;
2005.
[44] Cox LH. A constructive procedure for unbiased controlled rounding.
J Am Stat Assoc 1987;82:520e4.
[45] Aschpurwis þ Behrens GmbH. BIK-Regionen. Ballungsr€aume,
Stadtregionen, Mittel-/Unterzentrengebiete. Methodenbeschreibung
zur Aktualisierung 2000. Hamburg, Germany: Aschpurwis þ Beh-
rens GmbH; 2000.
[46] Kroenke K, Spitzer RL, Williams JB. The Patient Health
Questionnaire-2: validity of a two-item depression screener. Med
Care 2003;41:1284e92.
[47] Spitzer RL, Kroenke K, Williams JB. Validation and utility of a
self-report version of PRIME-MD: the PHQ primary care study. JA-
MA 1999;282:1737e44.
[48] L€owe B, Kroenke K, Gr€afe K. Detecting and monitoring depression
with a two-item questionnaire (PHQ-2). J Psychosom Res 2005;58:
163e71.
[49] Bracken BA, Barona A. State of the art procedures for translating,
validating and using psychoeducational tests in cross-cultural
assessment. Sch Psychol Int 1991;12:119e32.
[50] Kliem S, M€oßle T, Lohmann A, Zenger M, Hinz A, Br€ahler E. The
Gießener Beschwerdebogene 8: a brief measure of somatic symp-
tom burden. in prep.
[51] Kroenke K, Spitzer RL, Williams JB, Monahan PO, L€owe B. Anx-
iety disorders in primary care: prevalence, impairment, comorbidity,
and detection. Ann Intern Med 2007;146:317e25.
[52] Kroenke K, Spitzer RL, Williams JB, L€owe B. An ultra-brief
screening scale for anxiety and depression: the PHQe4. Psychoso-
matics 2009;50:613e21.
[53] White IR, Royston P, Wood AM. Multiple imputation using chained
equations: issues and guidance for practice. Stat Med 2011;30:
377e99.
[54] van Buuren S, Groothuis-Oudshoorn K. MICE: multivariate imputa-
tion by chained equations in R. J Stat Softw 2011;45:1e67.
[55] Grav S, Hellzen O, Romild U, Stordal E. Association between social
support and depression in the general population: the HUNT study, a
cross-sectional survey. J Clin Nurs 2012;21:111e20.
[56] Cohen S, Wills TA. Stress, social support, and the buffering hypoth-
esis. Psychol Bull 1985;98:310.
562 S. Kliem et al. / Journal of Clinical Epidemiology 68 (2015) 551e562
[57] Chou CP, Bentler PM, Satorra A. Scaled test statistics and robust
standard errors for non-normal data in covariance structure anal-
ysis: a Monte Carlo study. Br J Math Stat Psychol 1991;44:
347e57.
[58] Zijlstra WP, van der Ark LA, Sijtsma K. Outliers in questionnaire
data can they be detected and should they be removed? J Educ Be-
hav Stat 2011;36:186e212.
[59] Zijlstra WP, der Ark V, Andries L, Sijtsma K. Discordancy tests
for outlier detection in multi-item questionnaires. Methodology
2013;9:69.
[60] Molenaar IW. A weighted Loevinger H-coefficient extending Mokk-
en scaling to multicategory items. Kwantitatieve methoden 1991;37:
97e117.
[61] Tucker JW. Exploratory data analysis. Reading, MA: Addison-Wesley;
1977.
[62] Browne MW, Cudeck R, Bollen KA, Long JS. Alternative ways of
assessing model fit. Sage Focus Editions 1993;154:136.
[63] Lt Hu, Bentler PM. Cutoff criteria for fit indexes in covariance
structure analysis: conventional criteria versus new alternatives.
Struct Equ Modeling 1999;6:1e55.
[64] Meredith W, Teresi JA. An essay on measurement and factorial
invariance. Med Care 2006;44:S69e77.
[65] Millsap RE, Kwok O-M. Evaluating the impact of partial factorial
invariance on selection in two populations. Psychol Methods 2004;9:93.
[66] Chen FF. Sensitivity of goodness of fit indexes to lack of measure-
ment invariance. Struct Equ Modeling 2007;14:464e504.
[67] Rosseel Y. Lavaan: an R package for structural equation modeling. J
Stat Softw 2012;48:1e36.
[68] Radloff LS. The CES-D scale a self-report depression scale for
research in the general population. Appl Psych Meas 1977;1:
385e401.
[69] Aromaa E, Tolvanen A, Tuulari J, Wahlbeck K. Personal stigma and
use of mental health services among people with depression in a
general population in Finland. BMC Psychiatry 2011;11:52.