See discussions, stats, and author profiles for this publication at: https://www.researchgate.

net/publication/353107671

Is it the perceived stress scale (PSS) Undimimensional and invariant? A

Bifactor analysis in Mexican adults

Article in Current Psychology · July 2021

DOI: 10.1007/s12144-021-02067-x

CITATIONS READS

11 653

5 authors, including:

Arturo Juárez García Cesar Merino

Universidad Autónoma del Estado de Morelos University of San Martín de Porres
125 PUBLICATIONS 948 CITATIONS 331 PUBLICATIONS 2,652 CITATIONS

SEE PROFILE SEE PROFILE

José Félix Brito-Ortíz Alisma Monroy

Autonomous University of Baja California Universidad Autónoma del Estado de Morelos
18 PUBLICATIONS 55 CITATIONS 2 PUBLICATIONS 27 CITATIONS

SEE PROFILE SEE PROFILE

All content following this page was uploaded by José Félix Brito-Ortíz on 04 August 2021.

The user has requested enhancement of the downloaded file.

Current Psychology
https://doi.org/10.1007/s12144-021-02067-x

Is it the perceived stress scale (PSS) Undimimensional and invariant?

A Bifactor analysis in Mexican adults
Arturo Juárez-García 1 & César Merino-Soto 2 & José Félix Brito-Ortiz 3 & Martha Eugenia Nava-Gómez 3 &
Alisma Monroy-Castillo 4

Accepted: 1 July 2021

# The Author(s), under exclusive licence to Springer Science+Business Media, LLC, part of Springer Nature 2021Abstract

The perceived stress scale (PSS) has been widely used as a unidimensional global measure of stress despite its classical bi-
dimensional structure. However, a bifactor model seems to be the best structure alternative in terms of a possible method effect.
The few available studies on PSS validity (in both the 14 and 10 item versions) that test the bifactor model do not consider its
measurement invariance between occupational status and gender. The goal of this study was to test the bifactor structure and
measurement invariance between occupational status and gender in the PSS 14 and 10 item versions. 1149 Mexican adults
voluntarily answered the Spanish version of the PSS, 591 were students and 558 were health or education workers. Through a
SEM methodology, several models were tested (unidimensional, two factors, bifactor, positive and negative wording effect, and
random content). The results showed a bifactor model and a general factor with a subfactor for negative phrasing as the best for
both versions (14–10 items) (CFI > .998, RMSEA<.037). Measurement invariance for sex and occupational status was found.
The PSS-10 version showed slightly better indicators. In conclusion, PSS-10 and PSS-14 versions showed no evidence of gender
or occupational status bias. A bifactor structure was the best model in both versions; however, these findings suggest a clear
method effect in the PSS due to items with different directions (negative phrasing). Suggestions for future research and practical
implications are discussed.

Keywords Perceived stress scale . Bifactor . Validity . Mexican

Introduction study variable that has driven more inter and trans-
disciplinary efforts in the health sciences due to its
Stress is an emerging phenomenon that has captured the biopsychosocial complexity. It is associated with the most
interest of a wide diversity of disciplines. It may be the emerging public health diseases worldwide (e.g., cardio-
metabolic, immune, musculoskeletal, and mental diseases)
and has a salient role in overall mortality (Kiecolt-Glaser
* Arturo Juárez-García & Glaser 1999; Cohen, Janicki-Deverts & Miller, 2007,
arturojuarezg@hotmail.com Sapolsky, 2004). After the biological theory of Selye
(1976), the psychological theory of Lazarus and Folkman
César Merino-Soto
sikayax@yahoo.com.ar; cmerinos@usmp.pe (1984) earned its place as a way to conceive stress from a
more widely and necessary biopsychosocial perspective,
José Félix Brito-Ortiz adopting a transactional approach of the complex relation-
felix.brito@uabc.edu.mx
ship between people and their environment. This theory
Martha Eugenia Nava-Gómez considers three elements responsible for stress: the envi-
martha.nava40@uabc.edu.mx
ronment, the personal susceptibility of the individual, and
Alisma Monroy-Castillo mainly, the cognitive appraisal of events. For Lazarus and
alisma.monroy@uaem.mx Folkman (1984), stress is: “a particular relationship be-
1
tween the person and the environment that is appraised
Universidad Autónoma del Estado de Morelos, Cuernavaca, México
by the person as taxing or exceeding his or her resources
2
Universidad de San Martín de Porres, Lima, Peru and endangering his or her well-being” (p. 43); thus, it is
3
Universidad Autónoma de Baja California, Tijuana, México stated that stress is mainly a subjective phenomenon, a
4
Universidad Autónoma del Estado de Morelos, Cuernavaca, México pivotal feature to consider when measuring it.

Content courtesy of Springer Nature, terms of use apply. Rights reserved.


The Perceived Stress Scale (PSS; Cohen, Kamarck, and
Mermelstein, 1983) is one of the most widely used tools in
clinical and non-clinical scenarios to measure perceived stress.
It was developed based on Lazarus’ concept of cognitive ap-
praisal but in terms of an experience of overall stress (not of
specific events). In the PSS model, people assess the degree of
control they have or do not have over unpredictable or unex-
pected situations in their lives. This assessment includes the
perception of oppressive or burdensome situations, which in
the absence of adequate coping resources, translates into un-
ease and discomfort. This agrees with the transactional and
cognitive perspective of stress (Lazarus & Folkman, 1984;
Remor & Carrobles, 2001; Remor, 2006).
The original PSS consists of 14 items. 7 reversed (or neg-
atively worded) items favoring control over threatening situa-
tions, and 7 positively worded items focused on the loss of
control and its consequences (Cohen et al., 1983). The scale
was subsequently shortened; first to a 10-item (6 positive and
4 negative items), and later to a 4-item version (2 positive and
2 negative items; Cohen & Williamson, 1988). The original
PSS English version has been translated into several lan-
guages, including Arabic (Almadi, Cathers, Mansour, &
Chow, 2012), German (Reis, Lehr, Heber & Ebert, 2019),
Iranian (Safaei & Shokri, 2014), Korean (Lee & Jeong,
2019), Greek (Michaelides, Christodoulou, Kkeli, Karekla &
Panayyioutu, 2016), French (Lesage, Berjot & Deschamps,
2012), Japanese (Mimura & Grifiths, 2004), Swedish
(Eklund, Bäckström & Tuvesson, 2014), Serbian
(Jovanović, & Gavrilov-Jerković, 2015), and Spanish
(Remor and Carrobles, 2001; Remor, 2006), amongst others.
These wide range of translations point to the far-reaching in-
ternationalization of the PSS, and to the potential representa-
tional cognitions of stress and their universality (Fetvadjiev &
Van de Vijver 2015). For information on all translations
please visit the author’s website: https://www.cmu.edu/
dietrich/psychology/stress-immunity-disease-lab/scales/
index.html.
The factor structure of the PSS scale was not presented in
Cohen et al.’s seminal work (1983), but a later study (Cohen
& Williamson, 1988) provided 2 factors obtained via a prin-
cipal component method. Surprisingly, the second factor,
which contained only the recoded or negatively worded items
(items reflecting control over stress), was theoretically and
methodologically underestimated. This factor was considered
by the authors as “irrelevant”. They insist on the unidimen-
sionality of the scale and suggest the use of the total score.
However, this can be troublesome from a theoretical and psy-
chometric point of view (Reis, et al., 2019), and it is worri-
some given that the finding of a two-factor model is so
frequent.
Based on substantive studies, including those of the origi-
nal authors (Cohen & Janicki-Deverts, 2012; Cohen, Tyrrell,
& Smith, 1993; Cohen & Williamson, 1988), the structure of
Content courtesy of Springer Nature, terms of use apply. Rights reserved.
Curr Psychol
2 factors has been replicated by dozens of subsequent studies.
For example, in a systematic review of the psychometric prop-
erties of the PSS, Lee (2012) selected 18 reports spanning
from 1988 to 2012 and found that practically all the studies
reported a 2 factor-structure. It is important to note that in most
of these studies, Cronbach’s Alpha indices (α) evidenced ac-
ceptable thresholds (>.70); however, these were reported as-
suming the unidimensionality of the scale without testing it.
Likewise, other methodological assumptions were made with-
out testing. This trend continues in other studies after the pub-
lication of Lee’s review (Gónzález, Rodríguez, and
Hernandez, 2013; Safaei & Shokri, 2014; Eklund,
Bäckström & Tuvesson, 2014; Smith, Rosenberg & Haight,
2014, amongst others).
In general, some limitations can be identified in many of the
above-mentioned studies: a) reliability is estimated with
Cronbach’s α coefficients without verifying the compliance of
their assumptions; b) the exploratory solutions are reached with
common factor methods or with the “Little Jiffy” criterion,
Known as the Kaiser rule (Kaiser, 1970) (Principal Com-
ponents, eigenvalues equal to or greater than 1, and Varimax
rotation); c) CFA is estimated with the maximum likelihood
(ML) function; d) invariance of the scale across groups is as-
sumed, making direct comparisons between them (e.g. sex, age,
occupation, etc.); and e) there is no empirical evidence to sup-
port the use of total score with the use of suitable methods to
directly examine the uni- or multi-dimensionality of the scale.
Regarding the points discussed above, it should be noted
that: a) the Cronbach’s Alpha coefficient (α) is suitable if
assumptions such as tau-equivalence, are verified and com-
plied with (Trizano-Hermosilla, & Alvarado, 2016). The ome-
ga coefficient (ω), is a more robust indicator that can better
represent the reliable variance of the construct (McDonald,
1999); b) the use of the Little Jiffy criterion is currently the
least recommended method to validate constructs and mea-
sures of latent variables (Brown, 2015; Lloret-Segura,
Ferreres-Traver, Hernández-Baeza, & Tomás-Marco, 2014;
Park, Dailey & Lemus, 2002); c) although it is frequently
used, the ML function is sensitive to the breach of the multi-
variate normality assumption, and applying a CFA – SEM
without this feature miscalculates the estimated parameters
of the items and the latent covariance (Brown, 2015); d) it is
well known that the scores of any scale could change depend-
ing on the characteristics of a sample (e.g. sex or occupational
status). When such variance is relevant to the latent variable
assessed, there is a bias that invalidates any differences orig-
inally found between groups. Measurement invariance estima-
tion ensures that the meaning and levels that outline the con-
struct are equivalent across groups. This can be directly tested
through differential item functioning (DIF), or multigroup
confirmatory factor analysis (MCFA) (Reis et al., 2019;
Sass, 2011). Finally, e) the recommended procedures that
prove the uni- or multi-dimensionality are based on methods
Curr Psychol
like bifactor modeling (Reise, Morizot, & Hays, 2007), and
there are few available psychometric studies on PSS using this
approach.
After Lee’s review (2012), there is another set of studies
where the analysis of measurement invariance and bifactor
modeling have been applied to the PSS. For example, it was
recently observed that the scale is invariant or equivalent for
variables such as sex (Reis et al., 2019; Denovan et al., 2019);
language (Perera et al., 2017); type of participants (ie, clinical/
non-clinical; Jovanović & Gavrilov-Jerković, 2015), passing
time (Reis et al., 2019), or between athlete and non-athlete
students (Chiu, Lu, Lin, Nien, Hsu, & Liu, 2016). However,
no studies have analyzed the invariance across occupational
status, like professional employees (teacher and nurses) and
students, even though these groups have concentrated the ma-
jority of validation studies with the PSS (Lee, 2012).
Furthermore, given the biological or gender differences in
experiencing stress in different contexts, it seems necessary
to continue testing the invariance by sex in different samples,
thereby preventing wrong generalizations and procuring a
suitable treatment for each group (Messing et al., 2003).
Some studies have used a hierarchical CFA as a better fit
option for the two-factor model of the PSS (Golden-Kreutz,
Browne, Frierson, & Andersen, 2004; Siqueira Reis, Ferreira
Hino, & Romelio Rodriguez Añez, 2010). However, it has
long been observed that these two factors frequently covariate
highly (r > .50), suggesting the need for bifactor modeling.
Indeed, recent studies demonstrated that a bifactor structure
with a simultaneous general factor and two specific factors
(negative items and positive items in each factor) has been
superior to the two related factors model (Denovan et al.,
2019; Jovanović, & Gavrilov-Jerković, 2015; Lee, & Jeong,
2019; Park, & Colvin, 2019; Reis, Lehr, Heber, & Ebert,
2019; Reyna, Mola, & Correa, 2019; Wiriyakijja et al.,
2019; Wu, & Amtmann, 2013). Other studies have found a
better-fit model with a general factor and just one specific
subfactor which captures the negative items (i.e., recoded;
Michaelides, Christodoulou, Kkeli, Karekla, & Panayiotou,
2016; Perera et al., 2017). These findings suggest that there
is still no homogeneity in the psychometric bifactor structure
of the PSS. Besides, it should be noted that practically all of
these studies used only the 10-item version of the PSS, thus
the bifactor structure with the original 14-item scale is still
unknown, as is the case of a comparative evaluation including
both scales (see Table 1).
It is important to note that the interpretation of bifactor
modeling in these studies has been oriented towards its sub-
stantiality (theoretical aspect), rather than to the plausible ef-
fect of the method caused by the wording of the items (meth-
odological aspect). Bifactor modeling may also be interpreted
as a variant of the multitrait-multimethod approach (Markon,
2019; Reise, Moore & Haviland, 2010), where the covariance
of the substantive factors is parametrized and the method
Content courtesy of Springer Nature, terms of use apply. Rights reserved.
factors are maintained as orthogonal (correlated attributes –
non-correlated methods). Consistently, confirmatory factor
analyses through structural equation modeling (CFA –
SEM) have allowed verifying the effect of negatively worded
items (i.e., recoded items) together with positively worded
items. In general, this trend to share unique variance requires
to be modeled next to the substantive factors (Corwyn, 2000;
Horan, DiStefano, & Motl, 2003; Marsh, 1989). Some studies
with other self-reported measures in Hispanic populations
have also raised this concern (Dominguez-Lara, & Merino-
Soto, 2018; Tomas & Oliver, 1999; Tomás, Galiana,
Hontangas, Oliver, & Sancho, 2013; Tomás, Sancho, Oliver,
Galiana, & Meléndez, 2012; Suárez-Alvarez, Pedrosa,
Lozano, García-Cueto, Cuesta, & Muñiz, 2018). In evaluating
the effect of item wording, control strategies in structural
equation modeling (SEM) have been established for years,
such as the multitrait-multimethod approach (MultiTrait-
MultiMethod: MTMM; Marsh, 1989; Marsh and Grayson,
1995). The most frequent use occurs with the Correlated
Trait-Correlated Method approach (CTCM; Marsh, &
Grayson, 1995) which can be independent or complementary
to the bifactor analysis in general, and to another commonly
applied method called Correlated Trait-Correlated
Uniqueness (Marsh, 1989; Marsh & Bailey, 1991).
Based on all of the above, the purpose of this study is to
determine the factor structure of the Perceived Stress Scale
(PSS) in the 10- and 14-item versions, through bifactor model-
ing with a multitrait-multimethod approach, and to analyze the
measurement invariance across sex and occupational status
(workers and students) in a sample of Mexican adults.
Method
Population
A non-random sample of 1149 Mexican adult subjects from
two universities was included in this study. 558 participants
were healthcare and education workers, and 591 were
healthcare students. All of them were residents of the States
of Morelos and Baja California and were invited to participate
mainly by e-mail. The response rate was about 60%. All the
participants were enrolled after signing an informed con-
sent. This study was performed in line with the principles
of the Declaration of Helsinki. Approval was granted by the
Ethics Committees of the Health Sciences School at the
University of Baja California (2018/No F-PROMEP-38/
Rev-04) and the Center for Transdisciplinary Psychology
Research at the University of Morelos (2019/No.CEI-
210219-12). In addition to voluntary participation, the only
inclusion criterion was to be an adult over 18 years old. The
exclusion criteria were not completing the questionnaire ful-
ly, being under psychiatric treatment (detected through the
 Curr Psychol

Table 1 Studies of bifactor structure with the PSS scale

Study Sample Size and Major Results Invariance Remarks

(Country) Type

Wu, & N = 446 Versions: PSS-10 and PSS-14 Not reported • PSS-10 better than PSS-14
Amtmann, Multiple Language: English • The use of a total score is suggested
(2013) Sclerosis Best model: bifactor (1 general
USA Patients and 2 specific factors, 6 positive
Country: USA and 4 negative items)
Jovanović & N = 157 Versions: PSS-10 Not reported • PSS-10 better than PSS-14
Gavrilov- Psychiatric Language: Serbian • The use of a total score is suggested
Jerković, patients Best model: bifactor (1 general
(2015) N = 165 and 2 specific factors, 6 positive
Serbia Non-clinical and 4 negative items)
adults
Michaelides N = 320 Versions: PSS-10 Not reported • The use of a total score is suggested
(2016) Adults in general
Language: Greek
Greece Best model: bifactor (1 general and
2 specific factors, 6 positive and
4 negative items)
Perera et al. N = 5176 Versions: PSS-10 Not reported • The calculation of one score using a General Factor and
(2017) Hispanic/Latinos Language: Spanish and English another using the positive subdimension recoded is
USA Best model: bifactor (1 general suggested, due to a method effect found.
and 2 specific factors, 6 positive
and 4 negative items)
Reis et al. N = 11,939 Versions: PSS-10 Invariance • The use of the total score and that of subfactors is
(2019) German adults Language: German across suggested, depending on the interest of the investigator
Germany N = 1862 Best model: bifactor (1 general groups: sex • The scale shows temporary and sex invariance.
German adults and 2 specific factors, 6 positive Temporary
and 4 negative items) invariance
Denovan et al. N = 524 Versions: PSS-10 Group: sex • The use of a total score is suggested
(2019) University Language: English • The scale shows measurement invariance by sex
United Students Best model: bifactor (1 general
Kingdom and 2 specific factors, 6 positive
and 4 negative items)
Lee, & Jeong, N = 250 Versions: PSS-10 Not reported • The use of a total score is suggested
(2019) University Language: Korean
Korea Students Best model: bifactor (1 general
and 2 specific factors, 6 positive
and 4 negative items)
Wiriyakijja, N = 260 Versions: PSS-10 Not reported • The use of a total score is suggested
et al. (2019) Odontology Language: English
United students Best model: bifactor (1 general
Kingdom and 2 specific factors, 6 positive
and 4 negative items)
Park, & Colvin N = 373 Versions: PSS-10 Not reported • The use of a total score is suggested
(2019). Military Language: Korean
South Korea personnel Best model: bifactor (1 general
and 2 specific factors, 6 positive
and 4 negative items)
Reyna et al. N = 279 Versions: Version PSS 10 and Not reported • The use of a total score is suggested
(2019) Adults in general PSS 14
Argentina Language: Spanish
Best model: bifactor (1 general
and 2 specific factors, 6 positive
and 4 negative items)

self-report item: “are you under any psychiatric treat-
ment?”), or being identified as a multivariate outlier. After
the exclusion criteria screening (see the pretrial and item
analysis section), the effective sample size was made up
of 1093 participants. The demographic characteristics of
the participants were: average age was 39 years old (range
23–72 years old, SD = 10), 74.3% were females and the
most common academic level was undergraduate or gradu-
ate (51.9%), followed by middle school (20%), postgraduate
(17.5%), and high school (10.6%).

Content courtesy of Springer Nature, terms of use apply. Rights reserved.

Curr Psychol
Instrument
The Spanish version of the Perceived Stress Scale (PSS;
Cohen et al., 1983; Remor & Carrobles, 2001; González &
Landero, 2007) was completed by all participants. It consists
of 14 items, distributed in two factors on a Likert-type scale,
ranging from “never” (0) to “very often” (4). The scoring of
items 4, 5, 6, 7, 9, 10, and 13 are recoded to generate a new
score which indicates greater perceived stress. In the 10-item
version, 4,5, 12, and 13 items are removed. The questionnaires
were administered at the workplace sites through the author’s
contacts, and some were sent by email.
Procedure
Data Analysis
Potential Response Bias Multivariate outliers were examined
using D2 distance (Mahalanobis, 1936). In order to have a
balance between Type I and Type II errors, and due to possible
differences in data collection and its effects on the responses,
the significance level of D2 was preserved at 99% with the
Bonferroni correction (Penny, 1996), this is at α = .01 (critical
value, D2 ≥ 29.134). Also, individual variability was calculat-
ed to detect response patterns associated with the careless or
insufficient effort in responding (irv, greater or lesser
variability suggests random and/or careless responding;
Meade, & Craig, 2012). Analyses were performed using the
careless R program (Yentes, & Wilhelm, 2018).
Descriptive Characteristics of Items The descriptive character-
istics of items were analyzed as well as their association with
three demographic variables, (e.g. sex, age, and occupational
status). Glass rank-biserial correlations (King, Rosopa, &
Minium, 2011) and polyserial correlations were calculated to
assess these associations. We used the Rcompanion
(Mangiafico, 2020), and polycor (Fox, 2019) programs,
respectively.
Internal Structure The number of latent dimensions in the PSS
was firstly verified. The Exploratory Graph Analysis (EGA;
Golino & Epskamp, 2017), a new technique based on net-
works, was then applied. In this case, 500 parametric boot-
strap samples, and Glasso estimation of Gaussian graphical
models (Golino & Epskamp, 2017) were specified. Also, as
a complementary exploration of the number of factors, we
performed an optimum parallel analysis (Timmerman, &
Lorenzo-Seva, 2011). The EGAnet (Golino & Christensen,
2020) and Factor (Ferrando, & Lorenzo-Seva, 2017) pro-
grams were used.
The methodological approach used was confirmatory fac-
tor analysis (CFA), within the structural equation modeling
technique (SEM). Conceptually and empirically, the models
Content courtesy of Springer Nature, terms of use apply. Rights reserved.
specifying one factor to capture the method effect are more
recommended than the correlated uniqueness models
(Conway, Lievens, Scullen, Lance, 2004; Fan, & Lance,
2017; Lance, Noble, & Scullen, 2002). Thereby, this study
estimates the method effect by configuring the factors oriented
towards negatively or positively worded items, through two
basic substantive models: 1) the unidimensional model, where
all the items were modeled within a single factor, which rep-
resents the global construct of stress; and 2) the related factors
model, where the items are pooled in two factors linked to the
wording of the items (positive and negative) (Fig. 1).
Additionally, we estimated another group of models ad-
dressed to demonstrate the method effect using correlated
traits–correlated methods (CTCM). They were specified as
follows: 3) the bifactor model, which contained a general fac-
tor Fg (with all the negative items recoded, and the rest of the
positive items) and two specific factors (one containing the
negative items and the other with positive items); representing
also a model of correlated traits–correlated methods (Markon,
2019; Reise, Moore, & Haviland, 2010); 4) a model of one
substantive general factor (all of the items), and one factor
with the negatively worded items, and 5) a model of one
substantive factor (all of the items) and one factor with the
positively worded items. Both models 4 and 5, had the nega-
tively worded items recoded. These models are related to the
models of correlated traits–correlated methods minus one
(CT–C[M–1]; Eid, Lischetzke, Nussbeck, & Trierweiler,
2003), where one of the factors is considered as a reference
to interpret the general factor. Finally, according to
Schönberger and Ponsford (2010), the addition of one factor
in a model to improve its fit raises the risk of capitalizing on
the parameters added, even if the added factor is not substan-
tive. To corroborate the potential effect on the fit improvement
produced by the bifactor, another model was created 6): eight
items were chosen randomly (four positively worded and four
negatively worded) from the PSS to replace the method factor.
The fit deterioration in this model suggests the relevance of
the true method factors for positive and negative wording. Fig.
1 shows a graphic representation of the models tested.
The assessment of multidimensionality in the bifactor anal-
ysis was made with several descriptive coefficients. Firstly,
with the explained common variance (ECV; Rodriguez
et al., 2016a, 2016b) in the general factor (ECVg), in the spe-
cific factors (ECVf), and the items (I-ECV; Stucky, Edelen,
Vaughan, Tucker, & Butler, 2014). Secondly, factor reliability
was evaluated with the total reliable variance (ω) and the spe-
cific variance of each factor (ωh). Also, the degree of construct
replicability was evaluated by the H coefficient (Hancock &
Mueller, 2001). Thirdly, the absolute relative parameter bias
indicator was used (ARPB; Rodriguez et al., 2016a, 2016b).
This quantifies the degree of metric difference of the items
between a unidimensional model and the general factor of
the bifactor model. Finally, factor determinacy (FD,

Fig. 1 Six Measurement Models tested in the PSS – 14 and PSS – 10.
Note. G general factor. P: factor for positive items. N: factor for negative
items. n1 … n4: negatively worded items. p1 … p4: positively worded
Gorsuch, 1983; Rodriguez et al., 2016a, 2016b) and the higher
magnitude of these ratios (ECV > .70; ω ≥ .80; H ≥ .80; FD ≥
.90) indicate a greater power of the general factor (Quinn,
2014; Rodriguez et al., 2016a, 2016b), except for ARPB
(ARPB: between 10% and < 15%). These coefficients were
calculated through the Bifactor Indices Calculator in R pro-
gram (Dueber, 2020).
Modeling of both substantive and method effect models
were performed with WLSMV estimator (Muthén, du Toit,
& Spisic, 1997) and polychoric inter-item correlations. This
was considered appropriate because of its efficacy to treat
categorical ordinal variables within SEM modeling (Li,
2016). All the models were assessed using approximate fit
indices, including CFI (≥ .95), RMSEA (≤ .05) and its 90%
confidence intervals, SRMR (≤ .05), and WRMR (≤ .90; Yu,
2002). These were implemented using the lavaan (Rosseel,
2012) and semTools (Jorgensen, Pornprasertmanit,
Schoemann, & Rosseel, 2020) packages in R. The evaluation
of invariance was performed with the cumulative procedure of
added constraints (Brown, 2015). For categorical variables,
invariance estimation was performed according to Wu and
Estabrook specifications (2016), that is, from the
Content courtesy of Springer Nature, terms of use apply. Rights reserved.
Curr Psychol
items. A: factor for randomly selected items. Residuals not shown. (The
number of items shown is only informative).
unconstrained model (i.e., configural or weak invariance) to
the model with constrained intercepts (i.e., metric or strict
invariance). The fit assessment in these models was performed
using the differences in the fit ratios (Chen, 2007), including
ΔCFI (< .01), ΔRMSEA (< .015), and ΔSRMR (< .030). To assess
measurement invariance as an effect size measure, specific
measures were implemented for the degree of the impact on
the item, using the dmacs measure (Nye, & Drasgow, 2011),
which is a standardized measure equivalent in the interpreta-
tion of standardized differences (Nye, Bradburn, Olenick,
Bialko, & Drasgow, 2019).
Results
Preliminary and Item Analysis
In the preliminary analysis of the response biases, 60 subjects
with a D2 between 29.73 and 52.29 (i.e., 5.2% of the initial
sample) were detected. They accounted for an individual var-
iability out of the median (Md = .916), i.e., between irv = .196
and irv = 1.988. The response patterns seemed to be
Curr Psychol
inconsistent and therefore they were removed, thus, the effec-
tive final sample consisted of 1093 participants.
Regarding the items, the hypothesis of univariate (CVM >
6.00, p < .01) and multivariate (Royston’s – H = 1534.597, p <
.01) normal distribution were not validated with the data. On
the other hand, a correlation between the items of the scale and
three sociodemographic variables (sex, age, and occupation)
was found. This suggests that exposure to these conditions is
related to different response trends (Table 2) and demonstrates
sensibility to statistical and practical significance. For in-
stance, the average correlation between sex and the positively
worded items was lower (Mr = .028) than with the negatively
worded items (Mr = .192). This may show that females tend to
higher responses to the items emphasizing the maladaptive
stress response. In contrast, there was a higher linear covari-
ance between age (Mr = −.231) and occupation (Mr = .288)
for the positively worded items, in contrast with the correla-
tions between age (Mr = .036) and occupation (Mr = .093) for
the negatively worded items. In general, younger people per-
ceive more stress, and the occupational status (worker vs.
student) tends to show higher correlations, suggesting that
the perception of stress in workers is higher than that of stu-
dents (Table 2).
Modeling
Number of Dimensions For the 14-item version of the PSS,
EGA yielded a result of two dimensions with items
pooled in wording-based sets (Fig. 2). A possible factorial
complexity in items 2, 3, and 10 was also observed, sug-
gesting measurement ambiguity (items explained by more
than one dimension). The parallel analysis also suggested
two factors. For the 10-item version of the PSS, the re-
sults showed the same pattern on the number of latent
dimensions and the complexity of the items.
Model Fit and Measurement Invariance
PSS-14 For the fit indicators presented in Table 3, the unidi-
mensional model (1F) was not satisfactory at all. The two-
dimension model was only partially satisfactory (due to the
discrepancy between its CFI, SRMR, and RMSEA). In con-
trast, when considering the WRMR, both models showed a
WRMR >1.0 and cannot be considered satisfactory. At the
same time, when modeling method factors, the CFI was not
discriminative enough to identify the most acceptable model,
except for the model introducing a factor with random content
(1F + Fran). For this model (Fran consisted of items 1, 2, 3, 14,
4, 6, 9, and 13), all the fit indicators converged in showing a
substantial decrease of the fit, and consequently, this model
was also not considered adequate to the data. Overall, the set
of models for the method effect got the best-fit indices. In
order of relevance, the bifactor model was the best, followed
Content courtesy of Springer Nature, terms of use apply. Rights reserved.
by the unidimensional model with a method subfactor for
negative wording (items recoded) (1F + 1 fm neg), and then
the rest (Table 3).
Table 4 shows the parameters for the items. For the
bifactor model, the negatively worded items have a low
validity for the general factor Fg, while the factor for the
positive items (Fpos) carries heterogeneous factor loadings
with low to moderate magnitudes. Meanwhile, the specific
factor Fneg retained the highest factor loadings. This result
is reflected in the fit indices of the bifactor model, where
the power of Fneg was clear (I-ECV > .85, ω > .90, ωh >
.80, H > .90, and FD > .90), in contrast with Fg weak-
ness. The fit indicators of the items (RPB and I-ECV) also
confirm that the items do not represent a strong general
factor.
In assessing measurement invariance across groups
(Table 5), successive constraints did not produce great chang-
es in the parameters of the items and there was no need to add
modifications to improve the fit and establish the presence of
invariance. However, when comparing both groups (by sex
and occupational status) invariance of the intercepts seemed to
be absent for the ΔRMSEA difference. However, the effect size
indicators (at the item levels) were negligible: dmacs between
1.07e-16 and 8.89e-17, indicating that invariance between
groups remains.
PSS-10 The results for this version (also shown in Tables 2, 3,
4 and 5) were similar regarding the order of the better models
estimated (based on a WRMR). A closer inspection of the
WLSMV-□2 test and its associated fit indices shows that these
were comparatively better for the PSS-10 versus the PSS-14.
Nevertheless, the fit was also poor for the models 1F, 2F, and
the random factor model (Frandom composed of items 2, 8, 14,
6, 9, and 10).
Because of inadequate initial convergence to estimate the
bifactor model, the quasi-Newton L-BFGS-B (Byrd,
Lu, Nocedal, & Zhu, 1995) optimization algorithm with
optimal method = “L-BFGS-B” was performed. The param-
eters of the items presented in Table 4 show similar results
in both the PSS-10 and PSS-14 in terms of the fit of the
bifactor model and the size of the item factor loading.
Nevertheless, both items and fit indicators (EVC, ω,
ARPB, H, and FD) tended to be slightly superior for the
PSS-10 compared with the PSS-14. In summary, this ver-
sion of the PSS also confirms the weakness of an underlying
general factor and a higher power of the factor associated
with the negative wording (Fneg).
Regarding the invariance of the PSS-10, the changes in fit
indicators did not show substantial evidence of lack of invari-
ance. The discrepancy that calls for attention occurred in
ΔRMSEA, but the size effect indicators (dmacs) also pointed
out a negligible impact within a range similar to the results
of the PSS-14.
 Curr Psychol

Table 2 Descriptive statistics and correlations of the PSS items (n = 1093)

M SD sk ku CMV Correlation: sex Correlation: age Correlation: occupational status (CI 95%)
(CI 95%) (CI 95%)

rg Inf. Sup. rps Inf. Sup. rg Inf. Sup.

pss1* 1.842 .956 .160 −.190 9.870 .000 −.070 .070 −.238 −.295 −.181 .286 .223 .347
pss2* 1.603 1.009 .371 −.289 8.755 .061 −.013 .145 −.310 −.367 −.253 .379 .321 .436
pss3* 2.344 1.028 .018 −.663 8.126 .049 −.024 .123 −.283 −.338 −.228 .355 .299 .411
pss8* 1.838 .932 .134 −.364 9.581 .036 −.037 .111 −.218 −.275 −.161 .265 .197 .332
pss11* 1.994 .955 .218 −.302 9.759 .086 .014 .163 −.290 −.345 −.235 .351 .292 .407
pss12 2.781 .944 −.379 −.553 9.352 .036 −.039 .116 −.101 −.161 −.041 .160 .101 .232
pss14* 1.853 1.027 .143 −.576 7.776 −.071 −.144 .003 −.183 −.242 −.125 .225 .164 .295
pss4 2.510 1.077 −.450 −.448 8.035 .171 .090 .236 −.042 −.102 .018 .176 .112 .249
pss5 2.501 1.049 −.433 −.416 8.433 .224 .156 .292 .022 −.040 .084 .138 .062 .204
pss6* 2.543 1.155 −.531 −.543 7.795 .216 .141 .280 .042 −.020 .104 .113 .038 .179
pss7* 2.468 1.105 −.434 −.458 7.380 .194 .123 .268 .049 −.011 .109 .088 .019 .159
pss9* 2.551 1.037 −.526 −.346 10.018 .253 .182 .314 .037 −.025 .099 .134 .067 .201
pss10* 2.114 .941 −.025 −.356 9.485 .118 .039 .186 .116 .056 .176 −.065 −.125 −.003
pss13 2.309 1.012 −.254 −.519 8.378 .171 .105 .224 .033 −.027 .093 .070 .008 .135

Note. sk: skew. ku: kurtosis. rg: Glass’s biserial correlation. rps: polyserial correlation. Sex: male (female = reference category) occupational status:
worker (student reference category). *Items for PSS-10 version

Discussion and Conclusions
This study analyzed the psychometric properties of the PSS
(10 and 14 items) and focused mainly on unraveling the prob-
lem of its dimensionality. It was based on the internal structure
and measurement invariance examinations as relevant pieces
of evidence for validity and adequate interpretations of the
scale. The PSS was originally designed as a tool for measuring
global perceived stress through the simple metrical sum of its
items (Cohen et al., 1983; Cohen, & Williamson, 1988).
However, our results confirm a fundamental conclusion that
previous studies also have demonstrated (Table 1). It is nec-
essary to introduce another source of additional variability to
better understand the psychometric functioning of the PSS.
Here, the additional variability took the form of factors that
have to be modeled to separate their effect from the global
dimension. This effect was mainly observed in the reversed
or negatively worded items relative to the construct.
Although the negative phrasing and recodification proce-
dure have been unconditionally well accepted since the pivot-
al studies of the PSS, our findings suggest that it generates a
method effect. Other authors have correctly identified and

Fig. 2 Dimensionality explored

by EGA approach (n = 1093)

PSS-14
PSS-10

Content courtesy of Springer Nature, terms of use apply. Rights reserved.

Curr Psychol

Table 3 Fit of the models tested with the PSS – 14 and the PSS – 10 (n = 1093)

WLSMV-χ2 CFI RMSEA WRMR SRMR Ranking best fit

(gl)

PSS – 14
Substantive models
1) 1F 6478.932 .892 .276 6.639 .199 6
(77) (.270, .282)
2) 2F 404.186 .994 .063 1.658 .053 4
(76) (.057, .069)
Models for method effect
3) Bifactor 108.641 .999 .026 .86 .025 1
(63) (.017, .034)
4) 1F + 1 fm neg 209.306 .998 .043 .957 .037 2
(70) (.036, .049)
5) 1F + 1 fm pos 307.677 .996 .056 1.447 .045 3
(70) (.49, .062)
6) 1F + Frandom 3929.828 .935 .226 5.17 .156 5
(39) (.220, .232)
WLSMV-χ2 CFI RMSEA WRMR SRMR Ranking best fit
(gl)
PSS – 10
Substantive models
1F 3879.1000 .843 .317 6.755 .210 6
(.309, 326)
2F 204.191 .993 .068 1.55 .051 4
(34) (.059, .077)
Models for method effect
Bifactor 26.973 1.000 .009 .563 .018 1
(25) (.00, .026)
1F + 1 fm neg 70.989 .998 .034 .914 .028 2
(31) (.024, .045)
1F + 1 fm pos 187.707 .994 .071 1.486 .049 3
(29) (.061, .081)
1F + Frandom 2565.356 .897 .283 5.494 .171 5
(29) (.274, .292)

Note. Models: unidimensional (1F), bifactor (2F), unidimensional with method factor - negative items (1F + 1fm neg), positive item method (1F + 1fm
pos) and random (1F + 1Frandom)

interpreted these same results (Michaelides, Christodoulou,
Kkeli, Karekla, & Panayiotou, 2016; Perera et al., 2017). It
is worth pointing out that in a specific context of measurement
where the construct is oriented towards the maladaptive expe-
rience (i.e., perceived stress), the items worded in a
countersense lose coherence with the conceptual and opera-
tional definition of the original construct, which can be
troublesome.
Interestingly, the dimension of the negatively worded items
has received names such as “perception of control”, “coping
skills”, “resilience”, “perceived self-efficacy”, among others.
All these constructs are assumed as the automatic theoretical
opposites of perceived stress (and therefore re-coded), adopting
a view of two poles of a continuum in the stress process. This
contrasts with the theoretical literature that tends to interpret
these factors as completely independent constructs, even
though they are, to some extent, negatively related to stress
(Orkaizagirre-Gomara, Sánchez, Ortiz & Ortiz, 2020).
Methodologically, the results of this study highlight the
fact that one of the factors traditionally interpreted as a sub-
stantive dimension of stress in the PSS, confirmed herein as a
method factor. Furthermore, our findings call into question the
irrelevance that the authors of the scale have conferred to the
separation of two factors (Cohen, & Janicki-Deverts, 2012;
Cohen, Kamarak, & Mermelstein, 1983; Cohen, Janicki-
Deverts, & Miller, 2007; Cohen, & Williamson, 1988). One
consequence of that is the limited validity of the interpretation
when using a single score that incorporates irrelevant variance
of the construct. It should be noted that the total variance and
factor loadings of the negatively worded items were higher,
thus highlighting the positive experience of respondents in-
stead of a perception of maladaptation, as is expected in any
stress measure.
Our strategy confirms that the implementation of the
bifactor model served as a method of separation of two
sources of variance. One due to the method (i.e., caused by

Content courtesy of Springer Nature, terms of use apply. Rights reserved.

 Curr Psychol

Table 4 Parameters of internal structure for the items of the PSS-14 and PSS-10 (n = 1093)

PSS-14 PSS-10

1F Bifactor 2F 1F + 1 fm neg 1F Bifactor 2F 1F + 1 fm neg

Fg Fg Fpo Fne RPB I-ECV Fpo Fne Fg Fne Fg Fg Fpo Fne RPB I-ECV Fpo Fne Fg Fne

PSS1 .466 .595 .376 .217 .714 .701 .701 .586 .718 .039 .184 .997 .703 .702
PSS2 .527 .749 .415 .297 .765 .838 .837 .661 .901 −.017 .266 1.000 .840 .840
PSS3 .444 .466 .586 .046 .387 .730 .732 .573 .685 .201 .164 .921 .720 .722
PSS8 .413 .47 .511 .122 .458 .692 .691 .535 .592 .482 .096 .601 .690 .690
PSS11 .433 .452 .53 .043 .421 .690 .688 .551 .619 .294 .110 .816 .682 .681
PSS12 .188 .06 .638 2.145 .008 .442 .444 – – – – – – –
PSS14 .428 .489 .474 .124 .515 .683 .683 .545 .603 .380 .096 .716 .682 .682
PSS4r .827 .189 .838 3.374 .048 .850 .137 .845 – – – – – – –
PSS5r .891 .236 .878 2.77 .067 .906 .171 .892 – – – – – – –
PSS6r .904 .292 .876 2.096 .100 .923 .209 .899 .845 .229 .893 2.690 .062 .915 .220 .895
PSS7r .832 .333 .79 1.5 .150 .857 .243 .821 .800 .258 .834 2.101 .087 .872 .249 .836
PSS9r .879 .281 .851 2.13 .098 .896 .199 .874 .835 .221 .866 2.778 .061 .890 .214 .869
PSS10r .565 .442 .441 .277 .501 .579 .331 .495 .531 .346 .480 .535 .342 .595 .335 .484
PSS13r .662 .245 .656 1.7 .122 .700 .162 .682 – – – – – – – –
Fpo – – – – – 1.0 – – – – – – – – 1.0 – –
Fne – – – – – .245 1.0 – – – – – – – – .293 1.0 – –
ECV .286 .504 .871 – – .450 .931 .516 .149 .896 – – .577 .903
ω .897 .925 .873 .937 – – .921 .936 .908 .879 .895 – – .902 .895
ωh – .377 .468 .819 – – .517 .875 .638 .089 .801 – – .676 .807
ARPB – 1.203 – – – – 1.790 .902 – – .974
H .950 .785 .723 .934 – – .885 .944 .893 .379 .905 – – .881 .907
FD .825 .784 .959 – – .939 .973 .938 .664 .957 – – .937 .958

Note. Fg: general factor. Fpo: factor for positive items. Fne: factor for negative items (required for recodification). RPB: relative bias of the parameter. I-
ECV: explained common variance for the item. ECV: explained common variance. ω and ωh: omega reliability ratio. H: replicability ratio. FD: factor
determinacy. Models: 1F (unidimensional), Bifactor, 2F (two dimensions), 1F + Fm neg (unidimensional with method factor –negative items). Only the
most relevant 4 models are shown

the wording of the items), and the other within the unidimen-
sional interpretation of the PSS, represented by the general
factor (Fg). Although this approximation is similar to that of
previous studies where the bifactor technique has been used in
the PSS, the interpretation should not necessarily be oriented
towards the theoretical substantivity of its contents. A unidi-
mensional measure of these contents can also represent vari-
ance when there are items worded in both senses (i.e., positive
and negative). Literature about the effect of item wording in an
opposite sense to the construct has indicated that this problem
is rather usual (Conway et al., 2004; Corwyn, 2000;
Dominguez-Lara, & Merino-Soto, 2018; Horan et al., 2003;
Lindwall et al., 2012; Molina et al., 2014; Suárez-Alvarez,
et al., 2018: Tomás & Oliver, 1999; Tomás et al., 2013;
Tomás et al., 2012).
Although the negative wording is used to reduce acquies-
cence bias, the detected variance for this method factor was
relevant beyond the simple false addition of factors. In
general, a fit of any model may improve because of the addi-
tion of parameters, but by using the procedure suggested by
Schönberger & Ponsford (2010) in this work, the replicable
effect of the method factor founded in PSS was indeed con-
sistently superior. This effect seems to be more important than
the effect of any possible substantive or relevant factor. Other
internationally well-known measures have also shown a rep-
licable effect of the method, such as the Rosenberg Self-
Esteem Scale (Lima, & Souza, 2019) or the General Health
Questionnaire – 12 (Molina, Rodrigo, Losilla, & Vives,
2014).
There are practical implications in the assessment of the
PSS. All of them are related to the vulnerable validity of its
unidimensional approach. For instance, when cut-offs are sug-
gested for normative use (Cohen & Williamson, 1988;
González Rodríguez & Hernández, 2013), the consideration
of variance that does not belong to the construct generates
biased assessments, especially if the levels of reliability within

Content courtesy of Springer Nature, terms of use apply. Rights reserved.

 Curr Psychol

Table 5 Measurement invariance for PSS-14 and PSS-10 (n = 1093)

Sex group Occupational status group

WLSMV - χ2 CFI ΔCFI RMSEA ΔRMSEA SRMR ΔSRMR WLSMV - χ2 CFI ΔCFI RMSEA ΔRMSEA SRMR ΔSRMR
(gl) (90% IC) (gl) (90% IC)

PSS – 14
Configuration 268.90** .998 – .041 – .049 – 313.73 .998 – .048 – .047 –
(140) (.034, .048) (140) (.041, .055)
Threshold 291.60** .998 0.00 .037 0.00 .040 0.01 369.85 .997 0.00 .047 0.00 .047 0.00
(168) (.030, .044) (198) (.40, .053)
Loadings + threshold 564.40** .994 0.00 .061 −0.02 .052 −0.01 460.73 .996 0.00 .052 0.01 .050 0.00
(187) (.055, .067) (187) (.046, .058)
Intercept 891.26** .989 −0.01 .080 −0.02 .051 0.00 911.54 .990 −0.01 .081 0.03 .051 0.00
(200) (.074, .085) (200) (.075, .086)
PSS – 10
Configuration 95.24** .999 – .031 – .031 – 114.79 .998 – .040 – .037 –
(62) (.018, .043) (62) (.028, .051)
Threshold 116.46** .999 .00 .028 .00 .031 .00 158.52 .997 .00 .041 −.00 .037 .00
(82) (.015, .039) (82) (.032, .051)
Loading + threshold 297.82** .992 .00 .063 −.03 .047 −.01 215.01 .996 .00 .049 −.00 .043 −.00
(94) (.055, .071) (94) (.040, .057)
Intercept 497.06** .984 .00 .084 −.02 .045 .00 472.92 .987 .00 .081 −.03 .044 −.00
(103) (.076, .091) (103) (.074, .089)

**p < .01

Content courtesy of Springer Nature, terms of use apply. Rights reserved.


the heterogeneity of the factors of the PSS scale are not con-
sidered (Reis et al., 2019).
In conclusion, a two-dimension format of both Spanish
versions of the PSS instrument (10- and 14-item) does not
show bias across sexes or occupational status. However, this
Spanish version of the PSS scale fits better to a bifactor model
as has happened with the English (Wu & Amtmann, 2013),
Serbian (Jovanovic et al., 2015), Greek (Michaelides, 2016),
German (Reis et al., 2019), Korean (Park and Colvin, 2019)
versions, and even in other Latin samples (Reyna et al., 2019).
Unlike what these previous studies suggest about the preferred
use of a total score, a more careful assessment of the uni- or
multi-dimensional interpretation seems necessary, which
should be psychometrically justified (Reis et al., 2019).
Based on the method effects found, we suggest performing
more research to determine the advantages of using only the
contents of the positively worded items (towards the experi-
ence of stress). This approach is needed to avoid the method
effect of the negative items and will be closer to the theoretical
and operational framework of stress, as it happens with other
more recent and promising self-reported stress instruments
(Amirkhan, 2018).
If the PSS is used for research, then we suggest applying an
adjustment for the negative wording (in the opposite sense to
the experience of stress). Nevertheless, the interpretation of
the general score of the PSS seems weakened by the need of
modeling the method effect at calculating the scores, and thus
failing to obtain a valid measure. Although this would be a
reasonable analytical option for the investigator, we can con-
fidently say that it would be an important challenge for the
non-research practitioner, with insufficient training in multi-
variate analysis skills needed in the use of specialized soft-
ware. This challenge will also occur in the context of small
sample sizes, where the applicability of structural equation
models is not reliable.
One limitation of the study is the absence of a homoge-
neous and representative populated-based sample.
Nevertheless, the sample size relative to the magnitude of
the expected effects was reasonable. Given the time con-
straints of participants and organizations, we decided to apply
a short questionnaire, so another limitation was the lack of
external criteria that would allow assessing the convergent
and divergent validity of the factors and items of the PSS.
However, this study was firstly intended to contribute to the
unsolved problems of the scale’s dimensional structure, so we
can use objective criteria (e.g biomarkers) in future studies
with smaller samples. Concerning the items, the non-
normality of their distribution was notable, yet not unusual
in the investigation of psychosocial constructs (Bono,
Blanca, Arnau, & Gómez-Benito, 2017; Micceri, 1989), espe-
cially in categorical variables. This was not a problem that
limited our conclusions since the application of the WLSMV
and the polychoric correlations attenuated its possible effect.
Content courtesy of Springer Nature, terms of use apply. Rights reserved.
Curr Psychol
Authors’ Contributions AJG and CMS conceived the idea and design the
study, performed analyses, and drafted the manuscript. FBO, MNG, and
AMC were in charge of the implementation of the research logistics,
contributed and make suggestions to the design and the analysis of the
results, and reviewed the manuscript.
Funding No funds, grants, or other support was received for this study.
Data Availability Database is available in this repository: https://osf.io/
m4uzt/?fbclid=IwAR3L9oF6BsPxcqJ6uP_r5w5xYyvuKpwIr-
_TNiga91-m2sY1a3-NB638ZO0
Code Availability N/A
Declarations
Ethics approval The project was approved by the ethics committee of
the Health Sciences School at the Universidad Autónoma de Baja
California, with registration number F-PROMEP-38/Rev-04 and the
ethics committee of the Center for Transdisciplinary Psychology
Research at the Universidad Autónoma del Estado de Morelos
(2019/No.CEI-210219-12).
Consent to participate Informed consent was obtained from all individ-
ual participants included in the study.
Consent for publication Any kind of individual information of partici-
pants was included in the manuscript. Participants agreed and were in-
formed that the global report of the study would have academic purposes
and would be submitted in a scientific journal.
Conflict of interests The authors have no relevant financial or non-
financial interests to disclose. On behalf of all authors, the corresponding
author states that there is no conflict of interest.
References
Almadi, T., Cathers, I., Mansour, A. M. H., & Chow, C. M. (2012). An
Arabic version of the perceived stress scale: Translation and valida-
tion study. International Journal of Nursing Studies, 49(1), 84–89.
https://doi.org/10.1037/t25566-000.
Amirkhan, J. H. (2018). A brief stress diagnostic tool: The short stress
overload scale. Assessment, 25(8), 1001–1013. https://doi.org/10.
1177/1073191116673173.
Bono, R., Blanca, M. J., Arnau, J., & Gómez-Benito, J. (2017). Non-
normal distributions commonly used in health, education, and social
sciences: A systematic review. Frontiers in Psychology, 8, 1602.
https://doi.org/10.3389/fpsyg.2017.01602.
Brown, T. (2015). Confirmatory factor analysis for applied research
(2nd ed.). Guilford Press.
Byrd, R. H., Lu, P., Nocedal, J., & Zhu, C. (1995). A limited memory
algorithm for bound constrained optimization. SIAM Journal
Scientific Computing, 16(5), 1190–1208. https://doi.org/10.1137/
0916069.
Chen, F. F. (2007). Sensitivity of goodness of fit indexes to lack of
measurement invariance. Structural Equation Modeling, 14, 464–
504. https://doi.org/10.1080/10705510701301834.
Chiu, Y. H., Lu, F. J. H., Lin, J. H., Nien, C. L., Hsu, Y. W., & Liu, H. Y.
(2016). Psychometric properties of the perceived stress scale (PSS):
Measurement invariance between athletes and non-athletes and
Curr Psychol
construct validity. PeerJ, 4, e2790. https://doi.org/10.7287/peerj.
preprints.2268v1.
Cohen, S., & Janicki-Deverts, D. (2012). Who ' s stressed? Distributions
of psychological stress in the United States in probability samples
from 1983, 2006, and 2009. Journal of Applied Social Psychology,
42(6), 1320–1334. https://doi.org/10.1111/j.1559-1816.2012.
00900.x.
Cohen, S., & Williamson, G. M. (1988). Perceived stress in a probability
sample of the United States. In S. Spacapan & S. Oskamp (Eds.),
The Claremont Symposium on Applied Social Psychology. The so-
cial psychology of health (p. 31–67). Sage Publications, Inc..
Cohen, S., Janicki-Deverts, D., & Miller, G. E. (2007). Psychological
stress and disease. Journal of the American Medical Association,
298(14), 1685–1687. https://doi.org/10.1001/jama.298.14.1685.
Cohen, S., Kamarak, T., & Mermelstein, R. (1983). A global measure of
perceived stress. Journal of Health and Social Behavior, 24(4),
385–396.
Cohen, S., Tyrrell, D. A., & Smith, A. P. (1993). Negative life events,
perceived stress, negative affect, and susceptibility to the common
cold. Journal of Personality and Social Psychology, 64(1), 131–
140. https://doi.org/10.1037/0022-3514.64.1.131.
Conway, J. M., Lievens, F., Scullen, S. E., & Lance, C. E. (2004). Bias in
the correlated uniqueness model for MTMM data. Structural
Equation Modeling, 11, 535–559. https://doi.org/10.1207/
s15328007sem1104_3.
Corwyn, R. F. (2000). The factor structure of global self-esteem among
adolescents and adults. Journal of Research in Personality, 34(4),
357–379. https://doi.org/10.1006/jrpe.2000.2291.
Denovan, A., Dagnall, N., Dhingra, K., & Grogan, S. (2019). Evaluating
the perceived stress scale among UK university students:
Implications for stress measurement and management. Studies in
Higher Education, 44(1), 120–133. https://doi.org/10.1080/
03075079.2017.1340445.
Dominguez-Lara, S., & Merino-Soto, C. (2018). Efectos de método en el
Inventario de Depresión Estado-Rasgo (IDER): Un análisis SEM.
[method effects in the trait-state Depresion inventory]. Avances en
Psicología Latinoamericana, 36(2), 253–267. https://doi.org/10.
12804/revistas.urosario.edu.co/apl/a.4151.
Dueber, D. (2020). BifactorIndicesCalculator: bifactor indices calculator.
R package version 0.2.0. Recuperado de: https://CRAN.R-project.
org/package=BifactorIndicesCalculator
Eid, M., Lischetzke, T., Nussbeck, F. W., & Trierweiler, L. I. (2003).
Separating trait effects from trait-specific method effects in
multitrait-multimethod models: A multiple-indicator CT-C(M-1)
model. Psychological Methods, 8, 38–60. https://doi.org/10.1037/
1082-989X.8.1.38.
Eklund, M., Bäckström, M., & Tuvesson, H. (2014). Psychometric prop-
erties and factor structure of the Swedish version of the perceived
stress scale. Nordic Journal of Psychiatry, 68(7), 494–499. https://
doi.org/10.3109/08039488.2013.877072.
Fan, Y., & Lance, C. E. (2017). A reformulated correlated trait-correlated
method model for multitrait-multimethod data effectively increases
convergence and admissibility rates. Educational and Psychological
Measurement, 77(6), 1048–1063. https://doi.org/10.1177/
0013164416677144.
Ferrando, P. J., & Lorenzo-Seva, U. (2017). Program FACTOR at 10:
Origins, development and future directions. Psicothema, 29(2),
236–241. https://doi.org/10.7334/psicothema2016.304.
Fetvadjiev, V., & Van de Vijver, F. J. R. (2015). Measures of personality
across cultures. In G. J. Boyle, D. H. Saklofske, & G. Matthews
(Eds.), Measures of personality and social psychological constructs
(pp. 752-776). Amsterdam, the Netherlands: Elsevier. https://doi.
org/10.1016/B978-0-12-386915-9.00026-7.
Fox, J. (2019). polycor: Polychoric and polyserial correlations. R package
version 0.7-10}. Available in: https://CRAN.R-project.org/
package=polycor.
Content courtesy of Springer Nature, terms of use apply. Rights reserved.
Golden-Kreutz, D. M., Browne, M. W., Frierson, G. M., & Andersen, B.
L. (2004). Assessing stress in cancer patients: A second-order factor
analysis model for the perceived stress scale. Assessment, 11(3),
216–223. https://doi.org/10.1177/1073191104267398.
Golino, H. F., & Epskamp, S. (2017). Exploratory graph analysis: A new
approach for estimating the number of dimensions in psychological
research. PLoS One, 12(6), e0174035 journal.pone.0174035.
Golino, H., & Christensen, A. P. (2020). EGAnet: Exploratory graph
analysis: A framework for estimating the number of dimensions in
multivariate data using network psychometrics. R package version
0.9.5.
González, M., & Landero, R. (2007). Factor structure of the perceived
stress scale (PSS) in a sample from Mexico. The Spanish Journal of
Psychology, 10(1), 199–206.
González, M. T., Rodríguez, M. N., & Hernández, R. L. (2013). The
perceived stress scale (PSS): Normative data and factor structure
for a large-scale sample in Mexico. The Spanish Journal of
Psychology, 16, E47. https://doi.org/10.1017/sjp.2013.35.
Gorsuch, R. L. (1983). Factor analysis. Lawrence Erlbraum Associates.
Hancock, G. R., & Mueller, R. O. (2001). Rethinking construct reliability
within latent variable systems. In R. Cudeck, S. du Toit, & D.
Sörbom (Eds.), Structural equation modeling: Present und future—
A Festschrift in honor of Karl Joreskog (pp. 195–216).
Lincolnwood, IL: Scientific Software International.
Horan, P. M., DiStefano, C., & Motl, R. W. (2003). Wording effects in
self-esteem scales: Methodological artifact or response style?
Structural Equation Modeling, 10(3), 435–455. https://doi.org/10.
1207/S15328007SEM1003_6.
Jorgensen, T. D., Pornprasertmanit, S., Schoemann, A. M., & Rosseel, Y.
(2020). semTools: Useful tools for structural equation modeling. R
package version 0.5–3. https://CRAN.R-project.org/package=
semTools
Jovanović, V., & Gavrilov-Jerković, V. (2015). More than a (negative)
feeling: Validity of the perceived stress scale in Serbian clinical and
non-clinical samples. Psihologija, 48(1), 5–18. https://doi.org/10.
2298/psi1501005j.
Kaiser, H. F. (1970). A second generation little jiffy. Psychometrika, 35,
401–415. https://doi.org/10.1007/BF02291817.
Kiecolt-Glaser, J. K., & Glaser, R. (1999). Chronic stress and mortality
among older adults. Journal of the American Medical Association,
282(23), 2259–2260. https://doi.org/10.1001/jama.282.23.2259.
King, B. M., Rosopa, P. J., & Minium, E. W. (2011). Statistical reason-
ing in the behavioral sciences. John Wiley & Sons.
Lance, C. E., Noble, C. L., & Scullen, S. E. (2002). A critique of the
correlated trait-correlated method and correlated uniqueness models
for multitrait-multimethod data. Psychological Methods, 7(2), 228–
244. https://doi.org/10.1037/1082-989x.7.2.228.
Lazarus, R. S., & Folkman, S. (1984). Stress, appraisal, and coping.
Springer publishing company.
Lee, B., & Jeong, H. I. (2019). Construct validity of the perceived stress
scale (PSS-10) in a sample of early childhood teacher candidates.
Psychiatry and Clinical Psychopharmacology, 29, 76–82. https://
doi.org/10.1080/24750573.2019.1565693.
Lee, E. H. (2012). Review of the psychometric evidence of the perceived
stress scale. Asian Nursing Research, 6(4), 121–127. https://doi.org/
10.1016/j.anr.2012.08.004.
Lesage, F. X., Berjot, S., & Deschamps, F. (2012). Psychometric proper-
ties of the French versions of the perceived stress scale.
International Journal of Occupational Medicine and
Environmental Health, 25(2), 178–184. https://doi.org/10.2478/
S13382-012-0024-8.
Li, C. H. (2016). Confirmatory factor analysis with ordinal data:
Comparing robust maximum likelihood and diagonally weighted
least squares. Behavior Research Methods, 48(3), 936–949. https://
doi.org/10.3758/s13428-015-0619-7.

Lima, T., & Souza, L. (2019). Rosenberg self-esteem scale: Method effect
and gender invariance. Psico-USF, 24(3), 517–528. https://doi.org/
10.1590/1413-82712019240309.
Lindwall, M., Barkoukis, V., Grano, C., Lucidi, F., Raudsepp, L.,
Liukkonen, J., & Thøgersen-Ntoumani, C. (2012). Method effects:
The problem with negatively versus positively keyed items. Journal
of Personality Assessment, 94, 196–204. https://doi.org/10.1080/
00223891.2011.64593.
Lloret-Segura, S., Ferreres-Traver, A., Hernández-Baeza, A., & Tomás-
Marco, I. (2014). El análisis factorial exploratorio de los ítems: Una
guía práctica, revisada y actualizada [exploratory item factor analy-
sis: A practical guide revised and up-dated]. Anales de Psicología/
Annals of Psychology, 30(3), 1151–1169. https://doi.org/10.6018/
analesps.30.3.199361.
Mahalanobis, P. C. (1936). On the generalised distance in statistics.
Sankhya A, 80, 1–7. https://doi.org/10.1007/s13171-019-00164-5.
Mangiafico, S. (2020). rcompanion: Functions to support extension edu-
cation program evaluation. R package version 2.3.25. Available in:
https://CRAN.R-project.org/package=rcompanion
Markon, K. E. (2019). Bifactor and hierarchical models: Specification,
inference, and interpretation. Annual Review of Clinical Psychology,
15(1), 51–69. https://doi.org/10.1146/annurev-clinpsy-050718-
095522.
Marsh, H. W., & Bailey, M. (1991). Confirmatory factor analyses of
multitrait-multimethod data: A comparison of alternative models.
Applied Psychological Measurement, 15, 47–70. https://doi.org/10.
1177/014662169101500106.
Marsh, H. W, & Grayson, D. (1995). Latent variable models of
multitrait–multimethod data. In: R. H. Hoyle (Ed.), Structural equa-
tion modeling: Concept, issues, and applications (pp. 177–198).
Sage.
Marsh, H. W. (1989). Confirmatory factor analysis of multitrait-
multimethod data: Many problems Anda few solutions. Applied
Psychological Measurement, 13, 335–361. https://doi.org/10.1177/
014662168901300402.
McDonald, R. P. (1999). Test theory: A unified treatment. : Lawrence
Erlbaum Associates, Inc.
Meade, A. W., & Craig, S. B. (2012). Identifying careless responses in
survey data. Psychological Methods, 17(3), 437–455. https://doi.
org/10.1037/a0028085.
Messing, K., Punnett, L., Bond, M., Alexanderson, K., Pyle, J., Zahm, S.,
Wegman, D., Stock, S. R., & de Grosbois, S. (2003). Be the fairest
of them all: Challenges and recommendations for the treatment of
gender in occupational health research. American Journal of
Industrial Medicine, 43(6), 618–629. https://doi.org/10.1002/ajim.
10225.
Micceri, T. (1989). The unicorn, the normal curve, and other improbable
creatures. Psychological Bulletin, 105, 156–166. https://doi.org/10.
1037/0033-2909.105.1.156.
Michaelides, M. P., Christodoulou, A., Kkeli, N., Karekla, M., &
Panayiotou, G. (2016). Factorial structure of the perceived stress
scale and implications for scoring. Revue Européenne de
Psychologie Appliquée, 66(6), 309–316. https://doi.org/10.1016/j.
erap.2016.07.002.
Mimura, C., & Griffiths, P. (2004). A Japanese version of the perceived
stress scale: Translation and preliminary test. International Journal
of Nursing Studies, 41(4), 379–385. https://doi.org/10.1016/j.
ijnurstu.2003.10.009.
Molina, J. G., Rodrigo, M. F., Losilla, J. M., & Vives, J. (2014). Wording
effects and the factor structure of the 12-item general health ques-
tionnaire (GHQ-12). Psychological Assessment, 26, 1031–1037.
https://doi.org/10.1037/a0036472.
Muthén, B.O, du Toit, S., & Spisic, D. (1997). Robust inference using
weighted least squares and quadratic estimating equations in latent
variable modeling with categorical and continuous outcomes.
Content courtesy of Springer Nature, terms of use apply. Rights reserved.
Curr Psychol
Unpublished manuscript. Recuperado de: https://www.statmodel.
com/download/Article_075.pdf
Nye, C. D., & Drasgow, F. (2011). Effect size indices for analyses of
measurement equivalence: Understanding the practical importance
of differences between groups. Journal of Applied Psychology, 96,
966–980. https://doi.org/10.1037/a0022955.
Nye, C. D., Bradburn, J., Olenick, J., Bialko, C., & Drasgow, F. (2019).
How big are my effects? Examining the magnitude of effect sizes in
studies of measurement equivalence. Organizational Research
Methods, 22(3), 678–709. https://doi.org/10.1177/
1094428118761122.
Orkaizagirre-Gómara, A., Sánchez De Miguel, M., Ortiz de Elguea, J., &
Ortiz de Elguea, A. (2020). Testing general self-efficacy, perceived
competence, resilience, and stress among nursing students: An inte-
grator evaluation. Nursing & Health Sciences.(22) 3, 529–538.
https://doi.org/10.1111/nhs.12689.
Park, H. S., Dailey, R., & Lemus, D. (2002). The use of exploratory factor
analysis and principal components analysis in communication re-
search. eítems: Una guía práctica, revisada y actualizada. Anales
de Psicología, 30(3), 1151–1169. https://doi.org/10.6018/analesps.
30.3.199361.
Park, S. Y., & Colvin, K. F. (2019). Psychometric properties of a Korean
version of the perceived stress scale (PSS) in a military sample.
BMC Psychology, 7(58), 58. https://doi.org/10.1186/s40359-019-
0334-8.
Penny, K. I. (1996). Appropriate critical values when testing for a single
multivariate outlier by using the Mahalanobis distance. Applied
Statistics, 45(1), 73–81. https://doi.org/10.2307/2986224.
Perera, M. J., Brintz, C. E., Birnbaum-Weitzman, O., Penedo, F. J., Gallo,
L. C., Gonzalez, P., Gouskova, N., Isasi, C. R., Navas-Nacher, E. L.,
Perreira, K. M., Roesch, S. C., Schneiderman, N., & Llabre, M. M.
(2017). Factor structure of the perceived stress Scale-10 (PSS)
across English and Spanish language responders in the HCHS/
SOL sociocultural ancillary study. Psychological Assessment,
29(3), 320–328. https://doi.org/10.1037/pas0000336.
Quinn, H. O. C. (2014). Bifactor models, explained common variance
(ECV), and the usefulness of scores from unidimensional item re-
sponse theory analyses. [doctoral dissertation, the University of
North Carolina at Chapel Hill] https://doi.org/10.17615/t6ff-a088
Reis, D., Lehr, D., Heber, E., & Ebert, D. D. (2019). The German version
of the perceived stress scale (PSS-10): Evaluation of dimensionality,
validity, and measurement invariance with exploratory and confir-
matory bifactor modeling. Assessment, 26(7), 1246–1259. https://
doi.org/10.1177/1073191117715731.
Reise, S. P., Moore, T. M., & Haviland, M. G. (2010). Bifactor models
and rotations: Exploring the extent to which multidimensional data
yield univocal scale scores. Journal of Personality Assessment,
92(6), 544–559. https://doi.org/10.1080/00223891.2010.496477.
Reise, S. P., Morizot, J., & Hays, R. D. (2007). The role of the bifactor
model in resolving dimensionality issues in health outcomes mea-
sures. Quality of Life Research, 16(1), 19–31. https://doi.org/10.
1007/s11136-007-9183-7.
Remor, E. (2006). Psychometric properties of a European Spanish ver-
sion of the perceived stress scale (PSS). The Spanish Journal of
Psychology, 9(1), 86–93.
Remor, E., & Carrobles, J. A. (2001). Versión Española de la Escala de
Estrés Percibido (PSS-14): Estudio psicométrico en una muestra
VIH+.[Spanish versión of the perceived stress scale (PSS-14): A
psychometric study in a VIH+ sample] Ansiedad y estrés. 7(2–3),
195–201. https://psycnet.apa.org/record/2002-15104-002
Reyna, C., Mola, D., & Correa, P. (2019). Escala de Estrés Percibido:
análisis psicométrico desde la TCT y la TRI. [Perceived Stress
Scale: Psychometric analysis from the TCT and TRI]. Ansiedad y
Estrés, 25(2), 138–147. https://doi.org/10.1016/j.anyes.2019.04.
003.
Curr Psychol
Rodriguez, A., Reise, S. P., & Haviland, M. G. (2016a). Evaluating
bifactor models: Calculating and interpreting statistical indices.
Psychological Methods, 21(2), 137–150. https://doi.org/10.1037/
met0000045.
Rodriguez, A., Reise, S. P., & Haviland, M. G. (2016b). Applying
bifactor statistical indices in the evaluation of psychological mea-
sures. Journal of Personality Assessment, 98(3), 223–237. https://
doi.org/10.1080/00223891.2015.1089249.
Rosseel, Y. (2012). Lavaan: An R package for structural equation model-
ing. Journal of Statistical Software, 48(2), 1–36. https://doi.org/10.
18637/jss.v048.i02.
Safaei, M., & Shokri, O. (2014). Assessing stress in cancer patients:
Factorial validity of the perceived stress scale in Iran. Iranian
Journal of Psychiatric Nursing, 2(1), 13–22 https://www.sid.ir/en/
journal/ViewPaper.aspx?id=372609.
Sapolsky, R. M. (2004). Why zebras don ' t get ulcers: The acclaimed
guide to stress, stress-related diseases, and coping-now revised and
updated. Henry Holt paperbacks adn Company.
Sass, D. A. (2011). Testing measurement invariance and comparing latent
factor means within a confirmatory factor analysis framework.
Journal of Psychoeducational Assessment, 29, 347–363. https://
doi.org/10.1177/0734282911406661.
Schönberger, M., & Ponsford, J. (2010). The factor structure of the hos-
pital anxiety and depression scale in individuals with traumatic brain
injury. Psychiatry Research, 179(3), 342–349 https://doi.org/10.
1016/j.psychres.2009.07.003.
Selye, H. (1976). The stress concept. Canadian Medical Association
Journal, 115(8), 718.
Siqueira Reis, R., Ferreira Hino, A. A., & Romélio Rodriguez Añez, C.
(2010). Perceived stress scale: Reliability and validity study in
Brazil. Journal of Health Psychology, 15(1), 107–114. https://doi.
org/10.1177/1359105309346343.
Smith, K. J., Rosenberg, D. L., & Haight, T. (2014). An assessment of the
psychometric properties of the perceived stress Scale-10 (PSS 10)
with business and accounting students. Accounting Perspectives,
13(1), 29–59. https://doi.org/10.1016/j.adiac.2014.09.005.
Stucky, B. D., Edelen, M. O., Vaughan, C. A., Tucker, J. S., & Butler, J.
(2014). The psychometric development and initial validation of the
DCI-A short form for adolescent therapeutic community treatment
process. Journal of Substance Abuse Treatment, 46(4), 516–521.
https://doi.org/10.1016/j.jsat.2013.12.005.
Suárez-Alvarez, J., Pedrosa, I., Lozano, L. M., García-Cueto, E., Cuesta,
M., & Muñiz, J. (2018). Using reversed items in Likert scales: A
questionable practice. Psicothema, 30(2), 149–158. https://doi.org/
10.7334/psicothema2018.33.
Content courtesy of Springer Nature, terms of use apply. Rights reserved.
Timmerman, M. E., & Lorenzo-Seva, U. (2011). Dimensionality assess-
ment of ordered polytomous items with parallel analysis.
Psychological Methods, 16, 209–220. https://doi.org/10.1037/
a0023353.
Tomás, J., & Oliver, A. (1999). Rosenberg’s self-esteem scale: Two
factor or method effects. Structural Equation Modeling: A
Multidisciplinary Journal, 6(1), 84–98. https://doi.org/10.1080/
10705519909540120.
Tomás, J., Galiana, L., Hontangas, P., Oliver, A., & Sancho, P. (2013).
Evidencia acumulada sobre los efectos de método asociados a ítems
invertidos. [Acumulative evidence about the method effects by
inverted ítems]. Psicológica, 34, 365–381 https://www.redalyc.
org/pdf/169/16929535013.pdf.
Tomás, J., Sancho, P., Oliver, A., Galiana, L., & Meléndez, J. (2012).
Efectos de método asociados a ítems invertidos vs. ítems en
negativo.[ method effects asocciated to inverted ítems vs negative
ítems]. Revista Mexicana de Psicología, 29(2), 105–115 https://
www.redalyc.org/pdf/2430/243030190001.pdf.
Trizano-Hermosilla, I., & Alvarado, J. M. (2016). Best alternatives reli-
ability in realistic conditions: Congeneric and asymmetrical mea-
surements. Frontiers in Psychology, 7, 769. https://doi.org/10.
3389/fpsyg.2016.00769.
Wiriyakijja, P, Porter, S, Fedele, S, Hodgson, T., McMillan, R.,
Shephard, R., & Riordain, R. N. (2019). Validation of the HADS
and PSS-10 and psychological status in patients with oral lichen
planus. Oral Diseases, 1, 1–15. https://doi.org/10.1111/odi.13220.
Wu, H., & Estabrook, R. (2016). Identification of confirmatory factor
analysis models of different levels of invariance for ordered categor-
ical outcomes. Psychometrika, 81, 1014–1045. https://doi.org/10.
1007/s11336-016-9506-0.
Wu, S. M., & Amtmann, D. (2013). Psychometric evaluation of the
perceived stress scale in multiple sclerosis. International Scholarly
Research Notices., 2013, 1–9. https://doi.org/10.1155/2013/608356.
Yentes, R. D., & Wilhelm, F. (2018). careless: Procedures for computing
indices of careless responding. R package version 1.1.3.
Yu, C. (2002). Evaluating cutoff criteria of model fit indices for latent
variable models with binary and continuous outcomes (unpublished
doctoral dissertation). University of California, Los Angeles, CA.
Available in: http://citeseerx.ist.psu.edu/viewdoc/download;
jsessionid=890BA372486968AC957CF29C765395BA?doi=10.1.
1.310.3956&rep=rep1&type=pdf
Publisher’s Note Springer Nature remains neutral with regard to jurisdic-
tional claims in published maps and institutional affiliations.
Terms and Conditions
Springer Nature journal content, brought to you courtesy of Springer Nature Customer Service Center GmbH (“Springer Nature”).
Springer Nature supports a reasonable amount of sharing of research papers by authors, subscribers and authorised users (“Users”), for small-
scale personal, non-commercial use provided that all copyright, trade and service marks and other proprietary notices are maintained. By
accessing, sharing, receiving or otherwise using the Springer Nature journal content you agree to these terms of use (“Terms”). For these
purposes, Springer Nature considers academic use (by researchers and students) to be non-commercial.
These Terms are supplementary and will apply in addition to any applicable website terms and conditions, a relevant site licence or a personal
subscription. These Terms will prevail over any conflict or ambiguity with regards to the relevant terms, a site licence or a personal subscription
(to the extent of the conflict or ambiguity only). For Creative Commons-licensed articles, the terms of the Creative Commons license used will
apply.
We collect and use personal data to provide access to the Springer Nature journal content. We may also use these personal data internally within
ResearchGate and Springer Nature and as agreed share it, in an anonymised way, for purposes of tracking, analysis and reporting. We will not
otherwise disclose your personal data outside the ResearchGate or the Springer Nature group of companies unless we have your permission as
detailed in the Privacy Policy.
While Users may use the Springer Nature journal content for small scale, personal non-commercial use, it is important to note that Users may
not:

1. use such content for the purpose of providing other users with access on a regular or large scale basis or as a means to circumvent access
control;
2. use such content where to do so would be considered a criminal or statutory offence in any jurisdiction, or gives rise to civil liability, or is
otherwise unlawful;
3. falsely or misleadingly imply or suggest endorsement, approval , sponsorship, or association unless explicitly agreed to by Springer Nature in
writing;
4. use bots or other automated methods to access the content or redirect messages
5. override any security feature or exclusionary protocol; or
6. share the content in order to create substitute for Springer Nature products or services or a systematic database of Springer Nature journal
content.
In line with the restriction against commercial use, Springer Nature does not permit the creation of a product or service that creates revenue,
royalties, rent or income from our content or its inclusion as part of a paid for service or for other commercial gain. Springer Nature journal
content cannot be used for inter-library loans and librarians may not upload Springer Nature journal content on a large scale into their, or any
other, institutional repository.
These terms of use are reviewed regularly and may be amended at any time. Springer Nature is not obligated to publish any information or
content on this website and may remove it or features or functionality at our sole discretion, at any time with or without notice. Springer Nature
may revoke this licence to you at any time and remove access to any copies of the Springer Nature journal content which have been saved.
To the fullest extent permitted by law, Springer Nature makes no warranties, representations or guarantees to Users, either express or implied
with respect to the Springer nature journal content and all parties disclaim and waive any implied warranties or warranties imposed by law,
including merchantability or fitness for any particular purpose.
Please note that these rights do not automatically extend to content, data or other material published by Springer Nature that may be licensed
from third parties.
If you would like to use or distribute our Springer Nature journal content to a wider audience or on a regular basis or in any other manner not
expressly permitted by these Terms, please contact Springer Nature at

onlineservice@springernature.com

View publication stats