Psychological Assessment © 2013 American Psychological Association

2013, Vol. 25. No. 4, 1137-1145 1040-3590/13/$12.00 DOI: 10.1037/a0033323

The Impact of Acquiescence on the Evaluation of Personahty Structure

Beatrice Rammstedt Richard F. Farmer

GESIS-Leibniz Institute for the Social Sciences, Mannheim, Oregon Research Institute, Eugene, Oregon
Germany

Acquiescence, or the tendency to respond to descriptions of conceptually distinct personality attributes with
agreement/affirmation (acceptance acquiescence) or disagreement/opposition (counter-acquiescence), has
been widely recognized as a source of bias that can substantially alter interitem correlations within scales.
Acquiescence is also known to operate differently among some groups of persons; it is, for example, more
pronounced among individuals with less formal education. Consequently, the biasing effects of acquiescence
are of particular concern when the dimensionality underlying the item set of a measure is examined with
representative samples comprised of persons with varying levels of educational attainment and evaluated with
correlation-based statistical methods such as factor analysis. In the present study, we extended our earlier
research by investigating the biasing effect of acquiescence on personality factor structures derived from the
full-scale version ofthe Big Five Inventory (BFI) when administered to a large sample {N = 1,427) selected
to be representative of Germany's adult population. Consistent with previous findings based on a short-scale
version of the BEI, factor analyses of the unadjusted BEI item set failed to replicate the expected Big
Five-factor structure in the low/medium and high educational groups, with distortions in factor structure more
pronounced in the former group. Once acquiescence was controlled in the item responses for both groups,
however, the obtained factor structures were consistent with the Big Five framework. The implications of
acquiescence on the evaluation of the factor structure of personality inventories and for the validity of
personality assessments are discussed.

Keywords: personality assessment. Big Five, factor structure, acquiescence, education

Responses to personality inventory items have historically been
regarded as a joint function of construct-related item content and
non-content-related response sets or styles, including social desir-
ability, impression management, faking good or bad, and acquies-
cence. Response sets and styles are generally considered problem-
atic, as they constitute a source of noncontent variance and
correlated measurement errors. When responses to construct-
related content are confounded with response sets or styles, the
validity of personality assessment can be diminished or substan-
tially undentiined.
Acquiescence, or the tendency to respond to descriptions of
conceptually distinct personality attributes with agreement/affir-
mation (acceptance acquiescence) or disagreement/opposition
(counter-acquiescence), is an example of a response style whose
relevance to personality assessment has been passionately debated
historically. Accumulating data, however, are beginning to con-
This article was published Online First July 1, 2013.
Beatrice Rammstedt, GESIS-Leibniz Institute for the Social Sciences,
Mannheim, Germany; Richard F. Farmer, Oregon Research Institute, Eu-
gene, Oregon.
The authors thank Ingwer Borg for assistance with some of the data
analyses reported in this article and Lewis R. Goldberg for his comments
on an early draft of this report.
Correspondence concerning this article should be addressed to Beatrice
Rammstedt, GESIS-Leibniz Institute for the Social Sciences, P.O. Box 12
21 55, Mannheim D-68072, Germany. E-mail: beatrice.rarrunstedt@
gesis.org
1137
verge on the observation that even modest levels of acquiescent
responding can have substantial effects on item covariations and
factor structures (McCrae, Herbst, & Costa, 2C101; Rammstedt,
Goldberg, & Borg, 2010; Rammstedt cfe Kemper, 2011; Rammst-
edt, Kemper, & Borg, 2013; Soto, John, Gosling, & Potter, 2008;
Ware, 1978; Winkler, Kanouse, & Ware, 1982).
Historical Development of the Acquiescence Response
Style Concept
Within the fields of psychological and educational measure-
ment, definitions of acquiescence have shifted over time, as have
perspectives on the importance of acquiescence on the assessment
process. Cronbach's (1946, 1950) scholarly reviews are often
credited as important catalysts for subsequent research investiga-
tions into the potential biasing influence of response sets on item
responding and test scores. Cronbach (1946, p. 476) deñned a
response set as "any tendency causing a person consistently to give
different responses to test items than he [or she] would when the
same content is presented in different form." Acquiescence, re-
garded by Cronbach as a specific type of response set, was char-
acterized as the general tendency to agree with items by selecting
"yes" or "true" response options. His detailed analyses focused
attention on the possibility that factors other than item content
substantially influence responding, that individual differences in
response sets such as the tendency to agree or guess affirmatively
(i.e., acquiescence in his framework) were substantial and reliable,
and that biases associated with response sets can limit the validity
of a test if not statistically controlled.
1138 RAMMSTEDT AND FARMER
In their subsequent review of research on response sets, Jackson
and Messick (1958) distinguished between the content of items
and the style of responding. Whereas content referred to the topic
addressed by the item, style referred to one's characteristic mode
or style of expression. In making this discrimination, Jackson and
Messick also distinguished response style from response set, with
the former referring to potentially important modes of responding
and the latter denoting potential sources of measurement error that
should be minimized or eliminated. Acquiescence was regarded as
a response style and defined by Jackson and Messick in bipolar
terms as the tendency to either agree/endorse or disagree/oppose
items regardless of item content. Controversially, they also as-
serted that most of the major factor analytically derived dimen-
sions from personality inventories were largely infiuenced by
response styles rather than item content.
Jackson and Messick's (1958) suggestion of the centrality of
response style as a major determinant of item responding stimu-
lated considerable debate and dissent, most notably from Rorer
(1965) and Block (1965, 1971, 1972). These commentators
pointed to a collection of item-reversal studies that demonstrated
that when the keying of inventory items was reversed, responses to
questionnaire items were largely consistent across original and
reverse-keyed item sets. Such findings implied that the content of
the item was the primarily responsible for consistency in item
responding, not a response style. Specifically, acquiescence—
when conceptualized as the tendency to respond "yes" or "true" to
test items (e.g., Cronbach, 1946, p. 479) or as a "a tendency to
select some response category a disproportionate amount of the
time independently of the item content" (Rorer, 1965, p. 134)—
was shown to have minimal infiuence on responding.
Bentler, Jackson, and Messick (1971) subsequently conceded
that the tendency to consistently endorse the same response cate-
gory, which they termed agreement acquiescence, had at best only
a minor influence on responses to personality items. They sug-
gested, however, that acceptance acquiescence, or the tendency to
endorse or deny conceptually distinct personality attributes as
self-descriptive, constitutes a personality variable that can exert
considerable infiuence on item responding. Whereas agreement
acquiescence in their framework is posited to be response style
based on a true- or agree-response tendency and thus alterable
through item keying, acceptance acquiescence is related to the
tendency to ascribe various conceptually distinct characteristics to
oneself, regardless of the direction of item keying. With accep-
tance acquiescence, then, extreme "item acceptors" would accept
heterogeneous descriptions of personality attributes as "true" of
them and respond "false" to items that deny such characteristics,
whereas extreme "item rejecters" would respond in an opposite
fashion by denying various personality attributes while endorsing
as "true" items that deny or negate such attributes. With respect to
the present research, the conceptualization of the acquiescence
response style is most consistent with Bentler et al.'s (1971) notion
of acceptance acquiescence.
Potential Effects of Acquiescence on Personality
Measurement
From a measurement perspective, the acquiescent response style
can present several challenges to the validity of personality assess-
ment. This temporally stable response style (Billiet & Davidov,
2008; McCrae et al., 2001) can, for example, substantially alter
covariations among item pairs regardless of item content or the
direction of response keying (Bentler et al., 1971; McCrae et al.,
2001; Soto et al, 2008; Winkler et al., 1982), whereby positive
correlations become greater in magnitude in the positive direction
and negative correlations decrease in size by becoming less neg-
ative. The effect of acquiescence on item covariations can also
spuriously infiate the internal consistency of items sets and thus
imply greater content-related measurement accuracy than is war-
ranted.
Individual differences in acquiescence can also differentially
infiuence scale scores (e.g., infiate them among item acceptors),
thus presenting some challenges for test interpretation (Bentler et
al., 1971; McCrae et al., 2001). Similarly, when scores on predic-
tor and criterion measures are each confounded with acquiescence,
the resultant models might suggest greater predictive accuracy
related to content than is actually justified. Tests of group equiv-
alence in construct measurement may also be biased when item
responses or scale scores are more strongly confounded with
acquiescence in one of the comparison groups (e.g., among those
with less education, as is discussed later; see also Ware, 1978).
With respect to the primary focus of the present research,
acquiescence can also present significant problems when evaluat-
ing the underlying factor structure of a measure, either during the
scale development process or during assessment of the underlying
theories of personality inventories. Considerable distortions in
factor structure can result (McCrae et al., 2001; Rammstedt et al,
2010; Rammstedt & Kemper, 2011; Rammstedt et al., 2013; Soto
et al., 2008; Ware, 1978; Winkler et al., 1982), even when acqui-
escence has a relatively small effect on individual item responses
(McCrae et al., 2001). When item responses are strongly infiu-
enced by method, for example, resultant factor dimensions may be
primarily infiuenced by method-related features such as the direc-
tion of item keying in unbalanced item sets, whereby the highest
loading items on one factor are of a positive sign and the corre-
sponding loadings on another factor are of a negative sign (Jackson
& Messick, 1958; McCrae et al., 2001; Winkler et al., 1982).
The Current Study
Perhaps because of the previously noted effects on interitem
correlations, the acquiescence response style might be a central
factor underlying failures to replicate important findings in per-
sonality research with diverse and representative samples. For
example, recent studies on the predominant dimensional model of
personality structure, the Big Five factor structure, revealed that
self-report assessments of these personality domains are con-
founded by acquiescence, particularly among persons with lower
levels of educational attainment, resulting in failures to replicate
the Big Five factor organization (Rammstedt et al., 2010; Rammst-
edt & Kemper, 2011; Rammstedt et al., 2013). These studies, as
well as a study involving children and adolescents (Soto et al.,
2008), subsequently demonstrated unequivocal support for the Big
Five model once acquiescence was controlled, with this observa-
tion extended to different assessment modes (self-report and in-
terview; Rammstedt & Kemper, 2011) and to persons from differ-
ent countries (Rammstedt et al., 2013).
Although these studies have provided extensive evidence for the
distorting effects of acquiescence on Big Five factor structures.
 ACQUIESCENCE AND ITEM FACTOR STRUCTURE
they are also limited as they have almost exclusively relied on a
single abbreviated measure, the BFI-10 (Rammstedt & John,
2007), a 10-item short-scale version of the widely used Big Five
Inventory (John, Donahue, & Kentle, 1991; see also Benet-
Martinez & John, 1998; John & Srivastava, 1999). Soto et al.
(2008) provided the first evidence that acquiescence also affects
the psychometric quality of the full scale BFI. Their study, how-
ever, was based on data collected from children and adolescents
(mean age = 16.6 years), with evidence of acquiescence most
pronounced prior to age 18. Consequently, it is presently unclear if
the acquiescence response style is pervasive in the item endorse-
ments of the full scale BFI among adults and, correspondingly,
whether acquiescence distorts the factor structure of the 44-item
BFI item set.
As the extant evidence of the biasing effect of acquiescence on
Big Five factor structures is limited to a single abbreviated mea-
stire with adults and a single study with children and adolescents,
the present research was designed to investigate the generalizabil-
ity of previously reported findings with the full-scale BFI admin-
istered to a large community-based sample of adults with varying
levels of educational attainment. Previous research has also indi-
cated that the acquiescence response style is more commonly
observed among persons who have less formal education (Billiet &
McClendon, 2000; Rammstedt et al., 2010; Rammstedt & Kemper,
2011; Ware, 1978; Winkler et al., 1982). Consistent with these
earlier findings, we hypothesized that (a) the Big Five factor
structure would be distorted when uncorrected item scores were
subjected to a factor analysis; (b) distortion of the Big Five factor
structures would be more pronounced among those with lower
levels of educational attainment; and (c) the expected factor struc-
tures would emerge once acquiescence response tendencies were
controlled at the item level.
Method
Sampling Methods and Participants
Data for this research were collected as part of the Personality
and Election Behavior Survey conducted in Germany in 2003
(Schumann, 2004). The sampling procedure targeted adults (age ä
18) randomly selected from cotnmunities spanning all regions of
the country. A two-stage sampling procedure was followed.
Households selected for interview were first randomly identified.
Following household selection, the Kish grid method (Kish, 1949)
was used to randomly select household members for interview. A
commercial vendor (Marplan) organized the sampling procedure
and collected interview and questionnaire data. Participation in this
study was voluntary, and no financial compensation was offered.
As part of this survey, a total of 2,508 persons initially took part
in a 35-min in-person interview that largely emphasized the as-
sessment of demographic information, following which partici-
pants were provided with a packet of questionnaires (including the
BFI) to complete on their own and return in a self-addressed and
stamped envelope. Overall, 57% of the participants initially inter-
viewed {N - 1,427; 48% women; mean age = 49, SD = 17, range =
18-90) provided usable demographic data and subsequently com-
pleted and returned the questionnaire, and collectively serve as the
reference sample for this research. Complete descriptions of the
1139
sampling methods and study procedures are detailed in Schumann
(2004).
Measures and Procedures
Education. The highest level of educational attainment was
assessed during the interview. Participants were categorized ac-
cording to classifications consistent with the German educational
system into one of the following five groups: no secondary edu-
cation (Group 1; n = 31); lower secondary education (Group 2;
n = 653); intermediate secondary education (Group 3; n = 455);
higher secondary education, which is the entrance qualification for
general or applied universities (Group 4; n = 149); and completed
university degree (Group 5; n = 139).
Our previous research (Rammstedt et al., 2010; Rammstedt &
Kemper, 2011) has produced highly similar results for the factor
structure matrices for personality items for the low to medium
education groups (i.e.. Groups 1, 2, and 3 in the present study; n =
1,139); consequently, these groups were collapsed into a single
group {low/medium education) for data analytic purposes. Simi-
larly, the two highest educational groups (i.e.. Groups 4 and 5;n =
288) have also been observed in our earlier studies to produce
comparable results and were hence combined into a single group
{high education).
Big Five Inventory (BFI). Although not as widely used as the
NEO Personality Inventories (Costa & McCrae, 1992) and the
International Personality Item Pool Big Five Inventories (Gold-
berg, 1999; Goldberg et al., 2006), the relatively brief BFI (John et
al., 1991,as reprinted in Benet-Martinez & John, 1998; John &
Srivastava, 1999) is nonetheless a popular and widely used measure of
the Big Five model of personality. The BFI consists of 44 short-phrase
items, rated on a 5-point scale (from 1 = disagree strongly to 5 =
agree strongly), with each of the Big Five personality domains
(i.e., Extraversion, Agreeableness, Conscientiousness, Neuroti-
cism, and Openness) assessed with between eight and 10 items. As
will be described in more detail later, most items in the BFI are
balanced in content and keying (32 of 44 items, or 73%), with the
remaining items keyed in a positive direction (i.e., agreement with
the item is an endorsement of the personality characteristic).
The German version of the BFI used here was carefully trans-
lated and adapted to the German language based on translation and
back-translation methods by experts followed by reconciliation.
The resulting German version is fully comparable to the original
English-language version and demonstrates satisfactory psycho-
metric properties (Rammstedt, 1997; Rammstedt & John, 2005;
see also Lang, Liidtke, & Asendorpf, 2001). Internal consistencies
(coefficient alpha) for the items belonging to the Big Five scales in
the present research ranged between .74 (for Neuroticism) and .82
(for Openness), with an average of .79.
Acquiescence response style. In the present study, an overall
indicator of the acquiescent response style was derived from 16
matched pairs of items from each of the five personality domains
that are logically opposite in content and balanced in their keying
(e.g., "Is talkative" vs. "Tends to be quiet"). Mean acquiescence
scores were computed for each participant according to the pro-
cedures outlined by Soto et al. (2008). The use of balanced
matched pairs is particularly well suited for the study of acquies-
cence, as this approach involves the evaluation of consistent re-
sponding to items that are similar in content but expressed in
1140 RAMMSTEDT AND FARMER
opposite or balanced terms (Bentler et al., 1971; Winkler et al.,
1982). Acceptance acquiescence would be evident, for example, if
tbe mean score for a pair of balanced items was greater than the
value corresponding to the middle response option (i.e., coded as
3) along the 5-point Likert-response format, indicating that accep-
tance of the positive item is stronger than the rejection of the
negative item. Counter-acquiescence, or nay-saying, would be
evident if the mean score for a pair of balanced items was below
the value corresponding to the middle response option. In survey
research, acceptance acquiescence appears to be considerably
more common than counter-acquiescence (Winkler et al., 1982).
As operationalized in tbe current research, the acquiescence
response style is similar to Bentler et al.'s (1971, pp. 190-191)
notion of acceptance acquiescence described previously. Because
the BFI does not contain items that are strict negations of specific
experiences (e.g., "Not happy," "Not sad"), acquiescence on the
BFI would be evident when respondents demonstrate a tendency to
(a) consistently agree with items that describe personality charac-
teristics that are logical opposites (acceptance acquiescence) or (b)
consistently disagree with personality-descriptive items that are
logical opposites (counter-acquiescence).
Statistical Analyses
Ipsatization of individual item responses based on idio-
graphically derived acquiescence scores. Ipsatization is a com-
mon technique for removing variance due to acquiescent response
tendencies from personality questionnaires (Clemans, 1966; Cun-
ningham, Cunningham, & Green, 1977; Fischer, 2004; Ostendorf,
1990; ten Berge, 1999). In the present research, each respondent's
mean score derived from the 16 matched item pairs was subtracted
from bis or her score on each single BFI item, including the 12 BFI
items that were not among the 32 items used in the computation of
acquiescence scores. In analyses we will describe, botb unadjusted
BFI item responses and acquiescence-corrected BFI item values
were used in evaluations of the Big Five factor structure.
Factor analyses. To facilitate comparisons across similar
studies, we used the factor analytic procedures that were consistent
with the assumptions of the Big Five model and with methods used
in related studies (Rammstedt et al., 2010; Rammstedt & Kemper,
2011; Rammstedt et al., 2013). Specifically, we conducted a set of
principal component exploratory factor analyses for which we
specified Five-Factor extractions and orthogonal (varimax) simple
structure rotations. Noncorrected and acquiescence-corrected so-
lutions were generated separately for tbe two education groups,
low/medium and high.
Evaluation of factor structure congruence. As advocated by
Hopwood and Donnellan (2010) for evaluations of tbe internal
structure of personality inventories, tbe resultant varimax rotated
factors described were subsequently compared with the idealized
Big Five pattern (consisting of -I-1, - 1 , and 0 loadings) by use of
Procrustes rotation. The similarities between the unadjusted and
acquiescence-controlled five-dimensional factor solutions to the
idealized patterns were evaluated with the congruence coefficient
c (Burt, 1948). Guidelines provided by Lorenzo-Seva and ten
Berge (2006) for interpreting factor structure congruence were
followed, with congruence coefficients of at least .85 needed to
infer "fair" replication.
Results
Big Five Factor Structures for Low/Medium and
High Education Groups Based on Unadjusted
Item Responses
The 44 unadjusted BFI items were subjected to exploratory
factor analyses, separately for each of the education-based groups.
For botb subsamples, scree tests suggested factor structures with
six rather than five dimensions (upper half of Figure 1). To keep
our analyses aligned with the Big Five model, we extracted five
factors for both education groups, resulting in 47% of the variance
being explained in both subsamples. Tbe varimax-rotated factor
loading matrices for botb education groups are presented within
the first 10 columns of Table 1.
For the low/medium education group, tbe resulting factor struc-
ture is severely distorted, with no single Big Five dimension
unambiguously replicated. Departures from tbe expected factor
structures are also evident by the congruence coefficients of the
observed factors with the idealized factor structures. Congruence
coefficient values ranged between .53 and .86 for the rotated
Five-Factor solutions, with an overall matrix congruence of .72.
For the high education subsample, the matrix of factor loadings
similarly failed to conform to the expected Big Five pattern.
Except for the Agreeableness dimension, one or more items cor-
responding to each of the remaining four personality dimensions
demonstrated greater secondary loadings relative to tbe primary
loading. Overall, however, factor structures for the high education
subsample demonstrated greater correspondence to the idealized
factor matrix compared with the low/medium education group.
Factor congruence coefficients ranged from .58 to .93, with an
overall matrix congruence of .79.
Individual Differences in Acquiescence Response Bias
Research that we have reviewed previously suggests tbat level
of education attainment is related to individual differences in
acquiescence. We investigated this issue with the present sample
by comparing acquiescence response tendencies between the two
educational groups. As expected, variance in acquiescence scores
was larger in the low/medium education group compared with the
high education group, according to Levene's test for the equality of
variances (variances = .11 and .07, respectively), F(l, 1425) =
4.63, p < .05.
Big Five-Factor Structures after Controlling for
Acquiescence
To evaluate the possible infiuence of acquiescence on the dis-
tortion of the Big Five factor structure, we repeated the factor
analytic procedures used previously, this time replacing unadjusted
item scores with mean-corrected item scores adjusting for acqui-
escence. When based on acquiescence-controlled item values,
scree tests clearly revealed a five factorial solution for each
education-based subsample (see lower half of Figure 1). The
resultant Five-Factor solutions accounted for 44% and 47% of the
explained variance in the low/medium and high education groups,
respectively. Varimax-rotated factor loading matrices for botb
groups are presented in the last 10 columns of Table 1, each of
 ACQUIESCENCE AND ITEM FACTOR STRUCTURE 1141

Scree plot Scree plot

raw scores; low and medium educated raw scores: high educated

s 11 13 IS 17 19 21 23 2S 27 29 31 33 35 37 39 4( 43 1 3 5 7 9 11 13 15 17 19 21 23 25 2T 29 31 33 35 37 3S 41 43

Factor Factor

Scree plot Scree plot

mean-corrected scores; tow and medium educated mean-corrected scores; high educated

1 3 5 7 9 11 13 IS 17 19 21 23 25 27 29 31 33 35 37 39 41 43 1 3 5 7 9 11 13 ÎS 17 IS 21 23 25 27 2?) 31 33 35 37 33 41 43

Factor Factor

Figure 1. Scree plots from principal components analyses of raw data for low/medium and high education
groups (Panels a and b, respectively) and mean-corrected data for low/medium and high education groups
(Panels c and d, respectively).

which demonstrates clear and consistent Big Five patterns. Each
BFI item in the acquiescence-controlled analyses demonstrated its
highest primary loading on the factor to which it conceptually
belonged, with only a single exception (Item 29, "I can be moody,"
for the high education group). In each instance, congruence coef-
ficients for single factors equaled or exceeded .85, with an overall
matrix congruence of .86 and .87 for the low/medium and high
education groups, respectively.
Discussion
The internal structure of a personality measure, described by
Hopwood and Donnellan (2010, p. 332) as the "orderiiness of the
actual clustering of related elements . . . within and across theo-
retically substantive dimensions," can be regarded as an indication
of the plausibility of the psychological theory underlying a scale or
inventory (see also Loevinger, 1957). Previous exploratory and
confirmatory factor analytic studies of Big Five personality mea-
sures, summarized in McCrae et al. (2001), Rammstedt et al.
(2010), and Hopwood and Donnellan (2010), have occasionally
failed to reveal or confirm the expected Five-Factor internal sti-uc-
ture. The acquiescent response tendency has been recognized as a
potential source of bias that can create difficulties when one is
evaluating the internal structure of assessment measures, including
Big Five inventories. Findings irom this study in conjunction with
our previous adult-based studies with an abbreviated version of the
BFI (Rammstedt et al., 2010; Rammstedt & Kemper, 2011;
Rammstedt et al., 2013) and a single study with the full-scale BFI
with children and adolescents (Soto et al., 2008) clearly indicate
that acquiescence can have adverse impact on interitem correla-
tions that, in turn, produces distortions in the underlying factor
structure of a measure. In unadjusted analyses in the current study,
considerable cross-loadings among item sets were evident, and no
factor solution met or surpassed the critical congruence benchmark
of at least .85 with reference to the idealized Big Five factor
1142 RAMMSTEDT AND FARMER

t n n t m .Dr»nrm
ppp—'p— p p p p p p p r) cs
' ''' r' ' ' 'r ' ' ' ' ' ' ' ' ' ' ' ' ' ' ' p p p p p
r ' I i.H roooo^ONCNoo*;i-io^
ppp p p p —CN — — CNrn ^ „ _ p p —• p p —p — o

Tf • * O < n — O4 CN — >C Os ^ ON VO CN l~ 00 VD t~. t - r - M rt CN OMn « CN m t^ O — M CN CN in i/"! CN ^ in ÇS en — OO ^ ON

' '''' r '''' r
— pcNp^-rfcnp p — — p p p — — — r^inr^Nosci^vcTtin — — ppcNCjcNcN p — — p p p p — ppoo
. . . . ^. . ^. ^. ^. . ^. ^. . . . ^. . . ^. . ^. ^. . . . ^. ^. . ^. . . ^. . ^. . . . ^. . ^. ^. . . ^. .

.2
8 I I I I I

I ^ in ^ ^ *# in t
1 c*^ o ^^ ON ^^ csinvo — t^t^oor^r<i
oo2c I — csp p p p p p p p — — rnp
r ' r i'
p —
ooo—'r--cNor-'<tcN^i.
•^CN — c J r n - ^ p c s p p c
' r ' ' i' \' ' \' {
•s i t~~ in CN in o cN ^ '"' r~- cs r^
r '' I I
rrr
I I I I I

I r^ i ^ CN "^ "^
o o o ——
NomoNONOrtin«
rfvoin^NOTTinin

I I

I ^H t^ Tf (
in in >n ^
I I I I I I
r 'r ooo o o.
oocvivo
l p — CN C
o o oC
I I I II'

- f*^ CO ^ 00 o ^-
5 —. o o — —' CN OOOOO
I I I I I I I
ONr-inwoof'ïcNin
8 c*^pr
Ov —
in —vem —m —>o
I I I I r i r I I I
-a O'^ooç4.^c*^rn^ - —O\ —gin — g g
I I I I I I I I I I
!• CN
I
Ö—— <

I I I I I II II
§
^n^QO^O^'Or^CS

I
00 íN ^ t-^ 00 O —' 00 ^ ^ ^D r^ ^ cs o "^ ^ Vi I
O.—'iS'— — o o — o o—'O—'OCSCN'' n-¡priO'^fN004^- IOC
I

ONOONOrnO — —
O ~" o Ô o o íN p p pp5pp' ; p CN p

r ' r ' ' I' r ' r r r r r ' ' r I I I

rr
c§ I 00 m 00 in ^ m oo (^
in p in_ f i — ve — ve
I I r l'
—«inin^HTt^^c^^cN V j í. ^ ^ " ^ " O> O^ O^ O^ ^ ' O\ oo ©^ oo 00 r-- CN '—'CT\'—' O m
OCNCN — — T f O O
S —^So^or^o — 8 o o
1 1 1 1 1 1 1 1 1 1

t ~ - ' * C N N O ' * — CNOO 00 ^- VO 00 •<í O »n O r^ -^ 00

— <N — ^-CN^-CN — £2 12o — cN (N o •—1 es ^^ oo 00

I 1 II II

v^ CO r*^ ^^^ ""^ ^P c^ ""^ f^

1

£S cNin S ? £
1

in ^-' ^ tn ^ r^
's I r
Oo

-5?
T3

•a
II
s •a
§ gg I
to
II o
I 3 's: -K C

I *E°3»§22
I i3 S
# BBS'S « B S -^.
JJ M)
o t«
.5Í
S
"1
c
Si"
5 M
••a 'S
es
° i "u
onië'âS.-sl « O

<§ « .H I
S'a
^ va «5 t/3 H H V) "
j.ü(
• *CT\• * C
— I — I CN C

< U X
 ACQUIESCENCE AND ITEM FACTOR STRUCTURE
solution. Both personality characteristics (content) and method
(noncontent) likely infiuenced item covariations and the resultant
factor structures in these unadjusted analyses. Adjusted analyses,
however, revealed that distortions in factor structure could be
almost entirely eliminated following corrections for acquiescence
as evident by resultant factor structures that were unambiguously
interpretable in terms of the Big Five theory of personality orga-
nization. The recovery of the hypothesized factor structure follow-
ing the control for the acquiescent response style in the adjusted
analyses likely resulted from the predominant infiuence of content
with minimal infiuence from method on item covariations and
factor structures.
The present research also demonstrated that persons with less
formal education display greater variability in acquiescence re-
sponse tendencies and greater associated distorting effects on
factor structure compared with those with higher levels of educa-
tional attainment. These results, which are consistent with those
from other investigations (Billiet & McClendon, 2000; Rammstedt
et al., 2010; Ramtnstedt & Kemper, 2011; Rammstedt et al., 2013;
Ware, 1978; Winkler et al., 1982), suggest that findings frequently
obtained with highly educated samples (e.g., university students)
may not replicate in more representative samples such as we used
here unless acquiescence response tendencies are measured and
controlled. Despite greater acquiescence variability among the
low/medium education group, for example, the expected Big Five
factor structures were revealed with near-textbook-like clarity
once items were corrected for this response style.
Developers of personality scales and inventories can take steps
to minimize the potential impact of acquiescence during the initial
stages of scale development. During the item-writing phase, for
example, scale developers should avoid items that are ambiguous,
abstract, and moderately socially desirable content, as these item
features may increase the likelihood of acquiescent responding
(Cloud & Vaughan, 1970; Jackson & Messick, 1958). During the
item selection and retention process, items might be considered for
exclusion if they demonstrate relatively large associations with
acquiescence, as was done during the development of the NEO
inventories (McCrae et al., 2001).
Some test developers have included in their inventories scales
that explicitly measure response styles. One such example is the
True Response Inconsistency (TRIN) scales of the Multidimen-
sional Personality Questionnaire (Tellegen, 1982), the Minnesota
Multiphasic Personality Inventory-2 (MMPI-2; Butcher et al.,
2001), and MMPI-2-Restructured Form (MMPI-2-RF; Ben-
Porath & Tellegen, 2008). The TRIN scales consist of item pairs
that contain semantically opposite content and are particularly
useful for identifying persons whose responses are highly con-
founded with acquiescence responding. McCrae et al. (2001) pro-
vided examples in which TRIN scale scores were used to control
for acquiescence prior to factor analysis as a method for producing
unconfounded factor structures. Alternatively, researchers could
also simply elect to discard or deem as invalid data supplied by
individuals who score high on acquiescence measures such as
TRIN, although this approach may be less successful in eliminat-
ing the overall infiuence of acquiescent responding than statistical
approaches such as partialling (Winkler et al., 1982).
Another approach for minimizing acquiescence is to balance
item content and the keying of scales (Cloud & Vaughan, 1970;
McCrae, Costa, & Martin, 2005; McCrae et al., 2001; Winkler et
1143
al, 1982). As demonstrated here and elsewhere (Rammstedt et al.,
2010; Rammstedt & Kemper, 2011; Rammstedt et al., 2013; Soto
et al, 2008; Winkler et al, 1982), the inclusion of matched item
pairs within scales also provides a built-in means for assessing and
controlling acquiescent responding. Even when scales are balanced
in item keying, however, acquiescent responding is still possible
(Ware, 1978), particularly among individuals who are likely to
endorse a broad array of heterogeneous characteristics while op-
posing statements that negate such qualities (Bentler et al, 1971).
Statistical control of acquiescence may therefore be necessary in
some circumstances, even when scales are balanced.
Future research into the infiuence of acquiescence on Big Five
factor structures or other dimensional models of personality might
evaluate the extent to which this response style biases responding
across measures. The present study was based on data of the
German version of the BFl. Earlier research (Rammstedt et al.,
2013) has demonstrated the generalizability of the distorting effect
of acquiescence on the BFT across cultures. In need of further
evaluation, however, is the extent of the generalizability of acqui-
escence across different Big Five questionnaires (e.g., the NEO-
PI-R, NEO-FFI, or IPIP inventories) as well as among personality
inventories that assess different models of personality. In addition,
further research is needed to investigate the generalizability of the
effects across different response formats (e.g., a 4-point Likert
format instead of the 5-point scale utilized here that included a
neutral response option), and to corroborate limited reports that
suggest some response option formats more frequently occasion
acquiescent responding than others (Hui & Triandis, 1989).
Future research might also seek to illutninate psychological
processes that underlie the tendency to be acquiescent and to
identify when and among whom acquiescence is more probable.
Relatively little is presently known, for example, about individuals
who are especially prone to acquiescence. In addition to being
more common among persons with less formal education, acqui-
escence appears to be more frequent among persons who produce
lower IQ scores (Gudjonsson, 1990; Gudjonsson & Young, 2011)
or who reside at either end of the age continuum (Billiet &
McClendon, 2000; Soto et al, 2008; Winkler et al, 1982). In
accounting for age differences among youth, for example. Soto et
al. (2008) suggested that acquiescence might be less common
among persons with more clearly developed and differentiated
self-concepts who, in turn, have greater certainty about their pref-
erences, opinions, and personal experiences. Racial and cultural
factors also appear to be related to acquiescence (Bachman &
O'Malley, 1984; Hui & Triandis, 1989; Johnson, Kulesa, Cho, &
Shavitt, 2005; Marin, Gamba, & Marin, 1992; Rammstedt et al,
2013; Winkler et al, 1982). Smith (2004), for example, has sug-
gested that acquiescence is a style of communication that is more
common in cultures high on family collectivism or whose mem-
bers express preferences for highly structured lives with few
opportunities for uncertainty. Examinations of individual and cul-
tural differences in acquiescence can identify groups of persons
more or less inclined to utilize this response style and stimulate the
development of testable theories of acquiescent responding.
Finally, to establish the broader implications of acquiescence on
the validity of personality assessment, additional investigations are
needed on the effects of acquiescence beyond those related to
factor structures. Similar to studies on the effects of social desir-
ability in personnel selection (e.g.. Ones & Viswesvaran, 1998),
1144 RAMMSTEDT AND FARMER
acquiescence could also be examined as putative predictor, mod-
erator, mediator, or suppressor variables with respect to criterion
behaviors of interest. Evidence has been presented, for example,
which suggests that the importance of social desirability relative to
these various roles is, at best, modest and relatively trivial in
personnel selection contexts (Ones & Viswesvaran, 1998; Ones,
Viswesvaran, & Reiss, 1996). It is presently unclear if acquies-
cence would similarly display negligible importance in relation to
these roles in various personality assessment contexts.
In conclusion, the present research and our earlier related find-
ings suggest that acquiescence can have deleterious effects on the
internal structure of personality measures. When not taken into
account or controlled, the distorting effects of acquiescence on
internal structure can, in turn, produce uncertainty about the utility
of a inventory, raise questions about the validity of the personality
theory underlying a measure, pose problems in interpretation of
summary scores, or create difficulties in evaluations of correlations
between summary scores and external predictor or criterion vari-
ables. Findings from the present study suggest that replicable and
theoretically coherent internal structures of Big Five measures can
be achieved with diverse and representative samples when acqui-
escence response tendencies are measured and controlled. Given
findings from this and other studies that show that acquiescence is
generally a greater concern among those with less formal educa-
tion, caution is also suggested with respect to the exclusive use of
well-educated convenience samples during the development and
evaluation of personality questionnaires.
References
Bachman, J. G., & O'Malley, P. M. (1984). Yea-saying, nay-saying, and
going to extremes: Black-white differences in response styles. Public
Opinion Quarterly, 48, 491-509. doi:10.1086/268845
Benet-Martfnez, V., & John, O. P. (1998). Los Cinco Grandes across
cultures and ethnic groups: Multitrait multimethod analyses of the Big
Five in Spanish and English. Journal of Personality and Social P.9ychol-
ogy, 75, 729-750. doi: 10.1037/0022-3514.75.3.729
Ben-Porath, Y. S., & Tellegen, A. (2008). MMPI-2-RF (Minnesota Mui-
tiphasic Personality Inventory-2-Restructured Form): Manual for ad-
ministration and scoring. Minneapolis: University of Minnesota Press.
Bentler, P. M., Jackson, D. N., & Messick, S. (1971). Identification of
content and style: A two dimensional interpretation of acquiescence.
Psychological Bulletin, 76, 186-204. doi:10.1037/h0031474
Billiet, J. B., & Davidov, E. (2008). Testing the stability of an acquiescence
style factor behind two interrelated substantive variables in a panel
design. Sociological Methods & Research, 36, 542-562. doi:10.1177/
0049124107313901
Billiet, J. B., & McClendon, M. J. (2000). Modeling acquiescence in
measurement models for two balanced sets of items. Structural Equation
Modeling, 7, 608-628. doi:10.1207/S15328007SEM0704_5
Block, J. (1965). The challenge of response sets. New York, NY: Apple-
ton-Century-Crofts.
Block, J. (1971). On further conjectures regarding acquiescence. Psycho-
logical Bulletin, 76, 205-210. doi:10.1037/h0031475
Block, J. (1972). The shifting definitions of acquiescence. Psychological
Bulletin, 78, 10-12. doi:10.1037/h0033043
Burt, C. (1948). Factor analysis and canonical correlations. British Journal
of Statistical Psychology, 1, 95-106.
Butcher, J. N., Graham, J. R., Ben-Porath, Y. S., Tellegen, A., Dahlstrom,
W. G., & Kaemmer, B. (2001). MMPI-2 (Minnesota Muitiphasic Per-
sonality Inventory-2): Manual for administration and scoring (Rev. ed.).
Minneapolis: University of Minnesota Press.
Clemans, W. V. (1966). An analytical and empirical examination of some
properties of ipsative measures. Richmond, VA: Byrd Press.
Cloud, J., & Vaughan, G. M. (1970). Using balanced scales to control
acquiescence. Sociometry, 33, 193-202. doi: 10.2307/2786329
Costa, P. T., & McCrae, R. R. (1992). Revised NEO Personality Inventory
(NEO PI-R) and NEO Five Factor Inventory: Professional mantMl.
Odessa, FL: Psychological Assessment Resources.
Cronbach, L. J. (1946). Response sets and test validity. Educational and
Psychological Measurement, 6, 475-494.
Cronbach, L. J. (1950). Further evidence on response sets and test design.
Educational and Psychological Measurement, 10, 3-31. doi:10.1177/
001316445001000101
Cunningham, W., Cunningham, 1. C. M., & Green, R. T. (1977). The
ipsative process to reduce, response set bias. Public Opinion Quarterly,
41, 379-394. doi: 10.1086/268394
Fischer, R. (2004). Standardization to account for cross-cultural response
bias: A classification of score adjustment procedures and review of
research in JCCP. Journal of Cross-Cultural Psychology, 35, 263-282.
doi:10.1177/0022022104264122
Goldberg, L. R. (1999). A broad-bandwidth, public-domain personality
inventory measuring the lower-level facets of several five-factor models.
In I. Mervielde, I. J. Deary, F. De Fruyt, & F. Ostendorf (Eds.),
Personality psychology in Europe (Vol. 7, pp. 7-28). Tilburg, the
Netherlands: Tilburg University Press.
Goldberg, L. R., Johnson, J. A., Eber, H. W., Hogan, R., Ashton, M. C ,
Cloninger, C. R., & Gough, H. G. (2006). The international personality
item pool and the future of public-domain personality measures. Journal
of Research in Personality, 40, 84-96. doi:10.1016/j.jrp.2005.08.007
Gudjonsson, G. H. (1990). The relationship of intellectual skills to sug-
gestibility, compliance and acquiescence. Personality and Individual
Differences, U, 227-231. doi:10.1016/0191-8869(90)90236-K
Gudjonsson, G. H., & Young, S. (2011). Personality and deception: Are
suggestibility, compliance, and acquiescence related to socially desirable
responding? Personality and Individual Differences, 50, 192-195. doi:
10.1016/j.paid.2010.09.024
Hopwood, C. J., & Donnellan, M. B. (2010). How should the internal
structure of personality inventories be evaluated? Personality and Social
Psychology Review, 14, 332-346. doi:10.1177/1088868310361240
Hui, C. H., & Tdandis, H. C. (1989). Effects of culture and response format
on extreme response style. Journal of Cross-Cultural Psychology, 20,
296-309. doi:10.1177/0022022189203004
Jackson, D. N., & Messick, S. (1958). Content and style in personality
assessment. Psychological Bulletin, 55, 243-252. doi:10.1037/h0045996
John, O. P., Donahue, E. M., & Kentle, R. L. (1991). The Big Five
Inventory-Versions 4a and 54. Berkeley: University of California,
Berkeley, Institute of Personality and Social Research.
John, O. P., & Sdvastava, S. (1999). The Big Five trait taxonomy: History,
measurement, and theoretical perspectives. In L. A. Pervin & O. P. John
(Eds.), Handbook of personality: Theory and research (2nd ed., pp.
102-138). New York, NY: Guiiford Press.
Johnson, T., Kulesa, P., Cho, Y. I., & Shavitt, S. (2005). The relationship
between culture and response styles: Evidence from 19 countries. Jour-
nal of Cross-Cultural Psychology, 36, 264-277. doi:10.1177/
0022022104272905
Kish, L. (1949). A procedure for objective respondent selection within the
household. Journal of the American Statistical Association, 44, 380-
387. doi:10.1080/01621459.1949.10483314
Lang, F. R., Lüdtke, O., & Asendorpf, J. (2001). Testgüte und psy-
chometdsche Äquivalenz der deutschen Version des Big Five Inventory
(BFI) bei jungen, mittelalten und alten Erwachsenen. [Validity and
psychometric equivalence of the German version of the Big Five Inven-
tory in young, middle-aged, and old adults.] Diagnostica, 47, 111-121.
doi:10.1026//0012-1924.47.3.111
 ACQUIESCENCE AND ITEM FACTOR STRUCTURE
Loevinger, J. (1957). Objective tests as instruments of psychological the-
ory. Psychological Reports, 3, 635-694.
Lorenzo-Seva, U., & ten Berge, J. M. F. (2006). Tucker's congruence
coefficient as a meaningful index of factor similarity. Methodology, 2,
57-64.
Marin, G., Gamba, R. J., & Marin, B. V. (1992). Extreme response style
and acquiescence among Hispanics: The role of acculturation and edu-
cation. Journal of Cross-Cultural Psychology, 23, 498-509. doi:
10.1177/0022022192234006
McCrae, R. R., Costa, P. T., Jr., & Martin, T. A. (2005). The NEO-PI-3:
A more readable revised NEO personality inventory. Journal of Person-
ality Assessment, 84, 261-270. doi:10.1207/sl5327752jpa8403_05
McCrae, R. R., Herbst, J. H., & Costa, P. T., Jr. (2001). Effects of
acquiescence on personality factors structures. In R. Riemann, F. M.
Spinath, & F. Ostendorf (Eds.), Personality and temperament: Genetics,
evolution, and structure (pp. 217-231). Berlin, Germany: Pabst Science.
Ones, D. S., & Viswesvaran, C. (1998). The effects of social desirability
and faking on personality and integrity assessment for personnel selec-
tion. Human Performance, 11, 245-269.
Ones, D. S., Viswesvaran, C , & Reiss, A. D. (1996). Role of social
desirability in personality testing for personnel selection: The red her-
ring. Journal of Applied Psychology, 81, 660-679. doi:10.1037/0021-
9010.81.6.660
Ostendorf, F. (1990). Sprache und Persönlichkeitsstruktur. Zur Validität
des Fünf-Faktoren-Modells der Persönlichkeit. [Language and person-
ality structure. On the structural validity of the Five Factor Model of
personality]. Regensburg, Germany: Roderer.
Rammstedt, B. (1997). Die deutsche Version des Big Five Inventories
(BFI): Übersetzung und Validierung eines Fragebogens zur Erfassung
des Fünf-Faktoren-Modells der Persönlichkeit. [The German version of
the Big Five Inventory (BFI): Translation and validation of a question-
naire assessing the Five Factor Model of personality] (Unpublished
master's thesis [Diplomarbeit]). University of Bielefeld, Bielefeld, Ger-
many.
Rammstedt, B., Goldberg, L. R., & Borg, I. (2010). The measurement
equivalence of Big-Five factor markers for persons with different levels
of education. Journal of Research in Personality, 44, 53-61. doi:
10.1016/j.jrp.2009.10.005
Rammstedt, B., & John, O. P. (2005). Kurzversion des Big Five Inventory
(BFI-K): Entwicklung und Validierung eines ökonomischen Inventars
zur Erfassung der fünf Faktoren der Persönlichkeit. [The short-scale Big
Five Inventory (BFI-K): Development and validation of an economic
instrument for the assessment of the Big Five factors of personality.]
Diagnostica, 51, 195-206. doi:10.1026/0012-1924.51.4.195
1145
Rammstedt, B., & John, O. P. (2007). Measuring personality in one minute
or less: A 10-item short version of the Big Five Inventory in English and
German. Joumai of Research in Personality, 41, 203-212. doi:10.1016/
j.jrp.2006.02.001
Rammstedt, B., & Kemper, C. J. (2011). Measurement equivalence of the
Big Five: Shedding further light on potential causes of the educational
bias. Joumai of Research in Personality, 45, 121-125. doi:10.1016/j.jrp
.2010.11.006
Rammstedt, B., Kemper, C. J., & Borg, I. (2013). Correcting Big Five
personality measurements for acquiescence: An 18-country cross-
cultural study. European Journal of Personality, 27, 71-81. doi:
10.1002/per.l894
Rorer, L. G. (1965). The great response-style myth. Psychological Bulletin,
63, 129-156. doi:10.1037/h0021888
Schumann, S. (2004): ZA4052: Persönlichkeit und Wahlverhalten 2003
[ZA4052: Personality and voting behavior 2003] [Data file. Version
1.0.0]. Retrieved from the GESIS Data Archive (Cologne) at http://
info 1 .gesis.org/dbksearch 19/SDesc2.asp?no=4052&tab = 3&II = I0&
notabs=&af=&nf=l&db=E. doi: 110.4232/1.4052
Smith, P. B. (2004). Acquiescent response bias as an aspect of cultural
communication style. Joumai of Cross-Cultural Psychology, Í5, 5 0 - 61.
doi:10.1177/0022022103260380
Soto, C. J., John, O. P., Gosling, S. D., & Potter, J. (2008). The develop-
mental psychometrics of Big-Five self-reports: Acquiescence, factor
structure, coherence, and differentiation from ages 10 to 20. Joumai of
Personality and Social Psychology, 94, 1\%-11,1. doi: 10.1037/0022-
3514.94.4.718
Tellegen, A. (1982). Brief manual for the Differential Personality Ques-
tionnaire. Unpublished manuscript. Department of Psychology, Univer-
sity of Minnesota, Minneapolis.
ten Berge, J. M. F. (1999). A legitimate case of component analysis of
ipsative measures, and partialling the mean as an alternative to ipsati-
zation. Multivariate Behavioral Research, 34, 89-102. doi:10.1207/
sl5327906mbr3401_4
Ware, J. E., Jr. (1978). Effects of acquiescent response set on patient
satisfaction ratings. Medical Care, 16, 327-336. doi: 10.1097/00005650-
197804000-00005
Winkler, J. D., Kanouse, D. E., & Ware, J. E., Jr. (1982). Controlling for
acquiescence response set in scale development. Journal of Applied
Psychology, 67, 555-561. doi:10.1037/0021-9010.67.5.555
Received October 8, 2012
Revision received April 4, 2013
Accepted April 11, 2013 •
Copyright of Psychological Assessment is the property of American Psychological
Association and its content may not be copied or emailed to multiple sites or posted to a
listserv without the copyright holder's express written permission. However, users may print,
download, or email articles for individual use.
Copyright of Psychological Assessment is the property of American Psychological
Association and its content may not be copied or emailed to multiple sites or posted to a
listserv without the copyright holder's express written permission. However, users may print,
download, or email articles for individual use.