See discussions, stats, and author profiles for this publication at: https://www.researchgate.

net/publication/314271785

The Multidimensional Personality Questionnaire's Inconsistency Scales Identify

Invalid Profiles Through Internal Statistics and External Correlates

Article  in  Psychological Assessment · March 2017

DOI: 10.1037/pas0000452

CITATIONS READS

5 861

2 authors:

Stephen D Benning Andrew J Freeman

University of Nevada, Las Vegas State of Nevada
76 PUBLICATIONS   4,333 CITATIONS    39 PUBLICATIONS   419 CITATIONS   

SEE PROFILE SEE PROFILE

Some of the authors of this publication are also working on these related projects:

Comprehensive Assessment of Mental, Physical, and Social Health in Cirque du Soleil & National Circus School View project

All content following this page was uploaded by Stephen D Benning on 03 August 2017.

The user has requested enhancement of the downloaded file.

Running head: VALIDITY OF MPQ-BF INCONSISTENCY SCALES 1

The Multidimensional Personality Questionnaire’s Inconsistency Scales Identify Invalid Profiles

through Internal Statistics and External Correlates

Stephen D. Benning and Andrew J. Freeman

University of Nevada, Las Vegas

Author Note

Stephen D. Benning and Andrew J. Freeman, Department of Psychology, University of Nevada,

Las Vegas.

Funding for this work was provided through a Discovery Grant from Vanderbilt University. We

extend thanks to Emily A. Dowgwillo for coordinating data collection for this study along with Karen

Miller and Alan B. Storrow for giving us access to the emergency department for data collection.

Correspondence concerning this article (and requests for an extended version of this report)

should be addressed to Stephen D. Benning, 4505 S. Maryland Pkwy., Box 455030, Las Vegas, NV

89154-5030. Email: stephen.benning@unlv.edu

©American Psychological Association, 2017. This paper is not the copy of record and may not exactly

replicate the authoritative document published in the APA journal. Please do not copy or cite without

author's permission. The final article is available, upon publication, at:

https://doi.org/10.1037/pas0000452
VALIDITY OF MPQ-BF INCONSISTENCY SCALES 2

Abstract

Inconsistency scales represent a promising method for separating valid and invalid personality

profiles. In a sample of 1,258 participants in the waiting room of the emergency department of an urban

university hospital, we examined whether data from participants with profiles flagged as invalid (n = 132)

using the Variable Response Inconsistency (VRIN) or True Response Inconsistency (TRIN) scales of the

Multidimensional Personality Questionnaire’s brief form (MPQ-BF) differed from those that did not

exceed any validity cutoffs (n = 1,026). Invalid profiles’ scores on many scales were less internally

consistent and had less variability than those from valid profiles, especially for random and acquiescent

response styles. Scores on MPQ-BF primary trait scales from profiles featuring random responses

appeared more psychologically maladjusted than those on valid profiles. Compared to primary trait scores

on valid profiles, acquiescent profiles generally had higher scores, and counteracquiescent profiles had

lower scores. The higher-order component structure of invalid profiles was less consistent with published

MPQ-BF component structures than that of valid profiles, though negative emotionality was generally

reasonably well-preserved. Scores on primary traits associated with negative emotionality generally had

larger correlations with demographic criteria for valid profiles than invalid profiles. These results argue

that inconsistency scales meaningfully identify invalid profiles in normal-range personality assessment.

Keywords: validity, inconsistency, personality, emergency department, community sample,

Multidimensional Personality Questionnaire, substance use, socioeconomic status

Public Significance Statement

This study showed that normal-range personality trait scores from invalid profiles (due to random, yea-

saying, or nay-saying response styles) are distorted in many ways compared to valid personality profiles.

Furthermore, scores on invalid profiles are not related to demographic characteristics like scores on valid

profiles are. These results suggest that researchers and clinicians should not interpret personality profiles

that are flagged as invalid by scales assessing inconsistent responding.

VALIDITY OF MPQ-BF INCONSISTENCY SCALES 3

The Multidimensional Personality Questionnaire’s Inconsistency Scales Identify Invalid Profiles

through Internal Statistics and External Correlates

The clinical utility of scales designed to separate invalid profiles from valid profiles is

hotly debated. Profiles identified as invalid by positive or negative impression management

scales often demonstrate reduced criterion validity compared to invalid profiles (Caldwell-

Andrews, Baer, & Berry, 2000; Rohling et al., 2011), though some researchers hold this is not

always true (McGrath, Mitchell, Kim, & Hough, 2010; Piedmont, McCrae, Riemann, &

Angleitner, 2000). Invalid profiles identified through scales measuring the consistency of

responding are also frequently associated with weaker validity coefficients (Handel, Ben-Porath,

Tellegen, & Archer, 2010; Wetter, Baer, Berry, Smith, & Larsen, 1992). However, solely

considering rank-order associations among scores does not address important aspects of

construct validity (Mash & Hunsley, 2005). Internal consistencies of scales, mean trait scores,

and the factorial structure of multi-scale inventories may differ between valid and invalid profiles

and are likely to affect clinical utility (Swets, 1986). These statistics can determine whether

validity scales provide useful information to researchers and clinicians.

Inconsistent responding can distort scores on personality inventories (Burchett et al.,

2016). True Response Inconsistency (TRIN) scales on the MMPI-2 family of broadband

psychopathology instruments (e.g., Ben-Porath & Tellegen, 2008; Butcher et al., 2001), and the

normal-range Multidimensional Personality Questionnaire (MPQ; Tellegen & Waller, 2008)

detect fixed response patterns through endorsement of oppositely keyed item pairs in the same

direction. These inventories’ Variable Response Inconsistency (VRIN) scales assess random or

careless responding by evaluating endorsement of opposite extremes to highly correlated pairs of

VALIDITY OF MPQ-BF INCONSISTENCY SCALES 4

items that are keyed in the same direction (though the MMPI-2’s VRIN also has some same-

direction item pairs). Inconsistent responding can elevate scores on content-based validity scales,

such as those that assess either positive or negative self-presentation (Burchett et al., 2016).

Thus, profile inconsistency is associated with other forms of potentially deceptive responding.

In this study, we examined how profiles on the brief form of the MPQ (Patrick, Curtin, &

Tellegen, 2002) that were identified as invalid compared to those that were not. The MPQ-BF

consists of 11 primary trait scales that comprise three orthogonal higher-order components.

Positive Emotionality (PEM) comprises the primary traits of Well-being (feeling cheerful and

optimistic), Social Potency (enjoys dominant roles in the spotlight), Achievement (hard-working

and ambitious), and Social Closeness (interpersonally warm with many friends). Negative

Emotionality (NEM) comprises the primary traits of Stress Reaction (feeling anxious, moody,

and tense), Alienation (suspicious of and mistreated by others), and Aggression (enjoys violent

entertainment and willing to mistreat others for an advantage). Behavioral Constraint (CON)

comprises the primary traits of Control (preference for planning instead of acting), Harm

Avoidance (preference for dull experiences over thrilling and unsafe ones), and Traditionalism

(behavior influenced by parental, societal, and religious norms). The primary trait of Absorption

(proneness to cognitive involvement in sensory and imaginative experiences) does not load on

any specific higher-order component, but it is approximately equally related to PEM and NEM.

The MPQ-BF has three additional validity scales, two of which (VRIN and TRIN, as

noted above) detect inconsistent responding. It also has an Unlikely Virtues scale that assesses

moralistic (claiming social/intellectual status and managing agentic presentation) and egoistic

(denying socially deviant faults and managing communal presentation) self-deception (Paulhus
VALIDITY OF MPQ-BF INCONSISTENCY SCALES 5

& John, 1998). Judging profile validity with scales like Unlikely Virtues does not appreciably

alter criterion-related validity or clinical judgments about patients (Barthlow, Graham, Ben-

Porath, Tellegen, & Mcnulty, 2002). Rather, scales assessing self-deception are saturated with

content-related variance associated with psychological health; statistically controlling for them

often reduces criterion-related validity of personality scores (Paulhus, 1991). Thus, many

personality theorists have argued that scales like Unlikely Virtues should not be considered

genuine validity scales (Uziel, 2014). We omitted analyses involving Unlikely Virtues and

focused on invalid profiles due to inconsistent responding.

We used standard VRIN and TRIN cutoffs to identify invalid MPQ-BF profiles from

participants in an emergency department waiting room and compare invalid MPQ-BF profiles

with valid MPQ-BF profiles from participants in the same setting. This design allowed us to

examine invalid profiles occurring in an applied setting rather than with parametrically varied

proportions of random or fixed responding, as in (Burchett et al., 2016). We examined the

differences between these profile groups with reference to internal characteristics of the MPQ-

BF (i.e., internal consistencies, descriptive statistics, and component structure) and to the

differences in correlations among primary trait scales and demographic variables. We considered

the ability of MPQ-BF inconsistency scales to detect invalid profiles using analyses of the test

itself as well as its scores’ relationships with criterion-related variables.

Method

Participants were 1,258 adults present in the emergency department waiting area of a

private, urban, academic hospital. A total of 94 participants (7.5%) responded to fewer than 75%

of items and were excluded from further analysis, and 6 participants did not have sufficient data
VALIDITY OF MPQ-BF INCONSISTENCY SCALES 6

to compute scores on all 11 MPQ-BF primary trait scales. Vanderbilt University’s IRB approved

this study (protocol number 061251).

Of the 1,158 complete MPQ-BF profiles, 1,026 were valid; 132 profiles (11.5%) were

invalid due to extreme validity scale scores assessing inconsistent responding. Profiles were

invalid due to T scores > 78 on VRIN (n = 37; 28.0%), TRIN T scores > 75 (n = 32; 24.2%) or <

25 (n = 27; 20.5%), or a combination of both a T score on VRIN > 70 and a T score on TRIN

either > 70 (n = 20; 15.2% or < 30 (n = 16; 12.1%). Profiles identified as invalid solely for

having high VRIN scores were analyzed as “random” profiles, profiles invalid for having high

TRIN scores were classified as “acquiescent” profiles (n = 52), and profiles invalid for having

low TRIN scores were considered “counteracquiescent” profiles (n = 43). VRIN and TRIN

scores correlated -.04 (p = .148). Our data and analysis script are provided for reproducibility.

Participants were 35.5 years of age on average (SD = 12.7), and our sample was 57%

female. The racial and ethnic composition of the sample was 57.8% White, 35.8% Black, 2.8%

Hispanic, 2.8% Asian, 1.0% Pacific Islander, and 0.8% Native American, with 0.3% opting not

to report racial information (1% reported multiple races). Participants provided data using

netbook computers they checked out from a table that a research assistant staffed in the waiting

room. These netbooks administered demographic questions listed below before the MPQ-BF. A

two-second delay was imposed between responses to reduce the likelihood of terminating the

screening quickly to obtain the $5 cash compensation provided when the netbook was returned.

Participants inputted the largest number of alcoholic drinks they had consumed within

any 24 hour period in their lifetimes, the number of alcoholic drinks they consumed on average

per week, the number of caffeinated drinks they consumed on average each day, and their current
VALIDITY OF MPQ-BF INCONSISTENCY SCALES 7

and lifetime levels of nicotine use per day. Nicotine use was coded as follows: 0 = none, 1 = 10

cigarettes or less, 2 = 11-20 cigarettes, 3 = 21-30 cigarettes, 4 = 31 cigarettes or more. They also

reported on whether they had ever received treatment for a psychiatric diagnosis, their education

level along with that of their mother and father, whether they were currently employed, and their

yearly household income. Education level was coded as follows: 1 = never attended high school,

2 = attended high school but did not graduate, 3 = graduated from high school/received GED, 4 =

attended college but did not graduate, 5 = received 2-year college degree, 6 = received 4-year

college degree, 7 = received Master’s degree, 8 = received doctoral degree.

For random, acquiescent, and counteracquiescent profiles, we compared Cronbach’s α

values for each primary trait using cocron in R (http://comparingcronbachalphas.org). We

conducted independent samples t tests for scores each primary trait to examine how mean levels

of primary traits differed between those with invalid and valid profiles. We adjusted the degrees

of freedom in those analyses because many scores in the invalid profile groups had reduced

variance relative to the valid profile group according to Levene’s test (see Table 1). These

analyses had 80% power to detect population effect size ds from .40 (acquiescent responding)

to .47 (random responding).

Following (Patrick et al., 2002), we examined the component structures of the MPQ-BF

primary traits with principal component analyses (albeit followed by promax rotation to examine

component correlations) in each subsample separately. We computed congruence coefficients (rc)

to compare the correspondence of each component in this sample with the cross-validation

sample in (Patrick et al., 2002). Across domains, experts judge congruence coefficients of .70-.84

to represent “poor” congruence, those between .85-.94 indicate “fair” congruence, and those of .
VALIDITY OF MPQ-BF INCONSISTENCY SCALES 8

95 and above provide evidence for “good” congruence (Lorenzo-Seva & ten Berge, 2006). In

addition to hewing closely to (Patrick et al., 2002) data analytic strategy, we also chose to use

PCA instead of CFA because CFAs make strong assumptions about all non-substantive loadings

being zero that often do not hold (Borkenau & Ostendorf, 1990). Finally, we tested the

differences between valid and invalid profiles in their criterion-related correlations between

MPQ-BF primary trait scales and relevant demographic features. These analyses had 80% power

to detect a population correlation difference of .30 between valid and invalid profiles.

Results

Table 1 displays the internal consistencies, descriptive statistics, and component structure

of the MPQ-BF primary trait scales for participants with valid and invalid profiles. Compared to

primary trait scores for valid profiles, internal consistencies were significantly lower for 10/11

random profile scores, 8/11 acquiescent profile scores, and 2/11 counteracquiescent profile

scores. Likewise, variability was reduced for 8/11 random profile scores, 6/11 acquiescent profile

scores, and 2/11 counteracquiescent profile scores. Invalid profiles had substantially distorted

scores compared to valid profiles; scores on Well-being, Achievement, Alienation, and

Absorption differed from valid profiles across all three invalid profile groups. Random profiles

had lower scores on 3/11 primary traits (Cohen’s d range [-0.34, -0.53]) and higher scores on

Alienation (Cohen’s d = 0.36) and Aggression (Cohen’s d = 0.57). Acquiescent profiles had

higher scores on 6/11 primary traits (Cohen’s d range [0.24, 0.95]) and lower scores on Social

Closeness (Cohen’s d = -.43) and Control (Cohen’s d = -0.28). Counteracquiescent profiles had

lower scores on 7/11 primary traits (Cohen’s d range [-0.53, -1.43]). When data were pooled

across invalid profile groups, results largely mirrored those for the random response style group.
VALIDITY OF MPQ-BF INCONSISTENCY SCALES 9

The structure of the MPQ-BF for valid profiles largely paralleled that in (Patrick et al.,

2002); the first six eigenvalues in this analysis were 2.80, 1.98, 1.43, 0.87, 0.79, and 0.75. All

components showed fair congruence with normative MPQ-BF data. In contrast, the structure of

the MPQ for invalid profiles differed markedly from previous component structures. The first six

eigenvalues for random profiles were 2.67, 1.73, 1.32, 1.27, 1.08, and 0.87; those for acquiescent

profiles were 2.88, 1.75, 1.30, 1.04, 0.97, and 0.82; and those for counteracquiescent profiles

were 3.42, 2.70, 1.47, 0.77, 0.64, and 0.57. Random profiles’ PEM, acquiescent profiles’ CON,

and counteracquiescent profiles’ NEM components showed fair congruence with normative data.

However, the other component scores had poor or worse congruence to normative data. When all

three invalid profile groups were analyzed together, the resultant structure was remarkably

similar to the counteracquiescent group’s (PEM rc = .98, NEM rc = .98, CON rc = .89). These

components were also more strongly correlated (inter-component rs range [.19, .39]) than those

from valid profiles (inter-component rs range [-.17, -.10]).

In addition to having the most coherent component structure across invalid profile

groups, scales comprising NEM had the strongest average correlations with demographic

variables (median |r| range [.15, .17]; median |r| for other primary trait scales = .09). Thus, Table

2 gives the correlations of Stress Reaction, Alienation, and Aggression scores from valid and

invalid MPQ-BF profiles with substance use, psychiatric history, and socioeconomic variables.

Many more correlations were statistically significant in the valid profile group due to its

substantially greater power to detect correlations different from zero. Nevertheless, using a

Sidak-corrected p value of .0170 to account for analyzing 3 primary trait scales at once,

correlations in the valid profile group were more often of larger magnitude than those in the
VALIDITY OF MPQ-BF INCONSISTENCY SCALES 10

invalid profile group. This pattern was significant for Stress Reaction (binomial sign test p = .

006) and trended toward significance for Alienation and Aggression (binomial sign test ps = .

032). Four out of 33 correlations differed significantly between the groups, which was more than

would be expected by chance alone, binomial sign test p = .002. Current employment was

correlated with Stress Reaction scores in the valid but not invalid profile group, a pattern that

was also evident for the relationships between participants’ education levels and both Alienation

and Aggression. Aggression also correlated positively with lifetime cigarette use in the valid

profile group, but this correlation was negative in the invalid profile group.

Discussion

In this study, invalid MPQ-BF profiles were less internally consistent, less coherence in

the components, had lower scale variance, and had different means than valid profiles. Their

correlations with nicotine use, education, and employment were also reduced or of the opposite

direction than those for valid profiles. Our results were consistent with prior work suggesting

that invalid profiles demonstrate less factor coherence and lower internal reliability (Kam &

Meyer, 2015; Sherman, Krug, & Birenbaum, 1979) with some reduction in correlations (Archer,

Fontaine, & McCrae, 1998). Thus, inconsistency-identified profiles evidenced multiple threats to

internal validity. Additionally, invalid profiles showed substantially reduced external validity,

even though low αs need not substantially suppress criterion-related validity (Schmitt, 1996).

Though the VRIN scale does not assess specific personality constructs, random profiles

had lower scores traits associated with PEM and CON and higher scores on NEM-related traits.

This pattern of scores represents the inverse of moralistic self-deceptive responding (Paulhus &

John, 1998), indicating that random responding distorted scores in potentially substantial ways.
VALIDITY OF MPQ-BF INCONSISTENCY SCALES 11

Acquiescent responding tended to increase primary trait scores on invalid profiles, but scores on

Social Closeness and Control were lower on invalid than valid profiles. The latter two primary

traits are among the most balanced in their keying on the MPQ-BF, as most primary traits have

no more than 2/12 reverse-coded items. In contrast, 6 Social Closeness items are reverse-coded,

as are 4 Control items. Thus, true responses on reverse-coded items may have lowered their

scores. However, Harm Avoidance also has 6 reverse-coded items, and its mean was not affected

by acquiescent responding, so coding alone does not explain the associations of Social Closeness

and Harm Avoidance with lower scores during acquiescence. Counteracquiescent responding

lowered primary trait scores while preserving most scores’ internal consistencies (with the

exceptions of Social Potency and Social Closeness). Thus, each type of response style had

unique effects on trait scores (Handel et al., 2010). Clinical assessments should include separate

scales that assess random and fixed responding, unlike the PAI (Morey, 2007) and the PPI-R

(Lilienfeld & Widows, 2005), which blend both response styles into single inconsistency scales.

These results indicate that validity scales measuring the consistency of responses in

normal-range personality assessment have utility in both group and individual level decision-

making. Inconsistently endorsed profiles still had sufficient true score variance to permit

reasonable inferences to be drawn about the relationships among scores. Even so, scores from

invalid profiles typically yielded no relationships between personality traits and demographics,

whereas valid profiles demonstrated small but significant relationships with demographic

characteristics. Thus, assuming similar component structures, group-level research may need to

concern itself with the apparent consistency of responses. In addition, clinicians must make

inferences about a specific individual’s personality based on the individual’s score. These
VALIDITY OF MPQ-BF INCONSISTENCY SCALES 12

comparisons are affected by changes in the mean and shape of score distributions (Youngstrom,

2013). Inconsistency scales may help separate data from which reasonably reliable inferences

can be made from those that are psychometric dross, especially when considering random or

acquiescent response styles. It is unlikely that inferences made about scales that are saturated

with less than half of true score variance and that do not cohere would survive Daubert tests of

evidential admissibility in forensic proceedings. Nor is it likely that invalid profiles, with

reduced internal consistency and varying mean scores, would have similar levels of diagnostic

utility. Therefore, the utility of inconsistency-based validity scales is likely high.

This study was conducted in the waiting room of a busy emergency department with

people who were seeking treatment or associates of individuals seeking treatment. The increased

potential for distraction from using an atypical assessment environment could have increased the

rate of profile invalidity. However, our rate of profile invalidity was not above rates seen in

undergraduate samples (Sellbom & Ben-Porath, 2005), and our sampling methodology allowed

for a larger, more diverse sample than is typically explored with the MPQ. The number of

specific invalid profiles likely means that the PCAs for each invalid profile group were under-

powered. However, the overall invalid profile analyses were well-powered (MacCallum,

Widaman, Zhang, & Hong, 1999; Mundfrom, Shaw, & Ke, 2005). On the MMPI-2-RF, scores on

inconsistency scales are related to inconsistent reporting of symptoms on interview and reduced

intelligence; VRIN scores are also associated with education, and TRIN scores are additionally

associated with symptoms of depression and anxiety (Gu, Reddy, Green, Belfi, & Einzig, in

press). Similar work should be done with the MPQ’s validity scales to investigate reasons that

invalid profiles be generated. Furthermore, we only assessed profile invalidity with the MPQ’s
VALIDITY OF MPQ-BF INCONSISTENCY SCALES 13

inconsistency scales. It is unclear the degree to which different inventories’ validity scales – even

those that ostensibly measure the same types of invalid response styles – identify the same

individuals as producing invalid profiles (Nikolova, Hendry, Douglas, Edens, & Lilienfeld,

2012). Future work should examine whether specific inventories’ validity scales are particularly

good at identifying internally and externally invalid profiles, especially with respect to clinically

relevant criteria. Finally, additional studies should include criteria targeted to the constructs each

primary trait measures to assess the differential external validity of valid and invalid profiles.

In contrast to inconsistency-based validity scales, other personality inventories include

alternate measures of deceptive responding. In cases where feigning healthy is a risk, future scale

development might build on the MMPI scales that evaluate superlative self-presentation (Butcher

& Han, 1995), which assesses moralistic bias more purely (Detrick & Chibnall, 2008). The

evolution of scales designed to assess negative self-presentation may point the way to further

developments in this area. In the more typical case of negative self-deception, presenting as more

ill, the MMPI family of scales offer two alternative frameworks for scale development. The Fp

scale (Arbisi & Ben-Porath, 1995) uses infrequently endorsed items in inpatient units to detect

potential malingering in the psychiatrically ill. Conversely, the RBS scale (Gervais, Ben-Porath,

Wygant, & Green, 2007) comprises items that predict failing at least one cognitive effort test to

detect potential malingering in neurocognitively impaired patients. Outside of the MMPI family

of scales, the PPI-R Deviant Responding scale represents another method for identifying

inaccurate responding using items that are logically guaranteed to be true or false

(Marcus, Church, O’Connell, & Lilienfeld, 2016). Nevertheless, our study highlights the need to

flag invalid profiles in research studies and clinical assessments.

VALIDITY OF MPQ-BF INCONSISTENCY SCALES 14

References

Arbisi, P. A., & Ben-Porath, Y. S. (1995). An MMPI-2 infrequent response scale for use with

psychopathological populations: The Infrequency-Psychopathology Scale, F(p). Psychological

Assessment, 7(4), 424–431. https://doi.org/10.1037/1040-3590.7.4.424

Archer, R. P., Fontaine, J., & McCrae, R. R. (1998). Effects of two MMPI-2 validity scales on basic scale

relations to external criteria. Journal of Personality Assessment, 70(1), 87–102.

https://doi.org/10.1207/s15327752jpa7001_6

Barthlow, D. L., Graham, J. R., Ben-Porath, Y. S., Tellegen, A., & Mcnulty, J. L. (2002). The

appropriateness of the MMPI-2 K correction. Assessment, 9(3), 219–229.

https://doi.org/10.1177/1073191102009003001

Ben-Porath, Y. S., & Tellegen, A. (2008). Minnesota Multiphasic Personality Inventory-2 (Restructured

Form) manual for administration, scoring, and interpretation. Minneapolis, MN: University of

Minnesota Press.

Borkenau, P., & Ostendorf, F. (1990). Comparing exploratory and confirmatory factor analysis: A study

on the 5-factor model of personality. Personality and Individual Differences, 11(5), 515–524.

https://doi.org/10.1016/0191-8869(90)90065-Y

Burchett, D., Dragon, W. R., Smith Holbert, A. M., Tarescavage, A. M., Mattson, C. A., Handel, R. W., &

Ben-Porath, Y. S. (2016). “False feigners”: Examining the impact of non-content-based invalid

responding on the Minnesota Multiphasic Personality Inventory-2 Restructured Form content-

based invalid responding indicators. Psychological Assessment, 28(5), 458–470.

https://doi.org/10.1037/pas0000205

Butcher, J. N., Graham, J. R., Ben-Porath, Y. S., Tellegen, A., Dahlstrom, W. G., & Kaemmer, B. (2001).

Minnesota Multiphasic Personality Inventory – 2. Minneapolis, MN: University of Minnesota

Press.
VALIDITY OF MPQ-BF INCONSISTENCY SCALES 15

Butcher, J. N., & Han, K. (1995). Development of an MMPI-2 scale to assess the presentation of self in a

superlative manner: The S Scale. In J. N. Butcher & C. D. Spielberger (Eds.), Advances in

personality assessment, Vol. 10 (pp. 25–50). Hillsdale, NJ, England: Lawrence Erlbaum

Associates, Inc.

Caldwell-Andrews, A., Baer, R. A., & Berry, D. T. R. (2000). Effects of response sets on NEO-PI-R

scores and their relations to external criteria. Journal of Personality Assessment, 74(3), 472–488.

https://doi.org/10.1207/S15327752JPA7403_10

Detrick, P., & Chibnall, J. T. (2008). Positive response distortion by police officer applicants: Association

of Paulhus Deception Scales with MMPI-2 and Inwald Personality Inventory validity scales.

Assessment, 15(1), 87–96. https://doi.org/10.1177/1073191107306082

Gervais, R. O., Ben-Porath, Y. S., Wygant, D. B., & Green, P. (2007). Development and validation of a

Response Bias Scale (RBS) for the MMPI-2. Assessment, 14(2), 196–208.

https://doi.org/10.1177/1073191106295861

Gu, W., Reddy, H. B., Green, D., Belfi, B., & Einzig, S. (in press). Inconsistent responding in a criminal

forensic setting: An evaluation of the VRIN-r and TRIN-r scales of the MMPI–2–RF. Journal of

Personality Assessment, 1–11. https://doi.org/10.1080/00223891.2016.1149483

Handel, R. W., Ben-Porath, Y. S., Tellegen, A., & Archer, R. P. (2010). Psychometric functioning of the

MMPI-2-RF VRIN-r and TRIN-r scales with varying degrees of randomness, acquiescence, and

counter-acquiescence. Psychological Assessment, 22(1), 87–95. https://doi.org/10.1037/a0017061

Kam, C. C. S., & Meyer, J. P. (2015). How careless responding and acquiescence response bias can

influence construct dimensionality: The case of job satisfaction. Organizational Research

Methods, 18(3), 512–541. https://doi.org/10.1177/1094428115571894

Lilienfeld, S. O., & Widows, M. R. (2005). Psychopathic Personality Inventory – Revised. Lutz, FL:

PAR.
VALIDITY OF MPQ-BF INCONSISTENCY SCALES 16

Lorenzo-Seva, U., & ten Berge, J. M. F. (2006). Tucker’s congruence coefficient as a meaningful index of

factor similarity. Methodology, 2(2), 57–64. https://doi.org/10.1027/1614-2241.2.2.57

MacCallum, R. C., Widaman, K. F., Zhang, S., & Hong, S. (1999). Sample size in factor analysis.

Psychological Methods, 4(1), 84–99. https://doi.org/10.1037/1082-989X.4.1.84

Marcus, D. K., Church, A. S., O’Connell, D., & Lilienfeld, S. O. (2016). Identifying Careless Responding

With the Psychopathic Personality Inventory-Revised Validity Scales. Assessment.

https://doi.org/10.1177/1073191116641507

Mash, E. J., & Hunsley, J. (2005). Evidence-based assessment of child and adolescent disorders: Issues

and challenges. Journal of Clinical Child & Adolescent Psychology, 34(3), 362–379.

https://doi.org/10.1207/s15374424jccp3403_1

McGrath, R. E., Mitchell, M., Kim, B. H., & Hough, L. (2010). Evidence for response bias as a source of

error variance in applied assessment. Psychological Bulletin, 136(3), 450–470.

https://doi.org/10.1037/a0019216

Morey, L. C. (2007). Personality Assessment Inventory professional manual (2nd ed.). Lutz, FL: PAR.

Mundfrom, D. J., Shaw, D. G., & Ke, T. L. (2005). Minimum sample size recommendations for

conducting factor analyses. International Journal of Testing, 5(2), 159–168.

https://doi.org/10.1207/s15327574ijt0502_4

Nikolova, N. L., Hendry, M. C., Douglas, K. S., Edens, J. F., & Lilienfeld, S. O. (2012). The

inconsistency of inconsistency scales: a comparison of two widely used measures. Behavioral

Sciences & the Law, 30(1), 16–27. https://doi.org/10.1002/bsl.1996

Patrick, C. J., Curtin, J. J., & Tellegen, A. (2002). Development and validation of a brief form of the

Multidimensional Personality Questionnaire. Psychological Assessment, 14(2), 150–163.

Paulhus, D. L. (1991). Measurement and control of response bias. In J. P. Robinson, P. R. Shaver, & L. S.

Wrightsman (Eds.), Measures of personality and social psychological attitudes (pp. 17–59). San

Diego, CA, US: Academic Press.

VALIDITY OF MPQ-BF INCONSISTENCY SCALES 17

Paulhus, D. L., & John, O. P. (1998). Egoistic and moralistic biases in self-perception: The interplay of

self-deceptive styles with basic traits and motives. Journal of Personality, 66(6), 1025–1060.

https://doi.org/10.1111/1467-6494.00041

Piedmont, R. L., McCrae, R. R., Riemann, R., & Angleitner, A. (2000). On the invalidity of validity

scales: evidence from self-reports and observer ratings in volunteer samples. Journal of

Personality and Social Psychology, 78(3), 582–593.

Rohling, M. L., Larrabee, G. J., Greiffenstein, M. F., Ben-Porath, Y. S., Lees-Haley, P., Green, P., &

Greve, K. W. (2011). A misleading review of response bias: comment on McGrath, Mitchell,

Kim, and Hough (2010). Psychological Bulletin, 137(4), 708-712-715.

https://doi.org/10.1037/a0023327

Schmitt, N. (1996). Uses and abuses of coefficient alpha. Psychological Assessment, 8(4), 350–353.

https://doi.org/10.1037/1040-3590.8.4.350

Sellbom, M., & Ben-Porath, Y. S. (2005). Mapping the MMPI-2 Restructured Clinical scales onto normal

personality traits: evidence of construct validity. Journal of Personality Assessment, 85(2), 179–

187. https://doi.org/10.1207/s15327752jpa8502_10

Sherman, J. L., Krug, S. E., & Birenbaum, M. (1979). Checking the reliability and validity of HSPQ

profiles. Journal of Personality Assessment, 43(6), 644–647.

https://doi.org/10.1207/s15327752jpa4306_15

Swets, J. A. (1986). Indices of discrimination or diagnostic accuracy: Their ROCs and implied models.

Psychological Bulletin, 99(1), 100–117. https://doi.org/10.1037/0033-2909.99.1.100

Tellegen, A., & Waller, N. G. (2008). Exploring personality through test construction: Development of the

Multidimensional Personality Questionnaire. In G. J. Boyle, G. Matthews, & D. H. Saklofske

(Eds.), The SAGE handbook of personality theory and assessment, Vol 2: Personality

measurement and testing (pp. 261–292). Thousand Oaks, CA, US: Sage Publications, Inc.
VALIDITY OF MPQ-BF INCONSISTENCY SCALES 18

Uziel, L. (2014). Impression management (“lie”) scales are associated with interpersonally oriented self-

control, not other-deception. Journal of Personality, 82(3), 200–212.

https://doi.org/10.1111/jopy.12045

Wetter, M. W., Baer, R. A., Berry, D. T. R., Smith, G. T., & Larsen, L. H. (1992). Sensitivity of MMPI-2

validity scales to random responding and malingering. Psychological Assessment, 4(3), 369–374.

https://doi.org/10.1037/1040-3590.4.3.369

Youngstrom, E. A. (2013). Future directions in psychological assessment: Combining evidence-based

medicine innovations with psychology’s historical strengths to enhance utility. Journal of

Clinical Child and Adolescent Psychology, 42(1), 139–159.

https://doi.org/10.1080/15374416.2012.736358
 VALIDITY OF MPQ-BF INCONSISTENCY SCALES 19

Table 1

Descriptive Statistics and Component Structure of Valid and Invalid Multidimensional

Personality Questionnaire Profiles

MPQ primary trait Valid profiles (n = 1,026) Random profiles (n = 37) Acquiescent profiles (n = 52) Counteracquiescent profiles (n = 43)
Component loadings Component loadings Component loadings Component loadings
a M (SD) 1 2 3 a M (SD) 1 2 3 a M (SD) 1 2 3 a M (SD) 1 2 3
Well-being .83 49.2 (10.6) -.30 .71 .05 .58 44.7 (7.3) .03 .78 .07 .71 53.4 (8.8) .69 .05 -.19 .83 38.7 (10.4) .87 .01 .04
Social Potency .72 54.1 (7.1) -.23 .53 -.39 .43 52.4 (5.3) -.11 .12 -.60 -.01 55.3 (2.9) .17 .02 -.76 .47 53.0 (4.1) .04 -.05 .92
Achievement .74 52.8 (8.7) .05 .71 .24 .32 49.9 (5.5) -.25 .20 .56 .32 54.8 (4.1) .77 -.16 .21 .61 44.8 (7.5) .66 -.11 .39
Social Closeness .82 47.2 (9.8) -.56 .40 -.04 .30 45.0 (5.0) .10 .75 -.11 .21 43.1 (3.7) .23 -.42 -.46 .55 47.0 (6.7) .42 -.61 .07
Stress Reaction .84 53.3 (10.1) .83 -.02 .01 .45 53.4 (5.3) .79 -.01 .26 .48 61.7 (5.8) -.03 .77 .21 .72 47.6 (7.5) .20 .91 -.31
Alienation .84 63.0 (9.2) .83 .01 -.01 .20 66.2 (3.2) .75 -.07 -.18 .64 71.6 (5.3) .04 .87 -.03 .81 58.1 (8.2) .11 .85 .11
Aggression .85 53.6 (11.4) .45 .07 -.54 .62 60.1 (7.7) .65 -.49 .07 .75 64.2 (8.5) .16 .63 -.34 .81 52.2 (8.8) -.12 .65 .49
Control .69 47.1 (9.2) -.12 .21 .70 .35 44.7 (7.0) -.04 -.11 .05 .11 44.5 (4.7) .51 .28 .17 .59 40.1 (7.5) .67 -.38 .05
Harm Avoidance .70 46.0 (8.5) -.10 -.19 .59 -.08 44.1 (4.8) -.10 -.10 -.36 .48 44.0 (6.7) .16 .00 .58 .66 44.6 (7.4) .61 -.10 -.33
Traditionalism .61 47.6 (7.2) .30 .19 .67 .34 45.2 (5.8) .71 .47 .08 .15 48.6 (5.1) .36 -.07 .61 .39 41.7 (6.1) .81 .27 -.16
Absorption .71 55.5 (8.7) .48 .65 -.01 .02 50.9 (5.0) .61 .16 -.13 .41 63.6 (6.1) .68 .09 .01 .68 43.1 (9.0) .65 .40 .15

Congruence coefficients .94 .90 .94 .79 .85 .34 .80 .78 .85 .69 .92 -.64

Note. For each primary trait scale, underlined alphas or italicized means and standard deviations

differ significantly from those of valid profiles (Sidak-corrected p <.00465 to control family-

wise error at .05 across 11 primary trait scales). Component loadings |.40| or greater are

displayed in bold.
 VALIDITY OF MPQ-BF INCONSISTENCY SCALES 20

Table 2

Substance Use, Psychiatric History, and Socioeconomic Status Correlates of Valid and Invalid

Multidimensional Personality Questionnaire Profiles

SR AL AG
Maximum alcoholic drinks in 1 day .11* / .09 .10 / .15 .28**/ .10
Average alcoholic drinks per week .01 / .19 .07 / .10 .19**/ .07
Average caffeinated drinks per day .17**/ .04 .18**/ .07 .19**/ .03
Current cigarette use .23**/ .17 .15**/ .03 .13**/-.04
Maximum lifetime cigarette use .26**/ .06 .17**/ .00 .13**/-.13
History of psychiatric conditions .34**/ .16 .22**/ .12 .10* / .01
Education level: self -.18**/-.06 -.34**/-.08 -.27**/-.06
Education level: mother -.15**/ .02 -.16**/ .02 -.06 / .02
Education level: father -.13**/ .06 -.15**/-.03 -.09 /-.09
Currently employed? -.18**/ .05 -.23**/-.03 -.17**/-.03
Household income -.18**/ .06 -.26**/-.08 -.17**/-.05

Note. Correlations to the left of the slash represent those for valid profiles; ns range from 685

(maximum alcoholic drinks in one day) to 1,022 (history of psychiatric conditions). Correlations

to the right of the slash represent those for invalid profiles; ns range from 57 (maximum

alcoholic drinks in one day) to 131 (history of psychiatric conditions). SR = Stress Reaction, AL

= Alienation, AG = Aggression. * p < .00465 (Sidak-corrected p value to control family-wise

error at .05 across 11 demographic variables), ** p < .001. Underlined correlations differ

significantly between valid and invalid profile groups.

View publication stats