Received: 29 December 2019 Revised: 4 February 2020 Accepted: 27 February 2020

DOI: 10.1002/cpp.2442

RESEARCH ARTICLE

When's a story a story? Determining interpretability of Social

Cognition and Object Relations Scale-Global ratings on
Thematic Apperception Test narratives

Michelle B Stein1 | Solara Calderon2 | Jared Ruchensky1 | Christina Massey1 |

Jenelle Slavin-Mulford3 | Wei-Jean Chung1 | Laura A. Richardson1 | Mark A. Blais1

1
Department of Psychiatry, Massachusetts
General Hospital and Harvard Medical School,
Boston, MA, USA
2
Mental Health Service, VA San Diego
Healthcare System, San Diego, CA, USA
3
Department of Psychology, Augusta
University, Augusta, GA, USA
Correspondence
Michelle B. Stein, PhD, Psychological
Evaluation and Research Laboratory (PEaRL),
Massachusetts General Hospital and Harvard
Medical School, One Bowdoin Square, 7th
floor, Boston, MA 02114, USA.
Email: mstein3@mgh.harvard.edu
The Social Cognition and Object Relations Scale-Global Rating Method (SCORS-G)
measures the quality of object relations in narrative material. The reliability and valid-
ity of this measure have been well established. However, a psychometric oddity of
this scale is that default ratings are given to select dimensions when the relevant con-
struct is not present. This can result in narrative ‘blandness’ and may impact clinical
findings. The aim of these two studies is to understand these phenomena both psy-
chometrically and clinically. In the first study, we identified 276 outpatients who had
SCORS-G ratings for TAT Cards 1, 2, 3BM, and 14, set criteria for narrative ‘bland-
ness’ across all eight dimensions, and examined group differences. In Study 2, we
used a subset (N = 99) of Study 1 and examined how percentage of formal default
ratings for Emotional Investment in Values and Moral Standards (EIM), Experience
and Management of Aggressive Impulses (AGG), Self-Esteem(SE), and Identify and
Coherence of Self (ICS) impacted robustness of correlations across tests of intelli-
gence, psychopathology, and normal personality functioning. Taken together, we
identified clinical characteristics of patients who are more likely to produce ‘bland’
narratives and increased percentages of formal default ratings. Also, an excess of
default ratings per protocol impacts robustness of correlations and weakens signifi-
cant correlations. As cut-off scores increase (>25% and >28.12%), the likelihood of
being able to interpret EIM, AGG, SE, and ICS decreases. Psychometric and clinical
implications are discussed.

KEYWORDS

default ratings, object relations, performance-based tasks, SCORS-G, social cognition, TAT

1 | I N T RO D UC TI O N Standards (EIM), Understanding of Social Causality (SC), Experience

The Social Cognition and Object Relations Scale-Global(SCORS-G)
rating method (Stein & Slavin-Mulford, 2018) is a clinician-rated mea-
sure that assesses object relational processes, which mediate interper-
sonal functioning: Complexity of Representations of People (COM),
Affective Quality of Representations (AFF), Emotional Investment in
Relationships (EIR), Emotional Investment in Values and Moral
and Management of Aggressive Impulses (AGG), Self-Esteem(SE), and
Identity and Coherence of Self (ICS). The SCORS-G can be applied to
several narrative sources, the most popular being Thematic Appercep-
tion Test (TAT; Murray, 1943) narratives.
Although reliability, validity, and clinical utility of this measure
have been well established (Stein & Slavin-Mulford, 2018), psycho-
metric oddities are also evidenced. First, no specific criteria have been

Clin Psychol Psychother. 2020;27:567–580. wileyonlinelibrary.com/journal/cpp © 2020 John Wiley & Sons, Ltd. 567
568
set to determine whether there is sufficient information in a narrative
to code it with the SCORS-G. All narratives are coded regardless of
richness of object relational content or narrative length. Second, some
SCORS-G dimensions possess ‘default rating rules’. These rules require
the rater to assign a specific score when a narrative lacks scorable
content for that particular dimension. For example, when narratives
contain no aggressive content, raters assign a ‘default’ rating of four
for AGG. When a protocol only contains a single individual, raters
assign a two for EIR.
Default rating rules can sometimes contribute to challenges. For
some dimensions, a dimension's default rating rule and a rating based
on the actual narrative may result in the same score with two mean-
ings. For example, raters assign a score of four for EIM based on con-
tent of the moral quality of a narrative (i.e., scored based on content)
or because the narrative contained moral concerns that was bland
(i.e., scored based on the default rule for this dimension).
Although default rating rules create problems, they are necessary
as some narratives are ‘bland’, meaning they contain little ratable con-
tent. However, the SCORS-G does not provide raters with methods
for identifying bland, unusual, or questionable narratives or protocols.
Instead, default rating rules are given in these situations. When an
individual produces several bland narratives, their score will be driven
more by default rules (and less by the actual content of the narra-
tives). In research studies, this may attenuate associations and reduce
effect sizes. In clinical practice, a SCORS-G profile based on several
bland narratives may not accurately capture the patient's functioning.
Currently, it is unclear how frequently bland protocols occur or the
extent to which they may impact findings. The purpose of this paper
is to begin to understand and address these issues.
Numerous TAT rating systems have been developed. One of the
reasons for this is that there is not a unified scoring system to rate the
TAT, as there is, for instance, with the Rorschach Inkblot Test
(Exner, 2003). Jenkins (2008) edited a comprehensive text detailing
these TAT scoring systems. In reviewing these diverse scoring sys-
tems, all highlighted their reliability and administration procedures
(i.e., flexibility of card set, number of cards required, type of adminis-
tration, and use of prompts). Some scoring systems examined relation-
ships between scoring system constructs, word count, and/or Lambda
(Exner, 2003) on the Rorschach (Dana, 2008; Ritzler, 2008; Ronan,
Gibbs, Dreer, & Lombardo, 2008) and how this may or may not impact
scoring.
Most scoring systems have dichotomous as opposed to dimen-
sional variables. If a construct is not present, it is not scored. There is
not a middle category rating like there is with the SCORS-G. In
reviewing these manuals, there is one manual (Karon, 2008) that con-
siders the ratio between scorable and unscorable items in generating a
final score. However, there is very little discussion in these manuals as
to whether or not there is enough scorable material present in a pro-
tocol to interpret and later include in analyses. Reviewing this range of
diverse scoring systems highlights the variability in studying the TAT
and the difficulty in comparing findings across research studies.
Reviewing the literature on narrative-based measures proved
fruitful. We found two clinician-rated systems that quantify this
STEIN ET AL.
Key Practitioner Message
• Patients who produce ‘bland’ SCORS-G ratings on TAT
narratives report higher symptoms associated with drug
use, somatization, and cognitive features of depression,
indicating that these patients may be more emotionally
shut down and cognitively or physically preoccupied.
• Percentage of default ratings per protocol for SCORS-G
dimensions EIM, AGG, SE, and ICS have an impact on
robustness of correlations across measures of intelligence
and psychopathology. As cut-off scores increase (>25%
and >28.12%), the likelihood of being able to interpret
EIM, AGG, SE, and ICS decreases.
• Patients with externalizing psychopathology were more
likely to produce TAT protocols with ≤25% and ≤28.12%
per four card protocol.
• This study emphasizes the importance of examining the
percentage of default ratings per TAT protocol when
interpreting test findings.
process. The first is the Core Conflictual Relationship Theme Method
(CCRT; Luborsky & Crits-Christoph, 1998) and the second is the Com-
prehensive System (Exner, 2003) for the Rorschach Inkblot Test
(Rorschach, 1942). The CCRT identifies maladaptive relationship pat-
terns derived from relationship episodes (RE) elicited from psycho-
therapy narratives and narratives told during the Relationship
Anecdotes Paradigm Interview. The completeness of RE varies widely,
though criteria to identify RE specify that the interaction must have
(a) a beginning, middle, and end; (b) an interpersonal wish (e.g., to be
loved), response of other (e.g., rejecting), and response of self
(e.g., disappointment); and (c) a clear detailing of the patient's interac-
tion with the other person. After the RE has been identified, at least
two judges are asked to rate the RE on its degree of completeness
from 1 (least) to 5 (most). The cut-off for inclusion is ≥2.5. Thus, there
is specific criteria to identify both a RE and CCRT, in addition to
ensuring that the CCRT is rich enough to include in analyses.
Similarly, in Exner's Comprehensive System, there are criteria to
determine when there is sufficient information to code the Rorschach
Inkblot Test (i.e., no card rejection; minimum of 14 responses). Also,
Lambda is used to determine the extent to which an individual pro-
duced a meaningful and interpretable profile (>1.00 reflects a guarded
profile, and interpretations should be reviewed with caution).
To summarize, in reviewing TAT scoring system manuals, test
administration is considered in determining the scoring of narratives.
There is at least one scale that accounts for unscorable narratives
(Karon, 2008) in determining an overall rating. Most TAT rating sys-
tems are dichotomous, so if a select construct is not present, it is not
scored; this contrasts with the SCORS-G that is dimensional and as
such might not come up as a potential problem in other rating sys-
tems. Despite this, there is little information surrounding the extent to
STEIN ET AL.
which a protocol provides sufficient information (i.e., there is enough
of a construct present in the protocol within and across cards) to
interpret the rating(s) obtained. Examining the literature on narrative-
based measures at large provided some examples on how scale devel-
opers produced empirical-based methods for determining interpret-
ability of protocols.
The goals of this project are multi-fold. In Study 1, we want to
understand how frequently ‘bland’ narratives (i.e., as specified by
select scoring on all dimensions) occur in our dataset. Then, we want
to examine clinical differences between patients who tend to produce
‘bland’ narratives with those that do not. Study 2 will use a subset of
Study 1 to delve into how default ratings scattered throughout a TAT
protocol impact clinical findings.
2 | STUDY 1
The aims of Study 1 are to identify the frequency of ‘bland’ narratives
in our dataset and examine differences in patient characteristics for
those that produced ‘bland’ narratives versus those that did not using
a multi-method approach.
2.1 | Method
2.1.1 | Participants
Participants were 276 outpatients who were referred for psycholog-
ical assessment in the department of psychiatry within an academic
medical centre in the north-eastern United States between 2008
and 2018. The average age was 41 years (SD = 15.1), and the aver-
age education level was 14.6 (SD = 2.9) years. There were slightly
more males (n = 144; 52%) than females (n = 132; 48%). Patients
were predominately single (55%) with 26% married and 12%
divorced. Patients were primarily White (86%) with 5.5% self-
identifying as African American, 1.8% as Asian, and 4.8% as His-
panic. The most common referral diagnoses were depressive (42%),
bipolar (18%), and anxiety disorders (17%).
2.1.2 | Procedure
Our assessment clinic maintains a data repository approved by the
institutional review board, which contains assessment and demo-
graphic data for all evaluations conducted. Outpatients received a bat-
tery of standard clinical instruments, including the TAT. The vast
majority of patients completed their psychological assessments in a
single session. After the TAT cards were administered, they were tran-
scribed (written verbatim as patient was creating narrative),
deidentified, and scored by two independent raters using the SCORS-
G. The SCORS-G was used to rate all TAT narratives from each
patient. Raters (M. Stein and J. Slavin-Mulford) are expert raters and
previously completed manualized training on the SCORS-G
569
(Hilsenroth, Stein, & Pinsker, 2007). Good (>.60) to excellent (>.74)
reliability was attained by both raters in previous research (Stein
et al., 2016).
2.1.3 | Materials and measures
Thematic Apperception Test (TAT; Murray, 1943)
Outpatients completed a standard TAT protocol as part of their rou-
tine clinical assessment (see Stein, Slavin-Mulford, Sinclair, Siefert, &
Blais, 2012, Stein et al., 2016, for administration, scoring, and reliabil-
ity training). For this study, we used an abbreviated four card protocol:
(1) (A young boy is contemplating a violin which rests on a table in
front of him.); (2) (Country Scene: in the foreground is a young woman
with books in her hand, in the background a man is working in the
fields and an older woman is looking on); (3BM) (On the floor against
a couch is the huddled form of a boy with his head bowed on his right
arm. Beside him on the floor is a revolver); and (14) (The silhouette of
a man [or woman] against a bright window. The rest of the picture is
totally black) (Murray, 1943).
Social Cognition and Object Relations Scale-Global Rating Method
(SCORS-G; Stein & Slavin-Mulford, 2018; Westen, 1995)
The SCORS-G contains eight dimensions underlying object related-
ness, scored on a 7-point anchored scale. COM evaluates the pres-
ence, degree, and differentiation of the self and other. AFF examines
the emotional and perceptual lens through which one views their envi-
ronment. EIR assesses the depicted level of intimacy and emotional
sharing. EIM measures how a person views others and acts in relation
to morality and compassion for others. SC evaluates the extent to
which the person understands human behaviour, as well as the narra-
tive's coherence and reasoning. AGG explores the person's ability to
tolerate and manage aggression, and SE and ICS assess self-worth as
well as the degree to which a person has an integrated sense of who
he or she is. In Study 1, we set ‘bland’ criteria for all eight dimensions.
Personality Assessment Inventory (PAI; Morey, 1991)
The PAI is a 344 item self-report measure that assesses a broad range
of psychological constructs. It is a reliable and valid measure of psy-
chopathology (Morey, 1991, 1996). In Study 1, we used Negative
Impression Management (NIM), Positive Impression Management
(PIM), PAI Clinical Scales (Somatization [SOM], Anxiety [ANX],
Anxiety-Related Disorders [ARD], Depression [DEP], Mania [MAN],
Paranoia [PAR], Schizophrenia [SCZ], Borderline features [BOR], Anti-
social features [ANT], Alcohol Use [ALC], and Drug Use [DRG]) and
the Mean Clinical Elevation (MCE) score which is the average of the
11 clinical scales.
NEO Five-Factor Inventory (NEO-FFI; Costa &
McCrae, 1989, 1992b)
The NEO-FFI is a 60-itemself-report measure of personality traits
within the general population. This scale measures the ‘Big Five’
dimensions (as defined by Costa & McCrae, 1989, 1992b):
570
Neuroticism (N), Extraversion (E), Openness (O), Agreeableness (A),
and Conscientiousness (C). Correlations between the NEO-FFI and
NEO-PI-R scales range from .77 to .92. Internal consistency values
range from .68 to .86 (Costa & McCrae, 1992a, 1992b). In Study 1, we
used Neuroticism to denote emotional distress.
Rorschach Inkblot Test (Rorschach, 1921/1942)
The Rorschach Inkblot test is a performance-based measure used to
assess psychological functioning. In this study, the Rorschach was
administered and scored using the Exner Comprehensive Sys-
tem (2003; see Malone et al., 2013, for our lab scoring process). For
this study, we used Lambda and Number of Responses to capture
patient's level of engagement/productivity.
Wechsler Abbreviated Scale of Intelligence-2nd edition (WASI-II;
Wechsler, 2011)
The WASI-II is a brief measure of intellectual functioning. It consists
of four subscales, which produce Verbal Comprehension Index, Per-
ceptual Reasoning Index, and Full Scale IQ. The WASI-II has excellent
internal consistency and test–retest reliability. The average stability
coefficients range from .90 to .96 for the Composites
(Wechsler, 2011, p. 117). In Study 1, we used Full Scale IQ (FSIQ),
Verbal Comprehension Index (VCI), and Perceptual Reasoning Index
(PRI).
Psychiatric Diagnostic Screening Questionnaire (PDSQ;
Zimmerman & Mattia, 2001a, 2001b)
The PDSQ is a self-report measure of psychiatric symptoms that
screens for common Axis I disorders (American Psychiatric Associa-
tion, 1994). It yields 13 subscales (Depression, Post-Traumatic Stress,
Bulimia, Obsessive–Compulsive, Panic, Psychosis, Agoraphobia, Social
Phobia, Alcohol, Drug, Generalized Anxiety, Somatization, and Hypo-
chondriasis). Subscales have good to excellent test–retest reliability
ranging from .61 to .93 and high internal consistency (.66 to .94; Per-
key et al., 2017).
2.2 | Results
2.2.1 | SCORS-G interrater reliability
Interrater reliability for this sample is detailed elsewhere (Stein
et al., 2016). ICC (2.2) were reported to be in the ‘excellent’ range
(>.74; Shrout & Fleiss, 1979).
2.2.2 | Identifying ‘bland’ narratives
First, we identified SCORS-G dimensions that had a formal default
rating rules (EIM, AGG, and SE, ICS). EIM, AGG, and SE are provided
default rating rules of 4, and ICS is given a default rating rule of
5 when dimensional content is not present. AFF and EIR were more
challenging to categorize. Regarding AFF, a default rating rule of 4 is
STEIN ET AL.
provided when affective quality is absent or limited. This rating of
4, although technically a default rating rule, fits between the anchor
points of 3 (largely unpleasant) and 5 (mixed, though some positive is
necessary) in theoretically consistent ways. For this study, we did not
want to include only a rating of 4 as a default rating rule because
many of these cards pull for some negative affect, which cannot be
accurately reflected in a rating of 4. To account for this, we chose to
include the range of 3 to 4. This accounts for the patients who
depicted some painful affect but not too much which would be less
reflective of a ‘bland’ rating. For EIR, a default rating rule of 2 is given
when there is only one character depicted. For one-person cards
(1, 3BM, and 14), we included this rating of 2. For both one- and
multiple-person cards (1, 2, 3BM, and 14), we also included the rating
of 3 as in our clinical experience ‘bland’ narratives usually involve
patients describing characters in superficial ways. COM and SC do not
have formal default ratings rules. However, based on clinical observa-
tion, we have found that a rating of 3 reflects minimal information
surrounding internal states and self/other understanding.
Using our clinical judgement, scores in the following ranges for
respective dimensions were considered indicators of possible ‘bland-
ness’: COM (2.5–3.5), AFF (3–4), EIR (2 [one person cards] and 3 [all
cards]), EIM (4), SC (2.5–3.5), AGG (4), SE (4), ICS (4.5–5). A narrative
response was considered ‘bland’ when it fulfilled the above criteria.
After ‘bland’ narratives were identified based on the above criteria,
we reviewed these narratives to ensure they reflected ‘bland’
responses and not based on narrative content (i.e., AGG rating of a
4 represents content present in narrative as opposed to this rating
being reflective of absence of content). When this occurred, these
narratives were moved from the ‘bland’ to ‘non-bland’ group.
2.2.3 | Group differences
As can be observed in Table 1, 78% (N = 215) of protocols did not evi-
dence any ‘bland’ narratives; 18% of protocols exhibited one ‘bland’
narrative and 2.9% exhibited two ‘bland’ narratives per protocol. Less
than 1% of protocols exhibited greater than two ‘bland’ narratives.
When examining group differences, ‘bland’ protocols were identified
as having less than or equal to one bland narrative (22% of sample).
We examined group differences on tests of intellectual ability,
psychopathology, and performance-based measures of personality
(Table 2). Chi-squares were calculated to assess group differences for
TABLE 1 Number of ‘bland’ narratives per four card protocol in
Study 1
Number N Percentage of current sample
0 215 78
1 50 18
2 8 2.9
3 2 <1
4 1 <1
STEIN ET AL. 571

gender. There was no significant difference between the ‘bland’ and
‘non-bland’ group for gender, X2 = (1, N = 276) = 2.26, p = .13.
Independent t-tests were conducted to examine group differ-
ences for the following demographics: age and level of education
(Table 2).
There were no significant group differences in age, education
level, intellectual resources, impression management, or general dis-
tress. Lambda and Number of Responses were trending toward signif-
icance. The ‘bland’ protocol group exhibited higher Lambda and lower
number of responses, both of which are indicative of a guarded and

TABLE 2 Study 1 group differences

Demographics Group N M SD T p-value D

Age 0 215 40.9 15.5 0.34 n.s. .05
1 61 40.2 13.8
Educational level 0 210 14.7 2.7 1.32 n.s. .20
1 58 14.1 3.5
WASI-II
Full scale 0 211 104.2 15.7 0.84 n.s. .12
1 60 102.3 17.2
Verbal comprehension 0 210 105.2 16.8 0.65 n.s. .10
1 60 103.7 15.7
Perceptual reasoning 0 210 101.8 14.6 0.97 n.s. .14
1 60 99.7 17.2
Rorschach
Lambda 0 107 .73 .7 −1.66 .10a .34
1 32 1.0 1.3
Number of responses 0 107 21.8 6.8 1.67 .10a .34
1 32 19.6 6.1
TAT word count
Card 1 0 209 129.1 65.1 1.56 n.s. .23
1 57 114.0 63.2
Card 2 0 209 145.0 69.3 2.57 .01b .38
1 57 119.7 51.3
Card 3BM 0 209 109.8 55.5 3.33 .001b .50
1 57 83.8 38.0
Card 14 0 209 100.3 55.4 2.97 .003b .44
1 57 76.9 39.9
NEO-FFI
Neuroticism 0 170 63.7 12.5 −0.75 n.s. .13
1 41 65.3 12.4
PAI
MCE 0 201 59.6 8.6 −1.47 n.s. .22
1 57 61.5 8.6
NIM 0 206 61.0 14.9 −1.36 n.s. .20
1 60 61.1 16.8
PIM 0 206 42.0 11.6 1.34 n.s. .20
1 60 39.6 12.5

Note. 0 = non-bland group; 1 = bland group; n.s. = non-significant; WASI-II = Wechsler Abbreviated Scale of Intelligence-2nd edition; Lambda and Number
of Responses = Engagement and Richness; TAT word count = Verbal Productivity; PAI = Personality Assessment Inventory; MCE = Mean Clinical Elevation;
NIM = Negative Impression Management; PIM = Positive Impression Management. Cohen's d was calculated to determine effect size (d > .20–.50 = small,
.50–.80 = medium, and >.80 = large; Cohen, 1988) for these group comparisons.
a
t ≤ .10.
b
≤.01.
572 STEIN ET AL.

TABLE 3 Study 1 group differences: select psychopathology

PAI Group N M SD T p-value D

SOM 0 206 60.4 13.2 −2.32 .02 *
.34
1 60 64.8 12.3
ANX 0 206 66.2 13.6 −1.62 n.s. .24
1 60 69.5 14.5
ARD 0 206 62.5 16.2 −0.92 n.s. .13
1 60 64.6 13.2
DEP 0 206 71.2 15.6 −2.13 .03* .31
1 60 75.9 14.4
MAN 0 206 51.5 12.0 0.81 n.s. .12
1 60 50.1 11.0
PAR 0 206 56.9 12.0 0.07 n.s. .01
1 60 56.8 12.6
SCZ 0 206 62.6 13.7 −0.64 n.s. .09
1 60 63.9 13.0
BOR 0 206 63.9 12.8 −1.30 n.s. .20
1 60 66.4 12.9
ANT 0 206 52.4 11.5 −1.52 n.s. .22
1 60 55.0 12.0
ALC 0 206 54.0 12.3 0.27 n.s. .04
1 60 53.5 12.7
DRG 0 206 54.7 12.9 −3.06 .002** .45
1 60 60.9 16.7
SOM-H 0 202 60.9 13.1 −2.59 .01** .39
1 58 65.9 12.9
SOM-C 0 202 56.7 14.2 −0.36 n.s. .36
1 58 57.5 12.5
SOM-S 0 202 59.1 12.6 −1.49 n.s. .22
1 58 61.9 13.0
DEP-C 0 202 69.7 16.2 −2.24 .03* .33
1 58 75.2 17.7
DEP-A 0 202 71.2 16.5 −1.74 .08t .26
1 58 75.4 15.3
DEP-P 0 202 63.3 12.5 −0.70 n.s. .10
1 58 64.6 12.4
PDSQ
Depression 0 113 8.9 5.0 −1.80 .07* .38
1 29 10.8 4.2
Drug Use 0 113 0.4 1.3 −2.36 .02* .49
1 29 1.2 2.0
Somatization 0 113 1.3 1.4 −2.02 .04* .42
1 29 1.9 1.4
Hypochondriasis 0 113 0.7 1.3 −2.64 .01** .55
1 29 1.5 1.9

Note. 0 = non-bland group; 1 = bland group; PAI = Personality Assessment Inventory; SOM = Somatization; ANX = Anxiety; ARD = Anxiety-Related Disor-
ders; DEP = Depression; MAN = Mania; PAR = Paranoia; SCZ = Schizophrenia; BOR = Borderline features; ANT = Antisocial features; ALC = Alcohol Use;
DRG = Drug Use; SOM-H = Somatization-Health Concerns subscale; SOM-C: Somatization-Conversion; SOM-S = Somatization-Somatization;
DEP-C = Depression-Cognitive; DEP-A = Depression-Affective; DEP-P = Depression-Physical; PDSQ = Psychiatric Diagnostic Screening Questionnaire.
Cohen's d was calculated to determine effect size (d > .20–.50 = small, .50–.80 = medium, and >.80 = large; Cohen, 1988) for these group comparisons.
t = p ≤ .10.
*
p < .05.
**
p ≤ .01.
STEIN ET AL.
less meaningful profile. There were significant differences in verbal
productivity for Cards 2, 3BM, and 14. The ‘bland’ group's word count
was significantly lower than the ‘non-bland’ group for these cards.
Although there were no significant group differences on mea-
sures of general distress, results indicated significant group differ-
ences for SOM, DEP, and DRG such that those in the ‘bland’ group
reported higher rates of Somatization, Depression, and Drug Use.
Examination of the PAI subscales reveals that Somatization-Health
Concerns is driving SOM's elevation where those in the ‘bland’ group
are reporting increased health concerns. Regarding DEP, Depression-
Cognitive Features is driving this elevation such that those in the
‘bland’ group are reporting increased cognitive features of depression,
and Depression-Affective Features (DEP-A) is trending toward signifi-
cance (Table 3).
Lastly, we assessed group differences using the PDSQ to see if
similar findings emerged. We used scales targeting Drug Use, Depres-
sion, and Physical Health. Consistent with the PAI, the ‘bland’ group
reported higher rates of Drug Use, Somatization, and Hypochondria-
sis. Depression was trending toward significance.
2.3 | Discussion
Study 1's first goal was to assess the frequency of ‘bland’ narratives in
our database. The second goal was to examine clinical differences
between those individuals who produced ‘bland’ narratives and those
who did not. ‘Bland’ narratives were not as common as we anticipated
at the narrative (22%) or protocol level (18% exhibited one ‘bland’ nar-
rative per protocol).
However, clinical differences were found between patients who
produced ‘bland’ narratives versus those who did not. Clinically,
patients who produced ‘bland’ narratives tended to produce less ver-
biage on Cards 2, 3BM, and 14. There was not any significant differ-
ence in verbiage between groups on Card 1 (first card administered).
This indicates that there may be something about progression of cards
and the increasingly emotionally arousing nature of the task that con-
tributes to decreased verbal productivity in the ‘bland’ group. Another
possibility is that Card 1 may produce an anhedonic stance toward
the task.
Although no significant difference was found between the two
groups in terms of general distress, examining self-reported symptoms
of psychopathology illuminated that the ‘bland’ group endorsed higher
rates of somatization/health concerns, drug use, and features of
depression. Taken together, these findings suggest that those who
produce ‘bland’ protocols may be more ruminative and preoccupied
over their health and self-efficacy and depressed. They may use drugs
to avoid focusing on their internal world, be emotionally shut down,
and produce narratives that are simplistic and devoid of interpersonal
and emotional influences.
The clinical implications of these findings are useful, as this infor-
mation can help increase our patient understanding and formulate
treatment goals when working with patients and scoring a TAT proto-
col that meets ‘bland’ criteria.
573
3 | S TUD Y 2
The aims of Study 2 were to identify the frequency of formal default
ratings in our database. In addition, we wanted to assess how percent-
age of default ratings scattered throughout a TAT protocol impacts
clinical findings, using a multi-method approach.
3.1 | Method
3.1.1 | Participants
In Study 2, we used a subset of patients from Study 1. A power analy-
sis was conducted, which determined that we needed at least
85 patients to yield a moderate effect size. A random number genera-
tor identified TAT protocols that included Cards 1, 2, 3BM, and 14.
The final number of patients for Study 2 was N = 99. The average age
was 39.5 years (SD = 16.2), and the average education level was
14.6 years (SD = 2.6). There were slightly more males (n = 52; 52%)
than females (n = 47; 47%). Patients were predominately single (64%)
with 17% married and 10% divorced. Patients were primarily White
(82%) with 6% self-identifying as African American, 3% as Asian, and
4% as Hispanic. The most common referral diagnoses were depressive
(38%), bipolar (23%), and anxiety disorders (18%).
3.1.2 | Procedure
We took a more of a conservative approach in studying ‘bland’ proto-
cols in Study 2 and focused exclusively on the four SCORS-G dimen-
sions that possess formal default ratings (EIM, AGG, SE, and ICS).
After TAT protocols were identified using the random number
generator, we read through them to ensure that ratings for EIM,
AGG, and SE, and ICS represented a default rating and not reflec-
tive of a middle rating based on content expressed in the narrative.
Ratings that were not reflective of default ratings were excluded.
To determine specific cut-offs, we took our sample mean and
median into consideration and making sure we had sufficient num-
ber of protocols per group. Ultimately, we examined the utility of
two cut scores to identify questionable protocols, specifically eight
and nine default ratings per four card protocol. We then converted
these raw scores into percentages. Percentage of default ratings per
protocol consists of the number of default ratings for EIM, AGG,
SE, and ICS divided by 32 [8 SCORS-G dimensions × 4 TAT
cards = 32]. Eight default ratings per protocol converts to 25%
(e.g., 8 divided by 32 = .25), and nine default ratings per protocol
converts to 28.12%.
3.1.3 | Materials and measures
Thematic Apperception Test
Same as Study 1. We also examined TAT word count.
574 STEIN ET AL.

Social Cognition and Object Relations Scale-Global TABLE 4 Frequency of default ratings per four TAT card protocol
In Study 2, we examined EIM, AGG, SE, and ICS. for Study 2

Number of default ratings Default percentage Subset

Wechsler Abbreviated Scale of Intelligence-2nd edition per protocol per protocol frequency
In Study 2, we used VCI and PRI. 2 6.25 2
4 12.50 5
Personality Assessment Inventory 5 15.62 7
Given that Study 2 is a subset of Study 1, we needed to be mindful of 6 18.75 5
the measures and scales we used in analyses (to ensure we had ade-
7 21.87 9
quate power). As such, for the PAI, we focused on ‘bigger variables’
8 25.00 14
and opted to configure scales into higher order dimensions. Following
9 28.12 6
the Hopwood and Moser (2011) work, we calculated an Internalizing
10 31.25 15
dimension by averaging ANX, ARD, and DEP full scales and an Exter-
11 34.37 7
nalizing dimension by averaging ANT, ALC, and DRG full scales. To
create a reality-impairing dimension, we rationally selected PAR and 12 37.50 13

SCZ full scales and averaged them. We used MCE to denote general 13 40.62 5

distress. We included Borderline Features (BOR) given that the 14 43.75 9

SCORS rating system was initially created to assess presence of per- 15 46.90 1
sonality pathology, particularly Borderline Personality Disorder. 16 50.00 1

NEO Five-Factor Inventory

In Study 2, we used Neuroticism, Extraversion, Openness, Agreeable- similarity in patterns of correlations across groups. This approach
ness, and Conscientiousness. helps determine the degree to which default ratings may interfere
with correlational findings.

3.2 | Results
3.2.1 | Frequency of default ratings
Our first goal was to identify the frequency of default ratings for EIM,
AGG, SE, and ICS in this subset (Table 4). The mean percentage of
default ratings was 29.1% (SD = 9.8). This is equivalent to nine to
10 default ratings per protocol. The median and mode were 31.25%
(10 default ratings per protocol).
3.2.2 | Robustness of correlations
Our second goal was to ascertain the extent to which percentage of
default ratings per protocol impacts robustness of correlations across
tests of intelligence, psychopathology, and normal personality.
Tables 5 and 6 illustrate correlations across three groups: total sample,
subset of patients who exhibited ≤25% (Table 5) or ≤28.12% (Table 6)
default ratings, and subset of patients who exhibited >25% (Table 5)
or >28.12% (Table 6) default ratings per protocol.
We also computed double-entry intraclass correlations, which are
Pearson correlations between two double entered profiles, and pro-
vided an index of global similarity between two profiles (Furr, 2010).
In this study, we were interested in comparing each subset with the
total sample. First, we examined profile similarities between the total
sample with the subset that did not exceed the cut-off. Second, we
examined profile similarities between the total sample with the subset
that did exceed the cut-off. Stronger associations indicate greater
3.2.3 | Intellectual ability
Total sample
In the total sample, there were significant positive correlations
between VCI and SE (p = .03) and AGG and PRI (p = .05) (Tables 5 and
6).
25% cut-off
Significant WASI-II correlations were most prevalent in protocols that
exhibited ≤25% of default ratings (Table 5). VCI was significantly posi-
tively related to AGG (p = .03), SE (p = .02), and ICS (p < .01) and PRI
was significantly positively related to EIM (p = .01), AGG (p < .01), SE
(p = .05), and ICS (p = .04).
28.12% cut-off
Significant correlations were most evidenced in protocols that
exhibited ≤28.12% of default ratings (Table 6). VCI was significant
positively related to AGG (p = .03), SE (p < .01), and ICS (p < .01), and
PRI was significantly positively related to EIM (p < .01), AGG (p < .01),
and SE (p = .04).
3.2.4 | Psychopathology
Total sample
Significant PAI correlations were most prevalent in the total sample
(Tables 5 and 6). EIM was significantly negatively associated with
STEIN ET AL. 575

TABLE 5 Percentage of default ratings (cut-off: 25%) and correlational differences for WASI-II, PAI, and NEO-FFI

EIM AGG SE ICS

ALL ≤25 >25 ALL ≤25 >25 ALL ≤25 >25 ALL ≤25 >25
WASI-II
VCI .06 .22 −.11 .07 .33* −.19 .22* .35* .19 .12 .40** −.14
PRI .16 .38 *
−.05 .20 *
.45 **
.01 .18 .30 *
.14 .10 .32 *
−.12
PAI
Internalizing −.03 −.07 −.02 −.02 −.06 −.01 −.31** −.42** −.26* −.25* −.38* −.18
Externalizing −.27 **
−.24 −.11 −.24 *
−.22 −.08 −.21 *
−.20 −.05 −.22 *
−.17 −.12
Reality-impairing −.21 *
−.26 −.11 −.19 −.23 −.11 −.29 **
−.36 *
−.18 −.27 **
−.46 **
.03
MCE −.22* −.25 −.10 −.20 −.23 −.09 −.35** −.41** −.25 −.29** −.42* −.06
BOR −.13 −.16 −.12 −.11 −.13 −.08 −.28 **
−.41 *
−.14 −.26 *
−.43 **
−.04
NEO-FFI
N −.11 −.15 −.22 .05 −.05 .15 −.21 −.32 −.17 −.15 −.27 −.06
E −.07 .08 −.19 −.13 .02 −.25 .35 **
.52 **
.36 *
.36 **
.55 **
.36*
O −.13 −.09 −.12 −.16 −.04 −.26 .16 .30 .20 .00 .27 −.19
** * ** * *
A .36 .45 .20 .30 .36 .15 .20 .22 .08 .24 .26 .14
C .04 .04 .17 −.02 −.03 .09 .09 .26 −.06 .11 .17 .15
TAT word count .01 .28 .13 −.04 .18 .06 .03 .26 .12 −.05 .20 −.04

Note. N = 99; ≤25% = 42; >25% = 57; EIM = Emotional Investment in Values and Moral Standards; AGG = Experience and Management of Aggressive
Impulses; SE = Self-Esteem; ICS = Identity and Coherence of Self; WASI-II = Wechsler Abbreviated Scale of Intelligence-2nd Edition; VCI = Verbal Compre-
hension Index; PRI = Perceptual Reasoning Index; PAI = Personality Assessment Inventory; MCE = Mean Clinical Elevation; BOR = Borderline Features;
NEO-FFI = NEO Five Factor Inventory Short Form; N = Neuroticism; E = Extraversion; O = Openness to Immediate Experience; A = Agreeableness;
C = Conscientiousness;
*
p ≤ .05 small effect size.
**
p < .01.

externalizing (p < .01) and reality-impairing (p = .04) psychopathology
as well as MCE (p = .03). AGG was significantly negatively associated
with externalizing psychopathology (p = .01). SE and ICS were signifi-
cantly negatively associated with internalizing (p < .01 and p = .01,
respectively), externalizing (p = .04 and p = .03, respectively), and
reality-impairing (p < .01 and p < .01, respectively) psychopathology in
addition to MCE (p < .01 and p < .01, respectively) and borderline fea-
tures (p < .01 and p = .01, respectively).
25% cut-off
In the ≤25% cut-off group, SE and ICS were significantly negatively
related to internalizing (p < .01 and p = .01, respectively) and reality-
impairing (p = .02 and p < .01, respectively) psychopathology, general
distress (MCE; p < .01 and p = .01, respectively), and borderline fea-
tures (p = .01 and p < .01, respectively) (Table 5). One significant nega-
tive correlation was evidenced between SE and internalizing
psychopathology in the >25% group (p = .05).
28.12% cut-off
Identical findings were evidenced using this cut-off(Table 6). In the
≤28.12% group, SE and ICS were significantly negatively associated
with internalizing (p < .01 and p < .01, respectively), reality-impairing
psychopathology (p < .01 and p < .01, respectively), general distress
(MCE; p < .01 and p < .01, respectively), and borderline features
(p = .01 and p = . < .01). One significant negative correlation was
evidenced between SE and internalizing psychopathology in the
>28.12% group (p = .04).
3.2.5 | Normal personality traits
Total sample
EIM, AGG, and ICS were positively associated with Agreeableness
(p < .01, p < .01, and p = .04, respectively) (Tables 5 and 6). SE and ICS
were positively associated with Extraversion (p < .01 and p < .01).
25% cut-off
EIM and AGG were significantly positively associated with Agreeable-
ness (p = .01 and p = .04, respectively) in protocols that exhibited
≤25% of default ratings (Table 5). SE and ICS were significantly posi-
tively associated with extraversion in the ≤25% group (p < .01 and
p < .01, respectively) and >25% group (p = .02 and p = .02,
respectively).
28.12% cut-off
Identical findings were evidenced using this cut-off(Table 6). EIM and
AGG were significantly positively associated with Agreeableness
(p < .01 and p = .02, respectively) in protocols that exhibited ≤28.12%
576 STEIN ET AL.

TABLE 6 Percentage of default ratings (cut-off: 28.12%) and correlational differences for WASI-II, PAI, and NEO-FFI

EIM AGG SE ICS

ALL ≤28.12 >28.12 ALL ≤28.12 >28.12 ALL ≤28.12 >28.12 ALL ≤28.12 >28.12
WASI-II
VCI .06 .24 −.17 .07 .31* −.19 .22* .37** .13 .12 .39** −.17
PRI .16 .37 **
−.19 .20 *
.43 **
−.07 .18 .30 *
.05 .10 .26 −.11
PAI
Internalizing −.03 −.09 −.01 −.02 −.09 −.03 −.31** −.40** −.29* −.25* −.38** −.15
Externalizing −.27 **
−.24 −.08 −.24 *
−.21 −.07 −.21 *
−.16 −.12 −.22 *
−.15 −.19
Reality-impairing −.21* −.28 −.09 −.19 −.27 −.06 −.29** −.38** −.14 -.27** −.46** .07
MCE −.22 *
−.27 −.06 −.20 −.26 −.03 −.35 **
−.40 **
−.18 −.29 **
−.42* −.01
BOR −.13 −.16 −.12 −.11 −.15 −.05 −.28 **
−.36 *
−.26 −.26 *
−.42 **
−.05
NEO-FFI
N −.11 −.11 −.20 .05 −.04 .27 −.21 −.22 −.26 −.15 −.18 −.13
E −.07 .02 −.18 −.13 −.01 −.30 .35 **
.41 **
.46 **
.36 **
.47** .39*
O −.13 −.07 −.15 −.16 −.06 −.27 .16 .28 .16 .00 .17 −.17
** ** ** * *
A .36 .46 .05 .30 .38 .02 .20 .24 .01 .24 .25 .13
C .04 −.01 .19 −.02 −.01 −.05 .09 .10 .07 .11 .07 .24
TAT word count .01 .17 .13 −.04 .10 .02 .03 .13 .12 −.05 .05 −.02

Note. N = 99; ≤28.12% = 48; >28.12% = 51; EIM = Emotional Investment in Values and Moral Standards; AGG = Experience and Management of Aggres-
sive Impulses; SE = Self-Esteem; ICS = Identity and Coherence of Self; WASI-II: Wechsler Abbreviated Scale of Intelligence-2nd Edition; VCI = Verbal Com-
prehension Index; PRI = Perceptual Reasoning Index; PAI = Personality Assessment Inventory; MCE = Mean Clinical Elevation; BOR = Borderline Features;
NEO-FFI = NEO Five Factor Inventory Short Form; N = Neuroticism; E = Extraversion; O = Openness to Immediate Experience; A = Agreeableness;
C = Conscientiousness;
*
p ≤ .05 small effect size.
**
p < .01.

of default ratings. SE and ICS were significantly positively associated
with extraversion in the ≤28.12% group (p < .01 and p < .01, respec-
tively) and >28.12% group (p < .01 and p = .02, respectively).
TAT word count
There were no significant relationships between TAT word count
and SCORS-G dimensions across the three groups for both cut-
offs.
pattern of correlations between the total sample and subset which
exceeds the cut-off is highly dissimilar. Taken together, these results
indicate that having 9 (>25%) or more (>28.12%) formal default rat-
ings per protocol can exert a notable influence on the pattern of cor-
relations for EIM, AGG, and ICS in the total sample, though SE does
not follow this same pattern.
3.2.7 | Clinical differences

3.2.6 | Double-entry intraclass correlations
Comparing the total sample with those who fell below the 25% cut-
off, agreement was high for all dimensions (EIM = .83; AGG = .76;
SE = .91; ICS = .81). With the exception of ICS, results were similar
for the 28.12% cut-off group (EIM = .87; AGG = .81; SE = .95;
ICS = .60). This means that the pattern of correlations between the
total sample and those who did not exceed the cut-off demonstrated
strong similarity in profiles for most dimensions.
Comparing the total sample and the subset of patients who did
exceed default ratings cut-off indicated differences in the pattern of
correlations (with the exception of SE) across both the 25% cut-off
(EIM = .64; AGG = .58; SE = .90; ICS = .56) and the 28.12% cut-off
(EIM = .26; AGG = .30; SE = .89; ICS = .62). This indicates that the
Binary logistic regressions were calculated to assess whether there
were clinical differences in patients that produced ≤25% and ≤28.12%
default ratings to those that exhibited >25% and >28.12% default rat-
ings. The number of regressions were computed based on conceptu-
ally based combinations of variables. The following clinical themes
were used in separate analyses: intellectual ability, impression man-
agement, general distress, higher-order psychopathology, and verbal
productivity (mean TAT word count). For ease of comparison and to
reduce multicollinearity, these clinical variables were transformed into
a similar metric (z-scores; M = 0, SD = 1). With respect to the depen-
dent variable, a coding of 0 was provided to patients who produced
≤25% or ≤28.12% default ratings per protocol and a coding of 1 was
provided to patients who provide >25% or >28.12% default ratings
per protocol.
STEIN ET AL. 577

25% cut-off 3.3 | Discussion

Verbal productivity significantly predicted classification of those with
>25% default ratings per protocol, χ 2(1) = 8.93, p = .00 (Table 7). No
significant predictions were found for tests of intellectual ability,
χ 2(2) = .38, n.s., general distress, χ 2(1) = .56, n.s., and impression man-
agement, χ2(2) = .27, n.s. Regarding dimensional models of psychopa-
thology, although the overall model did not significantly predict
percentage of default rating per protocol, χ2(3) = 6.94, p = .07, the
externalizing dimension was a significant independent predictor
(B = −.48, p = .04) such that patients who reported greater externaliz-
ing psychopathology were more likely to produce ≤25% default rat-
ings per protocol.
28.12% cut-off
Verbal productivity significantly predicted classification of those
with >28.12% default ratings per protocol, χ2(1) = 5.19, p = .02
(Table 8). The dimensional models of psychopathology also signifi-
cantly predicted classification of those with ≤28.12% group,
χ (3) = 8.36, p = .04. In particular, patients who reported greater
2
externalizing psychopathology were more likely to produce ≤28.2%
default ratings per protocol (B = −.57, p = .02). No significant pre-
dictions were evidenced on tests of intellectual ability, χ 2(2) = .72,
n.s., general distress, χ (1) = .02, n.s., and impression management,
2
χ 2(2) = .89, n.s.
Study 2's first objective was to identify the frequency of default rat-
ings per a four TAT card protocol. The second objective was to iden-
tify a reasonable cut-off based on our first goal and examine how the
percentage of default ratings may impact clinical findings using a
multi-method approach. The frequency of default ratings ranged con-
siderably. We used these findings to identify reasonable cut-off points
and assist in ascertaining our second objective: the extent to which
the magnitude of correlations is impacted by the percentage of
default ratings per protocol.
The total sample and those who did not exceed the cut-off
exhibited the greatest number of significant correlations. The total
sample had the most significant correlations; however, this is likely
due to the fact that this group had a higher N. In some instances, the
correlations for these two groups were similar, but because of the
higher N, the correlations were significant in the total group and not
in the group that did not exceed the cut-off. In other instances, both
groups resulted in significant correlations, with the correlations being
consistently larger in the group that did not exceed the cut-off (≤25%
and ≤28.12% default ratings per protocol).
On the WASI-II, significant relationships between patients' verbal
comprehension and perceptual reasoning abilities were evidenced in
the total sample and groups that did not exceed the cut-off. In some

TABLE 7 Logistic regression results for predicting clinical TABLE 8 Logistic regression results for predicting clinical
differences in patients who produced less than and greater than 25% differences in patients who produced less than and greater than
default ratings per four card protocol 28.12% default ratings per 4 card protocol

Variable entered B SE Wald p-value Exp(B) Variable entered B SE Wald p-value Exp(B)
Intellectual ability (N = 97) Intellectual ability (N = 97)
VCI −.04 .30 .02 n.s. .96 VCI −.21 .30 .49 n.s. .81
PRI −.10 .30 .10 n.s. .91 PRI .05 .29 .03 n.s. 1.05
Constant .27 .20 1.74 n.s. 1.31 Constant .02 .20 .01 n.s. 1.02
Impression management (N = 97) Impression management (N = 97)
PIM −.09 .23 .16 n.s. .91 PIM −.11 .23 .23 n.s. .89
NIM .02 .24 .01 n.s. 1.03 NIM .11 .23 .24 n.s. 1.12
Constant .35 .21 2.96 n.s. 1.43 Constant .10 .20 .26 n.s. 1.11
General distress (N = 75) General distress (N = 75)
Neuroticism .18 .24 .55 n.s. 1.19 Neuroticism .04 .23 .02 n.s. 1.04
Constant .24 .23 1.08 n.s. 1.27 Constant −.08 .23 .12 n.s. .92
Dimensional models (N = 97) Dimensional models (N = 97)
Internalizing .55 .36 2.28 n.s. 1.73 Internalizing .59 .36 2.67 n.s. 1.81
Externalizing −.48 .24 4.07 .04 .62 Externalizing −.57 .25 5.30 .02 .56
Reality-impairing −.44 .36 1.54 n.s. .64 Reality-impairing −.40 .36 1.25 n.s. .67
Constant .37 .21 2.99 n.s. 1.45 Constant .10 .21 .22 n.s. 1.10
Verbal productivity (N = 97) Verbal productivity (N = 97)
TAT mean word count −.66 .24 7.68 .01 .51 TAT mean word count −.49 .23 4.71 .03 .61
Constant .33 .22 2.33 n.s. 1.39 Constant .06 .21 .08 n.s. 1.06

Note. VCI = Verbal Comprehension Index; PRI = Perceptual Reasoning Note. VCI = Verbal Comprehension Index; PRI = Perceptual Reasoning
Index; PIM = Positive Impression Management; NIM = Negative Impres- Index; PIM = Positive Impression Management; NIM = Negative Impres-
sion Management. sion Management.
578
instances, significant relationships were lost in the total sample, and in
other instances, using the cut-off resulted in the strengthening of
correlations.
On the PAI, SE and ICS were the two dimensions that were most
impacted by percentage of default ratings. Although both the total
sample and using the cut-off exhibited significant correlations, using
the cut-off (≤25% and ≤28.12%) resulted in the strengthening of cor-
relations for internalizing and reality-impairing psychopathology as
well as general distress and borderline features.
On the NEO-FFI, using the ≤25% and ≤28.12% cut-off resulted in
the strengthening of correlations between Extraversion, SE, and ICS
as well as Agreeableness with EIM. Correlational findings were similar
in the total sample and when using the cut-off between Agreeable-
ness, AGG, and ICS (small effect sizes).
Findings from the binary logistic regression indicate that patients
with greater rates of externalizing psychopathology are more likely to
fall into the ≤25% and ≤28.12% groups. An explanation for this is that
externalizing psychopathology is composed of the PAI scales targeting
Alcohol/Drug Use and Antisocial features. SCORS-G dimensions tap
underlying psychological processes associated with externalizing psy-
chopathology (i.e., morality and compassion, management of aggres-
sion, and self-reactivity). These patients may be more likely pulled to
produce TAT content with similar themes. Another possibility is that
this group is more disinhibited.
4 | GENERAL DISCUSSION
In these studies, we took initial steps toward understanding narrative
‘blandness’ and the extent to which percentage of formal default rat-
ings present in a protocol impacts clinical findings using a multi-
method approach. Study 1 informed us that there are select patient
characteristics that contribute to them being more likely to produce
bland protocols using specified criteria across all eight dimensions
(reduced word count, increased self-reported symptoms of Somatiza-
tion, Drug Use, and aspects of Depression), which is useful clinically.
This study also demonstrated that ‘bland’ narratives are not as com-
mon as we initially thought. However, it was less informative about
how formal default ratings psychometrically impact clinical findings.
This contributed to the execution of Study 2 where we took more of
a conservative approach and studied the percentage of default ratings
per protocol. We used the four SCORS-G dimensions that had a for-
mal default rating rule. Our findings suggest that limiting the number
of default ratings per protocol within study samples may produce
more meaningful and accurate findings that are driven less by default
ratings and more by actual narrative content. The inclusion of ques-
tionable protocols within a sample seemed to increase error and
attenuate associations with other variables (i.e., associations between
four SCORS-G dimensions and intellectual ability, SE and ICS with
select psychopathology and Agreeableness with EIM and Extraversion
with SE and ICS). Thus, removing these protocols reduces measure-
ment error and helps researchers identify and better characterize
meaningful associations.
STEIN ET AL.
Similar to Study 1, in Study 2, we found clinical differences
between patients who did and did not exceed the cut-off. Clini-
cally, individual differences like intellectual ability, impression man-
agement, general distress, internalizing, and reality-impairing
psychopathology did not relate to the percentage of default rating
group cut-offs. However, externalizing psychopathology was a sig-
nificant predictor. One of the reasons for this may be because the
majority of SCORS-G dimensions are theoretically (Stein & Slavin-
Mulford, 2018) and empirically (DeFife, Goldberg, & Westen, 2013)
connected to underlying processes associated with externalizing
psychopathology.
Given that word count was consistently linked to protocol ‘bland-
ness’ in Study 1 and the percentage of default ratings in Study 2, this
brings up a broader topic within the TAT literature regarding how we
as clinicians and researchers can maximize protocol richness without
significantly disrupting the patient's flow of associations. TAT adminis-
tration guidelines vary widely (Jenkins, 2008). We use a standard set
of prompts during TAT administration (Stein et al., 2012), though there
is room to think about a benefit to increasing the standardization of
re-prompting.
Based on these findings, we suggest interpreting protocols as
follows: ≤25% (8 default ratings) = high likelihood of interpretabil-
ity; >25% (9 or more) is questionable interpretability of four dimen-
sions; >28.12% (10 or more) = increasing concern that multiple
dimensions contain an excess of default ratings. However, it is
important to note that even at the ≤25% cut-off it is possible for
the 8 default ratings to cluster in two dimensions. As such,
researchers and clinicians should be mindful of this when making a
final determination. Likewise, protocols above these suggested cut
score should not be automatically thrown out. Instead, scores
above the cut score should cue clinicians or researchers to critically
examine the narrative and ratings. First, examine EIM, AGG, SE,
and ICS to see if one or two dimensions are driving elevations. For
example, if EIM and AGG are the dimensions that have all the
default ratings, then one can use SE and ICS for interpretation.
Second, we do not recommend using the composite rating if
default ratings exceed the cut-off. Third, the SCORS-G generates
higher-order factors (Stein & Slavin-Mulford, 2018). In a two-factor
model, the affective-relational factor (AFF, EIR, EIM, AGG, SE, and
ICS) may not be able to be used and the same applies in a three-
factor model when SE and ICS form a third factor. Fourth, COM,
AFF, EIR, and SC are not impacted by formal default ratings and
can be interpreted. Findings from Study 1 can help inform select
psychopathology that can be contributing to protocol ‘blandness’.
The number of TAT cards and the specific TAT cards used var-
ies considerably with the SCORS-G field (and TAT field more
broadly). The findings from this study only apply to this four card
TAT protocol (Cards 1, 2, 3BM, and 14). Cut-off raw scores and
the calculation of percentage of default ratings were based on this.
Replicating this study with increasing number of cards is important.
This will require researchers to identify reasonable cut-off raw
scores and then transform these scores into percentages. For
example, in a seven card protocol, one would calculate the number
STEIN ET AL.
of default ratings divided by 56 [8 SCORS-G dimensions × 7 TAT
cards = 56] = percentage of default ratings per protocol. The maxi-
mum number of default ratings using this approach is 28 (seven
cards × four dimensions). After this is calculated, researchers and
clinicians should review these protocols to ensure ratings reflect
true default scores and not reflective of thematic content before
generating a final cut-off score for each protocol. As far as future
research, we also encourage others to replicate this study (using
this 4 card TAT protocol) to see if similar findings emerge in addi-
tion to examining multiple cut-off points (to fine tune acceptable
levels of interpretability). Lastly, examining how other types of nar-
rative data impacts default ratings percentages is encouraged. We
do not suggest replicating this study with fewer than four cards
(Stein & Slavin-Mulford, 2018).
There are some study limitations that are worth discussing.
The total sample has approximately twice as many patients as each
cut-off group. SCORS-G ratings of TAT narratives are from one
dataset. It is not an ethnically diverse sample. A select group of
patients are referred to our clinic, are often clinically complex, and
have medical comorbidities. Replicating this study with other sam-
ples is suggested. We could not explore the nuances of how cut-
offs below ≤25% and above the >28.12% impact clinical findings.
ORCID
Michelle B Stein https://orcid.org/0000-0002-4179-0817
RE FE R ENC E S
American Psychiatric Association. (1994). Diagnostic and statistical manual
of mental disorders (4th ed.). Washington, DC: Author.
Cohen, J. (1988). Statistical power analysis in the behavioral sciences (2nd
ed.) (pp. 36–37). Hillsdale, NJ: Erlbaum.
Costa, P. T.Jr., & McCrae, R. R. (1989). The NEO–PI/NEO–FFI manual sup-
plement. Odessa, FL: Psychological Assessment Resources.
Costa, P. T.Jr., & McCrae, R. R. (1992a). Multiple uses for longitudinal per-
sonality data. European Journal of Personality, 6, 85–102.
Costa, P. T.Jr., & McCrae, R. R. (1992b). NEO PI–R professional manual:
Revised NEO Personality Inventory (NEO PI–R) and NEO Five-Factor
Inventory (NEO–FFI). Lutz, FL: Psychological Assessment Resources.
Dana, R. H. (2008). Manual for Objective TAT scoring. In S. R. Jenkins(Ed.),
A handbook of clinical scoring systems for thematic apperception tech-
niques (pp. 127–147). New York: Lawrence Erlbaum Associates.
DeFife, J. A., Goldberg, M., & Westen, D. (2013). Dimensional assessment
of self and interpersonal functioning in adolescents: Implications for
DSM-5's general definition of personality disorder. Journal of Personal-
ity Disorders, 27, 1–13.
Exner, J. E. (2003). The Rorschach: A comprehensive system. Volume 1: Basic
foundations and principles for interpretation (4th ed.). New York: John
Wiley.
Furr, R. M. (2010). The double-entry intraclass correlation as an index of
profile similarity: Meaning, limitations, and alternatives. Journal of Per-
sonality Assessment, 92(1), 1–15. https://doi.org/10.1080/
00223890903379134
Hilsenroth, M., Stein, M., & Pinsker, J. (2007). Social Cognition and Object
Relations Scale: Global Rating Method (SCORS-G, 2nd ed.).
Unpublished manuscript, The Derner Institute of Advanced Psychological
Studies, Garden City, NY: Adelphi University.
Hopwood, C. J., & Moser, J. S. (2011). Personality Assessment Inventory
internalizing and externalizing structure in college students: Invariance
579
across sex and ethnicity. Personality and Individual Differences, 50,
116–119.
Jenkins, S. R. (2008). A handbook of clinical scoring systems for thematic
apperception techniques. New York: Lawrence Erlbaum Associates.
Karon, B. P. (2008). Pathogenesis index. In S. R. Jenkins(Ed.), A handbook of
clinical scoring systems for thematic apperception techniques (pp. 347–
384). New York: Lawrence Erlbaum Associates.
Luborsky, L., & Crits-Christoph, P. (1998). Understanding transference: The
core conflictual relationship theme method (2nd ed.). Washington, DC:
American Psychological Organization.
Malone, J. C., Stein, M. B., Slavin-Mulford, J., Bello, I., Sinclair, S. J., & Blais,
M. A. (2013). Seeing red: Affect modulation and chromatic color
responses on the Rorschach. Bulletin of the Menninger Clinic, 77(1), 70–
93. https://doi.org/10.1521/bumc.2013.77.1.70
Morey, L. (1991). Personality assessment inventory professional manual.
Odessa, Fl: Psychological Assessment Resources.
Morey, L. (1996). An interpretative guide to the personality assessment inven-
tory (PAI). Odessa, FL: Psychological Assessment Resources.
Murray, H. A. (1943). Manual for the thematic apperception test. Cambridge,
MA: Harvard University Press.
Perkey, H., Sinclair, S. J., Blais, M. A., Stein, M. B., Neal, P., Pierson, A. D. &
Slavin-Mulford, J. (2017). External Validity of the Psychiatric Diagnos-
tic Screening Questionnaire (PDSQ) in a Clinical Sample. Psychiatry
Research, https://doi.org/10.1016/j.psychres.2017.12.011
Ritzler, B. (2008). The picture projection test. In S. R. Jenkins(Ed.), A hand-
book of clinical scoring systems for thematic apperception techniques
(pp. 229–257). New York: Lawrence Erlbaum Associates.
Ronan, G. F., Gibbs, M. S., Dreer, L. E., & Lombardo, J. A. (2008). Personal
problem-solvingsystem-revised. In S. R. Jenkins(Ed.), A handbook of
clinical scoring systems for thematic apperception techniques (pp. 181–
207). New York: Lawrence Erlbaum Associates.
Rorschach, H. (1942). Psychodiagnostics. Berne, Switzerland: Hans.
(Original work published 1921)
Shrout, P. E., & Fleiss, J. L. (1979). Intraclass correlations: Uses in assessing
rater reliability. Psychological Bulletin, 86, 420–428. https://doi.org/10.
1037//0033-2909.86.2.420
Stein, M. B., & Slavin-Mulford, J. (2018). The social cognition and object rela-
tions scale-global rating method (SCORS-G): A comprehensive guide for
clinicians and researchers. New York, NY: Routledge.
Stein, M. B., Slavin-Mulford, J., Sinclair, S. J., Chung, W. J., Roche, M.,
Denckla, C., & Blais, M. A. (2016). Extending the use of the SCORS-G
composite rating in assessing level of personality organization. Journal
of Personality Assessment, 100, 166–175. https://doi.org/10.1080/
00223891.2016.1195394
Stein, M. B., Slavin-Mulford, J., Sinclair, S. J., Siefert, C. J., & Blais, M. A.
(2012). Exploring the construct validity of the social cognition and
object relations scale in a clinical sample. Journal of Personality Assess-
ment, 94, 533–540.
Wechsler, D. (2011). Wechsler abbreviated scale of intelligence-second edi-
tion (WASI-II). San Antonio, TX: NCS Pearson.
Westen, D. (1995). Social cognition and object relations scale: Q-sort for pro-
jective stories (SCORS–Q). Department of Psychiatry, The Cambridge
Hospital and Harvard Medical School, Cambridge, MA: Unpublished
manuscript.
Zimmerman, M., & Mattia, J. I. (2001a). The psychiatric diagnostic screen-
ing questionnaire: Development, reliability and validity. Comprehen-
sive Psychiatry, 42, 175–189. https://doi.org/10.1053/comp.2001.
23126
Zimmerman, M., & Mattia, J. I. (2001b). A self-report scale to help make
psychiatric diagnoses: The psychiatric diagnostic screening question-
naire. Archives of General Psychiatry, 58, 787–794. https://doi.org/10.
1001/archpsyc.58.8.787
580 STEIN ET AL.

How to cite this article: Stein MB, Calderon S, Ruchensky J,

et al. When's a story a story? Determining interpretability of
Social Cognition and Object Relations Scale-Global ratings on
Thematic Apperception Test narratives. Clin Psychol
Psychother. 2020;27:567–580. https://doi.org/10.1002/cpp.
2442