BRIEF REPORT

The Challenges of Ecological Validity in the Measurement of
Social Perception in Schizophrenia
Anja Vaskinn, PhD,* Mark J. Sergi, PhD,†‡ and Michael F. Green, PhD‡§

Abstract: Ecological validity may be substantiated by verisimilitude (the
stimuli and cognitive processing of a test resemble the stimuli and cognitive
processing required in everyday life) and/or veridicality (a test is statistically
associated with everyday life functioning). The Interpersonal Perception
Task uses real-life social behavior as stimuli and has high verisimilitude. We
examined the Interpersonal Perception Task-15 in 72 persons with schizophrenia and 58 healthy controls. The test discriminated well between groups,
but failed to show associations with community functioning in participants
with schizophrenia, indicating poor veridicality. This study shows that the 2
approaches to ecological validity can dissociate. Hence, it is important for
investigators to conduct psychometric investigations when borrowing measures from social psychology for schizophrenia research.
Key Words: Schizophrenia, social cognition, social perception,
psychometrics, ecological validity.
(J Nerv Ment Dis 2009;197: 700 –702)

S

ocial cognition appears closely linked to functional status in
schizophrenia, and may mediate relations between neurocognition and functioning in the illness (Couture et al., 2006; Sergi et al.,
2006; Vaskinn et al., 2008). Despite the accelerating prominence of
social cognition as a construct in the schizophrenia research literature, concern exists regarding the psychometric properties and ecological validity of the measures used to assess social cognition
(Green et al., 2008), especially when these measures are borrowed
from other disciplines.
Ecological validity may be substantiated by verisimilitude
and/or veridicality (Chaytor and Schmitter-Edgecombe, 2003). Verisimilitude indicates that something appears to be “true” or “real.”
Based on theoretical knowledge and the judgment of experts, verisimilitude assumes that the stimuli of the measure and the cognitive
processing of these stimuli are both sufficiently similar to the stimuli
and cognitive processing of everyday occurrences of the psychological construct (Franzen and Arnett, 1997). Veridicality uses statistical approaches to associate performance on the measure of interest
with measures of everyday life functioning. Thus, a measure of
social cognition in schizophrenia would show evidence of high
veridicality if it correlated well with measures of functional status.
Social perception is an aspect of social cognition that involves
the interpretation of social cues such as facial expressions, gestures,
and voice tone to form impressions about others such as their

*Clinic for Mental Health, Oslo University Hospital - Aker, Oslo, Norway;
†Department of Psychology, California State University, Northridge, CA;
‡Department of Veterans Affairs VISN 22 Mental Illness, Research, and Clinical
Center (MIRECC), Los Angeles, CA; and §Department of Psychiatry and
Biobehavioral Sciences, Geffen School of Medicine, UCLA, Los Angeles, CA.
Supported by grants from the National Institute of Mental Health MH43292 (to
M.F.G.) and the Fulbright Foundation for Educational Exchange (to A.V.).
Reprints: Anja Vaskinn, PhD, Oslo University Hospital–Aker, Clinic for Mental
Health, Trondheimsvn. 235, 0514 Oslo, Norway. E-mail: anja.vaskinn@medisin.
uio.no.
Copyright © 2009 by Lippincott Williams & Wilkins
ISSN: 0022-3018/09/19709-0700
DOI: 10.1097/NMD.0b013e3181b3ae62

700 | www.jonmd.com

relative status, level of intimacy, motives, or veracity. Measures
commonly used to assess social perception in persons with schizophrenia (e.g., Social Cue Recognition Test, Profile of Nonverbal
Sensitivity) typically use stimuli involving restricted information
(e.g., 1 to 3 specific social cues). These measures are psychometrically sound and produce moderate associations with measures of
functional status. For example, Corrigan and Toomey (1995) found
correlations between Social Cue Recognition Test and social problem-solving skills ranging from 0.54 to 0.78. Sergi et al. (2006)
showed associations between Profile of Nonverbal Sensitivity and
functional status ranging from 0.11 to 0.36. Although these measures have veridicality, as illustrated through the moderate associations with functioning, they lack verisimilitude. Simply put, the
stimuli and the cognitive processing needed to complete these tests
are limited compared with the social perception demands commonly
experienced in daily life.
The Interpersonal Perception Task-15 (IPT-15; Costanzo and
Archer, 1993) is a measure of social perception with higher verisimilitude. Whereas other measures of social perception use actors to
display social cues, the videotape clips of the IPT-15 involve real
persons in real situations. While other measures of social perception
use brief exposures of a limited number of social cues, the videotape
clips of the IPT-15 involve highly varied social cues present in
everyday situations. These social cues include facial expressions,
gestures, voice tone, proxemics (spatial distance), haptics (touching), and verbal exchanges connoting subtleties in intimacy and
status. Whereas other measures tend to depict 1 or 2, typically
Caucasian, actors, the IPT-15 involves many people representing
diversity in race, age, and gender.
The aim of the present study was to examine the ecological
validity, specifically the veridicality, of the IPT-15. The IPT-15
clearly has high verisimilitude. However, because the IPT-15 only
involves 15 items, veridicality, the statistically determined aspect of
ecological validity, may be modest.

METHODS
Participants
The sample included 72 outpatients with a diagnosis of
schizophrenia or schizoaffective disorder (SZ) and 58 healthy comparison persons (HC). All participants provided their written informed consent. The SZ group was recruited through the clinics of
the Veterans Affairs Greater Los Angeles Health Care System as
well as local board-and-care facilities. Diagnosis was based on the
Structured Clinical Interview for DSM-IV Axis I Disorders (SCIDFirst et al., 1997). The HC group was recruited by advertisements in
newspapers and on the internet. All potential HC were administered
the SCID and selected sections of the Structured Clinical Interview
for DSM-IV Axis II Disorders (SCID-IFirst et al., 1996). They were
excluded if they had a history of any psychotic disorder, recurrent
depression, bipolar disorder, substance dependence, or if they met
criteria for any of the following Axis II disorders: Avoidant, Paranoid, Schizoid, and Schizotypal. Having a first degree relative with
a psychotic disorder was another exclusion criterion in the HC
group. All SCID interviewers were trained to administer the SCID in

The Journal of Nervous and Mental Disease • Volume 197, Number 9, September 2009

8 (2. Costanza and Archer (1989) have reported on the test-rest reliability (r ⫽ 0.The Journal of Nervous and Mental Disease • Volume 197. alogia. Unusual Thought Content. a b c The Role Functioning Scale (Goodman et al. The Scale for the Assessment of Negative Symptoms offers global ratings for affective flattening. The internal consistency (Cronbach’s alpha) of the IPT-15 was very poor for both groups (SZ ⫽ ⫺0. 2009). how long have they been dating.9 ⬍0. The SZ group displayed less social perception skill than their healthy counterparts after controlling for age and gender (F (1. The 15 scenes of the IPT-15 were selected from the 30-item Interpersonal Perception Task (IPT. the veracity of a person making 2 separate statements (e.. 30 to 90 second long. IL). HC ⫽ 0. Unfortunately. 1993) yields scores for 4 domains of everyday life functioning: work productivity. 2 weeks or 2 years?). but real persons in real situations. version 16. Small and nonsignificant associations were observed between social perception assessed with the IPT-15 and functional status assessed with the Role Functioning Scale. 129) ⫽ 17. Costanzo and Archer.086 0.g.9 (1.07).5) — — — 1. Costanzo and Archer. diverse in age. p ⬍ 0. and relationships with friends. Education and Clinical Center Treatment Unit and demonstrated agreement between their ratings and the consensus ratings of expert diagnosticians (minimum Kappa coefficient ⫽ 0. September 2009 TABLE 1. Clinical Symptoms The Brief Psychiatric Rating Scale ( Ventura et al. except differences in the gender distribution which was assessed with the chi square test.1 ⬍0. Independent samples t-tests were used to evaluate group differences for social perception performance and demographic variables.4 (1.001 0. Brekke et al. n ⫽ 56 due to missing data. 1993) is a videotape-based measure of social perception.. The relatively lengthy. However.8)b 61/11 40.75). which was the truth.6 (1. and inattention. SANS global scores.9 (7.com | 701 . as healthy persons performed at or near chance on several items of the 30-item IPT.7) ⫺4. race.1) 2. the veridicality (statistical associations suggestive of real world performance) of the IPT-15 is quite weak in this sample.7)a 11.001 n ⫽ 71 due to missing data. 1989) to improve the reliability and validity of the measure. n ⫽ 61 due to missing data. The IPT-15 yields a score between 0 and 15. anhedonia-asociality. items Hallucinations.2 (1..jonmd. The demographic characteristics of the 2 study groups are displayed in Table 1. The internal consistency of the IPT-15 was examined using Cronbach’s alpha for the SZ and the HC samples separately.. and Clinical Characteristics in SZ SZ (n ⴝ 72) HC (n ⴝ 58) Mean (SD) Mean (SD) Demographics Age Education Mother’s education Gender (M/F) Clinical status Positive symptoms Negative symptoms Affective flattening Alogia Avolition-apathy Anhedonia-asociality Inattention Social perception IPT-15 Challenges of Ecological Validity toms (Andreasen.g. Partly due to the length of the scenes. The Scale for the Assessment of Negative Symp© 2009 Lippincott Williams & Wilkins Table 1 displays the demographic features.7 11.0) 12. Each scene of the IPT-15 lasts 30 to 90 seconds and involves 1 to 4 persons. 1993) and has been used extensively in schizophrenia research (e.0) 2. and Conceptual Disorganization.6) 13. The scale has satisfactory psychometric properties (Goodman et al. Correlational analyses are presented in Table 2. or the intimacy level between persons (e.4 (1. and gender. In fact.7 (1.2) 0.7 (9. who won the racquetball game?). Community Functioning t/x2 p 46. Unusual Thought Content and Conceptual Disorganization. Kee et al. it discriminated well between the 2 groups of participants. Measures Social Perception The Interpersonal Perception Task-15 (IPT-15.001).7 ⫺1.001 2. DISCUSSION The main goal of this study was to examine the ecological validity (verisimilitude and veridicality) of the IPT-15 as a measure of social perception used in schizophrenia research.1. the first statement or the second statement?).7 (2. 1993) is an interview-based measure that assesses varied psychiatric symptoms. the features of the IPT-15 illustrate that these 2 aspects of ecological validity can be quite independent.3) 1. The verisimilitude (apparent and expert-based judgment of closeness to reality of both the stimuli and its social cognitive processing demands) of the IPT-15 is quite strong. The IPT-15 showed a statistically significant association with 1 symptom (affective flattening).7) 10. the measure has fewer items. relationships with family/spouse. Chicago. RESULTS the VISN 22 Mental Illness Research. and social perception performance of the 2 groups. This study shows that verisimilitude is no guarantee of veridicality.6 (1.g.. The association between social perception and symptoms was explored through correlational analyses (Pearson’s r).7 (1.7 ⫺1.099 0. videotape scenes of the IPT-15 add to the verisimilitude of the measure by making them look very contextual and real. The scenes of the IPT-15 were videotaped on a university campus and nearby community. symptoms. A positive symptom factor score was generated from the 3 items Hallucinations.7 (1. Number 9.g. Each scene is followed by 1 multiple-choice question about either the status of the persons (e.. 1984) is an interview-based measure that assesses negative symptoms. Veridicality was assessed by conducting bivariate correlations (Pearson’s r) between IPT-15 scores and community functioning in the SZ group.03. The lower number of items reduces www. independent living. 2003. Social perception assessed by the IPT-15 was not correlated with functional status assessed with the Role Functioning Scale in persons with schizophrenia.70) and internal consistency (Kuder-Richardson formula 20 statistic ⫽ 0. The persons in the scenes are not actors.52) of the 30-item IPT.0 (SPSS Inc. avolition-apathy.8 (1.6) 12. Ratings were completed by trained interviewers after a 30-minute interview with the participant. Negative symptoms. Variables were normally distributed and thus parametric analyses used. Demographic Characteristics and Social Perception Performance in Participants With Schizophrenia (SZ) and in Healthy Control Participants (HC)... Positive symptoms indicates BPRS..8)c 34/24 3.. Correlational analyses did not support the veridicality of the IPT-15. An analysis of covariance with age and gender as covariates was used to examine group differences in social perception because the SZ group was older and had more males than the HC group. Statistical Analyses All statistical analyses were conducted using the SPSS software.3)a — — — — — — — — — — — — — — — 8.

Leavitt N (1993) Assessing levels of adaptive functioning: the role functioning scale. IA: University of Iowa College of Medicine.Patient Edition. Williams JBW (1997) The Structured Clinical Interview for DSM-IV Axis I Disorders . Toomey R (1995) Interpersonal problem solving and information processing in schizophrenia. Gibbon M. New York: Biometrics Research. Costanzo M. it does not necessarily mean that a test is uninformative. In HW Reese. Silverstein SM. for useful advice during the planning of the study. Ringen PA. The results draw attention to a trade off. But the low veridicality and internal consistency show that these features do not guarantee that a measure is a useful research instrument for psychopathology. Sewell DR. J Nonverbal Behav. Penn DL. MD Franzen (Eds). Neuropsych Rev. A Guide for Researchers and Teachers. Couture SM. The Journal of Nervous and Mental Disease • Volume 197. Schiz Bull. Negative symptoms.02 0.jonmd.28 ⫺0. 32:S44 –S63. which makes it more difficult to identify associations with measures of functional status. Only one type of social judgment (of status. Bivariate Associations Between IPT-15 Scores and Community Functioning (Veridicality) and Symptoms in SZ IPT-15 Community functioning Work productivity Independent living Family network relationships Immediate social relationships Symptoms Positive symptoms Negative symptoms Affective flattening Alogia Avolition-apathy Anhedonia-asociality Inattention 0. Sergi MJ. Birkenaes AB. Archer D (1989) Interpreting the expressive behavior of others: the interpersonal perception task. Corrigan PW. New York: Biometrics Research. Vaskinn A.00 ⫺0. Rassovsky Y. Bentall R. functional change and service intensity during community-based psychosocial rehabilitation for schizophrenia. 29:119 –131. By itself. A low internal consistency in this case simply means that test does not measure one domain. veracity or intimacy) is required to answer any one of the IPT multiple-choice questions. Schiz Bull. Gaebel W.05 level. 14:279 –288. Park S. Hoe M. Archer D (1993) The Interpersonal Perception Task . such as social psychology. 29:487– 497. Number 9. Lieberman RP (1993) Training and quality assurance with the brief psychiatric rating scale: “The drift busters. In press. BA. Psychol Med. Friis S. Costanzo M. Kring AM. Nuechterlein KH. Mahwah. Green MF (2009) Neurocognitive change. and Mark McGee. Community Mental Health J. 3:221–244. Ventura J.” Int J Methods Psychiatr Res. Jo´nsdo´ttir H. REFERENCES Andreasen NC (1984) Scale for the Assessment of Negative Symptoms (SANS). before utilizing them in schizophrenia research. Arnett PA (1997) The validity of neuropsychological assessment procedures. September 2009 CONCLUSIONS TABLE 2. This underscores the importance of performing psychometric investigations of measures borrowed from other fields. Heinssen R (2008) Social cognition in schizophrenia: An NIMH workshop on definitions.13 ⫺0. With its objective accuracy criterion it has correct answers. and that this is reflected in the low internal consistency. 34:1211–1220. Williams JBW. items hallucinations. Positive symptoms. Gibbon M. Iowa City. assessment and research opportunities. for his help in preparing the data set. 21:395– 403. Horan. SANS global scores. Hence. Shaner A. Schiz Bull. The low internal consistency is concerning as it raises questions about whether the IPT-15 is assessing a single domain or construct.09 ⫺0.04 ⫺0.com The present study shows that the 2 approaches to ecological validity (verisimilitude and veridicality) can dissociate. Andreassen OA (2008) Emotion perception and learning potential: Mediators between neurocognition and social problem-solving in schizophrenia? J Int Neurops Soc. CA: University of California Extension. Chaytor N. Brekke JS. First MB. Sundet K. unusual thought content and conceptual disorganization.22 ⫺0. Green MF. 13:225–244. Spitzer RI. It is possible that understanding these different facets of social interactions involves somewhat different subprocesses. NJ: Erlbaum. Cooley EL. the IPT may actually assess more than one social perceptive domain. Brekke JS (2003) Is emotion perception a predictor of functional outcome in schizophrenia? Schiz Bull.17 a a Significant at the 0. Gur RC. Franzen MD.10 ⫺0. © 2009 Lippincott Williams & Wilkins . Green MF. Mintz J. 163:448 – 454. Am J Psych.13:181–197.Vaskinn et al. Carpenter WT.g. veridicality through the restricted range of scores and reduced sampling of items in the relevant domain. Spitzer RI. Green MF. Berkeley. Green MF (2006) Social perception as a mediator of the influence of early visual processing on functional status in schizophrenia. but are nevertheless highly informative regarding risk factors and community functioning. Benjamin L (1996) The Structured Clinical Interview for DSM-IV Axis II Personality Disorders. Goodman SH. First MB. Simonsen C. Biological and Neuropsychological Mechanisms: Life-Span Developmental Psychology (pp 51– 69).15.03 ⫺0. the continuous performance test for vigilance. Penn DL. ACKNOWLEDGMENTS The authors thank William P. It is well-tolerated by test takers and differentiates between people with and without presumed social cognitive difficulties. It should be noted that many tests used in schizophrenia research (e. 702 | www. Measures with a larger number of items would likely have better veridicality. Schmitter-Edgecombe M (2003) The ecological validity of neuropsychological tests: A review of the literature on everyday cognitive skills. or some tests of episodic memory) are not seen as suitable for internal consistency calculation. Roberts DL (2006) The functional significance of social cognition in schizophrenia: A review. BPRS. Community Functioning indicates role functioning scale. PhD. The IPT-15 is a measure highly similar to daily tasks. Kee KS.