See discussions, stats, and author profiles for this publication at: https://www.researchgate.

net/publication/247720594

Implicit Measurement of Attitudes, Stereotypes, and Self-

Concepts in Organizations: Teaching Old Dogmas New Tricks

Article in Organizational Research Methods · October 2006

DOI: 10.1177/1094428106286540

CITATIONS READS

50 494

2 authors:

Elizabeth L. Haines Kenneth E. Sumner

William Paterson University Montclair State University
17 PUBLICATIONS 839 CITATIONS 23 PUBLICATIONS 310 CITATIONS

SEE PROFILE SEE PROFILE

All content following this page was uploaded by Elizabeth L. Haines on 30 June 2015.

The user has requested enhancement of the downloaded file.

 Organizational Research
Methods http://orm.sagepub.com/

Implicit Measurement of Attitudes, Stereotypes, and Self-Concepts in Organizations:

Teaching Old Dogmas New Tricks
Elizabeth L. Haines and Kenneth E. Sumner
Organizational Research Methods 2006 9: 536
DOI: 10.1177/1094428106286540

The online version of this article can be found at:

http://orm.sagepub.com/content/9/4/536

Published by:

http://www.sagepublications.com

On behalf of:

The Research Methods Division of The Academy of Management

Additional services and information for Organizational Research Methods can be found at:

Email Alerts: http://orm.sagepub.com/cgi/alerts

Subscriptions: http://orm.sagepub.com/subscriptions

Reprints: http://www.sagepub.com/journalsReprints.nav

Permissions: http://www.sagepub.com/journalsPermissions.nav

Citations: http://orm.sagepub.com/content/9/4/536.refs.html

>> Version of Record - Oct 2, 2006

What is This?

Downloaded from orm.sagepub.com at William Paterson Univ of NJ on November 3, 2013

 Organizational
Research Methods
Volume 9 Number 4
October 2006 536-553
© 2006 Sage Publications
Implicit Measurement 10.1177/1094428106286540
http://orm.sagepub.com

of Attitudes, Stereotypes, and hosted at

http://online.sagepub.com

Self-Concepts in Organizations
Teaching Old Dogmas New Tricks
Elizabeth L. Haines
William Paterson University
Kenneth E. Sumner
Montclair State University

Implicit measurement using latencies is proposed as a complement to conventional measurement

to assess organizational constructs (e.g., job satisfaction), to assist in personnel decisions (e.g.,
selection), and to assess outcomes (e.g., diversity training). Latency-based measurements (i.e.,
the Implicit Association Test) use categorization tasks to measure attitudes, stereotypes, and
self-concepts. Its routine inclusion in organizational research can address measurement limi-
tations, enrich theoretical understanding of organizational phenomena, and inform practice by
better predicting behavior.

Keywords: IAT; implicit measurement; attitudes

I mplicit measurement of attitudes, stereotypes, and self-concepts using response latencies

may provide an alternative procedure for measuring constructs in organizations to
enhance prediction and understanding of work-related behaviors and to better understand
organizational phenomena (Haines & Sumner, 2003; Ingerick et al., 2002). Combining con-
temporary thinking in social cognition with technology, several computerized categorization
tasks have been developed to measure response latencies as a new form of implicit measure-
ment (e.g., Dasgupta & Greenwald, 2001; Dasgupta, McGhee, Greenwald, & Banaji, 2000;
Greenwald et al., 2002; Greenwald, McGhee, & Schwartz, 1998; Greenwald & Nosek, 2001;
Karpinski & Hilton, 2001). Work to date using latency-based implicit measurement has
focused on implicit attitudes and stereotypes of social constructs such as race and gender
(Devine, Plant, Amodio, Harmon-Jones, & Vance, 2002; Dovidio, Kawakami, & Gaertner,
2002; Fazio, Jackson, Dunton, & Williams, 1995; Greenwald et al., 1998; Rudman, Ashmore,
& Gary, 2001; Rudman & Glick, 2001; Wittenbrink, Judd, & Park, 2001) and has stimulated
a considerable amount of research over the past 5 years (Greenwald & Nosek, 2001;
Greenwald et al., 2002; Nosek, Banaji, & Greenwald, 2002). Application of this method
beyond social psychology and social cognitive research has been seen in marketing research

Authors’ Note: Parts of this article were presented to the Society for Industrial/Organizational Psychology in
2003 and 2004. The authors thank Bruce Diamond, Theresa Brown, and Denise Haines for their comments on
earlier drafts. Please address correspondence to Elizabeth L. Haines, Science Hall, William Paterson University,
Wayne, NJ 07470; e-mail: hainese@wpunj.edu.

536
 Haines, Sumner / Implicit Measurement 537

(Brunel, Tietje, & Greenwald, 2004; Maison, Greenwald, & Bruin, 2001), health and clinical
psychology (Marsh, Johnson, Scott-Sheldon, 2001; Teachman, Gapinski, Brownell, Rawlins, &
Jeyaram, 2003; Teachman, Gregg, & Woody, 2001; Teachman & Woody, 2003), personality
measurement (Asendorpf, Banse, & Mucke, 2002; Egloff & Schmukle, 2002; Egloff, Wilhelm,
Neubauer, Mauss, & Gross, 2002), and neuroscience (Phelps et al., 2000). The importance
of this work is further evidenced in the recent volume of the Journal of Personality and
Social Psychology that was devoted entirely to implicit attitude, stereotype, and self-concept
measurement (e.g., Blair, Ma, & Lenton, 2001; Dasgupta & Greenwald, 2001; Devine, 2001;
Wittenbrink et al., 2001).
In this article, we argue for a more general application of this methodology to assess
attitudes, stereotypes, and self-concepts that are important to organizational functioning
(Haines & Sumner, 2003; Sumner & Haines, 2004). In the review that follows, we define
implicit social cognition and describe its measurement, differentiating it from other social
psychological and personality measurement techniques such as self-report and projective
measurements. We then discuss what is known about the psychometric characteristics of
one flexible and widely used implicit measure, the Implicit Association Test (IAT).
Application of the IAT to assessing and understanding job attitudes, measuring and under-
standing stereotypes in organizations and their effects on decisions, and addressing the
potential use of implicit self-concepts in personnel testing and selection are examined.
Last, limitations and future directions for the use of implicit measurement to enhance appli-
cation and to aid in understanding factors important in organizational functioning are
discussed.

Implicit Social Cognition and Its Measurement

Although implicit social cognition has received an abundance of attention in the past
decade, the underlying constructs are not new to psychology. Indeed, as David Westen
(1998) clearly pointed out, the cognitive revolution revived interest in the unconsciousness
after a time when behaviorism declared the unconsciousness unnecessary. Even so, the psy-
chological unconsciousness is critical for understanding many psychological phenomena.
Furthermore, social cognitists have not fully credited their psychodynamic roots for many
of the ideas that have their basis in psychodynamic constructs, such as defense mecha-
nisms. For example, Baumeister, Dale, and Sommer (1998) showed that several defense
mechanisms are well supported by social cognitive research (e.g., attribution as projection,
counterfactual thinking as undoing); defense mechanisms, however, have been recast from
protecting threatening sexual/aggressive impulses to enhancing self-esteem (see also Cramer,
2000, for similar arguments).
Implicit social cognition refers to thought processes that (a) are influenced by past expe-
rience and (b) affect current processing but (c) lack introspective awareness (Greenwald &
Banaji, 1995). Implicit social cognition extends social cognition’s general focus on under-
standing the self in a social world (Fiske & Taylor, 1991) in two ways. First, implicit social
cognition focuses on how mental processes occur and affect behavior without conscious
guidance or control (e.g., Bargh, 1997; Bargh & Chartrand, 1999). Second, implicit social
538 Organizational Research Methods

cognition measures implicit attitudes, stereotypes, and self-concepts indirectly, based

on latencies to specially developed computerized tasks (Greenwald et al., 1998, 2002;
Greenwald & Nosek, 2001). Although the social cognition literature uses a variety of
methods to assess attitudes, stereotypes, and self-concepts, most of these measures rely on
self-reports or what are referred to as explicit measures.
The increased interest and use of implicit measurement may originate from two sources.
First, self-report questionnaires, or explicit measures, commonly used in organizational
research have well-known measurement limitations. Among these are problems that may
arise from (a) a participant’s lack of introspection (e.g., Asendorpf et al., 2002; Greenwald &
Banaji, 1995; Nisbett & Wilson, 1977) and/or (b) a participant’s potential motivation to
distort responses due to a variety of factors, including demand characteristics (Orne, 1962),
evaluation apprehension (Rosenberg, 1969), and/or impression management (Tedeschi,
Schlenker, & Bonoma, 1971).
Second, contemporary thinking on attitude-behavior linkages suggests that dual processes
may operate linking attitudes and behavior (e.g., Chaiken & Trope, 1999; Fazio, Towles-
Schwen, 1999). In 1989, McClelland, Koestner, and Weinberger proposed that dual motives
underlie preferences and behavioral tendencies: one explicit and one implicit. In this frame-
work, the explicit motives activate controlled behavior (i.e., behavior that is under conscious
direction and is most likely taught through socialization), and the implicit motive activates
automatic behaviors (i.e., reactions, emotions, nonverbal behaviors that often operate uncon-
sciously that originate in early affective experiences). Thus, it may be possible for an indi-
vidual to have an implicit motive that is different from an explicit motive. Thus, two
measures (one implicit and one explicit) assessing the same construct (e.g., a behavioral ten-
dency) can be related to different behaviors (see also Wilson, Lindsey, & Schooler, 2000, for
a discussion of dual attitudes). Perhaps a reason for the often weaker than expected
attitude-behavior linkages seen in many organizational research areas is due in part to a mis-
match of implicit and explicit predictors with controlled and automatic criteria. Personality
research recently addressed this mismatch in the personality domain including double dis-
sociation in the context of shyness (Asendorpf et al., 2002) and process dissociation in the
context of dependency (Bornstein, 2002). Thus, measuring implicit constructs may assist in
our understanding of conditions under which attitudes predict behaviors. From this, implicit
measures may generate a yet-untapped assessment of preferences, attitudes, stereotypes, and
self-concepts that advance prediction of behavior in organizations.

Implicit Measurements That Are Indirect but Not Projective

Implicit measurement, although indirect in nature, must be distinguished from other
indirect assessments, such as projective measures of personality (e.g., Rorschach Inkblot
Method, Thematic Apperception Test, Draw a Person Test). Whereas projective personal-
ity tests are based on the idea that the structure an individual places on ambiguous stim-
uli reflects aspects of the self (Anastasi & Urbina, 1997), implicit measurement is rooted
in social cognition where the response latencies used to evaluate the categorization speed
of words or pictures represent attitude, stereotype, or self-concept (Greenwald et al., 2002).
Furthermore, there is considerable debate regarding the scoring systems of projective
 Haines, Sumner / Implicit Measurement 539

assessment (Geiser & Stein, 1999; Hunsley & Bailey, 1999), whereas many scoring
systems of implicit measures are objective in nature and are based on aggregating a
participant’s latencies (e.g., Dovidio, Kawakami, Johnson, Johnson, & Howard, 1997;
Fazio et al., 1995, Gaertner & McLaughlin, 1983; Greenwald et al., 1998; Greenwald &
Nosek, 2001). Furthermore, psychometric characteristics of reliability, validity, and
fakability of projectives can be demonstrably weak (e.g., Garb, Wood, Lilienfeld, &
Nezworski, 2002; Grove, Barden, Garb, & Lilienfeld, 2002), whereas research on the psy-
chometrics of implicit measurement suggest these measures generally fall within accept-
able professional standards (Banse, Seise, & Zerbes, 2001; Greenwald & Nosek, 2001;
Kim & Greenwald, 1998; Nosek et al., 2002). Although many implicit measures exist
(e.g., Fazio, Sanbonmatsu, Powell, & Kardes, 1986) we focus on one implicit measure—
the IAT—that has generated considerable research and psychometric activity in the years
since its debut in 1998.

The IAT
Greenwald et al. (1998) introduced a method for assessing people’s attitudes: the IAT.
Using a straightforward method, they showed that implicit attitudes could be measured by
the strength of associations respondents held between a category (e.g., flowers) and an eval-
uation (e.g., pleasant). Furthermore, the implicit attitude (a) revealed more bias than the
self-reported attitudes did, (b) was weakly correlated to the explicit attitude, and (c) gener-
ated large effect sizes (e.g., d > 1.0). Several types of attitudes were demonstrated includ-
ing the implicit preferences (a) for flowers versus insects, (b) that Korean Americans have
for Korean versus Japanese names (and Japanese have for Japanese vs. Korean names), and
(c) that European Americans have for European names versus African American names.
Although response latencies had been used before to evaluate attitudes in implicit ways
(e.g., Bargh, Chaiken, Govender, & Pratto, 1992; Fazio et al., 1986, 1995; Gaertner &
McLaughlin, 1983; Kawakami, Dion, & Dovidio, 1998), the IAT has generated consider-
able attention since its introduction for a variety of practical, theoretical, and methodolog-
ical reasons.
The IAT has three critical features: implicitness, difference, and association. The IAT
measures automatic associations (Greenwald & Nosek, 2001) such as the automatic asso-
ciation between a concept and an evaluation (implicit attitude), a concept and an attribute
(implicit stereotype), or self and an attribute (implicit self-concept). These associations are
implicit because the participants’ attitudes (or self-concept or stereotype) is revealed
through latencies, rather than the participants’ reporting their attitudes themselves. An
implicit association, or IAT effect, is measured as the difference between two types of com-
bined categorization tasks in a similar way to the Stroop (1935) method. If a category and
a valence, such as African American and unpleasant, are strongly associated, then it should
be easier (i.e., shorter averaged latencies) to categorize African American and unpleasant
stimuli when they share a response key than when African American and pleasant (i.e., longer
averaged latencies) stimuli share a response key.

IAT procedure. The IAT directs participants to respond to words and/or pictures that are
presented to them sequentially on a computer screen. The participants are instructed to
540 Organizational Research Methods

Table 1
Implicit Association Test Steps for Race Attitude
Left-Hand Response (“e”) Right-Hand Response (“i”)

Step 1 European American African American

Step 2 Pleasant Unpleasant
Step 3 European American or African American or
(Combined task) Pleasant Unpleasant
Step 4 African American European American
Step 5 African American or European American or
(Combined task) Pleasant Unpleasant

Note: In Step 3, if the word diamond or a picture of a European American face was presented, the correct
response would be a left-hand response. In Step 5, if the word diamond was presented, the correct response
would be a left-hand response; if a picture of a European American was presented, the correct response would
be a right-hand response.

categorize the stimuli as quickly as possible according to several sorting instructions using
a left- (such as the “e” key) or right-handed (such as the “i” key) response. Thus, the main
measurement is a response latency: the time it takes in milliseconds for a participant to
categorize a stimulus by pressing a computer key. In an IAT task, the stimuli (e.g., pictures
or words) actually represent four categories. Category labels (e.g., pleasant, unpleasant,
African American, European American) stay on the computer screen throughout the task.
Implicit race attitudes have been one of most influential uses of the IAT (Dasgupta et al.,
2000; Dasgupta & Greenwald, 2001; Greenwald et al., 1998; Rudman, Ashmore, & Gary,
2001), and a race IAT clearly exemplifies the basic steps involved in taking an IAT and the
critical combined tasks. In an IAT, two categories represent a valence dimension such as
pleasant (e.g., diamonds, sunshine, joy) and unpleasant (e.g., stink, filth, hatred). The other
two categories represent a concept category such as African American and European
American (e.g., faces of unfamiliar African Americans and European Americans). A par-
ticipant is first instructed to sort the category items quickly using a left- or right-hand
response, where European American faces are assigned a left-hand response and African
American faces are assigned to the right-hand response (Step 1). Typically, participants
enter responses on a computer keyboard using the letter “e” as the left-hand response key
and the letter “i” as the right-hand response key. Then, a participant is instructed to sort the
attribute items quickly using a left-hand response for pleasant words or right-hand response
for unpleasant words (Step 2). As seen in Table 1, a combined task follows next in which
participants sort items when European American and pleasant items are paired together in
their response (i.e., left hand) and African American and unpleasant are paired together in
their response (right hand). That is, participants sort pleasant and European American
items using the same left-hand key and unpleasant and African American items using the
same right-hand key (Step 3). In the next task, the concept key designations are changed;
European American items are reassigned to a right-hand response and African American
items are reassigned to a left-hand response (Step 4). A final sorting task has participants
 Haines, Sumner / Implicit Measurement 541

sort items when European American items and unpleasant share a right-hand response and
African American and pleasant share a left-hand response (Step 5).
An IAT effect is calculated as the difference between the European American + pleasant
(Step 3) versus the African American + unpleasant (Step 5) sorting speeds. If a participant
had an implicit preference for European Americans, it should be easier for the individual to
categorize the European American and pleasant stimuli when they share the same response
key, resulting in relatively quick categorization speeds or short average latencies in this
combined task (Step 3). On the other hand, in Step 5, when African American and pleasant
stimuli share the same response key, the same individual would take longer to complete this
task, resulting in longer average latencies. Thus, the differences in sorting speeds are a
proxy for the strength of the association between a concept and evaluation, or implicit atti-
tude. It should be noted that IAT measures are relatively quick to administer, and the IAT
as described above would consist of approximately 140 presentations of stimuli that a par-
ticipant would categorize; categorization speeds for an item range typically between 300 and
3,000 milliseconds.

Beyond Implicit Attitudes: Implicit Measurement

of Stereotypes and Self-Concepts
The implicit attitude measures the strength of the association between a concept (e.g.,
social category) and an evaluation (positive or negative). The IAT can be used for mea-
surement of other social cognition variables as well (Rudman, Greenwald, Mellott, &
Schwartz, 1999).

Implicit stereotypes. An implicit stereotype can be measured as the association between

a group and a concept (e.g., Rudman et al., 2001; Teachman et al., 2003). An IAT con-
structed to measure implicit stereotypes measures the difference between sorting tasks
when one category is paired with a concept (e.g., African American + crime) as when con-
trasted with another social category (e.g., European American + crime). For example, mea-
surement of stereotypes that connect the social category men with work and women with
home can be assessed with an IAT. It is possible that this type of implicit stereotype may
underlie hiring and promotion decisions where these have favored men over women,
despite the tendency for paper-and-pencil measures to mask this possible stereotype.

Implicit self-esteem. Implicit self-esteem is measured by the association between self and
positive valence (e.g., me + pleasant) and contrasts it to self and negative valence (e.g., me +
unpleasant). Greenwald and Farnham (2000) have shown that an IAT assessing self-esteem
weakly correlated with traditional self-report measures, uniquely predicted reactions to suc-
cess and failure, and was overall a stronger measure of self-esteem than the self-report was
(as indicated by effect sizes). In organizational settings, implicit measures of self-esteem may
be useful for selection or promotion decisions or as an additional mediator/moderator of the
job satisfaction–job performance or job satisfaction–life satisfaction relationship.

Implicit self-concept. Implicit self-concept assesses the association between self and a
concept by contrasting the association between self and one concept (e.g., me + dominant)
542 Organizational Research Methods

and self and a contrast concept (e.g., me + subordinate). Although implicit and explicit
concepts are correlated with one another, implicit self-concept uniquely predicts behavior
(e.g., Asendorpf et al., 2002). Assessing implicit self–concepts may be important in iden-
tifying individual differences in work centrality or measuring personality characteristics
in personnel testing and selection such as conscientiousness or extroversion. Implicit self-
concept may also help us to understand how an individual’s self-concept may affect career
transitions, such as transitions between work and retirement.

Reliability, Validity, and Fakability of Implicit Measurement

The IAT has been shown to be a reliable and valid measure. In 2001, the psychometric
quality of the IAT was reviewed (Greenwald & Nosek, 2001; see also Cunningham,
Preacher, & Banaji, 2001) and was shown to be reliable over time as well as within the
measure. Test-retest reliabilities of .61 were reported by Cunningham et al. (2001) over a
6-week period. Others (e.g., Asendorpf et al., 2002; Cunningham et al., 2001) have reported
split-half internal consistency estimates of implicit measures between .78 and .89 for these
measures. Implicit measures are not affected by the number of items used, the intertrial
interval, or handedness (Greenwald et al., 1998). However, the order in which the tasks are
presented (e.g., European American + pleasant categorization task first or second) will
often produce order effects; increasing the number of practice trials before the fifth step
reduces order effects. IAT results converge with affective priming to the degree that affec-
tive priming is itself reliable (Cunningham et al., 2001; Rudman & Kilianski, 2000).
Construct validity evidence for implicit measures has been reported in the literature. For
example, IAT measures routinely correlate with self-report measures of the same construct
(e.g., Greenwald et al., 1998); whether this convergence is a strength or weakness of the
measure is of debate (see the Challenges, Considerations, and Future Directions section
below). Implicit measures have also been found to predict a variety of behaviors including
(a) nonverbal friendliness (McConnel & Liebold, 2001), (b) spontaneous shy behavior
(e.g., Asendorpf et al., 2002), (c) cardiovascular changes in anxiety reactions (Egloff &
Schmukle, 2002), (d) condom use (Marsh et al., 2001), and (e) amygdala activation in fear
responses to African American faces (Phelps et al., 2000). Interestingly, across these stud-
ies, these same behaviors were often not found to be related to the explicit measures (i.e.,
paper-and-pencil questionnaires) of the same constructs. Several studies have demonstrated
that faking instructions do not alter IAT effects (Banse et al., 2001; Egloff & Schmukle,
2002; Kim & Greenwald, 1998). For example, Asendorpf and colleagues (2002) told par-
ticipants to fake extroversion to obtain a sales position in a simulated job interview. The
self-shy association measured by the IAT was not affected, whereas the self-report results
changed under instruction.

Implicit Measurement in Organizational

Settings: A First Look
We argue that implicit measurement may tap unique components of attitudes, self-concepts,
and stereotypes (e.g., Asendorpf et al., 2002; Karpinski & Hilton, 2001; Rudman et al.,
 Haines, Sumner / Implicit Measurement 543

2001); previously proposed linkages between these variables and behavior in organizations
may be reexamined using IATs. The practical applications of implicit measurement include
the broad-based assessment of employee attitudes, the evaluation of training programs
in which attitudinal or stereotypic change is important, and use in personnel testing and
selection.

Implicit Measurement: Organizational Attitudes

Considerable research has examined organizational attitudes such as satisfaction and
commitment (Cranny, Smith, & Stone, 1992; Judge, Thoresen, Bono, & Patton, 2001; Locke,
1970; Porter, Steers, Mowday, & Boulian, 1974). Across these attitudinal constructs, all
measures have been paper-and-pencil self-report questionnaire measures or computerized
adaptations of such measures (Haines & Sumner, 2003, 2004). Work attitudes such as com-
mitment and satisfaction may be assessed via implicit measurement, and preliminary data
reported in Sumner and Haines (2004) support this claim.
In this research, an IAT to measure implicit job satisfaction was developed to measure
the association between the category, my job, and the valence, pleasant versus my job and
unpleasant. Results indicated significant correlations between the implicit job satisfaction
measure and the Work subscale (α = .90) of the Job Descriptive Index (JDI; Balzer et al.,
1990; P. C. Smith, Kendall, & Hulin, 1969; r = .38, p < .01) and between the implicit job
satisfaction measure and the Minnesota Satisfaction Questionnaire (MSQ; α = .89; Weiss,
Dawis, England, & Lofquist, 1967) short form (r = .36, p < .01). Nonsignificant correla-
tions were found between the implicit measure of work satisfaction and the Pay,
Promotions, Coworker, and Supervisor subscales of the JDI (all ps > .10), even though the
JDI subscales correlated with each other. This pattern of correlations suggests that implicit
measures relate to explicit measures but may possess unique measurement characteristics.
Self-monitoring (Snyder, 1974; α = .70; r = −.08, ns) and social desirability (Crowne &
Marlow, 1960; α = .74; r = .10, ns) were not associated with implicit satisfaction, sug-
gesting discriminant validity. Furthermore, when instructed to fake positive or negative
job attitudes, students were able to do so easily on the paper-and-pencil versions of
the JDI and MSQ but were not able to do so on the computerized implicit measure,
F(2, 70) = 2.32, p = .10. These results suggest that an implicit measure of job satisfaction
can assess job satisfaction and explore other facets of satisfaction such as coworker and
supervisor satisfaction. Subsequent research examining links among satisfaction, other
work attitudes, and performance measures might include an implicit measure of attitude
and compare the predictive efficiency of the implicit measure to that of the explicit mea-
sure. Findings might then enhance theoretical understanding among satisfaction, perfor-
mance, commitment, citizenship, loyalty, involvement, and practical applications to
enhance organizational effectiveness that are free from (a) the monomethod bias seen in
the empirical literature and (b) overly complicated models linking satisfaction to perfor-
mance (e.g., Judge et al., 2001).
Recently, Ziegert and Hanges (2005) investigated if overt, subtle, and implicit measures
of racial prejudice interact with organizational climate to predict discrimination directed at
544 Organizational Research Methods

a Black job applicant. In their experimental study, they manipulated the race of the applicant
(Black or White) and the organizational climate (no statement regarding climate or memo
from the company president stating preference for White candidates) and used explicit
measures of racial prejudice and a race attitude IAT to predict evaluations of job candidates.
Results indicated that the IAT interacted with organizational climate to predict discrimi-
nation. More specifically, participants with more implicit bias directed at Blacks and
who were presented with a biased organizational setting against Blacks evaluated Black
job applicants less favorably than those participants without a strong implicit preference in
neutral organizational climates. This finding underscores the need to evaluate implicit atti-
tudes in organizations as they may interact with climate variables to create prejudice toward
minority groups.

Implicit Measurement: Stereotypes in Organizations

In implicit social cognition, implicit stereotypes are evident in the differential speed in
pairing one group with an attribute (e.g., women + work) as compared to another group with
the same attribute (e.g., men + work). Assessing the relationship between the category,
women, and an attribute dimension important for work such as success, power, or leader-
ship would measure an implicit stereotype.
Although few people would admit holding overt negative stereotypes of women or ethnic
or racial minorities in the workplace, the extent that a person implicitly views being a
woman or African American as being inconsistent with success, power, or achievement may
affect organizational judgments (e.g., hiring, promotion, dismissal, salary). Furthermore,
these tacit attitudes may underlie subtle behaviors, such as social distancing, that are more
difficult to document as are overt biases. As a case in point, consider findings from McConnel
and Leibold (2001), in which they demonstrated that an implicit attitude measure predicted
lower nonverbal friendliness toward an African American confederate when explicit
measures did not.
Rudman et al. (2001) examined stereotype change as a function of diversity education.
In this quasi-experiment, Black-White stereotypes were assessed using both implicit and
explicit measures. Participants in the study were students enrolled in either a prejudice and
conflict course instructed by an African American male professor or a psychology methods
course instructed by a White female professor (control condition). Results indicated that
those in the conflict and prejudice seminar reduced both implicit and explicit attitudinal and
stereotype biases toward Blacks more than those participants in the control condition did.
Furthermore, results showed that explicit stereotype change altered by cognitive processes
(e.g., greater knowledge of prejudice faced by African Americans) and implicit stereotype
change found to be associated with affective processes (e.g., positive reactions to the course
and instructor) were associated with changes in implicit measures. These findings suggest
that implicit measures assess different aspects of attitudes and stereotypes and are affected
by different types of change. Examining the “glass ceiling” effect or reactions to affirma-
tive action programs using an implicit measure of stereotypes might provide insight into
this potential bias and demonstrate the usefulness of implicit measures in predicting and
changing judgmental behaviors.
 Haines, Sumner / Implicit Measurement 545

Implicit Measurement: Self-Concepts in Organizations

Implicit self-concepts can be thought of as attitudes and stereotypes one holds about the
self, rather than the attitudes and stereotypes one holds about others or objects. Research
examining implicit self-concepts has been reported and published on several self-attitudes
and stereotypes including self-esteem (e.g., Bosson, Swann, & Pennebaker, 2000; Farnham,
Greenwald, & Banaji, 1999; Greenwald & Farnham, 2000), shyness (Asendorpf et al.
2002), and anxiety (Egloff et al., 2002; Egloff & Schmukle, 2002).
Implicit measures relate to explicit measures, but they also predict different behaviors
than explicit measures or predict behavior differently than explicit measures. For example,
Asendorpf et al. (2002) reported data from two studies indicating that implicit and explicit
measures of the personality trait of shyness correlate moderately with each other, although
they differed in the extent they predicted shyness behavior. Shyness behavior in this study
was coded as spontaneous or controlled. Participants were separated into either sponta-
neous shyness groups as indicated by body posture and tension, facial adaptation (e.g.,
touching one’s face or neck), and body adaptation (e.g., rubbing one’s arm to reduce anxi-
ety) or controlled shyness groups as indicated by the use of movement to illustrate speech
or using movements with culturally defined meaning (e.g., gaze). Asendorpf et al. (2002)
reported that the explicit shyness measure predicted controlled shyness behavior, but the
implicit shyness measurement predicted spontaneous shy behavior. The researchers
described this difference as double dissociation: Two measures (i.e., one implicit and one
explicit) measuring the same construct (i.e., shyness) can produce two different behavioral
outcomes (i.e., controlled or spontaneous shy behavior).
A further demonstration of double dissociation was demonstrated by Egloff and col-
leagues (Egloff et al., 2002; Egloff & Schmukle, 2002), who contrasted and validated
implicit and explicit measures of anxiety. Results from their program of research indicated
that (a) the implicit measure was a better predictor of actual anxious behavior than were
self-reports of anxiety, (b) the implicit measure was found to be more resistant to faking
than the explicit anxiety measure, and (c) the implicit anxiety measure predicted cardio-
vascular reactivity more than the explicit measure did (which was unable to predict heart
rate or blood pressure).
Implicit self-concept measures may be useful for personnel testing and selection and to
job performance theories (Ingerick et al., 2002) because implicit self-concepts tend to be
reliable, valid, and resistant to faking. Given the resurgence of interest in personality mea-
sures as predictors of job performance in recent years (e.g., Cellar, Miller, Doverspike, &
Klawsky, 1996; Hogan & Holland, 2003; Ryan, Ployhart, & Friedel, 1998; Sackett, Gruys, &
Ellingson, 1998; Schmit & Ryan, 1993; D. B. Smith, Hanges, & Dickson, 2001), the appli-
cation of implicit measures of personality constructs is obvious: Implicit measures of
conscientiousness and extroversion need to be developed and examined for criterion-
related validity in predicting work behavior in organizations. Reiman, Bel-Bahar, and
Harbke (2003) reported the preliminary development of such measures on their Web site
and are continuing their work on the psychometrics of these measures. Furthermore,
the development of an implicit measure of employee reliability or integrity would be
useful in many hiring contexts because it may be less susceptible to faking, as with dis-
simulation effects (e.g., Ellingson, Smith, & Sackett, 2001; Mueller-Hanson, Heggestad, &
546 Organizational Research Methods

Thornton, 2003; Rosse, Stecher, Miller, & Levin, 1998; Stark, Chernyshenko, Chan, Lee, &
Drasgow, 2001).

Challenges, Considerations, and Future Directions

Despite the interest surrounding the use of implicit measures for attitudes, stereotypes,
and self-concepts, these measures have practical and theoretical considerations. Recently,
Greenwald (2001, 2004) addressed 10 limitations of the IAT; we will address several of the
considerations here, but we recommend that IAT users be mindful of Greenwald’s reflec-
tions and recent methodological advancements (Greenwald, Nosek, & Banaji, 2003) as
well as other researchers’ concerns regarding the interpretation and use of implicit mea-
sures (e.g., Brendl, Markman, & Messner, 2001; Olson & Fazio, 2004; Steffens & Plewe,
2001). These concerns, however, may stimulate additional measurement and theoretical
advancement of implicit social cognition in organizations.

Dualistic Categories
The IAT obliges researchers to choose categories that are dualistic or that force the
category into a dualistic scheme. For example, implicit race biases are investigated by
comparing one race group to another (e.g., African American vs. European American). Two
limitations arise from the use of dualistic categories: interpretation and measurement con-
struction. First, even though many researchers are interested in assessing race attitudes as
a contrast, the interpretation of the results remains unclear. Because the European
American + pleasant association is measured simultaneously with the African American +
unpleasant association, implicit bias may be a function of favoring European American not
a bias against African American per se.
Second, although many categories are easily dualistic (e.g., gender), many social con-
structs are not (e.g., sexual orientation, race/ethnicity). Greenwald (2001) recommended
limiting IAT usage to constructs that are naturally contrasting. To correct for these two
methodological limitations, it may be useful to seek other implicit measures to assess some
implicit attitudes. For example, the Go-No-Go Association Test (Nosek & Banaji, 2001) is
more flexible and can measure implicit attitudes without a comparison category (see also
De Houwer, 2003). In sum, we recommend that researchers using IATs be mindful of
how their category choices force comparisons or exclude some of the existing comparison
categories (e.g., Hispanic/Latino).

Matched Valence of Attribute Pairs

A similar problem of the use of two categories appears on the attribute end as well.
Because the IAT also uses two attribute dimensions in the combined categorization task,
selecting attributes is important because attributes that are unmatched in valence (e.g., work
vs. leisure) could lead to measuring implicit attitudes, not implicit stereotypes. In implicit
attitude research, the attributes consist of pleasant or unpleasant words. When constructing
 Haines, Sumner / Implicit Measurement 547

a stereotype or self-concept IAT, attribute selection is more complicated and may involve
attributes that are not opposite in nature. For this reason, researchers must be cautious in
creating IATs and interpreting the results. For example, when one attribute is more positive
than the other attribute (e.g., motivated vs. lazy), the IAT assesses implicit self-esteem not
solely implicit self-concept because it is easier to associate positive concepts with the self
than negative concepts (Greenwald & Farnham, 2000). In sum, the two attributes should be
balanced based on their positivity; IATs that have valence-unbalanced attributes create
alternative explanations for the meaning if the implicit measure.

Faking
Even though the IAT appears more resistant to faking instructions than traditional paper-
and-pencil measures do, it is possible that participants may try to alter their responses,
especially if there are repeated exposures to the same IAT. One way that a respondent can
manipulate his or her outcome is to slow down responses to the pairing that would be
easier or faster. For example, a race IAT test taker could respond unnaturally slower to the
European American + pleasant categorization block. In that way, the African American +
pleasant categorization would yield slow latencies as in the European American + pleasant
categorization. This type of faking takes considerable skill and understanding of what is
being measured and why, however. Most participants are compelled to do the task well and
feel that slow responding is “wrong.” Nevertheless, it is advisable to inspect combined test
blocks for participants with very long latencies as a test buster may be identified by exag-
gerated mean latencies in a prejudice-confirming block. Still, if the tests are used to make
employment decisions, test busting may become more prevalent and problematic.

Context Effects
A reliable measure of attitudes and traits should be consistent over time. However, the
IAT seems susceptible to context effects. For example, Karpinksi and Hilton (2001) showed
that implicit age bias changes as a function of the type of stereotypes that were recently
primed (youth = good or elderly = good) prior to an IAT administration. Dasgupta and
Greenwald (2001) showed that implicit race was reduced when participants were exposed
to pro-Black exemplars (e.g., Bill Cosby); this bias reduction persisted over a 24-hour
period. Haines and Kray (2005) showed that women’s self-identification with power was
stronger when they had just occupied a powerful role. Thus, when using IAT effects as an
individual difference measure, researchers must be cautioned to avoid priming and experi-
ence as it may affect responses. Greenwald (2001) recommended standardizing administra-
tion when participants take an IAT more than once and being careful in making comparisons
between IATs taken from the same respondent.

Questions Remain Regarding

the Implicit-Explicit Correlations
The varying correlations observed between implicit measures such as the IAT and
an explicit measure of the same construct deserves more attention and analysis. In one
548 Organizational Research Methods

comprehensive study (N = 6,836), 57 IAT-explicit measures correlations ranged from .70 to

−.05, with an average correlation of r = .36 (N = 6,836) and positive correlations found for
52 of 57 of the object pairs (Nosek, 2005). On one hand, low correlations between explicit
and implicit measures of the same construct indicate that each assesses different aspects of
that construct or support for discriminant validity. Therefore, incremental validity can be
gained by combining both types of measurement (as suggested by Bornstein, 2002) when
trying to predict criteria. On the other hand, a routine method for validating a measure is to
seek areas of convergence among different measures of the same construct (Campbell &
Fiske, 1959) to indicate convergent validity. Indeed, the improved scoring algorithm for the
IAT (Greenwald et al., 2002) uses implicit-explicit correlations to select the best IAT mea-
sure (Nosek, 2005). One possible solution to the explicit-implicit problem is to identify key
moderators that strengthen or weaken the implicit-explicit relationship as Bornstein (2002)
suggested. For example, Nosek (2005) showed that the correlation between implicit and
explicit measures varies as a function of self-presentation, attitude strength, the structure of
the attitude (bipolar or unipolar), and attitude distinctiveness (from others’ attitudes; see
also Nosek & Smyth, 2005, for extended data, discussion, and metaphor on implicit-explicit
convergence and divergence).

Conclusions and Recommendations: Taking Old Dogmas to Task

In summary, implicit measurement using latency tasks such as the IAT may advance
theory, prediction, and practice in organizational psychology because it may act as an enhance-
ment or corrective to traditional measurement. In this way, researchers and practitioners can
enrich their understanding of the implicit beliefs that (a) may underlie work-related behav-
iors and (b) can change in response to training and organizational change. We recommend
extreme caution, however, in using implicit measures by themselves to understand psycho-
logical phenomena and individual differences until more is known about implicit measures
including their construct, criterion, and content validity. In addition, we believe that an
implicit measurement’s use as an individual difference measure may sometimes result in an
unfair assessment of an individual’s attitudes, stereotypes, or self-concept because implicit
associations may change in response to context. Instead, implicit measures should be rou-
tinely used with explicit measures to broaden the scope of prediction, look for interactions
among measures to predict criteria, and extend the knowledge of the psychometric properties
of implicit-explicit comparisons and contrasts (see Greenwald et al., in 2003). Furthermore,
seemingly contradictory findings in research linking attitudes, stereotypes, and self-concepts
to behavior may be clarified when viewed through a double-disassociation or process-
dissociation framework. With these cautions and promises in mind, we believe that implicit
measures may be your dogma’s new best friend.

References
Anastasi, A., & Urbina, S. (1997). Psychological testing (7th ed.). Upper Saddle River, NJ: Prentice Hall.
Asendorpf, J. B., Banse, R., & Mucke, D. (2002). Double dissociation between implicit and explicit personality
self-concept: The case of shy behavior. Journal of Personality and Social Psychology, 83, 380-393.
 Haines, Sumner / Implicit Measurement 549

Balzer, W. K., Smith, P. C., Kravitz, D. A., Lovell, S. E., Paul, K. B., Reilly, B. A., et al. (1990). User’s manual
for the Job Descriptive Index (JDI) and the Job in General (JIG) scales. Bowling Green, OH: Bowling Green
State University.
Bargh, J. A. (1997). The automaticity of everyday life. In R. S. Wyer, Jr. (Ed.), Advances in social cognition
(Vol. 10, pp. 1-61). Mahwah, NJ: Lawrence Erlbaum.
Bargh, J. A., Chaiken, S., Govender, R., & Pratto, F. (1992). The generality of the automatic attitude activation
effect. Journal of Personality and Social Psychology, 62, 893-912.
Bargh, J. A., & Chartrand, T. L. (1999). The unbearable automaticity of being. American Psychologist, 54,
462-479.
Banse, R., Seise, J., & Zerbes, N. (2001). Implicit attitudes toward homosexuality: Reliability, validity, and con-
trollability of the IAT. Zeitschrift für Experimentelle Psychologie, 48(11), 45-60.
Baumeister, R. F., Dale, K., & Sommer, K. L. (1998). Freudian defense mechanism and empirical findings in
modern social psychology: Reaction formation, projection, displacement, undoing, isolation, sublimation
and denial. Journal of Personality, 66, 1081-1124.
Blair, I. V., Ma, J. E., & Lenton, A. P. (2001). Imagining stereotypes away: The moderation of implicit stereo-
types through mental imagery. Journal of Personality and Social Psychology, 81, 828-841.
Bornstein, R. F. (2002). A process dissociation approach to objective-projective test score interrelationships.
Journal of Personality Assessment, 78, 47-68.
Bosson, J. K., Swann, W. B., Jr., & Pennebaker, J. W. (2000). Stalking the perfect measure of implicit self-
esteem: The blind men and the elephant revisited? Journal of Personality and Social Psychology, 79, 631-643.
Brendl, C. M, Markman, A. B., & Messner, C. (2001) How do indirect measures of evaluation work? Evaluating
the inference of prejudice in the Implicit Association Test. Journal of Personality and Social Psychology,
81, 760-773.
Brunel, F. F., Tietje, B. C., & Greenwald, A. G. (2004). Is the Implicit Association Test a valid and valuable
measure of implicit consumer social cognition. Journal of Consumer Psychology, 14, 385–404.
Campbell, D. T., & Fiske, D. W. (1959). Moderators of the relationship between implicit and explicit evalua-
tion. Psychological Bulletin, 56, 81-105.
Cellar, D. F., Miller, M. L., Doverspike, D. D., & Klawsky, J. D. (1996). Comparison of factor structures and
criterion-related validity coefficients for two measures of personality based on the five-factor model. Journal
of Applied Psychology, 81, 694-704.
Chaiken, S., & Trope, Y. (1999). Dual-process theories in social psychology. New York: Guilford.
Cramer, P. (2000). Defense mechanisms in psychology today: Further processes for adaptation. American
Psychologist, 55, 637-646.
Cranny, C., Smith, P., & Stone, E. (Eds.). (1992). Job satisfaction: How people feel about their jobs and how it
affects their performance. New York: Lexington Books.
Crowne, D. P., & Marlow, D. (1960). A new scale of social desirability independent of psychopathology.
Journal of Consulting Psychology, 24, 349-354.
Cunningham, W. A., Preacher, K. J., & Banaji, M. R. (2001). Implicit attitude measures: Consistency reliability,
and convergent validity. Psychological Science, 12, 163-170.
Dasgupta, N., & Greenwald, A. G. (2001). Exposure to admired group members reduces automatic intergroup
bias. Journal of Personality and Social Psychology, 81, 800-814.
Dasgupta, N., McGhee, D. E., Greenwald, A. G., & Banaji, M. R. (2000). Automatic preference for
White Americans: Eliminating the familiarity explanation. Journal of Experimental Social Psychology,
36, 316-328.
De Houwer, J. (2003). The Extrinsic Affective Simon Task. Experimental Psychology, 50, 77-85.
Devine, P. G. (2001). Implicit prejudice and stereotyping: How automatic are they? Introduction to the special
section. Journal of Personality and Social Psychology, 81, 757-759.
Devine, P. G., Plant, E. A., Amodio, D. M., Harmon-Jones, E., & Vance, S. L. (2002). The regulation of explicit
and implicit race bias: The role of motivations to respond without prejudice. Journal of Personality and
Social Psychology, 82, 835-848.
Dovidio, J. F., Kawakami, K., & Gaertner, S. L. (2002). Implicit and explicit prejudice and interracial interac-
tion. Journal of Personality and Social Psychology, 82, 62-68.
550 Organizational Research Methods

Dovidio, J. F., Kawakami, K., Johnson, C., Johnson, B., & Howard, A. (1997). On the nature of prejudice:
Automatic and controlled processes. Journal of Experimental Social Psychology, 33, 510-540.
Egloff, B., & Schmukle, S. C. (2002). Predictive validity of an implicit association test for assessing anxiety.
Journal of Personality and Social Psychology, 83, 1441-1455.
Egloff, B., Wilhelm, F. H., Neubauer, D. H., Mauss, I. B., & Gross, J. J. (2002). Implicit anxiety measure
predicts cardiovascular reactivity to an evaluated speaking task. Emotion, 2, 3-11.
Ellingson, J. E., Smith, D. B., & Sackett, P. R. (2001). Investigating the influence of social desirability on
personality factor structure. Journal of Applied Psychology, 86, 122-133.
Farnham, S. D., Greenwald, A. G., & Banaji, M. (1999). Implicit self-esteem: Using the implicit associ-
ation test. In D. Abrams & M. Hogg (Eds.), Social identity and cognition (pp. 230-248). Oxford, UK:
Blackwell.
Fazio, R. H., Jackson, J. R., Dunton, B. C. & Williams, C. J. (1995). Variability in automatic activation as an
unobtrusive measure of racial attitudes: A bona fide pipeline? Journal of Personality and Social Psychology,
69, 1013-1027.
Fazio, R. H., Sanbonmatsu, D. M., Powell, M. C., & Kardes, F. R. (1986). On the automatic activation of
attitudes. Journal of Personality and Social Psychology, 50, 229-238.
Fazio, R. H., & Towles-Schwen, T. (1999). The MODE model of attitude-behavior processes. In S. Chaiken &
Y. Trope (Eds.), Dual process theories in social psychology (pp. 97-116). New York: Guilford.
Fiske, S. T., & Taylor, S. E. (1991). Social cognition. New York: McGraw-Hill.
Gaertner, S. L., & McLaughlin, J. P. (1983). Racial stereotypes: Associations and ascriptions of positive and
negative characteristics. Social Psychology Quarterly, 46, 23-30.
Garb, H. N., Wood, J. M., Lilienfeld, S. O., & Nezworski, M. T. (2002). Effective use of projective techniques
in clinical practice: Let the data help with selection and interpretation. Professional Psychology: Research &
Practice, 33, 454-463.
Geiser, L., & Stein, M. I. (1999). Evocative images: The Thematic Apperception Test and the art of projection.
Washington, DC: American Psychological Association.
Greenwald, A. G. (2001, October). What’s wrong with the IAT? Paper presented at the meeting of the Society
of Experimental Social Psychology, Spokane, WA.
Greenwald, A. G. (2004, January). Revised top 10 list of things wrong with the IAT. Paper presented at the
Attitudes Preconference of the fifth annual meeting of the Society of Personality and Social Psychology,
Austin, TX.
Greenwald, A. G., & Banaji, M. R. (1995). Implicit social cognition: Attitudes, self-esteem, and stereotypes.
Psychological Review, 102, 4-27.
Greenwald, A. G., Banaji, M. R., Rudman, L. A. Farnham, S. D., Nosek, B. A., & Mellott, D. S. (2002).
A unified theory of implicit attitudes, stereotypes, self-esteem, and self-concept. Psychological Review,
109, 3-25.
Greenwald, A. G., & Farnham, S. D. (2000). Using the Implicit Association Test to measure self-esteem and
self-concept. Journal of Personality and Social Psychology, 79, 1022-1038.
Greenwald, A. G., McGhee, D. E., & Schwartz, J. K. L. (1998). Measuring individual differences in implicit
cognition: The implicit association test. Journal of Personality and Social Psychology, 74, 1464-1480.
Greenwald, A. G., & Nosek, B. A. (2001). Health of the Implicit Association Test at age 3. Zeitschrift für
Experimentelle Psychologie, 48(11), 85-93.
Greenwald, A. G., Nosek, B. A., & Banaji, M. R. (2003). Understanding and using the Implicit Association
Test: I. An improved scoring algorithm. Journal of Personality and Social Psychology, 85, 197-216.
Grove, W. M., Barden, R. C., Garb, H. N., & Lilienfeld, S. O. (2002). Failure of Rorschach-comprehensive-
system-based testimony to be admissible under the Daubert-Joiner-Kumho standard. Psychology, Public
Policy, & Law, 8, 216-234.
Haines, E. L., & Kray, L. (2005). Self-power associations: The possession of power impact women’s self-concepts.
European Journal of Social Psychology, 35, 643-662.
Haines, E. L., & Sumner, K. E. (2003, April). Using Implicit Attitude Measurement in industrial organizational
psychology: Research and practice. Master tutorial, Society for Industrial Organizational Psychology,
Orlando, FL.
 Haines, Sumner / Implicit Measurement 551

Haines, E. L., & Sumner, K. (2004, April). The what, why, and how of implicit measurement: A primer. Paper
presented at the annual meeting of the Society for Industrial Organizational Psychology, Chicago.
Hogan, J., & Holland, B. (2003). Using theory to evaluate personality and job-performance relations: A socioanalytic
perspective. Journal of Applied Psychology, 88, 100-112.
Hunsley, J., & Bailey, J. M. (1999). The clinical utility of the Rorschach: Unfulfilled promises and an uncertain
future. Psychological Assessment, 11, 266-277.
Ingerick, M., Dudley, N. M., Cortina, J. M., Orvis, K. A., Lev-Arey Margalit, D., & Baughman, K. (2002,
April). Integrating implicit cognitions into theories of job performance: Issues of convergent, discriminant,
and predictive validity. Paper presented at the 17th annual conference of the Society for Industrial and
Organizational Psychology, Toronto, Canada.
Judge, T. A., Thoresen, C. J., Bono, J. E., & Patton, G. K. (2001). The job satisfaction-job performance rela-
tionship: A qualitative and quantitative review. Psychological Bulletin, 127, 376-407.
Karpinski, A., & Hilton, J. L. (2001). Attitudes and the implicit association test. Journal of Personality and
Social Psychology, 81, 774-788.
Kawakami, K., Dion, K. L., & Dovidio, J. (1998). Racial prejudice and stereotype activation. Personality and
Social Psychology Bulletin, 24, 407-416.
Kim, D. Y., & Greenwald, A. G. (1998, May). Voluntary controllability of implicit cognition: Can implicit atti-
tudes be faked? Paper presented at the meetings of the Midwestern Psychological Association, Chicago.
Locke, E. A. (1970). Job satisfaction and job performance: A theoretical analysis. Organizational Behavior and
Human Performance, 5, 484-500.
Maison, D., Greenwald, A. G., & Bruin, R. (2001). The Implicit Association Test as a measure of implicit con-
sumer attitudes. Polish Psychological Bulletin, 32(1), 61-69.
Marsh, K. L., Johnson, B. L., & Scott-Sheldon, L. (2001). Heart versus reason in condom use: Implicit versus
explicit attitudinal predictors of sexual behavior. Zeitschrift für Experimentelle Psychologie, 48, 161-175.
McClelland, D. C., Koestner, R., & Weinberger, J. (1989). How do self-attributed and implicit motives differ?
Psychological Review, 96, 690-702.
McConnell, A. R., & Leibold, J. M. (2001). Relations among the Implicit Association Test, discriminatory
behavior, and explicit measures of racial attitudes. Journal of Experimental Social Psychology, 37, 435-442.
Mueller-Hanson, R., Heggestad, E. D., & Thornton, G. C., III. (2003). Faking and selection: Considering
the use of personality from select-in and select-out perspectives. Journal of Applied Psychology, 88,
348-355.
Nisbett, R. E., & Wilson T. D. (1977). Telling more than we can know: Verbal reports on mental processes.
Psychological Review, 84, 231-259.
Nosek, B. A. (2005). Moderators of the relationship between implicit and explicit evaluation. Journal of
Experimental Psychology: General, 134, 565-584.
Nosek, B. A., & Banaji, M. R. (2001). The go/no-go association task. Social Cognition, 19(6), 161-176.
Nosek, B. A., Banaji, M. R., & Greenwald, A. G. (2002). Harvesting implicit group attitudes and beliefs from
a demonstration Website. Group Dynamics, 6(1), 101-115.
Nosek, B. A., & Smyth, F. L. (2005). A multitrait-multimethod validation of the Implicit Association Test:
Implicit and explicit attitudes are related but distinct constructs. Unpublished manuscript.
Olson, M. A., & Fazio, R. H. (2004). Reducing the influence of extra-personal associations on the Implicit
Association Test: Personalizing the IAT. Journal of Personality and Social Psychology, 86, 653-667.
Orne, M. T. (1962). On the social psychology of the psychological experiment: With particular reference to
demand characteristics and their implications. American Psychologist, 17, 776–783.
Phelps, E. A., O’Connor, K., Cunningham, W., Funayama, E., Gatenby, J., Gore, J., et al. (2000). Performance
on indirect measures of race evaluation predicts amygdala activation. Journal of Cognitive Neuroscience, 12,
729-738.
Porter, L. W., Steers, R. M., Mowday, R. T., & Boulian, P. V. (1974). Organizational commitment, job satisfac-
tion, and turnover among psychiatric technicians. Journal of Applied Psychology, 59, 603-609.
Reiman, A. S., Bel-Bahar, T. S., & Harbke, C. R. (2003). Implicit personality structure test: Reliability and
validity of response latency measurement. Retrieved June 29, 2003, from http://www.uaa.alaska.edu/
programs/documents/
552 Organizational Research Methods

Rosenberg, M. J. (1969). The conditions and consequences of evaluation apprehension. In R. Rosenthal &
R. L. Rosnow (Eds.), Artifact and behavioral research (pp. 279-349). New York: Academic Press.
Rosse, J. G., Stecher, M. D., Miller, J. L., & Levin, R. A. (1998). The impact of response distortion on
preemployment personality testing and hiring decisions. Journal of Applied Psychology, 83, 634-644.
Rudman, L. A., Ashmore, R. D., & Gary, M. L. (2001). “Unlearning” automatic biases: The malleability of
implicit prejudice and stereotypes. Journal of Personality and Social Psychology, 81, 856-868.
Rudman, L. A., & Glick, P. (2001). Prescriptive gender stereotypes and backlash toward agentic women.
Journal of Social Issues, 57, 743-762.
Rudman, L. A., Greenwald, A. G., Mellott, D. S., & Schwartz, J. L. K. (1999). Measuring the automatic com-
ponents of prejudice: Flexibility and generality of the Implicit Association Test. Social Cognition, 17(4), 1-29.
Rudman, L. A., & Kilianski, S. E. (2000). Implicit and explicit attitudes toward female authority. Personality
and Social Psychology Bulletin, 26, 1315-1328.
Ryan, A. M., Ployhart, R. E., & Friedel, L. A. (1998). Using personality testing to reduce adverse impact:
A cautionary note. Journal of Applied Psychology, 83, 298-307.
Sackett, P. R., Gruys, M. L., & Ellingson, J. E. (1998). Ability-personality interactions when predicting job
performance. Journal of Applied Psychology, 83, 545-556.
Schmit, M. J., & Ryan, A. M. (1993). The Big Five in personnel selection: Factor structure in applicant and
nonapplicant populations. Journal of Applied Psychology, 78, 552-566.
Smith, D. B., Hanges, P. J., & Dickson, M. W. (2001). Personnel selection and the five factor model:
Reexamining the effects of applicant’s frame of reference. Journal of Applied Psychology, 86, 304-315.
Smith, P. C., Kendall, L. M., & Hulin, C. L. (1969). The measurement of satisfaction in work and retirement:
A strategy for the study of attitudes. Chicago: Rand McNally.
Snyder, M. (1974). Self-monitoring of expressive behavior. Journal of Personality and Social Psychology, 30,
526-537.
Stark, S., Chernyshenko, O. S., Chan, K., Lee, W. C., & Drasgow, F. (2001). Effects of the testing situation on
item responding: Cause for concern. Journal of Applied Psychology, 86, 943-953.
Steffens, M. C., & Plewe, I. (2001). Items’ cross-category associations as a confounding factor in the Implicit
Association Test. Zeitschrift für experimentelle Psychologie, 48, 123-134.
Stroop, J. R. (1935). Studies of interference in serial verbal reactions. Journal of Experimental Psychology, 28,
643-662.
Sumner, K. & Haines, E. L. (2004, April). Implicit measurement in I-O psychology: Empirical realities and
theoretical possibilities. Symposium, Society for Industrial Organizational Psychology, Chicago.
Teachman, B. A., Gapinski, K. D., Brownell, K. D., Rawlins, M., & Jeyaram, S. (2003). Demonstrations of
implicit anti-fat bias: The impact of providing causal information and evoking empathy. Health Psychology,
22, 68-78.
Teachman, B. A., Gregg, A. P., & Woody, S. R. (2001). Implicit associations for fear-relevant stimuli among
individuals with snake and spider fears. Journal of Abnormal Psychology, 110, 226-235.
Teachman, B. A., & Woody, S. R. (2003). Automatic processing in spider phobia: Implicit fear associations over
the course of treatment. Journal of Abnormal Psychology, 112, 100-109.
Tedeschi, J. T., Schlenker, B. R., & Bonoma, T. V. (1971). Cognitive dissonance: Private ratiocination or
public spectacle? American Psychologist, 26, 685-695.
Weiss, D. J., Dawis, R. V., England, G. W., & Lofquist, L. H. (1967). Manual for the Minnesota Satisfaction
Questionnaire: Minnesota studies in vocational rehabilitation. Minneapolis: Industrial Relations Center,
University of Minnesota.
Westen, D. (1998). The scientific legacy of Sigmund Freud: Toward a psychodynamically informed psycholog-
ical science. Psychological Bulletin, 124, 333-371.
Wilson, T. D., Lindsey, S., & Schooler, T. Y. (2000). A model of dual attitudes. Psychological Review, 107,
101-126.
Wittenbrink, B., Judd, C. M., & Park, B. (2001). Spontaneous prejudice in context: Variability in automatically
activated attitudes. Journal of Personality and Social Psychology, 81, 815-827.
Ziegert, J. C., & Hanges, P. J. (2005). Employment discrimination: The role of implicit attitudes, motivation,
and a climate for bias. Journal of Applied Psychology, 90, 553-562.
 Haines, Sumner / Implicit Measurement 553

Elizabeth L. Haines is an assistant professor of psychology at William Patterson University in Wayne, New
Jersey. She is a graduate of the University of Delaware and City University of New York with postdoctoral work
at the University of Washington, Seattle. Her research focuses on stereotyping, power, racism, gender, and
implicit measurement.

Kenneth E. Sumner is an associate professor of psychology at Montclair State University in Upper Montclair,
New Jersey. He is a graduate of Wayne State University and Bowling Green State University. His research focus
is on workplace aggression, work satisfaction, and implicit measurement.

View publication stats